Score one more for the good guys!

According to this article on CNet News, the Federal Trade Commission has just shut down an ISP called Pricewert, which had sought to act as a one-stop shopping center for spammers, child porn, botnet operators, and virus and malware distributors.

Pricewert operated as a Web host under a bunch of different names–3FN.net, Triple Fiber, APS Communications, and a bunch of others.

I first became aware of 3FN back in February of 2008, when I started seeing spam for all kinds of porn sites hosted on their IP space. The spam I saw generally involved URLs hosted on 3FN that redirected to the affiliate sites of large pay-for-access porn sites–a common spam tactic I’ve seen before, especially from big-name offenders like Streamate.com.

Pricewert/3FN’s business extended well beyond spam, though, and into hosting for botnet command and control servers, virus droppers, malware distribution, and even kiddie porn. In other words, about business as usual for an ISP in a place like the Ukraine or Latvia, but somewhat surprising for an ISP in the US. (Somewhat surprising, at least, until you consider that the founder of Pricewert/3FN was from the Ukraine, where the business culture is such that hosting malware, child porn, and botnet control servers is part of any ISP’s normal revenue stream.)

And here’s the part where I get all Ranty McRanterson.

What’s really, really, really disappointing to me is how poor the US ISPs and backbone providers are at policing themselves, and how even egregiously illegal activity is tolerated by the vast majority of Internet service providers.

3FN’s upstream providers knew that 3FN was a rogue ISP hosting criminals involved in spam, viruses, and malware. I know for a fact that they knew this, because I told them myself, with detailed evidence. In February of 2008. And in March of 2008 (four times). And in June of 2008. And in July of 2008. And in…well, you get the idea.

There is, in the world of ISPs and Internet connectivity, a tacit understanding that any sort of illegal activity, including identity theft, malware, fraud, and computer virus distribution, will be tolerated so long as it doesn’t create too big an uproar and so long as ISPs occasionally move the offenders around from one IP address to another. Even child pornography is not going to create a problem so long as the hosting ISP removes or moves the child porn if they receive complaints.

ISP abuse employees do not generate revenue for an Internet company. In fact, they cost a company revenue. For that reason, ISPs will often hobble their own abuse teams (I sent seven complaints to one ISP about a hacked server on their network over a period of two months, only to be told that the abuse people were not permitted to take down the server until eight weeks after they had notified the owner to fix the problem–which is about like calling the fire department because your neighbor’s house is on fire and the flames are spreading to your house, only to be told that the fire department would mail a notice to your neighbors, and would send the trucks out in eight weeks if the neighbors hadn’t taken care of the problem themselves by then).

ISPs make money by selling hosting and bandwidth to people. Every site they take down is lost revenue; every downstream service provider they cut off is a lot of lost revenue. They’re not going to lose that revenue unless they’re forced to.

Case in point: The rogue hosting provider McColo, which was notorious for hosting child porn, computer viruses (they were a preferred host for the Russian Zlob gang and for the Asprox virus gang), and credit card identity theft rings (Fraudcrew hosted sites on McColo), yet remained merrily in business, with no problems from their upstream providers, for four years in spite of the fact that it was widely known and publicized that McColo catered exclusively to criminal clientele.

And, sadly, that’s the norm, not the exception. Upstream and backbone providers will cheerfully provide connectivity to known-rogue ISPs even though the rogue ISPs violate not only the law but also the upstream providers’ Terms of Service. Global Crossing, a mainstream, respectable business, knew that McColo was hosting computer viruses and child porn; they simply didn’t care. The money of organized crime spends just as well as the money of honest businesses, and often there’s more of it.

In the ISP world, often government intervention is the only way to shut down these operators. History has proven, conclusively, beyond all shadow of doubt, that ISPs and connectivity providers absolutely, positively can not be counted on to police themselves; left to their own devices, they will permit just about anything to happen on their networks. The ongoing corrupt business practices of US ISP Calpop, for example, is ample proof of that.

It pisses me off to no end to see an entire industry that has, for all intents and purposes, quietly agreed to permit organized crime, identity theft, and child pornography on their networks as long as there’s not too much of a fuss about it, and to take action only against the one or two most extreme offenders after many years of operation. While I do not normally see government intervention as a good way to solve business problems, in this case I do not believe the ISPs will ever police themselves effectively, or even want to; there’s too much money in allowing this sort of network abuse. Given how widespread the problem is, I do not think there is any solution other than tighter regulation of criminal activity on the backs of ISPs’ networks.

40 thoughts on “Score one more for the good guys!

  1. Starting out in the fraud/abuse field these last few years have really been interesting. You are absolutely right though. Money does indeed win out over security every time. Big enough account? “Oh, hey, what’s that shiny thing over there?”

    It’s terribly frustrating, but even with support up to and including our Director, it won’t change.

  2. Starting out in the fraud/abuse field these last few years have really been interesting. You are absolutely right though. Money does indeed win out over security every time. Big enough account? “Oh, hey, what’s that shiny thing over there?”

    It’s terribly frustrating, but even with support up to and including our Director, it won’t change.

  3. They’re illegal and they’re highly profitable, and they both rely on moving bits over the Internet. That’s about where the similarities end, but it seems that people who are willing to do one are willing to do the other.

    Leo “Badcow” Kuvayev, for example (head of the Russian Spam Gang affiliated with Russian organized crime), has his hand in everything from porn spam to virus writing to child porn. Ditto for Alex Polyakov, a botnet operator whose businesses include viruses, mortgage and fake “pharmaceutical” spam, and child porn; the Russian Artofit gang, who started out in botnets and spam before moving on to child porn distribution…and on and on.

    What’s interesting to me is that among American spammers, porn often seems off-limits; Alan Ralsky, once one of the world’s most prolific spammers (indicted last year on charges of conspiracy, fraud in connection with electronic mail, computer fraud, mail fraud and wire fraud, for using a botnet to spamvertise penny stocks in a “pump and dump” scheme), used to brag about not doing porn spam because of his “morals”. In Eastern Europe, on the other hand, child porn doesn’t seem to be seen as anything particularly special at all; spammers, organized crime, and ISPs alike will all profit from it without a second though.

  4. They’re illegal and they’re highly profitable, and they both rely on moving bits over the Internet. That’s about where the similarities end, but it seems that people who are willing to do one are willing to do the other.

    Leo “Badcow” Kuvayev, for example (head of the Russian Spam Gang affiliated with Russian organized crime), has his hand in everything from porn spam to virus writing to child porn. Ditto for Alex Polyakov, a botnet operator whose businesses include viruses, mortgage and fake “pharmaceutical” spam, and child porn; the Russian Artofit gang, who started out in botnets and spam before moving on to child porn distribution…and on and on.

    What’s interesting to me is that among American spammers, porn often seems off-limits; Alan Ralsky, once one of the world’s most prolific spammers (indicted last year on charges of conspiracy, fraud in connection with electronic mail, computer fraud, mail fraud and wire fraud, for using a botnet to spamvertise penny stocks in a “pump and dump” scheme), used to brag about not doing porn spam because of his “morals”. In Eastern Europe, on the other hand, child porn doesn’t seem to be seen as anything particularly special at all; spammers, organized crime, and ISPs alike will all profit from it without a second though.

  5. My suspicion is that they don’t give a fuck.

    Eastern Europe has a long and ignoble history of child porn and human trafficking; maybe it’s cultural conditioning.

  6. My suspicion is that they don’t give a fuck.

    Eastern Europe has a long and ignoble history of child porn and human trafficking; maybe it’s cultural conditioning.

  7. Toxic mortgage bankers, toxic ISPs, what’s the difference?

    For those same reasons, unless people like you turn them into the FBI, the problem will not stop, and we will never have a secure internet and web system in the US, either. If the industry won’t police itself against spammers and child pornos and identity theft, then you can be sure they won’t help against hackers, either.

    Rather like expecting mortgage bankers to police themselves, isn’t it?

  8. Toxic mortgage bankers, toxic ISPs, what’s the difference?

    For those same reasons, unless people like you turn them into the FBI, the problem will not stop, and we will never have a secure internet and web system in the US, either. If the industry won’t police itself against spammers and child pornos and identity theft, then you can be sure they won’t help against hackers, either.

    Rather like expecting mortgage bankers to police themselves, isn’t it?

  9. With all due respect, you seem like kind and caring person who has a conscience and a lot empathy for others. These people are criminals, and they have very little of those things. As tacit put it, the do not give a fuck. Some of them would whore their own sisters and mothers to make a profit.

    Honestly, if someone has no compunction about stealing thousands of dollars, destroying someone’s credit, and so and so on, they probably don’t care much about a few pictures of naked children.

    It’s sad. It’s wrong. But these people are not like you and me. They are scum of the earth, and they think only of themselves. They really don’t care how many people they hurt, as long as they can get whatever it is they think they’re entitled to.

  10. With all due respect, you seem like kind and caring person who has a conscience and a lot empathy for others. These people are criminals, and they have very little of those things. As tacit put it, the do not give a fuck. Some of them would whore their own sisters and mothers to make a profit.

    Honestly, if someone has no compunction about stealing thousands of dollars, destroying someone’s credit, and so and so on, they probably don’t care much about a few pictures of naked children.

    It’s sad. It’s wrong. But these people are not like you and me. They are scum of the earth, and they think only of themselves. They really don’t care how many people they hurt, as long as they can get whatever it is they think they’re entitled to.

  11. You’re sweet. A bit naive, but sweet.

    We’re not talking about people who shoplift because they are living on the street and starving. We’re not even talking about teenagers who are selling drugs on the corner because the gangs in their neighborhood convinced them that gang life was the only way to escape mind crushing poverty.

    We all need money. Or, at least, we need the things money can provide. Food, shelter, security… everyone needs those things. And most people find honest work to provide for themselves and their families. They’re also happy with the things I listed above, and not determined to drive expensive cars, live in mansions, and have all kinds of luxury items that are paid for by stealing, cheating, and otherwise taking whatever they want regardless who it hurts or who it belonged to.

    You may not like it, but these people made a choice. They may have souls, but they do not have morals. They do not care if someone else gets hurt, because they only care about themselves. That may not make them monsters, but it does make them different. These are career criminals we’re talking about. People who made a conscious choice to profit by harming others, either materially or otherwise.

    It’s odd how quick you are to make excuses for them, and how quick you are to say that me calling them “scum” is somehow on the same level as theft, fraud and child pornography. You have some very odd values if you truly believe that. And I’m guessing you’ve never been the victim of crime, either.

  12. You’re sweet. A bit naive, but sweet.

    We’re not talking about people who shoplift because they are living on the street and starving. We’re not even talking about teenagers who are selling drugs on the corner because the gangs in their neighborhood convinced them that gang life was the only way to escape mind crushing poverty.

    We all need money. Or, at least, we need the things money can provide. Food, shelter, security… everyone needs those things. And most people find honest work to provide for themselves and their families. They’re also happy with the things I listed above, and not determined to drive expensive cars, live in mansions, and have all kinds of luxury items that are paid for by stealing, cheating, and otherwise taking whatever they want regardless who it hurts or who it belonged to.

    You may not like it, but these people made a choice. They may have souls, but they do not have morals. They do not care if someone else gets hurt, because they only care about themselves. That may not make them monsters, but it does make them different. These are career criminals we’re talking about. People who made a conscious choice to profit by harming others, either materially or otherwise.

    It’s odd how quick you are to make excuses for them, and how quick you are to say that me calling them “scum” is somehow on the same level as theft, fraud and child pornography. You have some very odd values if you truly believe that. And I’m guessing you’ve never been the victim of crime, either.

  13. This is a good thing that they are being shut down.

    I am not sure that all of the Tier {1,2,3} carriers actually know (from a true corporate decision making point of view) when their downstream customers are doing “bad things”. As you point out, the abuse teams are hobbled — understaffed, under funded. Most abuse tickets get very little attention, if they are even read. There are all sorts of mid-level management type pressures that contribute the abuse problems never being properly dealt with.

    I’d wager that any “C” level management at those companies has no idea about those issues, being as clueless as about them as most other things actually going on in their businesses. And if they did know about it (or their corporate counsel knew about it), something most likely WOULD be done. In reality, none of these rouge customers represent all that much money to any of the carriers. Really, what is $2.5k (OC-3 level, 155mbits) or even $25k per month (1 or 2 GBit/sec+) in transit fees to a telecom doing tens or hundreds of millions per year in business?

    Perhaps you should be starting with the CEO’s office for your major complaint reporting? Too often I have tried to ‘trust the system’ and work through the ‘correct’ channels, and then had much better results aiming for the top.

  14. This is a good thing that they are being shut down.

    I am not sure that all of the Tier {1,2,3} carriers actually know (from a true corporate decision making point of view) when their downstream customers are doing “bad things”. As you point out, the abuse teams are hobbled — understaffed, under funded. Most abuse tickets get very little attention, if they are even read. There are all sorts of mid-level management type pressures that contribute the abuse problems never being properly dealt with.

    I’d wager that any “C” level management at those companies has no idea about those issues, being as clueless as about them as most other things actually going on in their businesses. And if they did know about it (or their corporate counsel knew about it), something most likely WOULD be done. In reality, none of these rouge customers represent all that much money to any of the carriers. Really, what is $2.5k (OC-3 level, 155mbits) or even $25k per month (1 or 2 GBit/sec+) in transit fees to a telecom doing tens or hundreds of millions per year in business?

    Perhaps you should be starting with the CEO’s office for your major complaint reporting? Too often I have tried to ‘trust the system’ and work through the ‘correct’ channels, and then had much better results aiming for the top.

  15. Shouldn’t this be viewed as a government matter, though, not an issue on behalf of the ISP? I’m years out of date, but I remember there used to be a lot of noise about how if ISPs policed themselves, they’d no longer be neutral, and thus they’d be responsible for anything they missed.

    Basically, government regulation ensures due process is followed, and that free speech isn’t squelched by ISPs – whether they be overzealous, misunderstanding the law, or actively grinding axes.

    That said… why the hell does it take FOUR YEARS for the government to act on this one? *grumbles* We can arrest a man for owning manga but we can’t crack down on servers with actual child porn 😛

  16. Shouldn’t this be viewed as a government matter, though, not an issue on behalf of the ISP? I’m years out of date, but I remember there used to be a lot of noise about how if ISPs policed themselves, they’d no longer be neutral, and thus they’d be responsible for anything they missed.

    Basically, government regulation ensures due process is followed, and that free speech isn’t squelched by ISPs – whether they be overzealous, misunderstanding the law, or actively grinding axes.

    That said… why the hell does it take FOUR YEARS for the government to act on this one? *grumbles* We can arrest a man for owning manga but we can’t crack down on servers with actual child porn 😛

  17. How about making a law that would allow to punish (big time) upstream ISP’s if it can be proved that they knew and didn’t act?

    You cannot count on anyone to police themselves. That’s what laws are for — to create/alter incentives. Morals can create strong enough incentives in some individuals, but very rarely they alone suffice to make whole societys act in a certain way. You need good laws.

  18. How about making a law that would allow to punish (big time) upstream ISP’s if it can be proved that they knew and didn’t act?

    You cannot count on anyone to police themselves. That’s what laws are for — to create/alter incentives. Morals can create strong enough incentives in some individuals, but very rarely they alone suffice to make whole societys act in a certain way. You need good laws.

  19. If it’s normal in your society? It’s easy to justify. Look how long we justified slavery in the US. In a hundred years, we’ll look back on today and find new things to be horrified by. There’s plenty of psuedo-acceptable ways to slide in to it, too – ageplay, lolicon, shotacon. A lot of people use them as releases for pedophilic desires.

    And once you’ve normalized the first step, “it’s okay to look at this”, well, why wouldn’t you share it with your pedophile friends? And what’s wrong with charging a bit for your expenses? And what’s wrong with making a profit? Then there’s a whole community around you, and they’re all providing moral support by buying from you. It’s obviously okay, right? My peer group said so!

    Do it step by step and it’s easy to ignore the massive disconnect between actions and principles. Our brains are good at this. Look at how many times in history perfectly sane, moral people have been talked in to genocide, slavery, torture, all sorts of atrocities.

    It’s easy to look the other way.

  20. If it’s normal in your society? It’s easy to justify. Look how long we justified slavery in the US. In a hundred years, we’ll look back on today and find new things to be horrified by. There’s plenty of psuedo-acceptable ways to slide in to it, too – ageplay, lolicon, shotacon. A lot of people use them as releases for pedophilic desires.

    And once you’ve normalized the first step, “it’s okay to look at this”, well, why wouldn’t you share it with your pedophile friends? And what’s wrong with charging a bit for your expenses? And what’s wrong with making a profit? Then there’s a whole community around you, and they’re all providing moral support by buying from you. It’s obviously okay, right? My peer group said so!

    Do it step by step and it’s easy to ignore the massive disconnect between actions and principles. Our brains are good at this. Look at how many times in history perfectly sane, moral people have been talked in to genocide, slavery, torture, all sorts of atrocities.

    It’s easy to look the other way.

  21. I am sadly reminded of a Frontline special on prostitution in Germany (where prostitution is legal). It turns out most of the prostitutes were brought from Eastern Bloc countries like Russia and the Ukraine, some under false pretenses, others through outright kidnapping, very few under their own volition.

    The main story from the episode revolved around a man trying to get his wife back. A family friend took her to East Germany for a shopping trip; he sold her to pimps while they were there. Only a burst of a conscience got him to confess what he had done to the husband. He kept the money, but did (IIRC) alert the husband to where he sold her.

    Sad, sad, sad.

  22. I am sadly reminded of a Frontline special on prostitution in Germany (where prostitution is legal). It turns out most of the prostitutes were brought from Eastern Bloc countries like Russia and the Ukraine, some under false pretenses, others through outright kidnapping, very few under their own volition.

    The main story from the episode revolved around a man trying to get his wife back. A family friend took her to East Germany for a shopping trip; he sold her to pimps while they were there. Only a burst of a conscience got him to confess what he had done to the husband. He kept the money, but did (IIRC) alert the husband to where he sold her.

    Sad, sad, sad.

  23. Well, hell. This is Livejournal. We’re not going to “fix anything” here, no matter how nicely we talk about them. If we do want to fix anything, we need to treat them like the criminals they are, and and not pretend they’re just like you and me. I’ve been homeless. I lost my job, my car, and a great deal of my self-dignity when I was forced to sleep on people’s couches, ask people for money, and do all kinds of things I never thought I would have to do.

    But I never turned to a life of crime. I never decided that hurting other people, stealing from them, ruining their lives, was an acceptable way to get where I wanted to be.

  24. Well, hell. This is Livejournal. We’re not going to “fix anything” here, no matter how nicely we talk about them. If we do want to fix anything, we need to treat them like the criminals they are, and and not pretend they’re just like you and me. I’ve been homeless. I lost my job, my car, and a great deal of my self-dignity when I was forced to sleep on people’s couches, ask people for money, and do all kinds of things I never thought I would have to do.

    But I never turned to a life of crime. I never decided that hurting other people, stealing from them, ruining their lives, was an acceptable way to get where I wanted to be.

  25. you’re partly right about the isps culpability here, but there are other things you’re probably not aware of…

    working for a tier 1 isp for the last 7 years, 5 of them as a manager i can tell you that at least our abuse department wasn’t the red-headed stepchild you describe above

    unfortunately, the reality is that shutting down an abuser is tremendously difficult – the legalities involved in proving that they (our client/subscriber) is knowingly hosting illegal content are tougher than hell – pulling routing or blacklisting a single ip is easy (and done by my former employer dozens of times a day) but to terminate their contract for abuse requires at a minimum a month of serious work by 2-3 different people, and that’s if the client doesn’t sick a lawyer on us for taking action

    the other problem we ran into a lot is that as a major bandwidth provider in both NA and the EU, a lot of our clients are resellers themselves, and sometimes their clients are also resellers which makes it ridiculously difficult to track – especially since so many of our customers have their own ip space; and there is also the reality that layer 2 services and tunneling are becoming major parts of the bandwidth we provide – so we literally cannot see the types traffic, the ips, or anything that our customers are sending across our network; all we see are the total amounts of traffic across the interfaces at each end of the tunnel

    there needs to be a better way to police internet traffic and a new set of protocols for handling abuse developed… we’re moving to IPV6 for more ip space, we’re now deploying equipment that will allow up to 160 10 gigabit traffic streams across a single strand of fiber optic cable when all the components are available; but we’re still handling abuse under the rules developed when compuserve was making money

    • I’m curious why it’s so difficult to terminate an abusive downstream. Aren’t the Terms of Service incorporated into the contract?

      I mean, we’re not really talking about gray areas here. McColo stayed afloat for four years in spite of the fact that they hosted thousands of malware sites for both the Zlob and Asprox gang, the command and control servers for the world’s largest botnet, and kiddie porn. Kiddie porn, fer Chrissakes! For four years nobody shut them down.

      nd look at cases like iPipe.net, which as near as I can tell hosts no legitimate Web sites at all. All of their servers appear to be operated by the Zlob gang; I can’t tell you how many malware droppers I’ve seen living in iPipe address space.

      Another great example: American ISP Calpop, which has become one of the preferred hosts for the Russian Zlob gang; they’re still hosting active redirectors to sites that try to drop the Zlob malware, like the one at http://hardcorepornovids.net/ (warning: site is live and will attempt a drive-by download of W32/Zlob) six months and one day after I first reported it to them.

      Hell, Calpop is hosting a whole NETWORK of redirectors, traffic managers, and virus droppers for the Zlob gang; the same IP addresses in Calpop space (216.240.139.236, 216.240.139.234, 216.240.139.238, 216.240.139.241) keep turning up like a bad penny whenever I track down another Zlob downloader.

      Calpop took over the role of corrupt black-hat host when ESThosts went dark, and they don’t even bother to move the malware sites from one IP address to another. They not only don’t shut down Zlob droppers, they don’t even bother to move the sites. How much more blatant does a company have to be in order to get shut down?

      • i get what you’re saying, and 4 years is crazy; i’m not sure who the upstreams are/were for these companies so i can’t speak to specifics on that

        unfortunately, while abuse and tos are laid out in the contracts, in order to be able to term a customer for tos violations we have to be able to prove (and the proof has to stand up if it gets taken to court) that they’re knowingly violating the tos – but not all providers take 4 years to pull a bad customer off their network (and at least the due diligence performed there kept us from selling to a lot of problem customers in the first place, much to our sales department’s frustration)

        as i’m not familiar with the companies you talked about i did a bit of digging into calpop; they’re a colo hosting provider with according to their website multiple upstream providers. however, their website lies about at least 1 of their upstreams (i know they’re not customers of or peers with my former company)… but they do have space in some of the large LA colo sites where most major NA isps have connectivity

        for example 600 7th is the handoff point for pretty much every commercial isp in LA as that is where Williams Communications Group (now part of Level3) landed all their fiber into/out of LA – and they’re the major provider of fiber optic cable in north america – so even companies who don’t use the WCG/L3 fiber wind up in 600 7th to peer with the other isps

        the traceroutes to both the calpop ips you list and the site http://www.calpop.com all go through XO no matter what trace tools i use – so if you’re not getting anywhere through XO or if you’re seeing them from a different provider, there are a couple of things you may want to try

        first, contact ARIN (they assign ips, and they may pull the IP space if calpop is violating their TOS)… and as they’re hosting malware and child porn, i’d contact the FBI as well – our abuse team worked with them on a few occasions and from what they reported the experience was a good one

        and i am going to assume you know about spamhaus and their rosko listings – they specifically blacklist spam – but if these companies are allowing child porn etc they’re probably also allowing spammers and that may be a way to get them pulled down – their blacklists are among the most used by the big isps as part of the due diligence checks on potential customers and one of the quickest ways we had to prove that our clients were violating TOS

        • Hmm. All my traceroutes to Calpop go thorugh Level 3:

          traceroute to http://www.calpop.com (64.27.14.228), 64 hops max, 40 byte packets
          1 10.111.25.1 (10.111.25.1) 1.182 ms 0.764 ms 0.765 ms
          2 72.54.146.105 (72.54.146.105) 1.834 ms 1.749 ms 1.689 ms
          3 172.16.226.125 (172.16.226.125) 4.481 ms 4.649 ms 4.154 ms
          4 192.168.14.133 (192.168.14.133) 4.568 ms 5.000 ms 5.029 ms
          5 192.168.14.135 (192.168.14.135) 4.954 ms 4.870 ms 4.840 ms
          6 192.168.14.137 (192.168.14.137) 5.300 ms 5.311 ms 5.885 ms
          7 192.168.14.104 (192.168.14.104) 5.417 ms 5.309 ms 5.258 ms
          8 192.168.14.102 (192.168.14.102) 6.101 ms 6.336 ms 6.018 ms
          9 192.168.14.100 (192.168.14.100) 6.693 ms 5.986 ms 5.963 ms
          10 192.168.14.122 (192.168.14.122) 6.433 ms 6.674 ms 6.346 ms
          11 te-3-3.car1.Atlanta4.Level3.net (4.53.232.73) 21.913 ms 21.780 ms 21.844 ms
          12 * ae-72-52.ebr2.Atlanta2.Level3.net (4.68.103.61) 26.699 ms 22.022 ms
          13 ae-73-70.ebr3.Atlanta2.Level3.net (4.69.138.20) 23.809 ms 21.371 ms 34.262 ms
          14 ae-7.ebr3.Dallas1.Level3.net (4.69.134.21) 27.774 ms 27.723 ms 35.138 ms
          15 ae-3.ebr2.LosAngeles1.Level3.net (4.69.132.77) 60.491 ms 68.894 ms 71.756 ms
          16 ae-4-90.edge1.LosAngeles9.Level3.net (4.69.144.202) 59.808 ms 60.312 ms 59.597 ms
          17 CALPOPCOM-I.edge1.LosAngeles9.Level3.net (4.53.228.66) 60.514 ms 60.241 ms 60.117 ms
          18 64.27.17.189 (64.27.17.189) 59.825 ms 60.645 ms 60.217 ms
          19 64.27.14.228 (64.27.14.228) 60.729 ms 59.828 ms 61.537 ms

          When I first discovered they were providing hosting for the Zlob crew in the wake of the ESThosts takedown, I sent a series of emails to them, and to Level 3 and XO, notifying them of the problem. 13 emails over a span of 5 months, with absolutely no action taken–the virus droppers are all still active, all still living on the same IP addresses. The first complaints went out in September of 2008.

          The complaints listed a total of 18 URLs and IP addresses being used to spread the W32/Zlob virus. All but one of the URLs, and every one of the IP addresses, is still being used to distribute the same malware. It seems to me that neither XO nor Level 3 is particularly concerned with the problem.

          Level 3 is the upstream for iPipe, which hosts exclsively for the Zlob gang (and in fact I believe may be owned by the zLob gang). Same deal–no action after repeated, detailed complaints of virus droppers living in iPipe address space. The registered owner of iPipe.net is Sergey Sabetyev, who is no stranger to spam blacklists; he’s also the owner of hqhosts.cn and hqhosts.net, which have a track record of being associated with hosting for spam sites including pharmacy spam and gambling spam, and hqhosts.cn has been used int he past to host malware as well.

          The Spamhuntress Web site has quite a bit on iPipe/hqhost; I would think that alone would be enough for Level 3 to be suspicious of this customer if they did any due diligence at all. Quite a few posts on news.admin.net-abuse.sightings about spam coming from or advertising domains (mostly malware domains masquerading as sex sites) hosted by iPipe.

          It really does seem to me that big ISPS do, in fact, willingly and knowingly provide services for bad apples.

  26. you’re partly right about the isps culpability here, but there are other things you’re probably not aware of…

    working for a tier 1 isp for the last 7 years, 5 of them as a manager i can tell you that at least our abuse department wasn’t the red-headed stepchild you describe above

    unfortunately, the reality is that shutting down an abuser is tremendously difficult – the legalities involved in proving that they (our client/subscriber) is knowingly hosting illegal content are tougher than hell – pulling routing or blacklisting a single ip is easy (and done by my former employer dozens of times a day) but to terminate their contract for abuse requires at a minimum a month of serious work by 2-3 different people, and that’s if the client doesn’t sick a lawyer on us for taking action

    the other problem we ran into a lot is that as a major bandwidth provider in both NA and the EU, a lot of our clients are resellers themselves, and sometimes their clients are also resellers which makes it ridiculously difficult to track – especially since so many of our customers have their own ip space; and there is also the reality that layer 2 services and tunneling are becoming major parts of the bandwidth we provide – so we literally cannot see the types traffic, the ips, or anything that our customers are sending across our network; all we see are the total amounts of traffic across the interfaces at each end of the tunnel

    there needs to be a better way to police internet traffic and a new set of protocols for handling abuse developed… we’re moving to IPV6 for more ip space, we’re now deploying equipment that will allow up to 160 10 gigabit traffic streams across a single strand of fiber optic cable when all the components are available; but we’re still handling abuse under the rules developed when compuserve was making money

  27. I’m curious why it’s so difficult to terminate an abusive downstream. Aren’t the Terms of Service incorporated into the contract?

    I mean, we’re not really talking about gray areas here. McColo stayed afloat for four years in spite of the fact that they hosted thousands of malware sites for both the Zlob and Asprox gang, the command and control servers for the world’s largest botnet, and kiddie porn. Kiddie porn, fer Chrissakes! For four years nobody shut them down.

    nd look at cases like iPipe.net, which as near as I can tell hosts no legitimate Web sites at all. All of their servers appear to be operated by the Zlob gang; I can’t tell you how many malware droppers I’ve seen living in iPipe address space.

    Another great example: American ISP Calpop, which has become one of the preferred hosts for the Russian Zlob gang; they’re still hosting active redirectors to sites that try to drop the Zlob malware, like the one at http://hardcorepornovids.net/ (warning: site is live and will attempt a drive-by download of W32/Zlob) six months and one day after I first reported it to them.

    Hell, Calpop is hosting a whole NETWORK of redirectors, traffic managers, and virus droppers for the Zlob gang; the same IP addresses in Calpop space (216.240.139.236, 216.240.139.234, 216.240.139.238, 216.240.139.241) keep turning up like a bad penny whenever I track down another Zlob downloader.

    Calpop took over the role of corrupt black-hat host when ESThosts went dark, and they don’t even bother to move the malware sites from one IP address to another. They not only don’t shut down Zlob droppers, they don’t even bother to move the sites. How much more blatant does a company have to be in order to get shut down?

  28. i get what you’re saying, and 4 years is crazy; i’m not sure who the upstreams are/were for these companies so i can’t speak to specifics on that

    unfortunately, while abuse and tos are laid out in the contracts, in order to be able to term a customer for tos violations we have to be able to prove (and the proof has to stand up if it gets taken to court) that they’re knowingly violating the tos – but not all providers take 4 years to pull a bad customer off their network (and at least the due diligence performed there kept us from selling to a lot of problem customers in the first place, much to our sales department’s frustration)

    as i’m not familiar with the companies you talked about i did a bit of digging into calpop; they’re a colo hosting provider with according to their website multiple upstream providers. however, their website lies about at least 1 of their upstreams (i know they’re not customers of or peers with my former company)… but they do have space in some of the large LA colo sites where most major NA isps have connectivity

    for example 600 7th is the handoff point for pretty much every commercial isp in LA as that is where Williams Communications Group (now part of Level3) landed all their fiber into/out of LA – and they’re the major provider of fiber optic cable in north america – so even companies who don’t use the WCG/L3 fiber wind up in 600 7th to peer with the other isps

    the traceroutes to both the calpop ips you list and the site http://www.calpop.com all go through XO no matter what trace tools i use – so if you’re not getting anywhere through XO or if you’re seeing them from a different provider, there are a couple of things you may want to try

    first, contact ARIN (they assign ips, and they may pull the IP space if calpop is violating their TOS)… and as they’re hosting malware and child porn, i’d contact the FBI as well – our abuse team worked with them on a few occasions and from what they reported the experience was a good one

    and i am going to assume you know about spamhaus and their rosko listings – they specifically blacklist spam – but if these companies are allowing child porn etc they’re probably also allowing spammers and that may be a way to get them pulled down – their blacklists are among the most used by the big isps as part of the due diligence checks on potential customers and one of the quickest ways we had to prove that our clients were violating TOS

  29. Hmm. All my traceroutes to Calpop go thorugh Level 3:

    traceroute to http://www.calpop.com (64.27.14.228), 64 hops max, 40 byte packets
    1 10.111.25.1 (10.111.25.1) 1.182 ms 0.764 ms 0.765 ms
    2 72.54.146.105 (72.54.146.105) 1.834 ms 1.749 ms 1.689 ms
    3 172.16.226.125 (172.16.226.125) 4.481 ms 4.649 ms 4.154 ms
    4 192.168.14.133 (192.168.14.133) 4.568 ms 5.000 ms 5.029 ms
    5 192.168.14.135 (192.168.14.135) 4.954 ms 4.870 ms 4.840 ms
    6 192.168.14.137 (192.168.14.137) 5.300 ms 5.311 ms 5.885 ms
    7 192.168.14.104 (192.168.14.104) 5.417 ms 5.309 ms 5.258 ms
    8 192.168.14.102 (192.168.14.102) 6.101 ms 6.336 ms 6.018 ms
    9 192.168.14.100 (192.168.14.100) 6.693 ms 5.986 ms 5.963 ms
    10 192.168.14.122 (192.168.14.122) 6.433 ms 6.674 ms 6.346 ms
    11 te-3-3.car1.Atlanta4.Level3.net (4.53.232.73) 21.913 ms 21.780 ms 21.844 ms
    12 * ae-72-52.ebr2.Atlanta2.Level3.net (4.68.103.61) 26.699 ms 22.022 ms
    13 ae-73-70.ebr3.Atlanta2.Level3.net (4.69.138.20) 23.809 ms 21.371 ms 34.262 ms
    14 ae-7.ebr3.Dallas1.Level3.net (4.69.134.21) 27.774 ms 27.723 ms 35.138 ms
    15 ae-3.ebr2.LosAngeles1.Level3.net (4.69.132.77) 60.491 ms 68.894 ms 71.756 ms
    16 ae-4-90.edge1.LosAngeles9.Level3.net (4.69.144.202) 59.808 ms 60.312 ms 59.597 ms
    17 CALPOPCOM-I.edge1.LosAngeles9.Level3.net (4.53.228.66) 60.514 ms 60.241 ms 60.117 ms
    18 64.27.17.189 (64.27.17.189) 59.825 ms 60.645 ms 60.217 ms
    19 64.27.14.228 (64.27.14.228) 60.729 ms 59.828 ms 61.537 ms

    When I first discovered they were providing hosting for the Zlob crew in the wake of the ESThosts takedown, I sent a series of emails to them, and to Level 3 and XO, notifying them of the problem. 13 emails over a span of 5 months, with absolutely no action taken–the virus droppers are all still active, all still living on the same IP addresses. The first complaints went out in September of 2008.

    The complaints listed a total of 18 URLs and IP addresses being used to spread the W32/Zlob virus. All but one of the URLs, and every one of the IP addresses, is still being used to distribute the same malware. It seems to me that neither XO nor Level 3 is particularly concerned with the problem.

    Level 3 is the upstream for iPipe, which hosts exclsively for the Zlob gang (and in fact I believe may be owned by the zLob gang). Same deal–no action after repeated, detailed complaints of virus droppers living in iPipe address space. The registered owner of iPipe.net is Sergey Sabetyev, who is no stranger to spam blacklists; he’s also the owner of hqhosts.cn and hqhosts.net, which have a track record of being associated with hosting for spam sites including pharmacy spam and gambling spam, and hqhosts.cn has been used int he past to host malware as well.

    The Spamhuntress Web site has quite a bit on iPipe/hqhost; I would think that alone would be enough for Level 3 to be suspicious of this customer if they did any due diligence at all. Quite a few posts on news.admin.net-abuse.sightings about spam coming from or advertising domains (mostly malware domains masquerading as sex sites) hosted by iPipe.

    It really does seem to me that big ISPS do, in fact, willingly and knowingly provide services for bad apples.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.