More than half of internet traffic this year has been bots

"Internet robots are known by many names — bots, spiders, web bots, robots, and crawlers. What all have in common is that they are software applications or scripts. Where they differ is in the tasks that they perform. They're purpose or benefit is simple: to quickly complete jobs, especially repetitive ones."

They can be used, for example, to visit websites to collect data such as e-mail addresses. These e-mail addresses can then be used to send out mass emails of spam — messages that are sent without consent. Other functions of these scripts include interactive interfaces used in Instant Messaging (IM) services. Advanced bots, also called chatter bots, may be able to communicate requested information to a user such as time, date, or weather. Some are used to censor and monitor profanity usage.

Security and cloud service provider Incapsula analyzed 1.45 billion page views from 20,000 sites over 90 days. Their sample, which included traffic from 249 countries by IP address, showed that more than 60 percent of internet traffic is computer generated, compared to less than 40 percent of traffic that is driven by human clicks. That's a pretty lame spread. In their infographic, Incapsula points out that there bots can have different end goals, and can mess stuff up in a variety of ways ranging from annoying to critical.

Source:
gizmodo.com...

Thought this update might be of interest. Is this increase in internet bot traffic a concern? Although this report is only over a 90 day period, I would assume it is representative of the year.

Is this increase in internet bot traffic a concern?

Yes. For a variety of reasons.
I have to admit I've written a few scripts (Indexing type things. Reading, not writing.) of my own but due to the limits of my machine and my ISP I don't think I've interfered with anyone's bandwidth.

Webbots. The drones of the internet.

reply to post by pandersway

Serious? That is weird. Some very good programmers on another site I play on have placed bots to do the job of ten or more users, except for one guy we suspect was born a bot. But malicious bots, why would anyone want to create problems for other computer users, I've never understood the meanness of that mentality.

Thanks for what looks like a good thread.

But malicious bots, why would anyone want to create problems for other computer users, I've never understood the meanness of that mentality.

reply to post by Aleister


I quite agree...the only thing lower, in my humble opinion, are hoaxers.

Yes. For a variety of reasons.

reply to post by Phage


Would you care to elucidate? Are they reasons a layman would not be aware of? Yes, like me.

WTH!

I'm competing with a damned Microchip for bandwidth!

No wonder my interwebs are slowing down!

reply to post by pandersway

Your OP covers it pretty well. Security issues being the main issue I would think. All it takes is a bit of a weak point on one server (Target stores) for a bot to find and bots are much more patient and persistent than humans.

I hinted at interfering with bandwidth. Must less dangerous but still a possible concern.

SLAYER69
WTH!

I'm competing with a damned Microchip for bandwidth!

No wonder my interwebs are slowing down!

Depending on where you live this could become an even greater concern due to the rates at which bandwidth is sold.

Trojans could be implemented to siphon bandwidth off of others to reduce the cost of access for malicious users. This would increase your bills significantly, especially in areas which charge by the byte.

Internet in Canada
en.wikipedia.org...

Usage-based billing (UBB)
See also: Bandwidth cap

Internet bandwidth limits and caps are considered by many to be too restrictive, due to the increasing popularity of online streaming media services such as Netflix, which require large amounts of bandwidth.[20]

The decision to impose bandwidth caps on smaller independent ISPs[21] caused controversy in 2011 when the Canadian Radio-television and Telecommunications Commission (CRTC), Canada's telecommunications regulator, approved a request by Bell Internet to begin, on 1 March 2011, to apply a bandwidth cap on the users of smaller independent ISPs who use Bell's last mile infrastructure. This new billing structure is called "usage-based billing" or UBB.

Bell pushed for a cap as small as 25 gigabytes of transfer per month, plus a $1–2 CAD surcharge for every GB over the limit. The stated intent was to prevent the customers of independent ISPs from congesting Bell's network,[22] because many independent ISPs offer service with unlimited bandwidth, while most major Canadian ISPs do not. The CRTC was criticized for allowing Bell to use anti-competitive practices to favour its own Internet and television offerings.[23] Bell is also under fire for forcing its own pricing structure and business on its wholesalers. Bell admits that more than 10 percent of its subscribers (at the time of said download cap) exceed their limit, resulting in additional billing.[24]

Many savvy Internet users also accuse Bell of falsifying information to the public regarding network congestion. Network congestion is primarily caused by many users accessing the Internet at the same time (after school/work, 5pm-10pm) and not by heavy users alone.

On 2 February 2011, industry minister Tony Clement and Prime Minister Stephen Harper called on the CRTC to reverse the decision. The next day, the CRTC announced that it would delay its decision by 60 days.[25]

There are some supporters for usage-based-billing (UBB) at lower rates instead of the current $2/GB. One example is TekSavvy, providing "Lite" cable Internet services (6 Mbit/s down, ¼ Mbit/s up) at $30.95/month with 300 GB, equivalent to around 10¢/GB.[26] Rogers Hi-Speed Internet offers Internet access at the same speed for $41.49/month but with only 20 GB, equivalent to around $2.07/GB.[27] The difference of $1.97/GB between the two providers is one key reason why consumer advocates oppose UBB. Some also claim that it costs the incumbents as low as 3¢/GB.[28] Supporters also suggest that instead of a penalty-based system (heavy users pay more), a credit-based system (light users be credited back monthly) would be much more consumer friendly and fair.

UBB could also be easily manipulated with BOTs by increasing traffic beyond tolerance to justify exponentially increasing costs to users.

-FBB

UBB could also be easily manipulated with BOTs by increasing traffic beyond tolerance to justify exponentially increasing costs to users.

reply to post by FriedBabelBroccoli


Right, so how could Bots be curtailed over the net by us the users?

pandersway

UBB could also be easily manipulated with BOTs by increasing traffic beyond tolerance to justify exponentially increasing costs to users.

reply to post by FriedBabelBroccoli

 

Right, so how could Bots be curtailed over the net by us the users?

Curtail BOTs?

The internet's response is thus;


I don't see it happening, the only solution is to implement effective consumer protections and literacy to prevent themselves being hijacked.

-FBB

reply to post by FriedBabelBroccoli


Hmm this is not very reassuring!! I'm thinking along the lines of protection against hacking Bots with regards to credit cards, transactions and the like.

This move towards too much automation concerns me.

Wow this is unbelievable to me. Like when I heard that 95% or some rediculous percentage of the Internet was porn. I guess we should not underestimate criminals. The more bots they have out there working, the higher their profit, I suppose is how that would work. I would expect this number to rise then as more people are desperate for ways to make easy money. How if at all can this be combated? Is there anything the average Web surfer can do?

reply to post by FriedBabelBroccoli

"UBB could also be easily manipulated with BOTs by increasing traffic beyond tolerance to justify exponentially increasing costs to users."

ah I see... so it's the service providers who are letting out all these bots to increase profits. It makes sense now.

wonder if bitcoin miners have anything to do with this

reply to post by pandersway

I believe these numbers. Sometimes I go to the help forums for software I own, and I can see who is visiting the site. I usually see more bots crawling the site than users like me. But those are search engine bots which are the "good bots".

That's an informative article about all the not so good or evil bots. I knew there were evil bots out there but I didn't know the percentage was that high.

I guess the bots are crawling this site a lot too, except for RATS where they are blocked, so if you really want to block them, you can. Unless it's the NSA in which case they can probably go wherever they want, they may even have a bot that can access RATS.

reply to post by Arbitrageur


How do RATS work? And how can they be used by people as a protection device from 'bad' bots?

pandersway
reply to post by Arbitrageur

 

How do RATS work? And how can they be used by people as a protection device from 'bad' bots?

For some reason RATS doesn't get scanned, and threads there aren't included in internet search engines. How that occurs is beyond my computer knowledge, which consists of "power on" and "will this erase everything on the page or just this word?"

pandersway
How do RATS work? And how can they be used by people as a protection device from 'bad' bots?

I haven't looked at the source code for RATS, but instructions for blocking bots like RATS does are easily available on the net. Here is one set of instructions:

How to use Robots.txt

Blocking Robots and Search Engines from Crawling

If you want to stop bots from visiting your site and stop search engines from ranking you, use this code:

I will stop there and refer you to the link because I don't want to paste any code here...it probably wouldn't cause a problem but I don't want to take any chances. But as you can see, it's pretty simple. Now if you're NSA and you want to crawl RATS anyway, it would be pretty easy to make your bot ignore that code.

If you want to protect your system against bad bots, you have to take a lot of security measures. In fact, some people have their computers taken over by malware that actually creates bad bots to attack other people's machines, and they are just innocent bystanders who don't even know their PC is now what's called a "zombie PC". So, one place to start is by locking down your PC to make sure it doesn't turn into a zombie PC that launches bot attacks against other PCs. Scanning your PC with the malwarebytes free anti-malware software should tell you if you have a zombie PC.

So you can prevent your PC from becoming a zombie, but the only real way to protect your PC from all bad bots including hacker bots is to disconnect it from the internet:

Personal Computer Security: Using Uncommon Sense

Most people are aware of the dangers, but not how to protect themselves. The truth is, if a hacker wants to get into your system, usually the only way to prevent that access is to completely cut the system off from the internet. Even then, there's still a remote possibility that access can be gained. Just recently, the US Department of Defense reported that a successful network intrusion had been accomplished through the use of a rogue USB flash drive.

But most people are going to stay connected so the rest of that gives a few simple pointers that don't really protect you completely, but they do help. I think the Target case is a good example of what can happen. If you're victimized by that, just take the necessary actions to resolve it...I don't see how you can prevent it short of never giving credit card info to any retailers. Here's another link with some tips on protecting yourself, which again is pretty generic but there's only so much you can really do:

Computer Security

reply to post by Arbitrageur

Assuming the Bot obeys the rules...

edit:

Some of that is confusing Bots with malware.

reply to post by pandersway


Great find.

Bump.