A Startup With a Way to Filter Botnet Traffic Gets Funding(bits.blogs.nytimes.com)
bits.blogs.nytimes.com
A Startup With a Way to Filter Botnet Traffic Gets Funding
http://bits.blogs.nytimes.com/2014/06/24/a-start-up-with-a-way-to-filter-botnet-traffic-gets-funding/?_php=true&_type=blogs&ref=technology&_r=0
8 comments
They're not looking for the presence/absence of JS/Flash but other things that can be measured by it. I assume their approach is similar to Spider.io (since acqui-hired by Google) which found (if I'm remembering correctly) that certain javascript calls ran differently or at different speeds depending on whether the browser instance was real. Similar to how software can detect whether it's running inside a VM because of subtle differences in how instructions are executed.
That said, if I had a botnet I'd just figure out what domains belonged to WhiteOPs and null-route them.
That said, if I had a botnet I'd just figure out what domains belonged to WhiteOPs and null-route them.
If that is what it does, they should have clearly stated that in the article. Instead, its in a completely separate press release :/
EDIT:
Its apparently stated here: http://www.marketwired.com/press-release/White-Ops-Secures-7...
"White Ops offers the world's first systematic solution for accurately isolating and eliminating bot-infected traffic. By applying the techniques of Side Channel Analysis across thousands of real-time user session variables, White Ops differentiates between a human and machine-driven request -- regardless of the sophistication of the bot programmer and across the full spectrum of impression and click fraud techniques. This approach is vastly more effective than methods generally employed by legacy fraud detection services, which rely largely on historical data and statistical and predictive analysis to detect bot activity. "
EDIT:
Its apparently stated here: http://www.marketwired.com/press-release/White-Ops-Secures-7...
"White Ops offers the world's first systematic solution for accurately isolating and eliminating bot-infected traffic. By applying the techniques of Side Channel Analysis across thousands of real-time user session variables, White Ops differentiates between a human and machine-driven request -- regardless of the sophistication of the bot programmer and across the full spectrum of impression and click fraud techniques. This approach is vastly more effective than methods generally employed by legacy fraud detection services, which rely largely on historical data and statistical and predictive analysis to detect bot activity. "
> That said, if I had a botnet I'd just figure out what domains belonged to WhiteOPs and null-route them.
That and bots that don't execute javascript would be a problem, yes. Our clients can account for that by comparing their traffic numbers with ours.
(full disclosure: I work at White Ops on the 'botprinting' team)
That and bots that don't execute javascript would be a problem, yes. Our clients can account for that by comparing their traffic numbers with ours.
(full disclosure: I work at White Ops on the 'botprinting' team)
I'd be very interested in knowing more about how the technology works. It's pretty difficult to assess it without more information. Specifically it feels like the good old software protection conundrum where you have to give the keys along with the lock. How do you prevent a bot operator to send the right signal (or not send it) to the (whichever) measuring party ?
Here's a short explanation of spider.io's approach that might interest you: https://www.youtube.com/watch?v=RSWhvNdjAAE
Ok so the library performs a kind of timing attack to detect whether it runs in a legitimate browser or not, and phones home { bot: true }. I make a bot that detects the script on the page and sends { bot: false }. You could try to obfuscate the signal by sending the raw collected browser behavior data but I still could reverse engineer its format or replay data from a legitimate session. I still don't understand how this could possibily work.
This is correct. If you ever look into how social media botnets are constructed and run, the good ones already do all the command and control through headless webkit tools. It's one of the primary reasons that it is so difficult to filter out whether or not a social profile is indeed a person or not from a statistical observation at mass.
I recommend reading about Dan a bit. He's not an amateur in the security world :-)
I don't care how well known he is. If that quote is accurate, he is peddling something that is known to not work in practice for the more sophisticated botnets.
These quotes are never accurate because it's marketology spread by a PR firm :-)
Then maybe he should invest in making sure the PR quotes him accurately?
Which makes it weirder, in a way. Surely such a security guru is going to be tweaking his personal browser, especially JS and Flash settings, and probably finding himself locked out of something for not looking enough like a typical clueless netizen. Or, he knows better than most how sophisticated and adaptive lucrative botnets are, so why would he think JS and Flash are deep magic the bots can't do?
But, as others have noted, it's probably been boiled down into something dumb for the layperson, and their algorithms are more complex than you would think from that quote.
Edit: Actually, given their target users, I guess he wouldn't be locked out, just falsely identified as a fake ad consumer.
But, as others have noted, it's probably been boiled down into something dumb for the layperson, and their algorithms are more complex than you would think from that quote.
Edit: Actually, given their target users, I guess he wouldn't be locked out, just falsely identified as a fake ad consumer.
Is there any research on how many botnets use Selenium? I would guess that most of them use something more basic just because it's a lot easier to put together.
How many use Selenium specifically? No.
It was just a random example that was intended to not point in an easily weaponized direction.
I figured it was both common and known in tech circles that Flash/Javascript executing botnets for ads existed and have for year(s):
http://www.spider.io/blog/2013/03/chameleon-botnet/
They were acquired by Google so it isn't just random blog spam:
http://techcrunch.com/2014/02/21/google-acquires-spider-io-t...
It was just a random example that was intended to not point in an easily weaponized direction.
I figured it was both common and known in tech circles that Flash/Javascript executing botnets for ads existed and have for year(s):
http://www.spider.io/blog/2013/03/chameleon-botnet/
They were acquired by Google so it isn't just random blog spam:
http://techcrunch.com/2014/02/21/google-acquires-spider-io-t...
Installing Selenenium on infected machines is not very stealthy. The new malware usually tries to stay in the kernel to avoid detection. Try stuffing Selenium in the kernel :-)
My personal experience with comment/forum spam bot networks is that at least 90% do not understand javascript. The ones that do tend to have extremely realistic traffic patterns and the ability to solve captchas. I suspect those are actually real people with incredibly crappy jobs, not super-intelligent Selenium-powered bots.
Comment/forum spam is different game than botnet ad fraud. The latter executes JavaScript by design since it's often necessary to load the ads.
see the charts from Incapsula in this article which show a third of the bots are not executing javascript
https://www.linkedin.com/today/post/article/20140623200557-8...
[deleted]
Is the startup's secret sauce simply:
- Scrape all IP's of server hosts from Arin.net
- if IP in that list, it's a bot.
- if not, it's human
That's a fairly noisy data source with lots of both false positives and false negatives.
I've actually been burned by use of that data as a user - the last time I was in Germany for a CCC camp, I needed to log into my bank account. Not trusting the WiFi, I bounced through my VPS to do so. It seemed to work fine, but a few days later the bank froze my account. When I called them they insisted that this activity meant my computer and email had been hacked and I needed to change my email password, virus scan my computer, change my username with the bank and change my password. There was no way for them to mark the use as 'legitimate'.
On the other side of the coin, there are lots of ad fraud bots running through residential ISPs.
I've actually been burned by use of that data as a user - the last time I was in Germany for a CCC camp, I needed to log into my bank account. Not trusting the WiFi, I bounced through my VPS to do so. It seemed to work fine, but a few days later the bank froze my account. When I called them they insisted that this activity meant my computer and email had been hacked and I needed to change my email password, virus scan my computer, change my username with the bank and change my password. There was no way for them to mark the use as 'legitimate'.
On the other side of the coin, there are lots of ad fraud bots running through residential ISPs.
what? ARIN tracks all American IPs (thus the 'a') - but if you are a residential ISP, you also get your IPs from ARIN. But the point is that you can't go to ARIN if you want information about European IPs; RIPE is the equivalent European org.
If an IP isn't allocated by a RIR, it is a "bogon" and usually dropped by your anti-spoof filters at your router.
If an IP isn't allocated by a RIR, it is a "bogon" and usually dropped by your anti-spoof filters at your router.
Interesting idea.
http://www.marketwired.com/press-release/White-Ops-Secures-7...
This is a more descriptive article on this funding round and what White Ops does. The company uses side channel analysis across an array of variables to detect behavioral differences between humans and bots.
This is a more descriptive article on this funding round and what White Ops does. The company uses side channel analysis across an array of variables to detect behavioral differences between humans and bots.
Side-channel analysis generally refers to analyzing emitted signals like RF emissions and power consumption patterns to defeat crypto systems. So it's an interesting turn of phrase considering it's referring to analysis of 'variables', implying in-channel data, rather than signals.
Editing for disclosure, I recently started working at White Ops as a data scientist.
> White Ops sells customers one line of code that allows them to differentiate between bot traffic and the real thing... White Ops tells website operators how much of their traffic is coming from humans and how much of it is coming from bots.
Looks like they only report on the percentage of bot users. While that can be useful in the long-run (especially if you're implementing ways to mitigate it), but it seems like it'd be much more useful if you could tell on a per-user or per-session level whether the user was a bot, so you could turn off features like ad-serving or message-sending or whatever.
Looks like they only report on the percentage of bot users. While that can be useful in the long-run (especially if you're implementing ways to mitigate it), but it seems like it'd be much more useful if you could tell on a per-user or per-session level whether the user was a bot, so you could turn off features like ad-serving or message-sending or whatever.
We have more detailed information about our detection decisions beyond bot percentage available to clients who want it, including statistics on a number of properties such as the type of bot and what bot markers are present.
This seems pretty much exactly like http://spider.io, now acquired by Google.
Trying to derive technical details from a business press release or an article in NY Times is silly. Unless you know about the tech and know how to read between the lines it will not make much sense.
So this is a piece of JavaScript. What happens when a bot blocks that JavaScript from running and/or emulates it but returns a result that indicates the user is human?
Posted this elsewhere in the thread already, but bots that don't run our code can be detected by comparing our traffic numbers with the client's traffic numbers.
It wouldn't be hard to make the javascript look like another ad tag. You could probably even serve the WhiteOps js through an ad tag or tag container.
I don't usually complain about titles (and I know it's the original one from the article), but I came here with the fascinated (though admittedly skeptical) hope that someone had improved radically on botnet DDoS mitigation techniques. It would be better if the title contained words like perhaps "ad" and "fraud".
Happy to change it if you or anyone can suggest a more accurate title.
That isn't how that works. That isn't how any of it works.
Selenium + https://code.google.com/p/flash-selenium/ == Has both.
People who block flash & javascript for security reasons? Also blocks both.
The sheer ease of bypass and the rate of false positives for any sophisticated operator is going to be huge.
Maybe I'm overly skeptical but I don't see this filtering any bot operation that is sophisticated enough to run something like selenium.
EDIT: Folks, I'm aware Selenium is a terrible example but it was the first thing that came to mind.
There are plenty of other headless toolkits that are easier to weaponize. Maybe I didn't want to point that out by linking to them/mentioning them? :|