I first built FPScanner during my PhD around 2017–2018, when I was doing research on browser fingerprinting and bot detection. After that I did not really maintain it for years.
I recently decided to revive it because things changed a lot. Automation is much easier now. With free automation libraries like Puppeteer/Playwright/Selenium + headless Chrome and cheap proxies, you can build decent bots very quickly. At the same time, open source defensive tooling is still quite limited, or very basic.
To be clear, FPScanner is not trying to be a silver bullet.
It is a small, self-hosted library that focuses on deterministic client-side signals:
- webdriver and automation flags
- CDP-related artifacts
- automation framework markers (Selenium, Playwright, etc.)
- JS cross-context inconsistencies: main JS context, iframes and workers
It also generates a JA4-inspired fingerprint ID for clustering sessions, and includes encrypted payload + simple anti-replay protections.
There is no ML here, no "AI detection", and no claim to block 100% of bots. The idea is just to expose strong, explainable signals and make automation a bit more expensive. I assume attackers can read the source code.
I tested it on different devices and browsers to avoid JS errors and obvious false positives, but I’m sure there are edge cases. If you try it on less common browsers or hardened setups and it breaks, please let me know or open an issue.
Happy to answer questions or discuss design choices / limitations.
I recently wrote about the limits of these kinds of fingerprinting tests. They tend to overly focus on uniqueness without taking into account stability. Moreover sample size is often really small which tends to artificially make a lot of users unique
Interesting article. I’ve been curious for a while about how residential proxy IPs are collected too. Many come from shady browser extensions or mobile apps, especially free VPNs (wink wink Hola VPN). People often don’t realize they are turning their device into an exit node.
Some time ago I started to track this as a side project (I work in bot detection and was always surprised by how many residential proxies show up in attacks). It started just out of curiosity. Now I collect proxy IPs, which provider they belong to, and how often they are seen. I also publish stats here:
https://deviceandbrowserinfo.com/proxy-api/stats/proxy-db-30...
For example, in the last 30 days I saw more than 120K IPs from Comcast and nearly 100K from AT&T.
I also maintain an open IP (ranges) blocklist, mostly effective against data center and ISP proxies. Residential IPs are harder since they are often shared with legit users:
https://github.com/antoinevastel/avastel-bot-ips-lists
Even if you can’t block all of them, tracking volume and reuse gives useful signal.
Author here: I work in bot detection, and wrote this post to explain why privacy-conscious users (VPNs, Brave, LibreWolf, etc.) often get flagged or blocked by anti-bot systems.
I’ve seen a lot of frustration in threads here, so I wanted to offer a technical perspective on why these false positives happen, and how detection systems interpret signals from non-mainstream setups.
Author here: There’ve been a lot of HN threads lately about scraping, especially in the context of AI, and with them, a fair amount of confusion about what actually works to stop bots on high-profile websites.
This post uses TikTok’s obfuscated JavaScript VM (recently discussed on HN) as a case study to walk through what modern bot defenses look like in practice. It’s not spyware, it’s an anti-bot measure designed to make life harder for HTTP clients and non-browser automation.
Key points:
- HTTP-based bots skip JS, so TikTok hides detection logic inside a JavaScript VM interpreter
- The VM computes signals like webdriver checks and canvas-based fingerprints
- Obfuscating this logic in a custom VM makes it significantly harder to reimplement outside the browser (and so to scale an attack)
The goal isn’t to stop all bots, it’s to push attackers into full browser environments, where detection is more feasible
The post covers why simple solutions like "just require JS" don’t hold up, and why defenders use techniques like VM-based obfuscation to increase attacker cost and reduce replayability.
The attacker had fully reverse engineered the signal collection and solved-state flow, including obfuscated parts. They could forge all the expected telemetry.
This kind of setup is pretty standard in bot-heavy environments like ticketing or sneaker drops. Scrapers often do the same to cut costs. CAPTCHA and PoW mostly become signal collection protocols, if those signals aren’t tightly coupled to the actual runtime, they get spoofed.
Yeah, not (too) surprising after a few years in the anti-bot industry. Last week I looked into a Binance CAPTCHA solver that didn’t use a browser at all, just a basic HTTP client. The attacker had reverse engineered the entire signal collection and response flow, including how the CAPTCHA was marked as solved. They were able to forge the expected telemetry despite some obfuscation.
https://blog.castle.io/what-a-binance-captcha-solver-tells-u...
This is pretty standard now in bot-heavy spaces like ticketing or sneaker drops. CAPTCHA often just ends up being a protocol to collect signals, and if those aren’t tightly bound to the browser/runtime, they get spoofed.
Also not surprised PoW isn’t holding up. Someone reverse engineered the PerimeterX PoW and converted it to CUDA to accelerate solving:
https://github.com/re-jevi/PerimiterXCudaSolver/blob/main/po...
At some point, it’s hard to make PoW slow enough for bots without also killing UX for humans on low-end devices.
Author here. A few weeks ago, someone posted a link on Reddit to an open-source CAPTCHA solver made for Binance’s slider challenge. It’s written in Python and works without using a browser. Just a custom HTTP client, some image matching, and basic reverse engineering.
I was curious and decided to dig into it. I wrote a long breakdown of how it works, how it solves the challenge, and what this says about how bots are built today. Many bots use headless browsers, but this one doesn’t, and it still gets through.
One of the main takeaways is how effective this kind of non-browser approach can be when CAPTCHA is deployed in isolation, without other layers like continuous behavioral checks.
Hi, author here.
I wrote a blog post where I analyze Hidemium, a popular anti-detect browser. I break down the techniques it uses to spoof fingerprints and show how JavaScript feature inconsistencies can reveal its presence.
Of course, JS feature detection isn’t a silver bullet, attackers can adapt. I also discuss the limitations of this approach and what it takes to build more reliable, environment-aware detection systems that work even against unfamiliar tools.
Using a 4G dongle makes it easier to hide in the crowd indeed. Since your traffic will go through heavily shared mobile IPs, probably with thousands of users behind them, anti-bot vendors won't/shouldn't block per IP, but per fingerprint/session cookie instead.
The DB contains different types of proxies:
- Residential
- ISP
- Data center
I don't include mobile proxies since they're heavily shared, so knowing that an IP address was used as a proxy at some point is basically useless.
Regarding your remark, indeed, there are several shared residential IPs, including IPs of legitimate users who may have a shady app that routes traffic through their device. That's why I don't recommend blocking using IP addresses as is. It's supposed to be more of a datapoint/signal to enrich your anti-fraud/anti-bot system.
However, regarding the block list, I analyze the IPs on bigger time frames, the percentage of IPs in the range that were used as proxies, and generate a confidence score to indicate whether or not it is safe to block.
So far I have ~ 3M distinct IP addresses per 30 days, with a lot of fresh proxy IPs, 1.7M. The DB contains only verified IP addresses through which I've been able to route traffic. It DOESN'T rely on 3rd party/open-source data sources.
Author here: I've been in the bot industry/bot detection field for ~ 10 years. I frequently see strong opinion about bot detection on Reddit and HN, in particular why it doesn't make sense for bot detection companies (I won't name who, but you will guess), to treat you so differently based on your user agent, and why it shouldn't matter when it comes to bot detection.
That's why I wrote a blog post about the role of the user agent in bot detection. Of course, everyone knows that the user agent is fragile, that it is one of the first signals spoofed by attackers to bypass basic detection. However, it's still really useful in a bot detection context. Detection engines should treat it a the identity claimed by the end user (potentially an attacker), not as the real identity. It should be used along with other fingerprinting signals to verify if the identity claimed in the user agent is consistent with the JS APIs observed, the canvas fingerprinting values and any types of proof of work/red pill
Author of the post: bot detection and bot-related topics are often discussed on HN, particularly lately with the increase of AI scrapers trying to scrape protected websites to gather training data for LLMs.
Most of the time, people just recommend using CAPTCHA or implementing rate limiting. In the article, I try to cover the most popular bot detection approaches, such as CAPTCHA, IP-based rate limiting, geo-blocking, and static JS fingerprinting, and discuss their main limits.
Hi HN! I’m Antoine, Head of Research at Castle. We’re a YC-backed Series A startup helping companies like Canva, Atlassian, and Rockstar Games fight fraud and abuse. We're building modern, AI-powered bot detection and abuse prevention tech, with a focus on real-time protection and behavioral intelligence.
We’re a small, fast-moving team looking to add two technical roles:
Senior ML Engineer – Fraud & Bot Detection
You'll work on real-time ML pipelines, anomaly detection, and adversarial ML techniques to stop bots and malicious actors. We’re combining traditional fraud signals with modern AI—LLMs, behavioral clustering, and vector-based search—to stay ahead of attackers. You’ll own models end to end and deploy quickly, with direct access to rich, structured behavioral and fingerprinting data.
Ideal for someone with strong experience in the anti-fraud/anti-bot field who loves cutting-edge ML and wants immediate impact.
Senior Detection Engine Engineer (R&D)
You’ll build and scale the core infrastructure behind our detection logic—rule engines, fingerprinting pipelines, and high-speed signal ingestion. We’re integrating AI-driven detection and auto-adaptive logic that reacts in real time. If you enjoy real-time systems, security, and automation, this is a highly technical role that’s at the core of our detection stack.
Perfect if you’ve worked in security, data pipelines, or real-time analytics and want to outsmart fraudsters.
We pay US salaries globally | Fully remote from Europe/EMEA timezone | Flexible hours | Unlimited PTO | Paid parental leave
We're actively hiring and committed to responding to every qualified applicant.
- multi-profiles management (to have separate identities linked to different fingerprints and locations)
In this blog post, I analyze the Undetectable anti-detect browser, the JS script it injects to modify the fingerprint, and lower level changes done at the c++ level within the browser. In the end, I show how you can craft a simple JS challenge that exploits side effects of Undetectable to detect it.
Disclaimer: I know anti-detect browsers can also be used for privacy purposes. The purpose of this article is to cover the main mechanisms used by these browsers that can be used to bypass security systems and how websites can protect against these techniques.
I first built FPScanner during my PhD around 2017–2018, when I was doing research on browser fingerprinting and bot detection. After that I did not really maintain it for years.
I recently decided to revive it because things changed a lot. Automation is much easier now. With free automation libraries like Puppeteer/Playwright/Selenium + headless Chrome and cheap proxies, you can build decent bots very quickly. At the same time, open source defensive tooling is still quite limited, or very basic.
To be clear, FPScanner is not trying to be a silver bullet.
It is a small, self-hosted library that focuses on deterministic client-side signals: - webdriver and automation flags - CDP-related artifacts - automation framework markers (Selenium, Playwright, etc.) - JS cross-context inconsistencies: main JS context, iframes and workers
It also generates a JA4-inspired fingerprint ID for clustering sessions, and includes encrypted payload + simple anti-replay protections.
There is no ML here, no "AI detection", and no claim to block 100% of bots. The idea is just to expose strong, explainable signals and make automation a bit more expensive. I assume attackers can read the source code.
I tested it on different devices and browsers to avoid JS errors and obvious false positives, but I’m sure there are edge cases. If you try it on less common browsers or hardened setups and it breaks, please let me know or open an issue.
Happy to answer questions or discuss design choices / limitations.