The thing I am curious about is vertical evals. BrowseComp and DRACO are useful, but they are not a finance, health, legal, or procurement workflow. If you have built a domain-specific research eval, I would like to compare notes!
Steel.dev | Founding Engineers (Infrastructure, Applied AI), Growth, GTM | Toronto, San Francisco | In-office
Steel is building open-source browser infrastructure for AI agents & apps. We make it easy for AI engineers to ship products that interact with the web using our Sessions API - An API that controls fleets of browser sessions in the cloud while we handle everything from scaling to antibot and more.
Our tech stack:
- Frontend: Vanilla React + TailwindCSS + Shadcn, built with Vite, deployed on Vercel
- Backend: Fastify, Node.js, Zod, NATs, Drizzle with Redis, PostgreSQL, and S3
With over 6,000 GitHub stars, dozens of paying customers, and millions of sessions served, we grew our platform 50x in 2025 purely through word-of-mouth and our open-source community. Backed by world-class investors, we're building a small, talent-dense team that's shaping the future of how humans & agents interact with the internet.
Current roles (all roles include competitive comp + equity):
- Infrastructure Engineer: Design, optimize, and scale our browser VM infrastructure
- Applied AI Engineer: Build on agent frameworks, integrations, and real-world AI use cases
- Founding Growth Lead: Own the growth machine to turn insane organic traction into a category-defining brand
- Founding GTM Lead: Be our first sales hire, closing deals with AI startups transforming entire industries
P.S. If you're feeling spicy, try our optional hiring challenge and submit the flag with your application once you reach the final step → https://challenge.steel.dev
With all the recent developments in AI web agents space (like OpenAI Operator and Manus), I’ve been connecting with countless engineers looking to navigate this space and build similar capabilities into their own products.
That’s why I created Awesome Web Agents - A comprehensive list of tools, frameworks, and resources focused on AI agents that can browse and interact with the web.
The list includes: Autonomous Web Agents (like OpenAI Operator, Browser-Use), AI Web Automation Tools & Dev Tools, AI-powered Web Scrapers, Research Benchmarks & Datasets, Tutorials & Implementation Guides.
The list is open source and maintained on GitHub. Stars, forks, and contributions welcome! If you’re currently working on a web agent tool or product, I’d love to feature you.
Would love to hear your thoughts/feedback. Any tools I'm missing?
Totally agreed. Only good as inputs to a stack decision but not the deciding factor. WebVoyager itself feels like it's approaching saturation though as scores as getting high and it only tests on a narrow domain of use-cases. We'll definitely see more challenging and interesting evals pop up in the next little bit.
IMHO, if you're building AI products, most of the time building and running your own evals is the only right way to build something good.
BTW - Arch looks super cool! Just starred and looking forward to playing around with it :)
Hey all! Wanted to share this leaderboard we put together to centralize some of the different models available in the AI browser agent space.
Since working on Steel, we've seen a ton of people have a hard time putting the browser agent space and how it's progressing into perspective and it felt odd to us that there were no centralized leaderboards like there were for so many other agentic use cases.
So we launched this leaderboard to help! It's open-source and we're open to any contributions we may be missing. We're committed to keeping this up to date as the space progresses (which it seems to be doing quite quickly).
Hey HN! We're the team behind steel.dev, and today we're excited to share Surf (https://surf.new) - an open-source playground where you can test and experiment with different web agents. Think of it as a sandbox where you can see how different AI agents/models interact with the web in real-time.
With OpenAI's Operator launch last week, web agents are having a moment. But with it being closed source, not having an API, not SOTA, and costing $200 a month, we felt like there was a gap in clearly understanding where the space is at today for both developers evaluating for prod and end-users. The open-source community is actually leading the way in performance (Shoutout Browser-use, currently SOTA on WebVoyager), but testing different approaches requires a bunch of work, setup, and debugging just to get started.
We built Surf to solve this. It's a hosted playground where you can chat with different web agents/models and test automation tasks - basically a "try before you deploy" sandbox for web agents. The most interesting challenge wasn't the agents themselves - it was designing Surf to be generalizable enough for contributors to plug in new agents and models easily.
Right now, you can switch between Browser-use’s agent and our experimental Claude Computer-use-based agent. But the real goal is to make it trivial for anyone building web agents to add their own. The whole thing is open source (https://github.com/steel-dev/surf.new) and built on our Steel's Sessions API. We built it with Next.js, Vercel AI SDK, & LangChain to keep everything clean, familiar, and hackable.
Heads up - it's new, and slow, and agents can fail in all sorts of ways. But that is kind of the point - we want to make it easier for everyone to understand the current state of web agents, whether you’re evaluating them for production use, just curious about how they work or want to automate some tasks for yourself. We also don’t yet support operator features like taking control and multiple tabs yet either.
Try it out at surf.new (no signup needed!), and let us know what breaks. We're actively maintaining it and would love your feedback on what you'd like to see next.
Hey HN! We're the team behind steel.dev, and today we're excited to share Surf (https://surf.new/) - an open-source playground where you can test and experiment with different web agents. Think of it as a sandbox where you can see how different AI agents/models interact with the web in real-time.
With OpenAI's Operator launch last week, web agents are having a moment. But with it being closed source, not having an API, not SOTA, and costing $200 a month, we felt like there was a gap in clearly understanding where the space is at today for both developers evaluating for prod and end-users. The open-source community is actually leading the way in performance (Shoutout Browser-use, currently SOTA on WebVoyager), but testing different approaches requires a bunch of work, setup, and debugging just to get started.
We built Surf to solve this. It's a hosted playground where you can chat with different web agents/models and test automation tasks - basically a "try before you deploy" sandbox for web agents. The most interesting challenge wasn't the agents themselves - it was designing Surf to be generalizable enough for contributors to plug in new agents and models easily.
Right now, you can switch between Browser-use’s agent and our experimental Claude Computer-use-based agent. But the real goal is to make it trivial for anyone building web agents to add their own. The whole thing is open source (https://github.com/steel-dev/surf.new) and built on our Steel's Sessions API. We built it with Next.js, Vercel AI SDK, & LangChain to keep everything clean, familiar, and hackable.
Heads up - it's new, and slow, and agents can fail in all sorts of ways. But that is kind of the point - we want to make it easier for everyone to understand the current state of web agents, whether you’re evaluating them for production use, just curious about how they work or want to automate some tasks for yourself. We also don’t yet support operator features like taking control and multiple tabs yet either.
Try it out at surf.new (no signup needed!), and let us know what breaks. We're actively maintaining it and would love your feedback on what you'd like to see next.
Steel is building open-source browser infrastructure for AI agents & apps. We make it easy for AI engineers to ship products that interact with the web using our Sessions API - An API that controls fleets of browser sessions in the cloud while we handle everything from scaling to antibot and more.
We're a small but fast-moving team looking for experienced engineers who thrive on autonomy and technical challenges. You'll work directly with our founding team to make Steel the go-to browser infrastructure powering the next billion AI agents. If you love dev tools, open-source, and/or AI, you're at the right door.
Current roles (all roles include competitive comp + equity):
- Full Stack Engineer: Build dev tools, APIs, and UIs to help devs control browsers at scale
- Infrastructure Engineer: Design, optimize, and scale our browser VM infrastructure
- Applied AI Engineer: Work on integrations, agents, and real-world AI use cases
With 2.8k+ GitHub stars, paying customers, and millions of API calls served, we're building a remote-first team that's shaping how humans interact with the internet of the future.
Our end goal is to build the LLM OS. We think the best place to start is the hardest but most hairy problem - getting agents to use an internet design for humans on our behalf. So, as we go, we want to keep building towards relieving blockers by handling auth securely, translating webpages, and creating the right toolset for driving sessions. And we want to do this in an open-source and communal way. That's also why we're attacking it bottom-up with infra, so we can build a community of people much smarter than us around getting there. Join the discord if you want to stay on top of our journey :)
ah, we could definitely do better in communicating that. Most of the work is ahead of us but we currently help ai devs in a few ways:
- designing with sessions as the atomic unit -- we wanted them to be spun up and discarded in a serverless way w quick boot times. That way you would have these quick or long-running sessions that agents can just jump in and out of or as a building block for custom tools. We also expose debug urls so you can embed viewable sessions into you app in a Devin-esque UI.
- In the Open-source API, we expose some features that allow you to turn webpages into more LLM-friendly formats for consumption like: markdown, readability, screenshots, and pdfs.
It does just so happen that it ends up being useful for non AI apps as well
That honestly sounds so fun! Who better to build your defences than your attackers ahaha
Tying into your concern, keeping IPs fresh and high quality will definitely be a balancing act as we get bigger. It'd be one today too if we were to try to offer super granular location controls because there's only so many proxies in X state, let alone X city. Currently, we get to aggregate & QA from multiple proxy providers, so our total pool is 300M+ IPs in the US and so far we've had a 99.95% rate of getting a fresh IP address in a session. So so far so good :)
As for that last point, I guess we'll see what the future has in store for us :P
Yeah, that’s totally fair! Not everybody will. In our experience, the problems we solve are the creeping kind that develop over time and show themselves as maintenance costs.
Let’s say we want to add web automation to our app, and we've set up a single instance of Chrome hosted on a server somewhere that we connect to and drive with Puppeteer. Great, now we need to ensure incoming requests/sessions are managed properly (>1 active session). If we have a ton of active requests, this becomes annoying because now the resources on our server are being eaten up — so we would need to scale horizontally (or vertically) but now that comes with potential downtime and cold start issues. If we decide to keep this instance of Chrome running 24/7 then we need to bake in resource management to handle memory leaks and connection issues. If we don’t keep them alive, then this comes with significant cold start times (10s+).
Now, let’s say we want to support a real-time scraping use case that requires multiple browsers in parallel. At this point, we would need to use something like Kubernetes with warm pods or maybe lambdas/cloud run. But even those have their own set of challenges/costs. Managing Kubernetes can get complicated and expensive quickly, especially if you’re not already using it for your app. On the other hand, serverless options like Lambda or Cloud Run introduce latency issues (cold starts) and often don’t provide the flexibility you need for long-running sessions or custom configurations.
Then, beyond the infra, we'll probably need to build capabilities to not get stopped out by anti-bot measures like proxy mgmt, fingerprint rotation, captcha solving, etc.
Pretty quickly, as you grow, a small-scale project can turn into a pretty big maintenance project. Building parsers/agents/scripts is hard enough, so we're hoping to make the infra side as easy as possible.
haha, I love this analogy! If we can do for deploying web automation to prod what Vercel did for deploying front-ends, I'd be a happy camper. With even friendlier pricing of course :P
The thing I am curious about is vertical evals. BrowseComp and DRACO are useful, but they are not a finance, health, legal, or procurement workflow. If you have built a domain-specific research eval, I would like to compare notes!