Techniques scraping vulnerability data with over 100 crawlers (without LLMs)(secalerts.co)
secalerts.co
Techniques scraping vulnerability data with over 100 crawlers (without LLMs)
https://secalerts.co/news/techniques-scraping-vulnerability-data-from-100-different-sources-without-llms/1iVmylCNLzLCdgOsnpkW34
1 comments
I wanted to share some of my techniques for managing and scaling 100+ unique web crawlers. I've tried a few approaches but these techniques stand out as having the most impact in terms of quick iteration and being resilient to breaking changes. Interested in hearing different approaches.