I haven't seen this approach after researching ways to implement a "Trending" query. We get super fast queries over an arbitrary time window at the cost of a slower but highly cacheable threshold calculation query.
I wanted to share some of my techniques for managing and scaling 100+ unique web crawlers. I've tried a few approaches but these techniques stand out as having the most impact in terms of quick iteration and being resilient to breaking changes. Interested in hearing different approaches.
I built an early HTML5 game engine in 2010 called CraftyJS when Facebook games were starting to become big. The project itself got me the job at a gaming startup and an offer at Zynga.
Our newest vulnerability alert service. We aggregate vulnerability data from over 50 sources into one consistent data schema and allow users to write detailed queries and setup alerting. We've even introduced a pretty good local scanner to keep your SBOM up-to-date.
I built SecAlerts https://secalerts.co a few years ago. Fairly straight forward SaaS product to send email alerts on new vulnerabilities matched to customer software.
I'm building a cybersecurity database that aggregates vulnerability information from about 100 different sources in one cohesive structured dataset that can be easily queried. It'll be much faster than CVEs and include way more things (remedy information, news, social media, forum, advisories).
Those are great, I'm more talking about a complex problem you're given and have to whiteboard the solution while making sure not to be silent for too long or you'll interrupted or given a hint.
My least favourite interview technique has to be "talk me through a complex problem on a whiteboard". It's not so much the stress but the fact that you can't have a train of thought longer than 5 seconds before someone finds the silence awkward or interprets it as you're lost and breaks the silence and train of thought. It's a test for solving complex problems in 5 second blocks while being constantly interrupted.
Working on a platform to collect unstructured security data from a variety of sources and aggregate into structured data that can be queried, monitored and used in various different tools like scanners. No link to show right now though.
We need to let you know about new Government regulation that will hurt how Australians use Google Search and YouTube.
A proposed law, the News Media Bargaining Code, would force us to provide you with a dramatically worse Google Search and YouTube, could lead to your data being handed over to big news businesses, and would put the free services you use at risk in Australia.
# The way Aussies search every day on Google is at risk from new regulation
You’ve always relied on Google Search and YouTube to show you what’s most relevant and helpful to you. We could no longer guarantee that under this law. The law would force us to give an unfair advantage to one group of businesses - news media businesses - over everyone else who has a website, YouTube channel or small business. News media businesses alone would be given information that would help them artificially inflate their ranking over everyone else, even when someone else provides a better result. We’ve always treated all website owners fairly when it comes to information we share about ranking. The proposed changes are not fair and they mean that Google Search results and YouTube will be worse for you.
# Your Search data may be at risk
You trust us with your data and our job is to keep it safe. Under this law, Google has to tell news media businesses “how they can gain access” to data about your use of our products. There’s no way of knowing if any data handed over would be protected, or how it might be used by news media businesses.
# Hurting the free services you use
We deeply believe in the importance of news to society. We partner closely with Australian news media businesses — we already pay them millions of dollars and send them billions of free clicks every year. We’ve offered to pay more to license content. But rather than encouraging these types of partnerships, the law is set up to give big media companies special treatment and to encourage them to make enormous and unreasonable demands that would put our free services at risk.
This law wouldn’t just impact the way Google and YouTube work with news media businesses — it would impact all of our Australian users, so we wanted to let you know. We’re going to do everything we possibly can to get this proposal changed so we can protect how Search and YouTube work for you in Australia and continue to build constructive partnerships with news media businesses — not choose one over the other.
You’ll hear more from us in the coming days — stay tuned.
I love the concept of making this public but I feel like the audit wasn't very thorough if all it suggested was two security headers that a free online scanner could have picked up.