"Tim wasn’t delivering software; Tim was delivering a team that was delivering software. The entire team became more effective, more productive, more aligned, more idiomatic, more fun, because Tim was in the team."
- This happened years ago to me, with a manager pulling me aside and asking why my Tickets resolved were so low... They only cared about the number of tickets worked on...
No one can answer that for you, you need to test it out your self. Depends on size of documents, number of keys, volume of reads, etc etc. Spin up 1 instance and test it out ;)
One way for example would be store all events you want to write to ElasticSearch to a Queue (like rabbit, Kafka, etc) then have a set of workers/importers (this can even be logstash with input from queue and output ES) that read the events from that queue and store them in ES.
This way your system becomes a sync, and your system above is not affected if ES becomes slow writing or goes down for a reason. Your events will remain stored in the queue waiting to be processed.
"If the Police have a legit reason to access a property they go to court and get a warrent, and if they need to they'll kick the door in to get in."
That's what currently takes, place, Government doesn't have the keys, they have to use force to get in, or other methods. (However there are physical limits to materials, so there is usually a way to break in)
But, by having a special key that opens all the doors, anyone could copy it - yes rules can put in place to who as access, etc, etc, but by knowing there is a "hole" in each device, every possible malicious agent will try and break it as soon as possible. Then what?
We have seen examples by Law Enforcement officers using accesses to gather data that would required a court order, but they didn't have one, and it was for personal reasons. So, how does that work out?
CTO Binaryedge here. For those wondering, We have detected more than 15k Kubernetes APIs with Auth. This post focuses on ~1.5k found without Auth, that are fully open.
It's not just a Kubernetes Problem. Like many have posted, many databases, other types of clusters, shares, are accessible without Auth for those that know how to look for them (not that hard now days), mainly malicious actors.
We scan the entire internet space and create real-time threat intelligence streams and reports that show what is exposed on the internet. Our objective is to help you answer questions like:
- What services is your organization exposing?
- How many IOT devices are accessible?
- How many services/databases are miss-configured?
- What DataLeaks have affected a specific e-Mail?