Immediately took me back to the scene from Firewall (2006) when Harrison Ford is copying the account data with an iPod and a fax machine https://www.youtube.com/watch?v=J13jKfVH01o ^_^.
At trivago, we rely on the ELK stack to support our logging pipeline. We use a mix of filebeat and Gollum (github.com/trivago/gollum) to write logs into Kafka (usually we encode them using Protocol Buffers) these are later on read by Logstash and written into our ELK cluster.
For us, ELK has scaled quite well both in terms of queries/s and writes/s and we ingest ~1.5TB of logs daily just in our on-premise infrastructure. The performance/monitoring team (where I work at) consists of only 4 people and although we take care of our ELK cluster it is not our only job (nor a component that requires constant attention).
trivago is a metasearch engine that compares accommodation offers from many booking sites e.g. independent hotels and online travel agencies.
- Site Reliability Engineer: Senior SRE for the Observability team. Experience with managing Elasticsearch/Opensearch clusters, Prometheus, Thanos, or similar systems. https://careers.trivago.com/job/r7959896002?gh_jid=795989600....
- Java/Kotlin Engineer: https://careers.trivago.com/job/r7536432002?gh_jid=753643200....
- Frontend Engineer: https://careers.trivago.com/job/r7927406002?gh_jid=792740600...
Read more about trivago: https://tech.trivago.com/ and https://life.trivago.com/.