Vulnerability Detection in Minutes-Here's Our Architecture Inspired by Sherlock(blog.appknox.com)
blog.appknox.com
Vulnerability Detection in Minutes-Here's Our Architecture Inspired by Sherlock
https://blog.appknox.com/architecture-behind-appknox-inspired-by-sherlock/
2 comments
Really impressive. Curious to know why are you using RabbitMQ and redis? You can use redis as a queue cluster with celery. Any specific reason?
Yes we could have gone with either one. It would have worked for having both.
RabbitMQ is very reliable (At the cost of performance). It persists the data on disk. Even if the RabbitMQ server goes down, we can attach the disk onto a new instance ant it would restore to the previous state. We needed this for task queues. They should work no matter what.
Redis is much more performant (at the cost of reliability). Because, by default, everything happens in-memory. Without additional settings for persistence (Which will bring down performance), if the Redis instance goes down, then all data is lost. We needed the performance for quickest delivery of notifications. milli-seconds matter.
Plus, now we can scale tasks and notifications independent of each other.
RabbitMQ is very reliable (At the cost of performance). It persists the data on disk. Even if the RabbitMQ server goes down, we can attach the disk onto a new instance ant it would restore to the previous state. We needed this for task queues. They should work no matter what.
Redis is much more performant (at the cost of reliability). Because, by default, everything happens in-memory. Without additional settings for persistence (Which will bring down performance), if the Redis instance goes down, then all data is lost. We needed the performance for quickest delivery of notifications. milli-seconds matter.
Plus, now we can scale tasks and notifications independent of each other.
Ok, I understand that.
Have you tried RabbitMQ in `RAM mode`. If yes then I wonder what is the performance ratio against radis?
Have you tried RabbitMQ in `RAM mode`. If yes then I wonder what is the performance ratio against radis?
Really impressive architecture, but, doesn't building the whole thing in Node, makes it more compatible for non-blocking calls?
It will certainly be non-blocking. But the primary libraries that we use to scan the apps is written in python. And we love python. So we went to python.