You can now directly sync Postgres with Redis(github.com)
github.com
You can now directly sync Postgres with Redis
https://github.com/redfly-ai-org/redfly.ai
7 comments
Since db cache consistency is a complex research space with things like https://readyset.io/ and https://www.usenix.org/system/files/conference/nsdi13/nsdi13... it would be great to hear more specifics about what this is giving up in order to sidestep all that complexity.
There’s also Polyscale as a service.
Can someone ELI5 where this is going to be valuable?
If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache?
I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tokens, and so it'll be for data that I wouldn't be storing in Postgres anyway?
If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache?
I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tokens, and so it'll be for data that I wouldn't be storing in Postgres anyway?
Probably useful for sharing stuff with services that already have access to redis, but you don't want to add SQL client dependencies for and/or don't want to give access to your database.
But you want to be able to store/have single source of authority and update them using a SQL database.
This would then eliminate the need to maintain a separate lambda or cron job or something like that to keep them in sync.
But you want to be able to store/have single source of authority and update them using a SQL database.
This would then eliminate the need to maintain a separate lambda or cron job or something like that to keep them in sync.
No Pricing information nor an open-source project. I don't know what to do with this product.
Assuming the link to the repo is correct: it's open source.
https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE
https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE
I believe this is another case of abusing GitHub for visibility, not actually doing anything meaningful open source
The code in the repo just seems to be connection code ("provides a way for anybody to test our Redis synchronization service on demand") rather than the sync service itself
The code in the repo just seems to be connection code ("provides a way for anybody to test our Redis synchronization service on demand") rather than the sync service itself
[deleted]
It'd be nice to see documentation of how it works. Is it using a replication channel for Postgres? The write-ahead log? It looks like it does schema introspection and then works off that. How does it handle schema changes? How does this work in scenarios where you have multi-master replication? What's the compatibility with popular Postgres extensions like PostGIS? What happens if the database is rebooted or crashes? Yes I realize the source is there.
> Yes, I realize the source is there
It isn't actually. The codein the repo just seems to be some scaffolding.
It isn't actually. The codein the repo just seems to be some scaffolding.
[deleted]
[deleted]
Did you really mean to leave your AES key in the code in RedflyEncryptionKeys?
> public const string AesKey = ...
Also, the way AES is used in the code is not good practice. It seems to be using plain AES ( https://learn.microsoft.com/en-us/dotnet/api/system.security... ), which isn't meant to be used bare like that. It needs to be coupled with a digest algorithm to protect the ciphertext. Maybe use AesGcm instead?
> public const string AesKey = ...
Also, the way AES is used in the code is not good practice. It seems to be using plain AES ( https://learn.microsoft.com/en-us/dotnet/api/system.security... ), which isn't meant to be used bare like that. It needs to be coupled with a digest algorithm to protect the ciphertext. Maybe use AesGcm instead?
The reason for this is that AES is not an authenticated algorithm so there is no way to determine if the ciphertext has been modified since encryption. The ciphertext could be modified/corrupted and you wouldn't know.
Also, AES is deterministic and will encrypt the same data the same way every time. This means if you are encrypting a lot of fields you will be able to do statistical attacks. Using an initialization vector with AES GCM is similar to salting a hash. This way there is no statistical method to determine the contents of the ciphertext.
Also, AES is deterministic and will encrypt the same data the same way every time. This means if you are encrypting a lot of fields you will be able to do statistical attacks. Using an initialization vector with AES GCM is similar to salting a hash. This way there is no statistical method to determine the contents of the ciphertext.