cwilper · HackerTrans

3 ปีที่แล้วdiscuss

I used to approach problems bottom-up, and I often prefer it when I'm working on my own because I do feel I have a much deeper understanding of the system as a whole that way. But on larger teams, and especially teams with junior developers or communication boundaries, I find a top-down approach to be much easier to get everyone on board with.

cwilper

3 ปีที่แล้วdiscuss

What would be great is a tiered storage service or library where oft-accessed data is in R2 and infrequently accessed has metadata in R2 but blobs in the cheaper S3 storage tiers or Glacier.

cwilper

3 ปีที่แล้วdiscuss

This is a fascinating topic that doesn't get the attention it deserves.

The idea of trusting a single entity (the family computer expert, a single company, or a government agency) to be the sole host or custodian of important records and family photos/videos is fraught with risk: Even if that entity has redundancy across multiple geographic locations, they can have security breaches, their software can fail, their business can fail, or they can otherwise become incapacitated.

Security breach risk can be drastically reduced by encrypting your files before you even store them in that entity's system(s). But you'll need to protect that key and keep it in multiple locations that multiple people have access to. And it's going to make certain functions (like search indexing or file format conversions, to battle file format obsolescence) a lot more complicated.

Data loss risk due to failing hardware, software, companies, or people, can be greatly reduced by copying your stuff to multiple storage providers who ultimately rely on physically different data centers (e.g. if company A and B both ultimately use S3, Amazon is now my single point of failure), and taking an "append-only" approach.

But who is going to spend the time to pre-encrypt and sync all of your family's records like that? Possibly the family computer expert, who you can't depend on long-term.

If you're lucky, you have one who will do all of that for now, and maybe give everybody in the family a copy on a durable USB drive or similar* every few years. Or maybe you can pay someone to do the same. I'd pay for it, because it's a lot of work I don't want to do.

* For longer-term archival storage that should last several decades without power, M-Disc (DVDs or Blu-ray discs) used to be a go-to option. But that depends on people having access to a Blu-ray player decades from now, which I suspect will be a lot like the process of getting access to a floppy drive today.

cwilper

3 ปีที่แล้วdiscuss

Yep, the guy has a gift. I've never touched a line of Clojure or looked at Datomic, but I'm repeatedly inspired by Rich's talks, which I come back to often.

cwilper

12 ปีที่แล้วdiscuss

I did something similar to this a while back, called qdupe[0], written in Python. It doesn't do the deleting for you, but is very fast at identifying duplicates if you have a lot to compare. Based on the fastdup algorithm.

[0] https://github.com/cwilper/qdupe

cwilper

15 ปีที่แล้วdiscuss

Very cool service you've got there. My advice is to get your order page going over SSL asap. Stripe's API is neat, but users don't care -- lack of encryption on a page asking me for a CC# gives off a shady vibe. Good luck!

comments