Simon Peter, at UT Austin, has been working on a file system that works better for systems with NVM. It looks like existing file systems (e.g. EXT4) are going to suck in this new paradigm (DRAM -> NVM -> SSD) . It’s a pretty interesting read and the benchmarks are damn impressive.
Hey y'all, I recently started working on a CI/CD product at an organization where the number of deployments per month is in the same ballpark as Netflix. In my experience, adopting systems for rapid deployments also requires adopting new processes. For example, products that are deployed frequently need to have developers on call during deployment windows. These developers tend to have a specialized skill set that makes them effective at handling botched deployments and bugs in production.
How y'all are planning to approach the cultural changes companies will have to adopt in order to leverage faster deployments?
A friend who works at the Dallas Animal Shelter (a non-profit) asked me to help automate their process of generating reports from data gathered via Google Forms, which they were doing by hand every month... As a developer who is used to high technical investment into complex systems, I had to take a completely different approach to storing their information.
Google Forms + Spreadsheet + Python works perfectly for use cases like this: it is free, scaling isn't a concern, users can view and modify data at any time, and users can create Google Forms at any time. If you are trying to help a non-profit and aren't planning on maintaining their systems long-term, this is a great approach.
The concept of immutable assets and links is at the core of IPFS, a distributed alternative to HTTP. Since Firefox inplements the concept of immutable assets now, it would be totally reasonable to load these assets in the browser peer to peer (see WebRTC and webtorrent). I think this would be a great way to retrofit some decentralization into webpages!
LinkedIn has a similar open source project that is much more mature. It builds on Gradle features to manage complex dependencies and build Python artifacts. If you include this LinkedIn Gradle Plugin [1] you can automatically run tests in a virtual env and source a file to enter the project's virtual env.
PyGradle [2]: "The PyGradle build system is a set of Gradle plugins that can be used to build Python artifacts"
You are correct that creating a program that verifies any other program is mathematically impossible. However, given some critical assumptions regarding the scope of the problem, we can use formal verification techniques to generate proofs for some programs. People in formal verification (https://en.m.wikipedia.org/wiki/Formal_verification) do research to find better ways of creating proofs for programs automatically.
I'm not sure that this experiment is all that useful in determining the practicality of using Raft instead of Paxos. Anybody implementing one of these algorithms is going to be spending more than one hour learning it. In my experience, there a lots of corollaries that must be proven in order to actually implement the spec described by the Raft paper (mainly due to weak assumptions that cannot be assumed in all production systems). Paxos is a bit more established, so this is not as big of an issue; however, I still do not have an intuitive understanding of Paxos. This can (and will) lead to non-deterministic bugs in your implementation. A more useful study would track bugs in Raft & Paxos implementations.
This is a really important point that a lot of people seem to have missed. A common assumption that stems from monotonously increasing clocks is that: all computers can guarantee the causality of their own internal events. Which is a critical assumption for a wide variety of distributed algorithms.
https://www.cs.utexas.edu/~simon/sosp17-final207.pdf