As someone who myself worked on a hobby-level Rust based Kafka alternative that used Raft for metadata coordination for ~8 months: nice work!
Wasn't immediately clear to me if the data-plane level replication also happens through Raft or something home-rolled? Getting consistency and reliability right with something home-rolled is challenging.
Notes:
- Would love to see it in an S3-backed mode, either entirely diskless like WarpStream or as tiered storage.
- Love the simplified API. If possible, adding a Kafka compatible API interface is probably worth it to connect to the broader ecosystem.
> In practice, many teams will use a pull request-based workflow because the tests could be too complicated to run locally or need to facilitate code reviews. For trunk-based development to fulfill its promise, it’s imperative that pull requests are kept small and that the team is committed to reviewing them quickly.
Would you consider adopting the STARS framework[1] ? It was designed to solve the questions you mentioned like “Are our key customers adopting the new feature we launched?” but at the same time it also incorporates qualitative feedback.
Lots of great additions. I will just highlight two:
Column selection:
When you have tons of columns these become useful. Clickhouse takes it to the next level and supports APPLY and COLUMN in addition to EXCEPT, REPLACE which DuckDB supports:
- APPLY: apply a function to a set of columns
- COLUMN: select columns by matching a regular expression (!)
I can't count how many times I've run into a problem with a trailing comma. There's a whole convention developed to overcome this: the prefix comma convention where you'd write:
SELECT
first_column
,second_column
,third_column
which lets you easily comment out a line without worrying about trailing comma errors. That's no longer necessary in DuckDB. Allowing for trailing commas should get included in the SQL spec.
I was delighted to see Opbeat on there (under Application Performance Monitoring).
Opbeat was acquired by Elastic, Inc and turned into their APM product. You can try the Elastic APM product for free as well: https://www.elastic.co/apm
(I was an Opbeat co-founder and part of the team that built Elastic APM)
The other problem here is that politicians are either oblivious or simple don't care about the cost associated with complexity in the laws. On top of that, the other unfortunate dynamic that exists is that politicians all work to get their little special case into new bills so they can point to that when their constituents ask how they influenced the new law.
Taken together, the complexity of the law just accumulates. Tax law in Denmark being a particularly gross example.
If you're using Apache Kafka or similar I'd love to understand how you're using it and which problems and annoyances you're running into. Please shoot me a mail at [email protected] if you're up for a brief Zoom call or email exchange.
I'm doing customer/product research on Apache Kafka and similar systems to inform a software project I'm working on.
taking this a bit further, the original authors could probably employ this as a strategy: Since it's an anonymous auction, the original author could win the auction and then refuse to pay repeatedly until all the real potential buyers have given up.
another contender is zstd: https://github.com/facebook/zstd. It typically offers better compression ratios than LZ4 at a slight (depending on your data) cost in speed. Additionally it offers a training mode to tune the algorithm to increase compression ratio on specific types of data, particularly useful for compression of small pieces of data.
> Not only is what the subject of the article did probably not illegal
Why do you say that?
The method they used consists of simply filing forms to get withheld taxes paid out that were never withheld. The amounts paid out corresponds to owning all stocks in the actual companies many times over.
Sanjay himself went drinking with the employee of the Danish tax authority in Copenhagen who was in charge of approving the applications while the (I'll go ahead and use the word) fraud was going on, presumably to ensure things would continue to go on[0]. The employee has since been jailed for approving fraudulent applications in another case[1]
American pension funds who similarly exploited this have settled with the danish authorities and returned all the money they received[2]
Sanjay says he suggested a settlement, offering to return 1.9B DKK (~311m USD) in 2018[3]
Yes, the danish authorities were sleeping at the wheel, but that does not make it legal.
Not really. The post is by Matt Asay who's title is Head of Open Source Strategy and Marketing. The post reads like an attempt at improving upon the negative image AWS has garnered in open source by making money on open source projects without paying, or sometimes without even crediting, the authors. Most recent examples include Headless Recorder[1] and before that Kafka, Elasticsearch etc.
Wasn't immediately clear to me if the data-plane level replication also happens through Raft or something home-rolled? Getting consistency and reliability right with something home-rolled is challenging.
Notes:
- Would love to see it in an S3-backed mode, either entirely diskless like WarpStream or as tiered storage.
- Love the simplified API. If possible, adding a Kafka compatible API interface is probably worth it to connect to the broader ecosystem.
Best of luck!