something doesn't make sense here - MySQL/InnoDB does put tables into files, but partitions get separate file.
MyRocks has a collection of files per each column family, and when you drop data it can quickly expunge files that don't contain data for other tables/partitions - and trigger compaction on neighbors, if needed.
this misses multiple points - lots of block access happens via internal caches (hyperclock/lru/etc) and filesystem isn't in the critical path as much as you'd think.
files are mapped out in relatively large chunks, especially compaction outputs - there's prealloaction, and usually you will just have a flat file->block conversion without huge trees or anything.
based on performance profiles filesystem doesn't do any heavy lifting, there's not that much fragmentation (and you usually keep some free space for flash GC anyway),
compaction output write is one logical operation on filesystem for tens of megabytes of data.
> filesystems are the unsung heroes underlying some LSM tree databases
Main use cases (social graph, messaging, ..) are on MySQL (and never left it). Storage engine is different, replication is improved, etc, but it is still tracking upstream MySQL tree.
The thing is - PG still relies on physical replication, and staying with logical one allows to use it for out-of-DBMS change data capture and reuse in other systems.
> MediaWiki is ...
Funny tidbit - I tried to move over Wikipedia to PG ages ago, and did the initial prototyping and made it work to a certain degree. But also I learned about InnoDB more at the time.
I'm not sure page level compression can get efficiency of LSM-Tree, and frequent page writes will be a write amplification problem if you end up doing any form of copy-on-write and try to do in-place overwrites.
¯\_(ツ)_/¯ optimizer guy says he did not do good job, shits on optimizer?
There're many reasons to pick MySQL over PG at large scale deployments (economics, replication strategies, etc) - and the fact that some queries will run better on PG may not outweight those benefits.
Don't get me wrong, I know many areas where MySQL sucks, and that is mostly in lacking execution strategies and optimizer problems. Indeed, in many of these ways MySQL is stuck years behind, but in other areas (MyRocks, modern hardware use, etc) it is far ahead of PG.
The thing is, optimizing has costs, and bypassing those costs is useful, if you're looking at economics of your system, and storage engines that pay attention to data patterns are ahead of just naive heaps of data.
It is very simple to diss MySQL when your business does not depend on managing petabytes of data.
P.S. We've migrated a major social network from PG onto our MySQL-based platform and our MySQL DBAs didn't even notice. :-)
So sad I can't see or reply to that comment anymore - I'm the author.
Sorry, HN, my personal blog rarely gets that wide circulation, I should've done better job at introducing the context.
This was just one of many systems that thousands of facebook engineers build and they did not think of some edge cases. We do know how to write queries or to operate our databases, in general.
It was conversion from standard MongoDB to RocksDB based MongoDB, so replica count calculations don't really apply.
And yes, Mongo did not optimize for space efficiency, which is why there're efforts to optimize for space efficiency.
Just like there were efforts to optimize InnoDB for space efficiency, that stalled and got derailed by these hole punching ideas.
InnoDB did not become shit, it just stalled and did not progress in an area where lots of innovation is happening elsewhere (and there were possible avenues for optimization), so it may become less relevant.
As the post [title] said, InnoDB lost its advantage.
No, that is not the highest praise you can get from me. It definitely is very interesting technology, and what it does well, it does really well - though I didn't get to benchmark parallel performance, as I had work to do :-)
I wouldn't have written that post the way it is if your marketing would be honest and with lots of actual technical merit. There're plenty other companies in the community that have invested much more into decent technical communication.
I agree that in some cases pure durability isn't needed and the best-effort with enforced maximum slippage is good enough, but still, pure durability implementation is extremely naive, and I can show that it hits benchmarks really badly :-)
Your optimizer does not optimize very simple things, and though you can claim that in-memory performance hides that, still, it is nowhere close a drop-in replacement of any kind.
I understand what you've done from technology perspective, and it is an interesting approach (albeit I'm not trying to evaluate applicability).
I just cannot approve the way you did initial publicity - unfair comparisons are unfair comparisons, and if you're using them that way, I can come up with my unfair comparison :)