gorset

6 miesięcy temudiscuss

One of the advantages of of protobuf I never see anyone highlight is how neat and well-designed the wireformat is, in terms of backward/forward compatibility and lowlevel stuff you can do with it. Very useful when building big and demanding systems over time.

For high performance and critical stuff, SBE is much more suitable, but it doesn't have as good of a schema evolution story as protobuf.

gorset

8 miesięcy temudiscuss

This brings back memories debugging an azul zing bug where an effectively final optimization ended up doing the wrong thing with zstd-jni. It was painful enough that I couldn’t convince the team to enable the optimization again for years after it was fixed.

gorset

9 miesięcy temudiscuss

I agree. It’s the enshittification of the internet. Luckily we still have infrastructure providers with more sensible offerings. We don’t have to use aws, gcp, etc.

gorset

10 miesięcy temudiscuss

I've used bazel for about 10 years, all in small orgs. Right now, single digit team.

I don't think there exists a better solution for a project mixing Java and C/C++ than Bazel. The new module system for bazel has matured a lot. As an example, it's trivial to add boringssl and access it from Java.

gorset

10 miesięcy temudiscuss

That’s the default in the monorepos I’ve worked on.

When a third party dep is broken or needs a workaround, just include a patch in the build (or fork). Then those patches can be upstreamed asynchronously without slowing down development.

gorset

11 miesięcy temudiscuss

I enjoyed both talks a lot.

I really appreciate the work done to evolve Java. They seem to be able to both fix earlier mistakes (and there were many!) and new ones from happening. While I see other languages struggling with the weight of complexity added over time, Java seems to get simpler and easier even with significant updates to the language spec.

gorset

11 miesięcy temudiscuss

I think the «best practices» found in Java enterprise has meant that a lot of people think all Java has to look like that.

High performance Java is awesome and can be competitive with C++ and Rust for non trivial systems, thanks to much better development cycle, debugging and high quality libraries.

Most the benefits is the JVM, so Kotlin has those too, but I don’t feel Kotlin is enough of an improvement to warrant the downsides (at least for me). But both Kotlin, Scala and Clojure are great for the Java ecosystem.

gorset

11 miesięcy temudiscuss

Mechanisms for getting the linux kernel out of the way is pretty decent these days, and CPUs with a lot of cores are common. That means you can isolate a bunch of cores and pin threads the way you want, and then use some kernel-bypass to access hardware directly. Communicate between cores using ring buffers.

This gives you best of both worlds - carefully designed system for the hardware with near optimal performance, and still with the ability to take advantage of the full linux kernel for management, monitoring, debugging, etc.

gorset

11 miesięcy temudiscuss

Isolating a core and then pinning a single thread is the way to go to get both low latency and high throughput, sacrificing efficiency.

This works fine on Linux, and common approach for trading systems where it’s fine to oversubscribe a bunch of cores for this type of stuff. The cores are mostly busy spinning and doing nothing, so it’s very inefficient in terms of actual work, but great for latency and throughput when you need it.

gorset

w zeszłym rokudiscuss

In a previous project we used fixedint32/64 instead of varint values in the schema for messages where performance was critical.

That left only varint used for tags. For encoding we already know the size at compile time. We also don’t have to decode them since we can match on the raw bytes (again, known at compile time).

A final optimization is to make sure the message has a fixed size and then just mutate the bytes of a template directly before shipping it off. Hard to beat.

In a newer project we just use SBE, but it lacks some of the ergonomics of protobuf.

gorset

2 lata temudiscuss

Nice work! I love the care and discussion around low level optimizations, as such things are often ignored.

There are a lot of interesting variants of rank/select and other succinct data structures which has interesting tradeoffs. Maybe most useful when compression is critical. At least my own data structures can’t compete with the query performance you are showing, but they are great when the memory footprint matters.

gorset

2 lata temudiscuss

The number of unique int32 values is not that great, and a full bitset would only be 512MB. Hard to bit a dense bitset in performance.

As a general purpose data structure, I would look at roaring bitmaps for this purpose, which has good trade-offs with great performance for most use-cases.

If only simple lookups are needed over a static set, then it's worth looking at minimal perfect hash functions (https://sux.di.unimi.it).

gorset

2 lata temudiscuss

Selective testing is great also for smaller projects to help keep velocity high for merging and working with branches with many commits.

I implemented selective testing using bazel for a CI some years ago, and it was painful to get it right. When finished even bigger branches would only take seconds-to-minutes to go through the pipeline, which was a significant improvement from the ~30 minutes build when I started working with the project, even though the project size grew a lot.

Glad to see mill-build is prioritizing this feature.

gorset

2 lata temudiscuss

I'm a big fan of this solution! It's always been annoying to perform all the ceremony involved in deploying a system with a bunch of files, with config, scripts and system written in a bunch of different languages.

In my current project I just wrote the installer and config generation as part of the main method. Gets rid of a lot of complexity, with a simpler build, and is arguably easier to maintain. Single language, single binary.

gorset

2 lata temudiscuss

I always found the existing bazel integrations annoying. My solution has been to generate .classpath based on a set of bazel queries. Not what's officially recommended, but it has worked great for the projects I've been involved in. Easy to understand, and keeps bazel completely separate and independent from the IDE experience which has some benefits.

gorset

2 lata temudiscuss

There's a lot of ceremony to run jdtls!

I find it easier to just install language servers using brew. Then the command can be simplified to

      cmd = { 'jdtls', '-data', home_dir .. '/.local/state/jdtls/' .. project_name },

My ftplugin/java.lua is about 100 lines, with most of it being settings and key bindings.

gorset

2 lata temudiscuss

They could also have gone the commercial route and gotten Zing with their pauseless GC. It’s been around forever and they even cover Cassandra in their marketing.

https://www.azul.com/technologies/cassandra/

gorset

2 lata temudiscuss

Patch series comes from the linux kernel workflow, which git was developed to support. https://kernelnewbies.org/PatchSeries

In this workflow you review every commit and not just the branch diff. Each commit is crafted carefully, and a well crafter series of commits can make even very large changes a brief to review.

It takes a certain skill to do this well. As the page above says > Crafting patches is one of the core activities in contributing code to the kernel, it takes practice and thought.

This is in contrast to using git more as a distributed filesystem where you don't care particularly much about the history, and you typically squash commits before merging etc. It's simpler and easier to work this way, but you lose some of the nice attributes of the linux kernel workflow.

gorset

2 lata temudiscuss

The use case is when you look at a branch as a series of patches.

Reviewing a clean set of commits is much easier than a branch full of mistakes and wrong paths taken.

Useful when we optimize for reviewing and good history for future maintenance. This has been important and useful when I’ve worked on big mission critical backend system, but I also understand it might not be the most important factor for a new project’s success.

gorset

2 lata temudiscuss

I struggle to use lmdb when using resource isolation (like docker, nomad exec driver). The hidden complexities really become apparent when you build largish systems using shared infrastructure. Anything is easy if you have a dedicated machine or have dataset smaller than your memory allocation.

Mmap is so simple for the application/developer, but the price you pay is giving up clear ownership of memory and clean separation between kernel page cache and application space. The interaction between the mmu, mmap, tlb and the linux kernel IO layer is complicated. You don't want to touch that complexity at all if you care about ultra low latency.

The article rephrases the question into

    "do you really want to reimplement everything the OS already does for you?"

but I would rephrase it again into questioning why fairly straight forward concepts like a buffer cache can't be implemented directly in the application?

Submissions

Radix sort is faster than quicksort for int arrays

comments