maxdamantus · HackerTrans

maxdamantus

no profile record

コメント

7 か月前議論

It's the same on Wayland. The client (usually part of a toolkit like Gtk/Qt) needs to subscribe to notifications [0] from the server so it can decide the raster size of the surface it wants to draw to. Qt does this on X11 by detecting when your window moves to a screen with another DPI and resizing/rescaling.

I guess the "third" program would be something like xrandr, so the Wayland analogue to that would be wlr-randr (for wlroots compositors), or some other DE-specific tool for configuring screen sizes. Again there's no fundamental difference here.

[0] https://wayland.app/protocols/fractional-scale-v1#wp_fractio...

7 か月前議論

> Per-display DPI settings

fwiw, Xorg already had this, since you can set the DPI for each display through RandR/xrandr. In both X11 and Wayland it's up to the toolkit to actually detect the setting and rasterise accordingly.

Wayland actually went backwards in this respect by using "integer scales" (eg, 1, 2, 3) instead of fine-grained DPIs (eg, 96, 192, 288), so using a scale of 1.5 would result in downscale blur (toolkit sees scale as 2, then the compositor scales it down to 75%), whereas in Xorg you could just set the DPI to 144, and the toolkit could theoretically render at the correct resolution. As far as I know Qt was the only toolkit to actually do this automatically, but that's not X11's fault.

Wayland has at least since fixed this in the form of "fractional scaling" [1], but here's [0] an old thread on HN where I complained about it and provided screenshots of the resulting blur.

[0] https://news.ycombinator.com/item?id=32021261

[1] Doing some quick searching it seems like this is still unsupported in Gtk3/Gtk4, maybe planned for Gtk5? Apparently Firefox has only just added support (December 2025), 3 years after the fractional scaling protocol was released. Seems ridiculous to me that Wayland failed to get this right from the start.

8 か月前議論

> It's true that any modern processor has a "compilation step" into microcode anyway, so in an abstract sense, that might as well be some kind of bytecode.

This.

> What I can imagine is a purpose-built CPU that would make the JIT's job a lot easier and faster than compiling for x86 or ARM. Such a machine wouldn't execute raw Java bytecode, rather, something a tiny bit more low-level.

My prediction is that eventually a lot of software will be written in such a way that it runs in "kernel mode" using a memory-safe VM to avoid context switches, so reading/writing to pipes, and accessing pages corresponding to files reduces down to function calls, which easily happen billions of times per second, as opposed to "system calls" or page faults which only happen 10 or 20 million times per second due to context switching.

This is basically what eBPF is used for today. I don't know if it will expand to be the VM that I'm predicting, or if kernel WASM [1] or something else will take over.

From there, it seems logical that CPU manufacturers would provide compilers ("CPU drivers"?) that turn bytecode into "microcode" or whatever the CPU circuitry expects to be in the CPU during execution, skipping the ISA. This compilation could be done in the form of JIT, though it could also be done AOT, either during installation (I believe ART in Android already does something similar [0], though it currently emits standard ISA code such as aarch64) or at the start of execution when it finds that there's no compilation cache entry for the bytecode blob (the cache could be in memory or on disk, managed by the OS).

Doing some of the compilation to "microcode" in regular software before execution rather than using special CPU code during execution should allow for more advanced optimisations. If there are patterns where this is not the case (eg, where branch prediction depends on runtime feedback), the compilation output can still emit something analogous to what the ISAs represent today. The other advantage is of course that CPU manufacturers are more free to perform hardware-specific optimisations, because the compiler isn't targeting a common ISA.

Anyway, these are my crazy predictions.

[0] https://source.android.com/docs/core/runtime/jit-compiler

[1] https://github.com/wasmerio/kernel-wasm (outdated)

8 か月前議論

Sir say "tray bean". Mercy.

9 か月前議論

> Oh, wait, there is:

As far as I can tell this is a transliterator, not a translator. It's just turning latin letters into hieroglyphs as you type them. I don't know how accurate the transliteration is.

It would be like coming up with a sequence of Chinese characters that sounds like an English sentence when pronounced by a Mandarin speaker. Nothing really to do with translation.

9 か月前議論

Indeed, my understanding (which is backed up by your link) is that the hieroglyphs aren't just pictograms that try to draw the meaning but they tend to have particular pronunciations, and the selection of glyphs will usually depend on both the sound and the meaning of the word.

I guess Chinese characters work similarly, where eg, each character has a particular sound in Mandarin (with some characters having the same sound), but you spell words using certain characters based on the (sometimes historical) semantic association of components (radicals) within each character.

I'll admit I'm not an expert in either system, so sorry if either description seems like an oversimplification (I'm pretty sure there are exceptions in both cases).

This also leads to one of my favourite tables on Wikipedia [0], showing correspondences between various scripts, including Egyptian hieroglyphs and Arabic/Hebrew. Not all hieroglyphs are included, but you can see that each letter in Arabic/Hebrew ultimately derives from some hieroglyph which would have had a similar sound. The name of the Arabic letter ع sounds the same as the Arabic word for "eye" (ʿayn, عين) and the corresponding hieroglyph also looks like an eye.

[0] https://en.wikipedia.org/wiki/Phoenician_alphabet#Table_of_l...

9 か月前議論

> Exceptions in transactional logic are often used to represent a "rollback persistent store changes made thus far" scenario.

Handling can be added to change the transaction to be read-only if the inner code throws a particular exception, but the consistency should still be checked through a `commit` phase (at least in an OCC setting), so the `continue` in `finally` is still the correct way to do it.

> could be expressed in a clearer manner by having it within the `try` block as described by IntelliJ here[0].

> 0 - https://www.jetbrains.com/help/inspectopedia/ContinueOrBreak...

Wrong link? The only solution I see there is to add a comment to suppress the warning, which sounds fine to me (eg, analogous to having a `// fallthrough` comment when intentionally omitting `break` statements within `switch`, since I can agree that both of these things are uncommon, but sometimes desirable).

9 か月前議論

You want to at least check that the exception was raised in the absence of read anomalies. The check for read anomalies in OCC happens during the commit phase.

Setting a transaction to read-only on error is possible using the code (using a rethrowing catch within the transaction), but this is not universally desirable.

If you're using transactions to run fairly arbitrary code atomically (assuming no static state outside of the transaction), the expected behaviour would be that modifications leading up to an exception (in a non-anomalous transaction) are still persisted. Eg, imagine the code within the transaction is updating a persisted retry counter before performing a fallible operation. In this case you want the counter to be updated so that the transaction doesn't just fail an infinite number of times, since each time you roll back on error you're just restoring the state that leads to the error.

Another case would be where the exception is due to something that was written within the transaction. If the exception were raised but the writes were not persisted, it would at least be confusing seeing the exception, and possibly logically incorrect depending on how the irrelevant exception is handled (since it's due to data that theoretically doesn't exist).

9 か月前議論

I wasn't thinking of JDBC SQL transactions specifically, but sure, different APIs can denote retriable transaction failures differently. Instead of:

  if (!tx.commit()) { continue; }

you do:

  try { conn.commit(); } catch (SQLTransactionRollbackException e) { continue; }

and the principle still applies. The simplest solution still involves a `continue` within the `finally` block.

Whether it's a good idea to actually do this directly using SQL connections is another question .. SQL databases usually use pessimistic locking, where the transaction failures are actually "deadlocks" that are preferably avoided through careful ordering of operations within the transaction (or more commonly, YOLOing it using an isolation level that allows read anomalies). Without going into all the details, this probably has a large influence over the design of the SQL APIs you're referring to.

9 か月前議論

> A transaction failing is the opposite of an unexpected event.

That's why it's denoted by a non-exceptional return value from `tx.commit()` in my sample code. When I've talked about exceptions here, I'm talking about exceptions raised within the transaction. If the transaction succeeds, those exceptions should be propagated to the calling code.

> Exceptions are for unexpected events such as the node running out of memory, or a process failing to write to disk.

Discussing valid uses of exceptions seems orthogonal to this (should OOM lead to a catchable exception [0], or should it crash the process?). In any case, if the process is still alive and the transaction code determines without error that "yes, this transaction was invalid due to other contending transactions", it should retry the transaction. If something threw due to lack of memory or disk space, chances are it will throw again within a successful transaction and the error will be propagated.

[0] As alluded to in my first post, you might want to add some special cases for exceptions/errors that you want to immediately propagate instead of retrying. Eg, you might treat `Error` subtypes differently, which includes `OutOfMemoryError` and other cases that suggest the program is in a potentially unusable state, but this still isn't required according to the intent of the transactional logic.

9 か月前議論

A result of an inconsistent transaction should be discarded whether it's a return value or a thrown exception. If it runs out of tries another error should be thrown. This should only happen due to contention (overlapping transactions), not due to a logical exception within the transaction.

You can add extra logging to show results or exceptions within the transaction if you want (for the exception this would simply be a `catch` just before the `finally` that logs and rethrows).

I've omitted these extra things because it's orthogonal to the point that the simplest way to express this logic is by having the `continue` control flow unconditional on whether the code was successful .. which is what you use `finally` for.

If you did this in Rust noone would complain, since the overall result is expressed as a first-class `Result<T, E>` value that can naturally be discarded. This is why Rust doesn't have `finally`.

Rust is also a lot more permissive about use of control flow, since you can write things like `foo(if x { y } else { continue }, bar)`.

Personally, I prefer when the language gives a bit more flexibility here. Of course you can write things that are difficult to understand, but my stance is still that my example code above is the simplest way to write the intended logic, until someone demonstrates otherwise.

I don't think this is a restriction that generally helps with code quality. If anything I've probably seen more bad code due to a lack of finding the simplest way to express control flow of an algorithm.

I'm sure there's some train of thought that says that continue/break/return from a loop is bad (see proponents of `Array.prototype.forEach` in JS), but I disagree with it.

9 か月前議論

Sounds like the right intent to me. To pinpoint your existing quote from the documentation:

> The finally block always executes when the try block exits. This ensures that the finally block is executed even if an unexpected exception occurs.

The intent of the transaction code is that the consistency is checked (using `tx.commit()`) "even if an unexpected exception occurs".

I'm not sure how else to interpret that to be honest. If you've got a clearer way of expressing this, feel free to explain.

9 か月前議論

And JavaScript .. And Python (though as sibling posts have mentioned it looks like they're intending to make a breaking change to remove it).

EDIT: actually, the PEP points out that they intend for it to only be a warning in CPython, to avoid the breaking change

9 か月前議論

Interesting .. from the post above:

> The projects examined contained a total of 120,964,221 lines of Python code, and among them the script found 203 instances of control flow instructions in a finally block. Most were return, a handful were break, and none were continue.

I don't really write a lot of Python, but I do write a lot of Java, and `continue` is the main control flow statement that makes sense to me within a finally block.

I think it makes sense when implementing a generic transaction loop, something along the lines of:

  <T> T executeTransaction(Function<Transaction, T> fn) {
    for (int tries = 0;; tries++) {
      var tx = newTransaction();
      try {
        return fn.apply(tx);
      } finally {
        if (!tx.commit()) {
          // TODO: potentially log number of tries, maybe include a backoff, maybe fail after a certain number
          continue;
        }
      }
    }
  }

In these cases "swallowing" the exception is often intentional, since the exception could be due to some logic failing as a result of inconsistent reads, so the transaction should be retried.

The alternative ways of writing this seem more awkward to me. Either you need to store the result (returned value or thrown exception) in one or two variables, or you need to duplicate the condition and the `continue;` behaviour. Having the retry logic within the `finally` block seems like the best way of denoting the intention to me, since the intention is to swallow the result, whether that was a return or a throw.

If there are particular exceptions that should not be retried, these would need to be caught/rethrown and a boolean set to disable the condition in the `finally` block, though to me this still seems easier to reason about than the alternatives.

9 か月前議論

Doesn't look difficult: https://www.fbi.gov/wanted/seeking-info/ballot-box-fires (yes, that's in Oregon)

9 か月前議論

Here in NZ when I've been to vote, there are usually a couple of party affiliates at the voting location, doing what one of the parent posts described:

> You can stay there and wait for the count at the end of the day if you want to.

And if you watch the election night news, you'll see footage of multiple people counting the votes from the ballot boxes, again with various people observing to check that nothing dodgy is going on.

Having everyone just put their ballots in a postbox seems like a good way remove public trust from the electoral system, because noone's standing around waiting for the postie to collect the mail, or looking at what happens in the mail truck, or the rest of the mail distribution process.

I'm sure I've seen reports in the US of people burning postboxes around election time. Things like this give more excuses to treat election results as illegitimate, which I believe has been an issue over there.

(Yes, we do also have advanced voting in NZ, but I think they're considered "special votes" and are counted separately .. the elections are largely determined on the day by in-person votes, with the special votes being confirmed some days later)

9 か月前議論

The point about inducing segmentation faults is interesting and sounds like it could work to implement the `hint_read` mechanism. I guess it would mostly be a question of how performant userfaultfd or SIGSEGV handling is. In any case it will be sub-optimal to having it in the kernel's own fault handler, since each userfaultfd read or SIGSEGV callback is already a user-kernel-user switch, and it still needs to perform another system call to do the actual reads, and even more system calls to mmap the bits of memory again.

Presumably having fine-grained mmaps will be another source of overhead. Not to mention that each mmap requires another system call. Instead of a single fault or a single call to `readv`, you're doing many `mmap` calls.

> I tried implementing your "hint_read" years ago in userspace in a search engine I wrote, by having a "readahead thread" read from pages before the main thread got to them.

Yeah, doing it in another thread will also have quite a bit of overhead. You need some sort of synchronisation with the other thread, and ultimately the "readahead" thread will need to induce the disk reads through something other than a page fault to achieve concurrent reads, since within the readahead thread, the page faults are still synchronous, and they don't know what the future page faults will be.

It might help to do `readv` into dummy buffers to force the kernel to load the pages from disk to memory, so the subsequent page faults are minor instead of major. You're still not reducing the number of page faults though, and the total number of mode switches is increased.

Anyway, all of these workarounds are very complicated and will certainly be a lot more overhead than vectored IO, so I would recommend just doing that. The overall point is that using mmap isn't friendly to concurrent reads from disk like io_uring or `readv` is.

Major page faults are basically the same as synchronous read calls, but Golang read calls are asynchronous, so the OS thread can continue doing computation from other Goroutines.

Fundamentally, the benchmarks in this repository are broken because in the mmap case they never read any of the data [0], so there are basically no page faults anyway. With a well-written program, there shouldn't be a reason that mmap would be faster than IO, and vectored IO can obviously be faster in various cases.

[0] Eg, see here where the byte slice is assigned to `_` instead of being used: https://github.com/perbu/mmaps-in-go/blob/7e24f1542f28ef172b...

9 か月前議論

I think the model I described is more precise than madvise. I think madvise would usually be called on large sequences of pages, which is why it has `MADV_RANDOM`, `MADV_SEQUENTIAL` etc. You're not specifying which memory/pages are about to be accessed, but the likely access pattern.

If you're just using mmap to read a file from start to finish, then the `hint_read` mechanism is indeed pointless, since multiple `hint_read` calls would do the same thing as a single `madvise(..., MADV_SEQUENTIAL)` call.

The point of `hint_read`, and indeed io_uring or `readv` is the program knows exactly what parts of the file it wants to read first, so it would be best if those are read concurrently, and preferably using a single system call or page fault (ie, one switch to kernel space).

I would expect the `hint_read` function to push to a queue in thread-local storage, so it shouldn't need a switch to kernel space. User/kernel space switches are slow, in the order of a couple of 10s of millions per second. This is why the vDSO exists, and why the libc buffers writes through `fwrite`/`println`/etc, because function calls within userspace can happen at rates of billions per second.

9 か月前議論

Just to clarify, I think the parent posts are talking about non-failing page faults, ie where the kernel just needs to update the mapping in the MMU after finding the existing page already in memory (minor page fault), or possibly reading it from filesystem/swap (major page fault).

SIGSEGV isn't raised during a typical page fault, only ones that are deemed to be due to invalid reads/writes.

When one of the parents talks about "no good programming model/OS api", they basically mean an async option that gives the power of threads; threading allows concurrency of page faults, so the kernel is able to perform concurrent reads against the underlying storage media.

Off the top of my head, a model I can think of for supporting concurrent mmap reads might involve a function:

  bool hint_read(void *data, size_t length);

When the caller is going to read various parts of an mmapped region, it can call `hint_read` multiple times beforehand to add regions into a queue. When the next page fault happens, instead of only reading the currently accessed page from disk, it can drain the `hint_read` queue for other pages concurrently. The `bool` return indicates whether the queue was full, so the caller stops making useless `hint_read` calls.

I'm not familiar with userfaultfd, so don't know if it relates to this functionality. The mechanism I came up with is still a bit clunky and probably sub-optimal compared to using io_uring or even `readv`, but these are alternatives to mmap.

9 か月前議論

> I can say that the display & overall performance is noticeably faster on the two actual computers I tested on than under qemu on my Linux system.

You'd probably want to use `-enable-kvm` so it's not doing full software emulation. Assuming you're running this on another x86-64 machine.