dzaima · HackerTrans

9 uur geledendiscuss

Unaligned loads/stores aren't super bad; if still within a cacheline, there's zero penalty, and on crossing cachelines it's alike two ops (except page crossing, which is more bad).

So, for 32B loads/stores and 64B cachelines, it's 1.5x more L1 cache ops (as half of the ops will cross a cacheline); perhaps bad if you're L1-cache-throughput-bound, but less so if you're at L2+ as the extra work sits in L1.

dzaima

3 dagen geledendiscuss

Where I eat doesn't have a "WC" sign, walls separating it from public areas, and a toilet present. Codebergs private repos have a clear "private" label, infrastructure for making them, acknowledgement and rules for it in ToS, and they are kept private.

Anyway, originally I didn't quite agree with fs111 on straight up calling your post misinformation, but I guess it is indeed your intent to completely ignore facts and just say random garbage.

dzaima

4 dagen geledendiscuss

...in what way whatsoever is this "They dont allow private repos"?

The size limit is........a size limit, something every host should have; there's one for public codeberg repos too.

And while there are restrictions on what private repos are allowed, there are also restrictions on what public repos are allowed too, and it's extremely clear that neither requirement is equivalent to "they don't allow [public / private] repos".

My reading of that section is that, besides the size limit, the rules on private repos are, to an extent, less strict than of public repos; anyone who falls under your second quoted sentence couldn't have any public repos either, by the nature of public repos needing to be FOSS.

dzaima

9 dagen geledendiscuss

> For example any such function can be marked with __attribute__((pure)) in compilers like gcc and clang.

https://gcc.gnu.org/onlinedocs/gcc/Common-Attributes.html states:

> However, functions declared with the pure attribute can safely [...] in a way that does not affect [...] or the observable state of the program

So an exception-aware FP op cannot be __attribute__((pure)) in a program that may read the FP status later, or has trapping on FP exceptions enabled.

This is obvious from the fact that gcc & clang will entirely delete a call to a pure function if the result isn't used, thereby making potential FP traps not happen, or accrued exceptions not mutated.

(also traps on exceptions would make FP ops clearly not pure by the definition in functional programming too)

> If exceptions were enabled only for a part of the program, then the rest of the program can still generate special values and you need to test for them everywhere. So you get the worst of both choices, a complex program that can be crashed by an error.

Would be perfectly fine if combined with separate NaNful and NaNless types; design the language s.t. exception handling state must always be (or is automatically set to) the necessary one, and you never get NaNs in the NaNless types.

dzaima

9 dagen geledendiscuss

> Floating-point operations are stateful in any CPU made during the last 40 years.

But sane programming languages entirely hide that (to the same extent that they hide integer flags and registers in general), at the very least by default; and in such sane languages, programmers do not need to be aware of any of this mess.

e.g. C's requirement of "#pragma STDC FENV_ACCESS ON" for non-default FP status, and lack thereof in 99.99% of code; C spec even explicitly notes:

> With these conventions, a programmer can safely assume default floating-point control modes (or be unaware of them). The responsibilities associated with accessing the floating-point environment fall on the programmer or program that does so explicitly.

(never mind basically every other language, which don't even allow anything other than round-nearest-ties-even + no-exceptions)

> Enabling some or all kinds of FP exceptions has absolutely no influence on auto-vectorization. It also has no influence on speed, in well-designed CPUs.

If you want to have deterministic semantics (i.e. want to be a sane language), it absolutely does; it means that, in `a+b; x[i];`, the `x[i]` load must not happen if `a+b` results in a trap. So you cannot ever execute the next loop iteration before being sure the previous doesn't trap. Entirely forbidding autovectorization.

If you want to still have autovectorization, you'd need to have some washy semantics of like "the trap of an fp op can be delayed to an arbitrarily-later point in time, up to [insert some messy ambiguous incomplete attempt at establishing some limit to this]", which, besides making basic code non-deterministic (yikes! we don't need more of that in the world), will make proving compiler optimization correctness extra-messy.

Non-trapping exceptions (i.e. accruing) would be simpler to preserve through vectorization (allowing reordering up to reading the status, which is explicit in code), but you're back to allowing observable NaNs and making programmers insert manual checks. (and even then, you'd run into issues if you wanted to vectorize the tail of a loop on architectures without maskable FP SIMD ops)

(and, of course, unrelated to any of this, giving the programmer control over FP status means compilers are massively-restricted on optimizing code - can't DCE float ops, can't insert new FP ops even if it would be beneficial, etc)

dzaima

12 dagen geledendiscuss

> A decent programming language should have 2 distinct types for floating-point numbers, one for those that cannot be NaNs (to be used only in a program that enables and handles invalid operation exceptions) and one for floating-point numbers that may be NaNs (to be used in programs that disable the invalid operation exception, like most programming languages wrongly do by default).

You've already explained why such a type separation is a bad idea - those types being only usable with specific setup; so, completely utterly breaking composability.

Never mind this making float ops impure & stateful (also forbidding autovectorizing anything with more than one potentially-NaN-producing op if you don't want to break semantics).

Perhaps if hardware supported embedding exception behavior in individual instructions this'd not be insane, but I haven't heard of any architecture having such, making it a complete non-option for sane languages designed to be used.

(there is the option of making compilers insert the necessary fp state transitions, though then you'll have the desire to embed it into calling conventions & function types to avoid transitions when a certain state is expected to stay for a prolonged period of time, which, while certainly possible and would be quite neat to have (also for rounding modes, FTZ/DAZ, etc), is very much in the territory of something almost noone will bother doing, and as such things would just quietly be slower)

While partially-ordered types are neat, what can you really sanely do with a comparison over such? Seems like a rather pointless thing to have.

dzaima

15 dagen geledendiscuss

Rather depends on definitions; GPL does contain:

> The "source code" for a work means the preferred form of the work for making modifications to it.

With that definition, there's definitely space for arguing that the AI tooling for modifying the code is necessary for the modification process to be sane therefore "preferable" for any human, if the code is "designed" (or lack of design thereof) around the idea of being AI-maintained.

Otherwise, it's not source-code, it's not meaningfully-modifiable, it's basically equivalent to just decompiling a binary. (similarly-bad quality may of course be human-produced too, though then at least you have direct proof of it being the preferred form for at least one person - the author)

dzaima

19 dagen geledendiscuss

FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16 (allowing the bitcast to be O(1) by itself). (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs)

The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, but mostly only AVX2 on mainline; and no arm SVE)

That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway.

(didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing; and, of course, if some of the columns are bools/int8/int16/int32, it'd be beneficial to store & load them as such instead of float64)

dzaima

20 dagen geledendiscuss

Perhaps some early things in the pipeline could be easier with a saner language, but I can't imagine it affecting much of the core optimizations, which'd all benefit from being ran on some messy low-level intermediate form equivalent to what C converts to anyway (if not more complex if preserving precise aliasing info). Would be curious to hear about that external visibility thing, doesn't feel like it could make optimizations (beyond inlining, perhaps?) harder.

dzaima

21 dagen geledendiscuss

Formal proofs for specific instruction sequence substitution (e.g. LLVM's instcombine) are simple-ish enough via just throwing SMT at it, ...as long as the source pattern and target replacement are in a format from which both the compiler code and verification source+target can be automatically derived from; though you'd still need manual proofs for anything with unbounded configurations that can't be exhaustively checked if you aren't satisfied with just checking some subset.

Larger-scale things operating over an unbounded amount of instructions require significant amounts of effort of verification on each pass. CompCert apparently has 1200LoC of proving DCE, one of the simplest whole-function passes - https://github.com/AbsInt/CompCert/blob/02fc017cf69210db5fd5...

dzaima

21 dagen geledendiscuss

..except, if you want to add logging/metrics/other logic, it's quite possible you'll want it to be conditional on the boolean anyway, bringing branching back, now mixed with the non-branching code.

And even if you don't need to keep an if statement in mind, you still need to keep the variable in mind anyway.

dzaima

25 dagen geledendiscuss

Note that glibc does provide a malloc_usable_size to query the size of a malloc'd block; not standard though of course.

A problem with just directly exposing such is it makes precise sanitizing impossible, as you'd have to tolerate some out-of-intended-bounds reads/writes. (and making the sanitizer always give exact-size allocations would also be bad as that'd end up not testing code paths that may break when they're not)

dzaima

25 dagen geledendiscuss

Not that I know of; and such would necessarily have false-positives (...or, rather, entirely consist of potential false-positives) because you may actually want the re-read.

dzaima

25 dagen geledendiscuss

The crappiness of shrink-wrapping in gcc and clang (but especially clang) annoys me a lot. It feels like there should be a quite decent amount of general performance to be gained from properly pushing more into slow paths (or, not necessarily even slow paths, but generally paths with high register pressure / uninlined function calls), never mind calling conventions in general.

On the push impl in the article - for non-x86 (and perhaps even on x86 for performance, though not size/instruction count) it'd be better to allow the size increment to reuse the size read done by the capacity check; with C++'s lack of suitable aliasing information, the interleaved memcpy/store prevents the compiler from deciding this itself.

dzaima

26 dagen geledendiscuss

Ampere Altra is for cloud/datacenters/servers where multithreaded throughput is approximately all that matters. Apple M series is for consumers.

dzaima

26 dagen geledendiscuss

That'd only help for one object per address space. Main thing needing relocation - shared libraries - needs arbitrarily-many segment bases.

And when you're not a library, relocation is just a mild probabalistic security improvement (...that'd be massively-more bypassable than it already is if the program was littered full of gadgets of "read register as unrelocated offset and use it with its correct base" instructions).

dzaima

27 dagen geledendiscuss

OP links you to POSIX explicitly denoting it being implementation-defined - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/e..., and https://pubs.opengroup.org/onlinepubs/9699919799/utilities/e... literally says "It is not possible to use echo portably across all POSIX systems unless [specific setup]"

Interestingly, that doesn't allow implementation-defined behavior for "echo -e", for which bash does have special behavior.

dzaima

27 dagen geledendiscuss

ECMAScript has a pretty massive amount of fully-specified behavior though; the things that differ between those implementations is nearly-entirely limited to fresh additions like `require` or whatever.

The echo thing would be like if ECMAScript allowed stuff like `"123" == 123` to give either false or true; and then indeed many things would probably break if moved across implementations.

C is the closer comparison, and indeed much software that could easily be portable (and might claim it is) often depends on implementation-specific things like 8-bit bytes, 32-bit int, assuming int8_t/etc in stdint.h exist, twos complement (before C23 at least), arithmetic shift right, etc.

dzaima

27 dagen geledendiscuss

That seems to be be an entirely-different question - `echo "c:\\new"` still differs in behavior between bash and dash - dash parses backslashes in both the double-quoted string, and then echo does another backslash parsing pass, still printing a newline; whereas bash prints a backslash + n.

dzaima

2 maanden geledendiscuss

> But this has never been a condition in the FOSS world, as far as I'm aware. I've only ever seen attribution requirements attach to redistribution of source, not usage of the software.

AGPL requires that even users using the software even across a network must be provided with a way to get the license (i.e. attribution) and source. Never mind that LLMs consume the source code instead of "using" the software anyway. (and of course things go more downhill for LLMs for licenses more restrictive than AGPL)

Otherwise, I'd say that, for many, the ideal condition for (copyleft) FOSS would be that anything that utilizes source code in any form also provides said source code and license/attribution. Sometimes that can even extend to outputs of software (and e.g. gcc takes time to explicitly state that its compiled code output does not count as being derived from gcc's code).

> whether training an LLM is redistribution of the underlying code

There's a funky side-note of whether LLM training can even be done on material with improperly-followed licensing; if you don't even have the permission to modify the material (as properly following MIT/GPL/etc would give you), it might be illegal to even tokenize it, never mind use it for training.

> That's literally all LLMs do. That's what tokenization is.

It's clearly not that simple, otherwise "split source into 10-char chunks, reverse that list, reverse it back, join this fun list we've gotten" would be enough to circumvent copyright.

> all you'll see on the LLM side is probability matrices representing correlations between decomposed units of knowledge aggregated across the entire dataset as an integrated whole.

Yeah, you need at least that, tokenization is irrelevant. But jury's out on this one - of course a good chunk is some form of "abstract knowledge", but other parts could be just encoding material in some compressed form (and surely gzipping a source code file doesn't circumvent copyright) that at the very least can apply to weights.

> The only intent ever in play is that of the user. LLMs are just software.

So my split-into-words-and-join-back is valid circumvention of copyright, if the user of some software doing that isn't informed that it's just effectively directly copying material. (I'll grant that perhaps, in such, the accidental-infringer might get a smaller penalty and/or get to defer punishment to whoever mismarketed the software to them,...but that wouldn't apply to anyone who knows that LLMs are very much just directly trained on copyrighted material. Don't know about legally derived, but surely mathematically derived)

Never mind that, for some things, learning some specific copyrighted code is the desired thing (humans do do this after all!), at which point at the very least the weights of the model are as copyright-infused as a gzipped source code file is.

If intent determination is on the user, and the user is aware that LLMs are very much technically capable of producing copyrighted works to some extent (which they better be), it would be on the user to ensure that any specific code they end up using is not, which is...a rather non-trivial task (a human that writes code can also reasonably-reason about whether they're infringing on whatever they learned from, but splitting into LLM writing + human checking fundamentally makes that basically infeasible).

comments