Those numbers are wrong for most use cases. Likely the LLM did not take SWA (Sliding Window Attention) of G4 into account. Without SWA those numbers could be correct, I can't load a q8 without SWA on a 24GB card.
I tested this with the at the time newest llama.cpp master on a Linux system with 2 3090 24GB, only one was used for testing. q8 without any KV quant, 256k context, mmproj loaded takes less than 20GB VRAM. This runs at about 1.5k to 2k tok/s pp and 40-50 tok/s gen (slightly lowered power limits & undervolted). q8 with 64k non-quant context and mmproj takes just under 16GB VRAM. Drop down to the q6k model, no mmproj, 64k non-quant context and it fits in 12GB VRAM. All the way down to q4km and some batch size tweaking and it barely fits into 8GB VRAM.
64k context is the minimum for Hermes agent, so a vision capable "agentic" model fits into a 16GB card. This is very impressive. I am currently testing how smart the model is and it does decently so far, had one looping issue it recovered after a lot of tokens, did some basic tool calling.
Ah, ok. I took a look at the 3090 numbers and they list 400 tok/s prefill, so if I normalize my expectations to that base line the numbers you posted do make sense. I haven't dug deep into that site's methodology, but their estimates seems way off. Especially since they don't take into account cache quant when deciding whether or not you can run a model. Overall I found that website a bit confusing, but maybe the UX just didn't click with me.
> The MI100 is roughly double the performance on Qwen 3.5 35B A3B Q5_K_M to the R9700 (462 token/s prefill vs 239 tokens/s, 217 tokens/s vs 118 token/s for inference)
Those prefill numbers look really low to me. I can run nearly that same model (qwen 3.6) at q4km with q6 cache on a single 3090 and get 2.3k-4.4k prefill and 100-170 generation. Just based on raw numbers I would expect the R9700 to land around 70-90 generation (about 2/3 of memory bandwidth of a 3090) and at least the same or higher prefill (nearly 3x FP16 TOPS on the R9700). That means the numbers really don't add up. Is the benchmark done with some special settings, e.g. parallel requests or with very low prompt length?
Yes, $1.3M in token cost in less than 30 days and some days were even off-peak, if you can call it that with that insane scale that likely hides quite a lot of tokens in the lower bars.
Without having firm data, I can see a few factors that are different. After the collapse of the GDR, it was easier for eastern Germans to move to west Germany than for Polish to move to a different country in the west. Mostly younger and educated people would have made that move, hampering future generations. With the Reunification also came the whole Treuhand issue which essentially sold off a good chunk of eastern Germany for pennies to western investors, because eastern investors had no capital. That meant the east lost out on the profits from its economy as they would accumulate in the west instead. Even today a large part of east German rentals are owned by western landlords or corporations. Then the industrial base of west Germany was setup far more for competing on the open world market with automotive companies in the NW (VW), SW (Daimler) and SE (BMW) plus the big industrial area Ruhrgebiet. So you naturally got an economic focus even after Reunification on the old BRD with the previous GDR requiring decades to hopefully catch up to the rest of the new country.
And a minority of those kids will get curious about the How and Why. Those are the security nerds of the future securing the networks against both the kids they were themselves and actual malicious actors.
Source: Early interest in wifi security, including in other people's networks, lead me down an education and career in security
In 2019 there was a talk about data mining the DB arrival data [1] (yes, this problem is nothing new). One of the takeaways was that on some connections you can actually buy a "Sparticket" (cheaper, but only valid for a specific train), but get it upgraded to a "Flexticket" (more expensive, can take any train on the route) for free. This works because a delay of more than X minutes removes the specific train requirement and some routes are nearly always delayed by at least that threshold.
Related, there is a stable way to detect whether your .exe is running under Wine and even which version by looking up the address of the export wine_get_version in ntdll [1]. Years ago I actually had to do this to work around some weird path bug when we were testing our Windows build under Wine (easier to setup Wine than a full Windows CI).
SteamVR works ok, but last I checked it still performs worse than on Windows. If you are feeling adventurous, you can try a FOSS VR stack [1]. It works for Steam games running Proton and when it works it provides better performance. I had some troubles with it, sometimes you need to switch versions or you get some artifacts in games, or some games just don't work at all. Good thing is, switching between FOSS and SteamVR is as simple as launching either first before starting the VR game in Steam.
I guess the Linux VR stack might get a bit of love from Valve for the Steam Frame, so things might improve in the near future.
At this point escalating, or threatening to, might be the better option. But I can't help trying to figure out how to solve a people/organizational problem with a technological solution.
Github is still famously IPv4 only. I don't know if there is a split between the SSH (if you use SSH to access the repos) and HTTPS (the tarballs) setup on their end, so maybe you get full speed on IPv6 and limited on IPv4 (or the other way around). Try disabling IPv6 on your end, if the speeds match then this might be it. If IPv6 is fast using an IPv4 gateway that tunnels via an IPv6 VPN might be a workaround.
I also had a similar problem a while back. Some speedtests showed more bandwidth than I could get in regular HTTPS downloads. I could get multiple downloads running at the same time that in total added up to the expected speed. In my case the line was just lossy enough (TCP retransmits in Wireshark) for TCP to never scale up its window size properly beyond a certain limit per connection. I verified this by running iperf in TCP and UDP against a gigabit server, UDP reached near full speed because it didn't care about a few lost packages. Working around that issue might be a bit harder, maybe [1] via [2] can provide some ideas to look into.
As much as I hate where this might lead, I can't help but amuse myself over a scene like this: You walk up to a bearded fellow in a dark alley and quietly whisper "Stallman..." only for him to quietly respond "... was right.". He then opens his coat for you to choose the install USB of your favorite distro (Arch of course!). He dutifully hands you the stick and a printed copy of the GPLv5 for you to hide in your coat as you walk past the telescreens back to your home.
That part also caused my tin foil hat to heat up. At least they get the credit of including it directly instead of adding it in a later revision that gets even less news coverage. It is hard not to grow cynical when you see this.
I am also worried about another detail:
> The states also want to prevent the circumvention of blocking orders by erotic portals ... using so-called mirror domains – i.e., the distribution of identical content under a minimally changed web address. For a page to be treated as a mirror page and quickly blocked without a new procedure, it must essentially have the same content as the already blocked original.
Note the part "quickly blocked without a new procedure" so there is a way to block sites with even less process and oversight. That just invites overblocking without accountability.
I managed to find some statistics on hull losses per million departures [1, p. 13]. Seems like indeed MD-11s have a highish rate of incidents by that metric compared to other types, even if they are not catastrophically less safe than other planes. That metric stacks the statistics a bit against cargo planes, which most (all?) MD-11s are now. These planes tend to fly longer haul instead of short hop, so you get more flight time/miles but less departures. There are also likely some other confounding factors like mostly night operations (visibility and crew fatigue) and the tendency to write off older planes instead of returning them to service after an incident. Plus these aircraft have been in operation long enough that improvements in procedures and training would impact them less than more modern types, as in they already had more accidents before these improvements.
At some point it comes down to probabilities. With so many flights going on, one in a million incidents become a certainty. For example UA232 [1] suffered failure in all 3 redundant hydraulic systems due to an uncontained engine failure. Any of the 3 systems would have been enough to retain control of the aircraft. Of course this lead to some investigations on why all 3 systems could be impacted at the same time and what can be done to limit failures.
Besides the technical aspects that flight is an impressive example of resilience and skill. Bringing that plane down to the ground in nearly one piece was essentially impossible and a one in a million chance in itself.
Given that the report only mentioned a single other seemingly related accident in 1979 I am not sure that objectively this is a reason to discontinue flying these planes. The fact that these planes have been in service since the early 70s is a testament to their safety and reliability in itself. Of course public perception, especially with the videos of huge fireballs from hitting one of the worst possible locations, might put enough pressure on airlines to retire the planes anyway.
I agree on the end of an era. Hearing something else besides just Airbus- or Boeing-something always gives me a bit of joy. Even though MDs and DCs are of course Boeings in a sense now as well.
Grounding all MD-11s and DC-10s is a major move. I guess it makes sense as a big factor was the fatigue cracks on the pylon (lugs), despite the pylon not being behind on inspections. I am wondering what the inspections of pylons in other planes will yield, likely that will determine whether the grounding will continue.
But beyond figuring out why the engine mount failed, I am very interested in what caused the actual crash. "Just" losing thrust in a single engine is usually not enough to cause a crash, the remaining engine(s) have enough margin to get the plane airborne. Of course this was a major structural failure and might have caused additional damage.
EDIT: It seems there was damage to the engine in the tail, even though this was not specified in the preliminary report, likely because it has not been sufficiently confirmed yet.
Had to check, but that is actually beyond what DNS allows. Labels (the part between dots) are limited to 63 characters. We could sneakily drop an s somewhere in there and then it would fit.
I tested this with the at the time newest llama.cpp master on a Linux system with 2 3090 24GB, only one was used for testing. q8 without any KV quant, 256k context, mmproj loaded takes less than 20GB VRAM. This runs at about 1.5k to 2k tok/s pp and 40-50 tok/s gen (slightly lowered power limits & undervolted). q8 with 64k non-quant context and mmproj takes just under 16GB VRAM. Drop down to the q6k model, no mmproj, 64k non-quant context and it fits in 12GB VRAM. All the way down to q4km and some batch size tweaking and it barely fits into 8GB VRAM.
64k context is the minimum for Hermes agent, so a vision capable "agentic" model fits into a 16GB card. This is very impressive. I am currently testing how smart the model is and it does decently so far, had one looping issue it recovered after a lot of tokens, did some basic tool calling.