fleventynine

hierdiscuss

This machine needs better networking options to be useful, such as built-in 100Gbit QSFP28. There is not enough space between the x4 PCIe slot and the power supply for a NIC to fit, and even if there was, you'd have to cut a hole in the back of the case to get at it.

fleventynine

il y a 5 joursdiscuss

It's pretty clear to me that coding agents are a significant cause of the current glut of comp-sci graduates unable to find their first job...

fleventynine

il y a 5 joursdiscuss

I think there's room for win-win solutions here; the cost of placating the locals is a rounding error compared to the cost of these data centers.

1. Local governments can require that if the data center peak load is 1.0GW, they must provision transmission and generation capacity for the local region of 1.2GW; the extra power can be given to the neighbors at substantial discount.

2. Construct a public swimming complex nearby, heated with waste data center heat.

3. Local governments can enforce strict limits on environmental noise at the property boundary. I've personally stood right next to a 150MW-class datacenter and experienced sound levels less than traffic from a typical suburban 4-lane street; making the datacenter quieter than any other industrial use is a choice the builder can make if they have the right incentives.

Setup directional microphones around the perimeter, and fine them $25,000 per minute per dB above the agreed threshold. Distribute the fines to the neighbors. Failure to pay promptly means the sheriff shuts down the power.

All of these options are far cheaper than launching the data center into space...

fleventynine

il y a 5 joursdiscuss

I like Rust and use it full-time professionally. Unsafe is not the same as unsound. Unsound means that the unsafe code is not maintaining the aliasing invariants on references required by the language, and thus undefined behavior can leak into safe code (that is, the safe code can be miscompiled).

Known unsound code should not be merged, let alone released to production. If you have good enough tests and run them under MIRI or ASAN, maybe you can get away with it for a time, but most Rust experts would not sign-off on such a project.

If somebody put a gun to my head and told me to make the best of such a codebase, I would try to figure out how to turn off the LLVM optimizations that assume the Rust references don't alias. With these optimizations this codebase is scarier than most C or C++ code.

fleventynine

il y a 6 joursdiscuss

I don't think this does enough to address to supply-chain security concerns. Before I use a crate, I want some assurance that the transitive deps have had a proper security audit, and that I'm not a single "cargo update" away from getting malware installed in my machine.

fleventynine

il y a 12 joursdiscuss

I stopped using Facebook back in 2021 when the majority of my feed was reshared political content with 20+ comments from my friends fighting about divisive social/political issues. It wasn't fun, and it wasn't fostering community, so I left. A few years later I logged in again to see that most of my Facebook friends had also stopped engaging.

fleventynine

il y a 13 joursdiscuss

Is the RTL for the FPGA available to be tinkered with?

fleventynine

il y a 16 joursdiscuss

> Every corporate network to which I've connected worked just fine without it.

Just because it appears to be working fine doesn't mean you are in control of it. Without hardware attestation, how do you know the machines are running the software you think they are?

fleventynine

il y a 2 moisdiscuss

If regular people can repurpose old hardware, so can shared providers, who can extract more value from the hardware and thus afford to pay more.

In a constrained market, supply and demand favors folks who can most efficiently extract rent. Local models only make sense in a world with abundant compute and energy.

fleventynine

il y a 2 moisdiscuss

My point is that dram demand is mostly orthogonal to whether everyone is using open weight models or secret weight models. Heavy demand for local models (whether secret or open weight) will require even more aggregate DRAM than for shared.

Demand will only go down if people reduce their use of these AI tools. Given how much folks here complain about quotas, I'm very skeptical that will happen willingly.

fleventynine

il y a 2 moisdiscuss

If local models are good enough, doesn't that increase demand for DRAM as everyone buys DRAM for their poorly utilized local machines?

Surely it is a more efficient use of DRAM to run inference on shared hardware with large batch sizes and more utilization.

fleventynine

il y a 2 moisdiscuss

I wonder how good LLM agents are at reverse engineering FPGA bitstreams...

I want a robust open-source ecosystem where anyone can take my hardware projects and modify them without needing to deal with licensing friction.

fleventynine

il y a 2 moisdiscuss

There isn't enough hardware in the world for everyone to run their own SoTA model. The only hope we have is if we work together to host these on shared infrastructure, benefiting from >50x economies scale due to batching, etc. That infrastructure doesn't have to be owned by greedy corporations.

fleventynine

il y a 2 moisdiscuss

That DRAM would get even more use if it was removed from these machines and placed into a shared pool :) I joke, but thanks to the brutal DRAM market there has been some movement in this direction lately...

fleventynine

il y a 2 moisdiscuss

My point is that it is WAY more efficient if we put the world's DRAM supply into a shared inference pool instead of stranding it in local machines where it won't have as high of batch size or utilization.

The cost of not being efficient is even higher DRAM costs than we have now, given supply and demand.

fleventynine

il y a 2 moisdiscuss

I think folks in this thread are underestimating how expensive it is to serve a SoTA model at 100 tokens a second. In addition to the $500k in capital costs, you also have significant electricity costs.

This stuff is expensive because supply is much lower than demand. If everyone was to run their own hardware with a batch size of 1, we'd have 100x more demand for inference hardware and electricity than we do now, and people would be even more frustrated. Efficiency is everything, and we need all the economies of scale we can get to meet demand.

fleventynine

il y a 3 moisdiscuss

Many small crates published by large, trustworthy projects are fine and preferable to one large crate that "does everything".

fleventynine

il y a 3 moisdiscuss

Good luck finding reasonably priced switches and low power PD ICs that support type 3 or type 4 PoE.

Also, supporting those tiny pulses requires large capacitors to hold a charge in between pulses. That plus the required magnetics make PoE sensors way more bulky and expensive to manufacture than old fashioned RS-485 sensors.

fleventynine

il y a 3 moisdiscuss

PoE is lousy for sensors. The switch will cut the power if you draw less than 10mA (480 mW), so regardless of PHY efficiency (which is terrible compared to most RS-485, CAN, or even radio ICs), you are REQUIRED by the spec to generate heat that will mess up your sensor measurements.

fleventynine

il y a 4 moisdiscuss

Even more interesting, the developer manual:

https://discreetfx.com/documents/NewTekVideoToasterDeveloper...

I remember using this thing when I was a kid, trying to figure out how all the switching effects worked, so stumbling on this manual many years later was really satisfying...