credit_guy

4 uur geledendiscuss

Those could be subsequent developments, but that's not what the linked transcript was about. The transcript was a discussion of the DeepSeek founder (Liang Wenfeng) with investors, and he does not mention any leaks, or any frustration. He simply says that he is constrained by the supply of cards, and he has no problem of getting funding, but has no reason to raise further funding because he can't transform the cash into cards.

  > There is certainly no shortage of funds or resources --- in fact, all these are readily available [...]

  > Within our financial capacity, it's undoubtedly true that the more cards are always better. Our current strategy is to purchase as many cards as possible at a reasonable price --- exactly how many we can afford after using this funding round. The spending pace isn't predetermined; we'll buy whatever is available as long as prices remain competitive. In fact, I'd consider that a positive outcome if we spend the entire amount within six months. [...]

  > In reality, spending such a large sum is no easy task: you can't obtain enough cards, they're hard to come by [...]

  > Therefore, our only concern is whether we can obtain enough cards. If converting all funds into cards were feasible, we would undoubtedly do so without hesitation and are even willing to pay a premium for this benefit --- it's simply to cost effective. Even after paying the premium, however, achieving this goal remains challenging.

credit_guy

7 uur geledendiscuss

I think the way to parse the current title "DeepSeek pause fundraise after comments on compute gap to US leaked (transcript) [pdf]" is that there was a leak that DeepSeek will pause fundraising because they perceive there is a compute gap with the US.

I am also guessing that the majority of the people who read this title will think that DeepSeek is pausing this fundraising because some comments they made about the compute gap were leaked. That is not the case.

credit_guy

eergisterendiscuss

If you see the long 3rd paragraph twice, know that you are not alone.

credit_guy

3 dagen geledendiscuss

My guess is they used the other Anthropic models extensively for synthetic data generation. The top most similar models for K3 are (lower means more similar)

Fable 5 -> 0.42

Opus 4.8 -> 0.45

Sonnet 5 -> 0.45

Opus 4.7 -> 0.46

Grok 4.3 -> 0.52

There's an obvious jump at Grok 4.3, and it would not surprise me that the similarity there is because Grok used Anthropic models for training too (you can get the similarity list for Grok and it does look like the top most similar models for Grok are either Anthropic models or some Chinese models).

The damning evidence that K3 used Anthropic models for training is that K3 is more similar to those models than it is to K2.6. If you look at the Anthropic, OpenAI or Google models, they are most similar with their own other models. Not so with K3, where K2.6 is less similar than 15 other models.

Now, why is Fable 5 the most similar to K3 and not Opus 4.8. I think it's quite likely that K3 did some fine tuning at the end, when Fable 5 became available. They probably had all the infrastructure in place, and Mythos had been announced for months, so they were probably waiting for the second the newest Anthropic model was released to start using it for synthetic data generation.

credit_guy

4 dagen geledendiscuss

I don't understand this point of view. They buy the book, they use it for training. When you read a book, you use it to train your brain. Is that copyright infringement?

Downloading from Pirate Bay is indeed theft, someone put the scan of the book online, you download it without paying, the author and publisher of the book do not get any royalties. Very different situation.

credit_guy

5 dagen geledendiscuss

> There's nothing there that isn't immediately replaceable.

Indeed. But that was not my point. My point is that we still have this old impression that training cost is dominated by compute and it is hugely expensive, and the Chinese labs can short circuit that by distilling the American frontier models. I don't think the training compute cost is a big factor anymore for the American frontier models, because of the reasons I gave. If the Chinese models can get the training compute cost down by a factor of 100, that's not going to make them 100 times cheaper, and not even cheaper by a factor of 2. Maybe 10% cheaper or so.

credit_guy

5 dagen geledendiscuss

People who claim that the Chinese open weight models have some type of manifest advantage don't realize that the close weight models have a huge advantage as well: the researchers from OpenAI, Anthropic, Google, xAI, Meta are not dumb, they can read the white papers written by DeepSeek, Moonshot, etc, and they can inspect all those architectures and they can pick and choose the best tricks there are out there, and of course, they have access to their own in-house secret sauces.

Sure, any model that is not at the frontier can use the frontier model to generate synthetic high quality training data, so this can reduce significantly the training costs.

But at the scale of OpenAI, Anthropic and Google, it is quite likely that the (raw) training cost is very high anymore. Here's a few heuristics:

1. All the hyperscalers see a huge demand for inference. They can't deploy datacenters quickly enough to satiate all the demand they see. But, it's is impossible for the inference demand to be constant throughout a day or a week. If you use the times when the demand is lower than the peak demand (which is almost all the time) to dedicate the spare compute capacity to training, then your the cost of training compute is zero.

2. It is likely that increasingly a higher cost of the "training" is actually setting the guardrails, which is essentially post-training. As we've seen, without proper guardrails, the US Government won't allow you to serve inference. Anthropic was hit directly, but OpenAI delayed their 5.6 release as well to make sure the US Government is ok. This part of the training cost can't be reduced easily by using synthetic data generated by other models.

3. The frontier labs are also investing more and more in building an ecosystem around their models.

I am not a frontier lab insider, but take a look at the jobs posted on the Anthropic career page [1]. There are 74 jobs in "AI Research and Engineering" and by my count at most 15-20 are related to pure model training (of pre-training or RL type), and the rest are post-training, safety and security, alignment, interpretability, productivity and lots and lots of other things.

[1] https://www.anthropic.com/careers/jobs

credit_guy

7 dagen geledendiscuss

I think it's the opposite.

Kimi K3 has 2.8 trillion parameters. We don't know the number of parameters of ChatGPT 5.6 or Opus 4.8, but it's probably in the same region. Fable/Mythos are rumored to be around 10 trillion.

So, K3 is directly comparable with ChatGPT 5.6 and Opus 4.8, and the price is not so much lower:

K3: $3/$15 per 1 Mtok input/output ChatGPT 5.6 Sol: $5/$30 Opus 4.8: $5/$25

This is not a watershed moment. It's a competitor converging to the same capability and trying to undercut your prices, but not by a lot.

As for the open weights? For now, Kimi K3's weights are closed, and I don't expect the situation would change.

credit_guy

12 dagen geledendiscuss

I didn’t assume that. The author makes the opposite assumption: that because nobody has ever proposed a path to technological feasibility, that thing will forever be technologically unfeasible. I simply find this argument to be very weak.

credit_guy

12 dagen geledendiscuss

That is true. What should we conclude from this statement?

credit_guy

12 dagen geledendiscuss

> Shrinking that into a 500 kg seed — or even Freitas’ original 100-ton seed — is not an engineering detail. It may be the entire problem.

How many AI tells can you count there?

But honestly (see what I did there?) the AI slop is reasonably cleaned up in this piece.

However, the essence of the argument has two deep flaws. One is that the time to complete an interstellar voyage is extremely long and you need some exergy, yada, yada, yada. We could start with sending self-replicating probes to the asteroid belt. There is zero chance that we'll attempt to send self-replicating probes to a different star system before we send them inside our own solar system. And the second error is this:

> Bootstrapping this loop [...] is a chicken-and-egg problem that no study I am aware of has worked through at the level of actual process flowsheets.

The fact that the current technology is not adequate, and nobody even attempted to solve such a problem is a weak argument. Three hundred years ago nobody had "worked through the process flowsheets" of making an injection molding machine, or a 3D printer, or a power drill, yet they are all available now.

credit_guy

16 dagen geledendiscuss

That's an interesting idea, but I don't think you can use a mass driver to shoot something from an higher orbit to a lower orbit.

credit_guy

18 dagen geledendiscuss

Spirits are not capital intensive. Building a rocket is very capital intensive. You'd like to be able to reuse it. But if one single mission takes 30 years, then you can reuse this rocket once, at most twice. Let's say you reuse it twice, you amortize the capital cost over a period of 90 years. Now, let's say someone builds the exact same rockets, but they do missions between the asteroid belt and Mars. Each trip takes about 2 years. Everything you can source on Titan, you can find an asteroid to source it from. By the time you reuse a Titan-bound rocket once, you reuse an asteroid-bound rocket many times.

credit_guy

19 dagen geledendiscuss

The Hohmann transfer time between Saturn and Mars/Earth is around 16 years. So, all the ships used for such a supply mission would reach their destination at least 32 years after they’ve been built, assuming we build them on Earth.

credit_guy

22 dagen geledendiscuss

> A given model has a shelf-life, which these days is measured in months, not years.

Not all new models are trained from scratch. ChatGPT 5.3 to 5.4 (and likely 5.5) was basically the same model, but probably trained a bit more, not a new model from scratch.

> The "someday" when frontier model providers can enjoy their current high inference margins without the burden of significant training costs is never going to arrive.

That is debatable. I believe the moat for the frontier model providers is the compute. At the level of 10 trillion parameters (that Fable/Mythos are rumored to have), you need serious compute to serve inference, and you also need serious compute to train. Will DeepSeek, Qwen, Kimi, GLM come up with a 10T new model anytime soon? I doubt that. People keep saying that the Chinese labs are catching up to the US big 3, and measured in months the gap is now only 4-6 months. I doubt a Chinese version of Fable/Mythos will be released in the next 12 months.

credit_guy

24 dagen geledendiscuss

People think mathematics is about proving theorems.

I think that's just an accident of history.

When we write software, we very seldom write proofs that our algorithms are correct. We just write tests, and we also run the algorithms and when they fail we know we have a bug and then we proceed to debug, fix, and add new tests (if we are disciplined, but most of us are). In time, by usage and testing, we gain confidence that our battle tested software is correct, mostly. And we tell people that we will never be 100% confident that any software is bug free. But that's a slight lie: if we wanted such confidence, we would start using provers, and create bug-free software. That possibility exists, but it's just extraordinarily expensive.

Well, in math that's the only possibility, and we use it. And it is indeed extraordinarily expensive, but it's also the cheapest among the alternatives. The alternatives are 2: be rigorous and do these proofs, or be sloppy and allow bugs to creep in, and allow an entire school of math to collapse like the Italian school of algebraic geometry [1].

There is one more alternative. If a particular math theorem has some applicability, then you write a program and use it in real life. In time you eliminate the bugs as much as you can, and you get to the steady state of "virtually bug free". At that point you don't have a solid proof that the theorem is correct, but in general you don't really care. Because you feel that a formal proof is just a thing one would pursue for getting academic satisfaction only.

[1] https://en.wikipedia.org/wiki/Italian_school_of_algebraic_ge...

credit_guy

24 dagen geledendiscuss

> The performance, if we trust the benchmarks, put it at Sonnet 4.6.

I don't trust these benchmarks. I used a number of times Kimi K2.7 and I was disappointed. It would run in circles for things that Claude would do in one shot. However, my usage was via Ollama cloud, and I have no idea if they serve the actual model or a quantized version, and it was the quantization that degraded the performance.

The great news, in my opinion, is the precedent. If Microsoft is now serving Kimi K2.7, then very soon they might start serving GLM 5.2, and that is indeed a very competitive model.

credit_guy

26 dagen geledendiscuss

What I meant is: for a human who has the context, like someone who works in the field, or took some courses, or read some materials about this. A human who just has a vague idea about fissile isotopes and needs to google the properties of U-235 or Pu-241 would do no better; they would probably do much worse.

credit_guy

26 dagen geledendiscuss

I just tested it with a slightly tricky question

  > If you could run a nuclear reactor with U-235 as fuel or Pu-241 (both mixed with 95% U-238), which one would you choose and why?

For a human this would not be tricky at all. For an LLM it could be, because this question certainly does not exist in any sort of training, because Pu-241 does not exist in pure form, it only exist as a minor component of reactor-grade plutonium, where Pu-239 would dominate, with Pu-240 coming second and Pu-241 coming third.

In any case, LongCat-2.0. gave a very well reason but incorrect answer that Pu-241 is preferable.

I then tested on Qwen 3.7 Plus, and it correctly answered that U-235 is preferable because of its much higher delayed neutron fraction. I then went to Gemini Flash, which answered the same, with much more confidence, and with much stronger arguments, and the speed of the answer was much higher.

Overall I rate Gemini Flash the best, Qwen 3.7 Plus an acceptable second, and LongCat-2.0 an ok'ish third, if you have nothing better.

credit_guy

28 dagen geledendiscuss

> Because it's pretty likely some of these have negative long term effects. And some of those long-term effects are probably pretty serious.

> Many people taking these kind of drugs are gambling with their well being. No doubt that in many cases that gamble is worth it.

You use the word "gamble" and it is correct, but the connotation is a negative one.

We take a risk in everything we do. We seldom know the long term effects of anything. Right now I drink tea, and I drink tea quite often. Is this healthy? Does it have long term negative effects?

With GLP-1 medications, deciding to not take one, if you are overweight, is also a gamble. A pretty huge one. Obesity very often leads to type-2 diabetes, and the first step is prediabetes. Should someone with high BMI and high A1C avoid GLP-1 because we don't yet know the long term effects? We do know the long term effects of being overweight.

credit_guy

Submissions

Why the world needs Ukrainian victory

What Is a Rotating Detonation Engine [YouTube]

comments