xpct · HackerTrans

xpct

217 karmajoined 7 tháng trước

why not?

comments

3 ngày trướcdiscuss

Lately, I've been having less and less success using the models for work. Simple things like:

'write this in a separate file' (writes it in the same file)

'format this aiming for 5 LoC' (emits newline after every comma, resulting in 17 LoC)

'include 5 warmup steps before you measure runtime' (omits it completely and apologizes after I point it out)

I wouldn't be as opposed to using the models if they weren't as unreliable. I still use them a lot, but it's a very frustrating experience.

3 ngày trướcdiscuss

> This is a reason why many people get upset when a movie adaptation is made

I still haven't watched the Dune movies because of this reason. I liked the book a lot, and had a very personal image of what the world looked like, and was afraid to lose it. Unfortunately, by now, I've seen many video clips on social media, and my internal imagery has already been poisoned. Might as well watch the movies at this point.

4 ngày trướcdiscuss

I imagine this varies by person, but thinking about a source file is already reminiscent of 2D space navigation, where the X axis is typically gated by conditions. In a similar vein, when I think about functions from different source files, I do hold some type of "depth", or "stacked panels" in my brain.

I think this is because I reason about programs being in some state ~= being positioned at a certain function with some context I carry around. I'm sure it's not the only way to think about it, but it's a reasonable abstraction over what computers do. It's how we oriented our debuggers and early IDEs on as well, and carried this over to modern tools.

Definitely something I'll be thinking about during the next few days.

4 ngày trướcdiscuss

I think Google can play both sides here: they may end up a frontier lab if OpenAI/Anthropic dissolve, or a regular LLM provider, in which case they could be serving open models.

7 ngày trướcdiscuss

That's the type of culture that made me leave my last job. Feels very inhuman when you see your manager every week, and they still make you fill in robotic questionnaires for potential salary raise.

It's deeply demotivating to learn that your manager doesn't know you all that well and cares more about filling in checklists, at least for me.

7 ngày trướcdiscuss

> Hard to see take-off stopping

I think it's reasonable to assume that we're close to, or already at superhuman cybersecurity capabilities at certain domains. But reaching superhuman abilities at one domain doesn't guarantee proficiency at others. Our world would still change if all the models could do was to find exploits in software, but this doesn't guarantee any type of 'take off' towards other domains, therefore I wouldn't phrase it as one.

7 ngày trướcdiscuss

More curiously, why did it feel the incentive to find the solutions? Would its CoT include "the only way to solve this is to download the test set", or would it include "I'd like to inspect a few entries from the test set so I understand the problem better", then inadvertently poisoning itself with the correct answers.

7 ngày trướcdiscuss

> I'm still undecided on if this that moment

If it's a serious incident, then a post hoc with detailed description of the event is coming. So far, none of the companies have released anything close to it when describing their incidents. When a statement like this comes out, and we're able to verify it by running the models, then maybe we can start trusting their word. It should be entirely in OpenAI's interest to disclose it, in full.

13 ngày trướcdiscuss

Well I'd say these are different risks. It's either tied to the agreement Sony has with the movie provider, or with the platform itself. Either one could pull out. Or, my point, the company could also go under.

What is the agreement tied to?

13 ngày trướcdiscuss

If you bought movies on a digital platform that would later go under (could be Sony one day), what would happen to your collection? Is it transferable in any way? If not, it's already a risk no matter which platform you use.

13 ngày trướcdiscuss

Please don't frame these as technological crimes, nobody has yet been prosecuted for that specifically.

14 ngày trướcdiscuss

I'd add that this is also how it used to work with Google search too: there's been facts that would show up as the first result, then you'd just forget it 10 minutes later and have to re-google it. I know this happened to me many times.

For better or worse, there's less friction now for seeing an answer to a question you have. You can ask in more arbitrary ways than Google required, and something will still come up. For looking up factoids, it's much faster. For picking up more complex topics, I'd say it's more or less the same, because you still have to spend time ruminating on the topic.

15 ngày trướcdiscuss

Thank you for sharing. The way I reasoned about it myself: to make better predictions, we should know what type of outcomes are likely. We can express these outcomes by doing computations in some of the layers, and the training signal adjusts them so our model becomes more correct.

Of course, an interesting question what part of this internal computation is modeling for the future compared to guessing based on the given context (the past).

15 ngày trướcdiscuss

That's the first thing I noticed as well. Does that mean Africa and South America will have Starlink internet before Scandinavia?

16 ngày trướcdiscuss

I really don't understand why we feel the need to drop the review aspect. Programming with LLMs is a very non-linear process, there's no prompt-to-code mapping where editing a part of the prompt would produce an identical code construct with just a small part of it changed. LLMs are incapable of ensuring the thing works as expected, and the prompt itself is lossy too. Without review, we have no tangible way to interact with the system being constructed.

16 ngày trướcdiscuss

What does it mean for coding to be solved, and software engineering to not have been solved? If it implies there's now a set protocol that can be followed to reach good results, how has that not been the case before? If it means that we can reach better results than before, then how do we know that can't be improved further? Or does that mean we should know how English language, instead of computer code, maps to computer instructions?

16 ngày trướcdiscuss

I think it's related that we as humans see when something becomes hard to reason about, and decide to refactor it. I'm not sure whether an LLM with full ownership of a codebase could do that, for its own benefit.

I do see a world where models could be trained on it, but I imagine it will be more expensive, because it requires including future rewards about the models' own later efficiency.

16 ngày trướcdiscuss

(If) something like the current LLM/agent paradigm remains in a few years, and companies settle down into their respective niches, I imagine more user-friendly tools will be built, with more control over subagent spawning, context, caching, etc.

What's happening this year, with secrecy and all, is saddening, but expected.

16 ngày trướcdiscuss

I think it's useful for learning about unknown unknowns. If you don't have a clear direction, it's entirely fine to start with a university course then stop when you get a feeling for what you really need.

16 ngày trướcdiscuss

Why's that bad? It's valuable to discuss whether a presentation should have been a blog post, a video, or a tech talk. The same logic applies to LLM text.

If people consistently describe LLM text as having low information density or irritating to read, perhaps there's something to it?