Just because something is written in Rust, doesn't mean it's good Rust.
I have noticed, in general, LLMs tend to "fix" problems by shoving them under the rug (like adding a cast) or writing a super-local "fix" instead of taking the time to understand the deeper problem or structure.
In Rust, when you get stuck in a complicated borrow-checking problem, Rust people will tell you it's a Good Thing(tm) because it forces you to think about the higher level architecture of your code. An LLM, on the other hand, might bash its head a couple of times trying to "fix" the problem, and then just throw in a Refcell (or other workaround), see that it compiles, and call it a day.
Refcells "move borrow checking to runtime", meaning that the code will compile, and will crash at runtime if the object is tried to be accessed from two different places at the same time. In most "normal" programming languages it's not an issue -- but it's a crash in Rust.
Now, maybe the models have gotten better, and maybe you can get around this problem by using a good system prompt/"tools" and a good testing methodology. What I am saying however is that you shouldn't automatically take "rewritten to Rust by AI" at face value of it being good Rust code, or a testimonial of "Rust and AI being a good match". (Go is better I think)
If we go along with the though that our own subjective experience is the only thing we truly know, and that we cannot really know if any other humans are having the same experience (and any belief of that sorts is purely extrapolation), then there isn't a fundamental difference between LLMs and "other humans" in terms of whether or not they're "conscious". Sure, it appears more likely that "other humans" are real conscious beings, but there's no fundamental difference.
>whether these systems do or could in the future exhibit something similar?
I think the whole discussion is based on the idea that consciousness isn't something you can "exhibit". (Tell me, how can you "exhibit consciousness"?)
I'm starting to agree with you; I found the plan/develop/review workflow to work quite well, but I'm not at the point of not looking at the code at all yet.
I guess you actually review and actively participate in making the plan, you just don't review the code afterwards?
Could you share some more details on the specifics of your workflow? (What models/harnesses? do you use the same or different context windows? How exactly do you run the review, and how do you pass along and act upon the information from the review?) Also, how big are the changes you usually implement with one plan/develop/review cycle?
There's a difference between addictiveness and enjoyment, and definitely between addictiveness and satisfaction.
While the thing that gives you quick dopamine might win in the very short term, you can still step back and recognize when it's not satisfying in the long term and you're not even enjoying it that much.
And people aren't stupid. Junk food exists, yet lots of people choose to eat more wholesome food as the majority of their diet.
The problem with instagram or youtube is that you can't separate the good from the bad.
It's like if every time you went to store Y to buy milk, you would be exposed to highly manipulative marketing trying to get you to buy junk food. You would probably want to go to a different store instead.
What I'm suggesting is the possibilities of different stores, with different philosophies and standards, so that people can choose where they go. Corner stores (where almost everything is junk food) exist, yet people still choose to go to real supermarkets.
>Isn't anyone strong enough to choose a less addictive algorithm necessarily strong enough to not scroll Instagram for hours in the first place?
Absolutely not. It's much easier to make a one-time switch than to be continuously resisting temptation. Changing the things in your environment is an important tool to break bad habits. The book "Atomic Habits" talks about this at length.
The solution to this would be a law forcing these sites to allow third-party suggestion algorithms, so that you can choose who and how content is being suggested to you.
It could be perhaps as simple as allowing third-party websites and apps for watching Youtube on your phone. And it's okay if this would be a premium paid feature, so there's no counter argument that "it costs them money to host videos".
This is not an entirely new idea either. Before Spotify became popular, people would integrate Last.FM into their media players to get music recommendation based on their listening history, and you could listen to music via YouTube directly on the last.fm website.
As the author said: "Our sense that something is weird is often accurate, but our stories about precisely what the weirdness represents are often way, way off." It might have been better to start off with the general impression you had.