bwest87

vor 2 Monatendiscuss

He forgot the tokens!

It's not simple weights and numbers all the way down. The available output is pre-set by the tokens we allow it to predict.

There was a whole bit in there about not having a language module or using words. But it does. We tell it.

Humans do not come pre programmed with a set of possible "tokens". We just figure it out and I believe that fact captures something very essential. Maybe the missing piece of AGI. The fact that humans can just be awash in pure sense data, and somehow just figure out what is important and what to do. Never ceases to amaze me.

bwest87

vor 7 Monatendiscuss

Sure there is some minimal marginal cost, but it's so close to zero that it's usually negligible, and the incentive is to basically give it away and "monetize" something else. Your point about games actually just makes my original point. Software is already usually free or dirt cheap, which is why reducing the cost to make the software can't create some "low cost / low value" quadrant. Unless your talking about bespoke software that has such a small market size it isn't worth making today. I could maybe see that area opening up, but even that software would not fit the OP's description of software that "has no owner and is not meant to be maintained"

bwest87

vor 7 Monatendiscuss

But focusing on production cost is silly. The cost to consumers is what matters. Software is already free or dirt cheap because it can be served at zero marginal cost. There was only a market for cheap industrial clothes because tailor made clothes were expensive. This is not the case in software and that's why this whole industrialization analogy falls apart upon inspection

bwest87

vor 7 Monatendiscuss

One thing that has become clearer to me over the years is that reasoning by analogy (like this article does) sounds a lot smarter than it is. If you look from first principles, it's clear that physical goods and software don't share the same properties and thus the analogy falls apart.

Physical goods like clothes or cars have variable costs. The marginal unit always costs > 0, and thus the price to the consumer is always greater than zero. Industrialization lowered this variable cost, while simultaneously increasing production capacity, and thus enabled a new segment of "low cost, high volume" products, but it does not eliminate the variable cost. This variable cost (eg. the cost of a hand made suit) is the "umbrella" under which a low cost variant (factory made clothes) has space to enter the market.

Digital goods have zero marginal cost. Many digital goods do not cost anything at all to the consumer! Or they are as cheap as possible to actively maximize users because their costs are effectively fixed. What is the "low value / low cost" version of Google? or Netflix for that matter? This is non-sensical because there's no space for a low cost entrant to play in when the price is already free.

In digital goods, consumers tend to choose on quality because price is just not that relevant of a dimension. You see this in the market structure of digital goods. They tend to be winner (or few) take all because the best good can serve everyone. That is a direct result of zero marginal cost.

Even if you accept the premise that AI will make software "industrialized" and thus cheaper to produce, it doesn't change the fact that most software is already free or dirt cheap.

The version of this that might make sense is software that is too expensive to make at all because the market size (eg. number of consumers * price they would pay) is less than the cost of the software developer / entrpreneurs time. But by definition those are small markets, and not anything like the huge markets that were enabled by physical good industrialization.

bwest87

letztes Jahrdiscuss

This video was fascinating. I didn't know about "open endedness" as a concept but now that I see it, of course it's an approach.

One thought... in the video, Ken makes the observation that it takes way more complexity and steps to find a given shape with SGD vs. open-endedness. Which is certainly fascinating. However...

Intuitively, this feels like a similar dynamic is at play with the "birthday paradox". That's where if you take a room of just 23 people, there is a greater than 50% chance that two of them have the same birthday. This is very surprising to most people. It seems like you should need way more people (365 in fact!). The paradox is resolved when you realize that your intuition is asking how many people it takes to have your birthday. But the situation with a room of 23 people is implicitly asking for just one connection among any two people. Thus you don't have 23 chances, you have 23 ^ 2 = 529 chances.

I think the same thing is at work here. With the open-ended approach, humans can find any pattern at any generation. With the SGD approach, you can only look for one pattern. So it's just not an apples to apples comparison and sort of misleading / unfair to say that open-endedness is way more "efficient", because you aren't asking it to do the same task.

Said another way, I think with the open-endedness, it seems like you are looking for thousands (or even millions) of shapes simultaneously. With SGD, you're kinda flipping that around, and looking for exactly 1 shape, but giving it thousands of generations to achieve it.

bwest87

letztes Jahrdiscuss

I did a chat with Gemini about the paper, and tldr is... * They introduce a loop at the beginning between Q, K, and V vectors (theoretically representing "question", "clues" and "hypothesis" of thinking) * This loop contains a non linearity (ReLU) * The loop is used to "pre select" relevant info * They then feed that into a light weight attention mechanism.

They claim OOM faster learning, and robustness acro domains. There's enough detail to probably do your own PuTorch implementation, though they haven't released code. The paper has been accepted into AMLDS2025. So peer reviewed.

At first blush, this sounds really exciting and if results hold up and are replicated, it could be huge.

bwest87

vor 4 Jahrendiscuss

Appropriately prioritizing problems has never really been society's strength...

bwest87

vor 4 Jahrendiscuss

I think we're not factoring in that people will react. We're already all starting to realize that the free for all is getting quite hard to navigate. My hunch is that within 10 years, we will start to see an "information immune system" develop. This could take many forms. For example, self regulatory organizations for news, or actual regulations. Like we have with food products, the use of certain words is regulated. Or it could be trusted information filters becoming the norm, the way we trust our browsers to warn us of insecure websites. Or simply some changing cultural norms, like we saw happen with cigarettes. Like it's totally fine today for news outlets to just use Twitter as a source. And maybe the bar will get higher over time. I'm spit balling about solutions, but I don't think society can or will tolerate some dystopian world where truly no one knows what's real for very long.

bwest87

vor 4 Jahrendiscuss

Ok, echoing my top level comment... An alternative framing that I've come to find more helpful is to take your life expectancy, and cut it by 2/3. For example, if you're 20 years old and your life expectancy is 80 (ie. 60 more years), pretend that you only have 20 more, so you'll only live until you're 40. It's nice cause it naturally adjusts as you get older. You'll have smaller windows to work with.

This approach strikes a nice balance. It gives you enough time to be able to really do something and change directions if you want. But not so much time that you can really waste any. It forces you to ask the hard questions about whether your day to day is truly connecting with your dreams, and whether you're on a path to get there.

Of course, Seneca didn't have life expectancy tables to work with. But I think he would have approved. :)

bwest87

vor 4 Jahrendiscuss

> You should organize each day as if it were your last, so that you neither need to long for nor fear the next day.

I've come to find this "live each day like it's your last" advice to be pretty unhelpful. My favorite quote about it is, "all that goes to show you is some people would spend their last day giving you stupid advice".

The problem is that if it actually was your last day, most people would give the finger to all of their responsibilities and go party, eat cake, see friends, familiy, lovers, etc. Which is simply not an actual way to live your life. It's a way to exit your life.

An alternative framing that I've come to find more helpful is to take your life expectancy, and cut it by 2/3. Now what do you do? For example, if you're 20 years old and your life expectancy is 80 (ie. 60 more years), pretend that you only have 20 more, so you'll only live until you're 40. It's nice cause it naturally adjusts as you get older. You'll have smaller windows to work with.

This approach strikes a nice balance. It gives you enough time to be able to really do something and change directions if you want. But not so much time that you can really waste any. It forces you to ask the hard questions about whether your day to day is truly connecting with your dreams, and whether you're on a path to get there.

Of course, Seneca didn't have life expectancy tables to work with. But I think he would have approved. :)

bwest87

vor 5 Jahrendiscuss

Guided aspects... he sent over a questionairre ahead of time with a lot of broad questions. We then did a one hour zoom call going over the questions and getting to know him. It's all designed to help you figure out what you want the session to be about for you personally at that moment in time in your life. And then the session itself lasts 4-6 hours, and he will ask you many questions, but also will follow the journey wherever it takes you, and there's ups and downs and everything in between. It's all very specific to you and the guide and where you want to go with it. And lastly there's an "integration session" the following week where you talk with him for an hour to go over how it went, and what it means. Can discuss more if you want. Email me at bwest87 at gmail.com if you'd like to discuss further.

bwest87

vor 5 Jahrendiscuss

One session meaning one time. Both of us think it would be valuable to do, but on the timescale of like... once/year or once every few years. But there are people who do it once/month for several months if they have a lot of specific things to work through. Our guide actually works with a number of 'regular' therapists, and they pass clients on to him if they think a guided session is the right move. He says therapists will sometimes say, "please take this person on a journey once / month for the next 3 months" (or something along those lines)

bwest87

vor 5 Jahrendiscuss

I'm located in San Francisco. I asked around a bunch of friends, got intros, and talked to a few potential guides. Eventually got linked up with someone who's been doing it for a number of years, and we vibed. We had a few phone calls through Signal, and then decided on a date/time/place.

bwest87

vor 5 Jahrendiscuss

This actually took me a bit of time. I had to search around. But can give advice. If you're serious, email me at bwest87 at gmail.com

bwest87

vor 5 Jahrendiscuss

A friend and I did a "guided" psychedelic session earlier this year. We did it individually over one weekend. It was great. She did it for more therapeutic reasons. I did it more for philosophy/spiritual reasons. But the two main things I took away are 1.) Guided sessions are qualitatively different than recreational sessions, and 2.) It is such a crying shame that this isn't an accepted "tool in the toolbox" for therapists.

It's not about having crazy life altering, world-bending experiences (though that can happen). It's just about helping you get into a state of mind that allows for an effective therapy session. Sort of like... would you want to do your therapy session in a crowded bar, next to your mom? No, probably not. We all recognize that such a setting would not be conducive to good therapy. So similarly, we should be able to recognize that having the right setting, both mentally and physically can affect the quality of your session. Psychadelics can do exactly this.

It's also worth noting my friend has done "regular" therapy for 2 years, and she felt like there was a step change after the guided session. Her therapist noticed it as well.

When you consider that pain meds have ruined literally millions of lives through addiction, and that also virtually (maybe literally?) no one has ever died due to overdose of psilocybin, it's very confusing why one is prescribed all the time, and the other is considered incredibly dangerous. The U.S.'s perspective on drugs is so very backwards.

bwest87

vor 6 Jahrendiscuss

Oakland relaxed a lot of zoning, and permitting regulations about 5 years ago, and so now you're starting to really see production sky rocket. This article [0] mentions 2019 was had 15x more units completed than 2018. And 3x the total units from 2013-2018 combined. Anecdotally, I have several friends who've all moved to Oakland in the last year or so. It will be interesting to see how this plays out, and maybe, hopefully, SF will take the hint.

[0] - https://www.city-journal.org/oakland-rezoning-california-hou...

bwest87

vor 6 Jahrendiscuss

yes, I should have clarified. They still do hack weeks AFAIK (I left about 6 months ago), but they no longer do continuous "20% time"

bwest87

vor 6 Jahrendiscuss

Eh, maybe. Your core responsibility is to ship on time. Especially at a startup, it's not only "ok", it's usually the "right" call to ship faster, and accumulate technical debt. This is constant, and never really goes away. So you tend to have to intentionally bake in this time, or else it never happens. And I think this is OK! It's not always clear what is and isn't technical debt at the time it's being introduced. A lot of things seem "gross", but aren't actually much of a problem. Or you think, "oh god, this will never work when we add X feature", but then you just never add X feature, and so it's totally fine 2 years later.

But to your point, of "well shouldn't that intentionality be part of normal duties?" Sure, but shouldn't coming up with Gmail? Or fixing bugs or UX improvements also be part of normal duties? Yes, they all should. The truth is it's hard to prioritize small things like that, and it's really hard to get a sense for their value as a centralized management team. They aren't close enough to the code or the product to always know. So I think 20% time is a great way to just decentralize that, and allow the engineers closest to the issues to pick what's important to work on.

bwest87

vor 6 Jahrendiscuss

You can do it a few ways. But in my experience, the simplest and best way is to force people to demo something at a pre-specified time. If you're doing it "hackathon" style, then it should be at the end of the hackathon. If you're doing it 20% style, then probably end of the quarter? You could do it with zero accountability, but having tried to run these programs (see my other top-level reply here), I think you'll mostly get a lot of people not doing much if you don't have any accountability.

bwest87

vor 6 Jahrendiscuss

I never worked at Google, but I did work for a different, fairly well known consumer tech company doing a lot of work around "hack time". While there, they didn't have any such program, and I really wanted there to be one. So I helped implement a program for "hack time", which was pretty close to 20% time. I did surveys of over 100 engineers, put together a presentation for leadership, got directors of various eng departments on board, did a pilot program, then did post-surveys and presentations to gauge impact. We also spent time with managers to make sure they were on board, and made it clear to their direct reports that they were on board, and that it was OK to use hack time. The point is, we really tried to do this right, and get people to use the time, and assess if it was actually valuable.

We did this for a quarter across a few different large eng teams. We eventually shut it down. My high level takeaways were the following...

  * Way more engineers *say* they're into doing hack projects than actually ended up doing them. We had huge initial survey response of people saying they wanted to do something, and only ended up having like 5-ish projects that were seen through. And there were probably 20ish projects that started at the beginning of the quarter. So large drop-off.
 * BUT! That can be totally fine! Many people who actually used hack time were some of the best engineers at the company. Other ones were some of the newest at the company. You're really helping job satisfaction for those top engineers, and really helping mentorship and learning for the new engineers.

 * I now believe it's better to have regular highly condensed hack weeks (or hack days if you're a small company), rather than a spread out "20% time". Even people who really liked hack time found it hard to actually take the time when deadlines were approaching. But when you just eliminate those things with a condensed period of hacking, then people can focus. You also tend to increase participation considerably through this method. (I know cause we did this method as well)
 * Stop trying to create Gmail in your hack time! A lot of the most valuable stuff to come out of hacking was internal tools, refactors, and little things that improved daily quality of life around the company. (ie. someone made a batch upload tool for the customer support team that they *loved*!). Or small UX improvements for customers. Some people did try to create certain new products, but it's super rare that those make it into the main product Not saying it can't, or won't, or even that you shouldn't do it. But as a general rule, the best you can hope for there is you've de-risked a new feature, or gotten your manager excited about the possibility, and they'll slot in time to "do it for real" in the next quarter. Realistically, people tend to have more fun just making stuff that they actually can see the value of the next day. Or when they get to hear thank you emails immediately from a whole other team.
 * You get a lot of other value from hack time besides break-through products. Specifically, you get people across teams mingling with each other. You get new and experienced engineers hanging out. You get a feeling of autonomy. People learn new tech or new parts of the company. For example, I made one of my most lasting relationships at that company by just randomly deciding to join his hack team for a hackathon project. I also got to use GraphQL and React for the first time on that project.

Overall, I think hack time/ 20% time / whatever you want to call it is very very valuable, and companies of all sizes should do it. But you have to do it right, and you have to go in with the right mindset about why you're doing it. Do it because it's fun. Do it because you help your company meet one another. Do it because you'll probably improve the daily quality of life in small tangible ways for a lot of people for your customers, or for other employees. Do it to give your engineers some autonomy. Don't do it in order to get your next Gmail.

Submissions

[untitled]

Solidity Learnings: How to Save 50% on Gas Costs

Crypto Won't Fix Music Streaming: A Lesson from History

Crypto Won't Music Streaming: A Lesson from History

A Token Doesn't Make You Decentralized, So What Does?

Perfection Premium

$600M in 10 Days: How Crypto Project Yam Shows the Future of Finance

The Influence of Organizational Structure on Software Quality [pdf]

Why Consumers Will Love the Decentralized Web

Show HN: Convolutional Neural Net in Google Sheets

Show HN: Virtual Piano Performance with any MIDI file

comments