tristanm · HackerTrans

If you start with a weak model that doesn't contain all the knowledge you have available, your anomalies will contain many irrelevant or already known things. If you start with a strong model representing the best current understanding, then correcting the model is not so straightforward.

tristanm

7 anni fadiscuss

This is a good illustration of the core problem we have in "anomaly detection" in data science. Often we are presented with a challenge that if solved, would negate the presence of the challenge itself: We have to look for events that aren't explained or predicted to exist by our current understanding of the given system. To find them, we collect all events and evaluate their likelihood under our best model, taking the least-likely as our "anomalous" events. Then, once found, we have to explain them. But to explain them requires that we understand the system well enough to predict the existence of those events. If we did, we could have produced a better model, and that model would have rated those events as more likely. So they wouldn't have shown up. This contradiction seems to be inherent to the whole concept of anomaly detection.

tristanm

7 anni fadiscuss

Two points I forgot to add:

One, there seem to be a lot of people who commute via car / bart / caltrain from places in the south peninsula, san jose, or south-east bay (fremont area roughly). In these places, rent is lower than SF but not notably lower, especially in the peninsula. And traffic / commute times are notoriously horrible coming from there.

Two, if people are rational about real estate values, it suggests that people truly don't expect any expansion at all to happen in the greater bay area. Not only that, but they don't expect work-from-home or telecommute to become more commonplace. Either of those things would substantially lower the cost of distant living, either by increasing the (future) value of potential real estate investments made now, or by reducing the current cost via reducing lost hours. I can think of arguments why we should expect more WFH, and to a lesser degree more expansion, but that is a whole other debate.

tristanm

7 anni fadiscuss

There is one way that it would not solve the problem to build more housing. That way would be that most renters and homebuyers are basically irrational, or systematically overvaluing the location of their residence when better options exist. I would argue that this is actually the case in the SF Bay Area.

For example, I live in Pittsburg, almost at the very end of the yellow line (it recently got extended to Antioch). I work in SF making a decent salary. My commute is pretty long, nearly an hour one way, sometimes longer. My rent? Well, for a two bedroom, two and a half bath condo with a garage, porch, and backyard, it costs me roughly $2100/month, and we don't have rent control here. My rent has been raised only twice in the 3 years I've lived here, and only by about $50 each time. The same amount of space in San Francisco, by my best estimate, would cost somewhere between $4k and $5k a month. Also, that space would probably be in a much much older building, in a denser and more dangerous place than my town.

So, one could immediately ask, if I'm able to work in SF with an SF-commensurate salary, and pay this low in rent, why is no one else doing this? As far as I can tell, people even living in SF are very lucky to have less than a 30min commute one way. So I tack on an extra hour per day of commute, or, lets say, roughly 20-25hrs per month.

People do vary in their subjective valuation of a long commute. I probably do consider it to be less of a problem than most people. But do people really consider it to be worth nearly $2k-$3k a month? For someone making near what I am that would be more than my average hourly salary for time spent on the train. Again, people do vary in their respective valuations of time spent doing something other than optimally, but I would be surprised if it was worth that much.

Also consider that housing is continuing to be built on the other side of the mountains in the east bay, and that in Pittsburg and especially in Antioch it is possible to get a lot of space for very cheap, still. Its not unreasonable to expect transportation to get better over time, either.

tristanm

7 anni fadiscuss

In the normal case of one innovation in the free market, there is a punctuated moment of growth followed by a plateau. The owners of the innovation get the biggest share of the rewards, but after the plateau the benefits become more distributed. They would need to keep innovating if they want to gain further.

In the case of AGI, there is no plateau after the initial acceleration. With recursive self improvement, there is just an exponential explosion of growth. Unless this event was initiated such that distributed rewards were specifically intended to occur, the feedback loop would prevent external pressures from incentivizing any kind of distribution of rewards.

tristanm

7 anni fadiscuss

You think so? I think if he thought hard takeoff was a real risk, he would be devoted to actually making their research "open", in the original sense of the word when Musk was in charge. According to the hard takeoff theory, it is more likely to occur due to "hardware overhangs", where big organizations accrue lots of compute infrastructure over time, but AI progress occurs in sudden jumps. Then all that infrastructure is just sitting there waiting to be eaten up by a hungry AI. These sudden jumps are more likely to occur if AI research is generally undertaken in a secretive manner, with leaks or espionage resulting in staggered spread of knowledge. This is at least my understanding.

tristanm

7 anni fadiscuss

Yes. This is in the scenario where the benefits are widely distributed. But in that case the value of the entire planet goes up; You would not necessarily need to make an investment in OpenAI in order to reap benefits. Unless you think that scenario is more likely to happen with OpenAI and not via the usual academic research + shared industry research (which I don't see why that would be the case).

tristanm

7 anni fadiscuss

It's interesting that in order for his pitch to work (if you invest in OpenAI, you will get up to 100x returns), assuming they do build AGI, it still requires that their AGI acquires a very stable, virtually guaranteed advantage of large magnitude. This very strongly requires that they cannot share anything they discover whatsoever. Especially since they apparently plan on using it to make strategic investments to beat the market by a huge margin. That would mean they obtain information (about the economy, world affairs, technology, the future, etc.) not possessed by anyone else, or that information would be reflected in the market already. Any information leakage, whether regarding their AI or whatever it learns about the world, would compromise that advantage.

In other words, what Altman says about "we can't only let one group of investors have that" can't be true, or at least not sincere. The more investors who have access to it, the more its returns get distrubuted across society more evenly (which would be a good thing, obviously), but lowers the incentive for initial investments. They will want to keep it contained within a small group of investors for as long as possible.

tristanm

8 anni fadiscuss

I think that bursts of technological progress follows more from the model of individualized intelligence, whereas continuous progress follows from the distributed, networked model of intelligence.

A promoter of the distributed model of intelligence might argue that Einstein was only able to produce the general theory of relativity because of the knowledge already contained within society, such as the mathematics and physics that had already been built up to that time. All the stuff from Euclid to Newton to Gauss to Poincare and Minkowski that Einstein's work relied upon.

Does that imply that Einstein wasn't really smart? If you narrow your focus to just the innovation Einstein made, where did that come from? Did it come from the "hivemind" or was Einstein himself doing something special that allowed him to develop the insight?

More individualized intelligence would predict that we would see smaller intervals between bursts as society increases in size and connectedness (more chances for Einsteins to appear, more likelihood that they can work together). But if intelligence is somehow an emergent process from the network of all humans itself, then as society grows we shouldn't see many bursts at all, just a fairly continuous increase in knowledge as little bits and pieces get absorbed and distributed.

tristanm

8 anni fadiscuss

The "hivemind" argument seems to predict that as society scales up (either through massive population growth or through faster and better interconnectedness, such as through the internet) that as a result we should be seeing much faster gains in technological progress especially in the last few decades or so. However there are quite a few observations that a lot of this progress has sort of slowed down compared to the early 20th century (see the arguments for "technological stagnation"). At the very least, technological progress hasn't increased linearly with population growth and better communication. In other words, the rate of technological progress looks more discontinuous and not obviously a function of societal coherence.

comments