clickok

3 lata temudiscuss

Damn. I bought rl.ai when I was a grad student but it's been laying fallow since I can't really blog about the stuff I'm working on right now. How does one go about selling their domain for millions of dollars?

clickok

4 lata temudiscuss

According to others, that story is severely embellished:

https://savingjournalism.substack.com/p/i-talked-to-elon-mus...

clickok

5 lat temudiscuss

I see a lot of really divergent results with these time series database benchmarking posts. Timescale's open source benchmark suite[0] is a great contribution towards making different software comparable, but it seems like the tasks/metrics heavily favor TimescaleDB.

This article has Clickhouse more-or-less spanking TimescaleDB, but the blog post it references[1] is basically the reverse. Are the use cases just that different?

-----

0. https://github.com/timescale/tsbs

1. https://blog.timescale.com/blog/what-is-clickhouse-how-does-...

clickok

5 lat temudiscuss

A good idea and very cleanly implemented. I imagine that there's a ton of other possible applications that don't require much modification to the code. Thanks for sharing!

clickok

5 lat temudiscuss

None of these solutions are ideal, although Zenodo's better than most. As far as I can tell, they're all targeted more towards the final, authoritative release, so it seems you're still out of luck during the paper writing process. What if I'm just trying to share a dataset/pre-trained model with remote collaborators?

I ran into this when doing some OCR experiments[1], finding acquiring data and pre-trained models to be the most time-consuming part of the enterprise. This ended up adding enough additional hassle that I didn't manage to get anything really interesting going, although figuring out how to containerize other peoples' code was educational. Personally, I think I'll be relying on some combination of institutional repositories + torrents/IPFS for any large datasets/models I end up releasing in the future.

-----

1. https://github.com/rldotai/ocr-experiments

clickok

5 lat temudiscuss

Who, whom?

Less tersely: this article is one in a long procession of journalists trying to exert control over tech. The opening example (Speech2Face, which they aver is transphobic) is inflammatory and utterly unrepresentative of the usual topics of AI conferences. The other references are far better, but the choice is revealing-- it's not so much an abstract concern about an unaccountable few exerting control from the shadows, but alarm that someone else might be muscling in on their territory.

clickok

5 lat temudiscuss

This was great, actually. I don't program in Scala, but it was very interesting to hear about the difference between types as abstractions vs types as they are used.

For unfamiliar topics or when presented with uncommon insight, I believe rants, monologues, even diatribes are actually some of the best things to read.

clickok

6 lat temudiscuss

Depends what you mean by "practical". Assuming you've done CV/ML stuff before, you could probably get something working pretty well over a weekend, and I think I could solve it completely with a bit more effort via 3D scanning + synthetic dataset generation... but unless you have cubic meters of lego to sort, doing it by hand would be faster, albeit less fun.

clickok

6 lat temudiscuss

In the words of John von Neumann, there's a lot more that's known than is proved.

It's often frustratingly difficult to go from the known to the proven; still, just because something's not proved doesn't mean that scientists are ignoring it.

clickok

6 lat temudiscuss

It would be nice if citing repositories were easier-- either for generating a reference for my own code or acknowledging when I've used someone else's code in my research.

There's tons of math and physics blogs that contain useful results that the author wanted to make available but didn't manage to incorporate into a paper. I wonder if there'd be any interest in a sort of GitHub for proofs? It could even use git, since (assuming consistency) isn't math just a DAG anyways (and therefore isomorphic to a neural net, as are all things).

clickok

6 lat temudiscuss

> A theory is nothing but a way to turn (a finite number of) measurements into predictions.

That's a very nice way of putting it. Is there a particular book, lecture, or other resource you could recommend for this interpretation of theorizing?

clickok

6 lat temudiscuss

Furthermore, over the long term it can change the texture of the Reddit community. What sort of person quits using Reddit after becoming frustrated with yet another imposed hassle? What sort of person stays, or is retained by pop-up notifications and whatever other tedious blandishments their app/redesign provides?

There are other sites with similar features and layout to Reddit (Voat, Ruqqus, Saidit, Raddle, communities.win, etc.) but none are really in competition to be "the front page of the Internet". This is because Reddit has a huge community with varied interests which provides self-perpetuating advantages. For example: the activity level is generally higher; if you start a subreddit for a niche topic you have a chance at finding an audience; great comments/submissions will garner thousands of upvotes rather than a few dozen (smaller sites have a proportionately tinier vote ceiling, making it hard to differentiate between "good" and "amazing").

I can see increased retention/monetization maybe leading to a larger user-base in the short term, but if it makes good content less likely to crop up, or leads to less discerning users (ingenious effortposts lose out to low-effort pandering memes), or even if it just narrows the interests/hobbies represented on there, then over time it will destroy Reddit's main source of value.

clickok

6 lat temudiscuss

It's consistent with it.

Rich used to be very bullish on neural nets, then somewhat dismissive of them (due to the fragility/inadequacy of FCNs), and then increasingly enthusiastic as the renewed interest demonstrated that those problems could be overcome-- e.g., through better initialization, training, and (as you note) different architecture choices. His main concern was whether a method could keep working as more resources became available, as otherwise you would tautologically end up with something short of true artificial intelligence.

The important thing is that the technique can scale with increasing data or compute without hitting a hard or soft limit.

clickok

6 lat temudiscuss

You have misunderstood Sutton's argument. Quoting Rich:

  One thing that should be learned from the bitter lesson is 
  the great power of general purpose methods, of methods that 
  continue to scale with increased computation even as the
  available computation becomes very great.

The point isn't that improvements in our algorithms is unnecessary or unhelpful, rather that the algorithms we should focus on will be capable of scaling with arbitrary amounts of compute/data. Such as, for example, neural networks, where we see an almost constant rate of improvement (for the appropriate architecture) as more resources are added.

clickok

6 lat temudiscuss

It's a loathsome book for some people, even angsty teenagers, because not all of them are angsty for the same reasons.

I didn't enjoy it either when I was reading it, but later appreciated that it depicted a certain kind of angst— that of someone leaving childhood for adulthood, feeling anxious and threatened by this shift, and wishing to just remain as the most adult-like child (the "catcher in the rye" that gives the book its title) instead of having to go through that awkward period of being the most childlike adult.

Under that reading, it's much less irritating, because Holden isn't supposed to be valorized, just understood.

clickok

6 lat temudiscuss

I believe "crank" suffices, or possibly "poseur".

clickok

6 lat temudiscuss

It depends– if you're trying to run a specific algorithm on your new supercomputer, then you'd almost certainly be better off paying for researchers to optimize or improve on that algorithm. If that's the situation (which it is for e.g. weather forecasting or computational fluid dynamics), then a billion pound supercomputer is likely to be more of a boondoggle than a sharp-eyed investment. A good implementation on a desktop can beat a bad one running on a supercomputer.

But if it's a time-sharing system, then it might not matter as much. The supercomputer at my university tends to run a lot of one-off jobs like an experiment repeated thousands of times with different parameters. On a desktop that might take weeks, but if run in parallel it's like a couple hours. Tightly optimized code might bring that down to an hour on the cluster (or a mere week on my home PC) but I wouldn't bother because making the code more efficient might itself take a week or more. So the fastest way to get the results I need would be to just run it on the supercomputer.

clickok

7 lat temudiscuss

Hey Will-- congratulations on the publication. It's very cool to see an algorithm reflected in the brain, and the turnaround (from idea to implementation to experiment) in... what, three years? is astonishingly fast.

On a more technical note, I was curious about whether the distribution is being approximated "properly" (in the sense of probabilities summing to one, no negative probabilities) via expectile regression. Does that hold here? I'm less than an expert on neurology so I am unsure if that would be necessary in vivo (since it's clearly not needed for good performance in silico).

clickok

7 lat temudiscuss

Slow compared to what? You could write equivalent verbiage for the slow progress of mathematics, chip design, or cinematography-- the only difference is that uninformed randoms feel less bold when critiquing fields these fields (as opposed to AI), or perhaps people are less inclined to waste time reading such half-formed speculation.

The reason progress towards general artificial intelligence is "slow" is because it's a hard problem.

Formulating a definition of intelligence precisely enough that it can be optimized is incredibly difficult. We can capture facets of it, in some settings, but also have to deal with the cost of hardware and the difficulty of acquiring data. Sometimes you can get inspired by examining how animals behave, or analyzing the brain, or considering what it means to learn in abstract, but considering one thing in isolation means that you tend to hit a wall eventually.

We refine ideas, we try things, and are buoyed by the advances in technology that makes some strategies possible, but for every major milestone we have a ton of things that people tried and couldn't get to work. I personally have spent months working on stuff that ultimately yielded a minor improvement; I've spent days proving results that ended up taking half a page in some papers. It's not easy, and no one expects it to be easy, although some jerks shilling something singularity-related might make such claims.

But saying (to pick an example from the article) "oh, you just need to incorporate emotions, bro" into a learning agent is just about the dumbest thing I have ever heard.

clickok

7 lat temudiscuss

My word choice is more driven by exposure to the dataset that I'm working with. There's been some recent successes with autoencoders trained on virtual sensory input (i.e., video games) with surprising results, e.g., neural networks that can simulate the dynamics of these environments with surprising fidelity.

Of course, learning to play video games at a high level is trivially easy, as everyone in the field now knows. The next challenge is, naturally, to make money doing this. But how? After the traditional thirty seconds of research before undertaking a major project, I determined that the only way to make money from video games is to become a popular streamer.

So now I am training an agent to generate video of it playing and reacting to an imaginary game and equally fictitious Twitch viewers, with a dataset drawn from the top Fortnite streamers. The reward function is comprised of a blend of subscribers, donations, and (logarithmically scaled) misogyny in the chat. Thus far, I've only managed to create some sort of window into hell, where the "game" consists of unceasing violence, murder after murder after murder as towers of mismatched material swell and fall in ever transforming locations on the isle while the chat endlessly subscribes, spams, and emotes in cackling glee and the superimposed webcam video features a... thing with too many eyes and hands screaming incoherently.

At first I thought it was a problem with my dataset, so I started watching some of the streams myself. This has not yielded insight into the whole "nightmare vision" output of my model, but it has expanded my vocabulary on the twin subjects of combustibles and comestibles, which I feel is a reasonable trade-off for the sanity battering associated with this whole endeavour.

Submissions

IFSTile – Iterated function systems in the browser via WASM

Mathematical Methods for Economic Theory

Deep Learning Summer School Video Lectures (2015)

Countdown to 2016

Reinforcement Learning Neural Turing Machines

comments