Apr 29, 2023·edited Apr 29, 2023

Overnight? Very probably not.. But few futurists are actually suggesting it'll happen "overnight" in the literal sense. They're often speaking comparatively, to progress/advancements of the past.

The thing about technological progress over time, is humans are rather poor at differentiating between linear progress, and so called "exponential progress".. Which while trickier to understand, tends to better describe the way some technologies enable greater advancements more rapidly than before, than thinking about progress in an easy linear way, and up until recently, there wasn't much functional difference in the pace of change between the two.

Someday, imo likely soon, with one of the next iterations of GPT or some other LLM.. Someone will figure out the right combination of prompting loop-back methods, or the right way to use a team of AIs with various API access, and they'll start automating portions of a business, then someone will use it to automate the management of an entire business. And if it generates more profit than human managers, we'll rapidly see entire industries begin automating using autonomous ai agents. Unless we change the profit motives, it's an obvious chain of events.. And all it takes is the right industries to automate, before it becomes a self-improving feedback loop. Again, that process doesn't happen overnight, but after looking back on a year's worth of progress, it'll be shocking how much happens.

Expand full comment

You make some very interesting points! The whole idea that the data is going to be a bottleneck made me think. As you said, we need to either get bigger better datasets from somewhere, or figure out which datapoints are actually important for learning and focus on those. I wonder if that second problem might produce some solutions that spill over into our ability to teach things to humans as well.

Expand full comment

I thought the recent "Adaptive Agent" paper from DeepMind was really impressive progress in DRL.


Am I missing something?

Expand full comment

Thank you for calming these spicy waters of AI hyperbole!

Expand full comment

Your argument seems to be based on the claim that with enough data we can model anything with a neural network. To me that looks like saying that a movie is just made of pixels and those can be tracked from frame to frame, and one can make sense of them. While that may be true, the amount data and effort to represent the world at enough level of resolution can be truly outrageous, not talk about the amount of compute and training time needed. To me that looks impractical, and you outlined above some difficulties.

This is also not how we people do things. We reason and create toy models, rather than doing exhaustive data collections in our head.

Then, a lot of the knowledge that you want to create and embed in the neural network already exists as well-defined models implemented in software, and as models in physics, math, etc.

A much more efficient, more robust, and more understandable system would be not to push for ever more data, but to have reasoning, world models, and employ existing knowledge and libraries are needed for specific tasks.

Easier said than done, of course.

Expand full comment

"We Will Eventually Be Bottlenecked By Our Datasets." OpenaAI has already fixed this issue by releasing future gtp 4.1, 4.2, 4.3 and so on. At each version several millions users will give data. So new data will always be available in millions.

Expand full comment
Apr 29, 2023·edited Apr 29, 2023

> There are superhuman deep learning chess AIs, like AlphaZero; crucially, these models acquire their abilities by self-play, not passive imitation. It is the process of interleaving learning and interaction that allows the AI’s abilities to grow indefinitely.

I think the crucial ability allowing this leap to superhuman ability by AlphaZero was a built-in understanding of the rules of the game (go, chess, shogi).

The "rules" of reality are logic, math, physics (in some sense). What if Large Language Models can learn the rules of the game by tackling logic and math in the form of language and programming language coherence, then self-improve on that, opening the door to doing the same with physics/microbiology (and hence computational advancement/improvement)?

Well, a blueprint for self-improvement is already provided here: https://arxiv.org/abs/2212.08073 in the form of Constitutional AI. A constitution is just another way of defining the rules of the game. If a constitution requires an AI to abide by logic, reason, and mathematical coherence and left to self-improve, what could happen? Seems to me there is some low-hanging fruit here in our current paradigm and this experiment is happening today.

Expand full comment

Interesting post. As you say, it has been evident for some time that data will be the key bottleneck to LLM scaling. Couple of thoughts on this:

Firstly, to the extent that human experts are never perfect, a LLM might be able to become weakly superhuman by just completely avoiding mistakes and imperfection (as you allude to in the post). For example, maybe being a good rocket engineer requires you to understand propellant chemistry + materials science + supply chain issues + manufacturing process design, but any given human rocket engineer has a less than perfect understanding in all of these areas. Nonetheless we might expect that a LLM appropriately trained+prompted would be able to marshal the understanding of the best human expert in each of these individual areas, and thus act as a superhuman example of a rocket engineer.

I say "weakly" superhuman above because it seems less likely that a mixture-of-best-experts like this could obtain, say, 10x human IQ. But maybe there could be some super-additive effect where the model is able to spot hithero unexploited links between fields and come with some truly remarkable innovations?

Secondly: since data is the key problem, perhaps a bootstrapping approach could be useful. i.e. you could use an existing LLM to classify the existing input data we have into "high expertise" and "low expertise" subsets. If you then retrain on the "high expertise" subset you should get a LLM that will output higher-quality token streams. Then, use this "stage-2" LLM to generate+filter for new "high expertise" training data for the training of a stage-3 LLM etc.

Perhaps this process could help us train a weakly superhuman mixture-of-best-experts as outlined above, or perhaps the process of recursive bootstrapping could continue, to train something of unbridled expertise?

I guess the key problems are likely to be that 1) inference costs for this bootstrapping approach seem like they will be quite high -- though maybe still cheaper than many other ways of obtaining new training data -- and 2) the bootstrapping could go haywire, so the stage-N model is working with some definition of "expertise" which is unrecognizably distant from what we actually intended.

Expand full comment