Richard Sutton is a scholar who wrote The Bitter

Richard Sutton is a scholar who wrote The Bitter Lesson, which is cited almost like a textbook by artificial intelligence researchers. Watching his recent YouTube lecture video makes me think a lot.

To sum up…

It is said that the history of AI is not a clever human idea, but simply an increase in computational resources has won.

Today, large language models (LLM) are the heirs of these lessons, performing more GPU operations to improve their performance. However, Sutton points out that LLM does not follow the true lessons written. The reason is that LLM relies entirely on finite and biased human data called Internet text.

This point sends an important message to industrial strategy beyond simple academic debate. Internet data is not infinite and has copyright and bias problems, making it difficult to grow simply by securing data. This is why GPT-5, which has learned much more than before, does not perform so well.

LLM can significantly increase service and productivity in the short term. However, as Alan Turing said, in the long run, a ‘baby-type AI’ that interacts with the environment and learns on its own, reinforcement learning, and autonomous agents, are needed. At least like AlphaGo, which does not only learn by looking at human notation, but also plays Go on its own and finds its way.

Today’s AI is more like a ghost that replicates human knowledge. Ghosts are not as autonomous as living things, but they are very useful industrially right now. On the other hand, baby-type AI, which learns by colliding with the environment, has the potential to change the paradigm in the long run, although it is still a long way off. In this way, it increases the likelihood of reaching AGI (Universal Artificial Intelligence).

Therefore, businesses and governments should not choose just one side. Long-term research to develop autonomous learning AI should be invested in the same way that it is now used quickly to produce short-term results.

AGI, which government policies and companies are currently doing, will be completed and AI hegemony will be achieved quickly by governments and companies that balance this from securing GPUs to ecosystem investments, from data utilization to autonomous learning research, and from short-term practical use to long-term innovation.

링크: https://youtu.be/21EYKqUsPfg