[A phase of the AI cycle?]

1.
I had a few conversations with a friend who is a professor at NYU the day before yesterday, who collaborated and wrote a thesis together. We talked about various things, and he said, “Recently, AI research has been too biased toward LLM/VLM-related research, so topics have become a little boring.”

I agreed to some extent. To overdo it a little bit, when it comes to AI (to non-researchers), people often think of LLM/VLM including chatGPT like an equation. I’m also doing AI-new drug development research, and breakthroughs like AlphaFold3 are constantly coming out, but in itself, it’s not a situation that is revolutionizing the profitability of the industry.

(If I ask a few simple questions, can we develop a new drug with AlphaFold3 given a novel target? Can new modality drugs such as ADC/TPD be developed with AlphaFold3? What kind of contribution can AI make in the post-clinical phase? How many tens to hundreds of thousands of H100s do we need the technology and resources? It’s not necessarily No, but it’s only about partially yes.)

​2.
The day before yesterday, there was a Google conference saying, “Over-investing is better than under-investing,” and “Despite this large capex investment, there is no lock effect, killer app using AI such as LLM,” or “It is not as useful as capex investment.”

In fact, it was something I was constantly curious about while researching AI.
Everyone studies LLM and says that research cannot be done without GPU resources, but it was not easy to draw when asked, “What is the future these will show?” Some say it is an AI bubble, but the expression bubble is so vague that I didn’t even agree with it.

3.
Nevertheless, I can feel that AI research is too focused on foundational AI, and JD in various job postings is also focused on this side day by day.
Wouldn’t the biggest theoretical/technical problem be the “generalization” that keeps coming out? LLM’s Hallucination eventually comes out as a lack of generalization ability, and can AI be applied without doubt in the real world, not in benchmarks? If you answer based on your own experience, I don’t think so.

So, how can the generalization problem be further improved? Algorithms/methodologies, when I was in the thick of papers, optimizer, semi-/self-supervised learning, etc. There have been many reports of better cases when using them than when not, but in the end, supervision using labeled data has to be added. LLM also eventually needed a tremendous amount of data and a model capa to digest it, so needed more GPUs. With an explicit scale.

If the direction of LLM improvement in the future is like that, doesn’t it mean that the cost will eventually grow? In a place like AI new drug development, everything about acquiring labeled data is money, so what is the speed and cost?
Compared to the current capex investment, the killer app may not be out yet, or another application such as LLM may come from one domain or another if you have a lag.