Many people are surprised by the outcome

Today’s timeline was covered with Sora and Gemini 1.5, and the reaction was roughly like this. (There was also Meta V-JEPA ^^)

Many people are surprised by the outcome beyond their imagination, and it’s becoming clear that this kind of surprise will be experienced more often in the future. And it’s becoming clear that the goal is AGI. And on OpenAI’s Sora introduction page, it says…

Sora serves as the foundation for models that can understand and simulate the real world, and this feature is believed to be a significant milestone in achieving AGI. (Sora services as a foundation for models that can understand and simulate the real world, a capability we believe will be an important milestone for reaching AGI.)

What can we do in the AGI era that is coming this fast?

Before the AGI era, I thought the most important thing was to secure safety/reliability, and I proposed as a new task how to preemptively standardize and respond to these issues, but how much we can understand and sympathize with these important issues is another matter.

A series of measures such as the establishment of an AI safety summit in the UK, the establishment of an AI safety lab, and the establishment of an AISIC in NIST in the United States are also intended to preemptively respond to these AGI risks.

The AI Safety Lab is the first state-funded institution to focus on cutting-edge AI safety for the public interest. It is stated that its mission is to minimize damage to the country and citizens caused by the rapid development of AI. Although the topic has been suggested in Korea, it seems to be becoming a “problem in which a tangerine becomes a tanga when it crosses the collection” such as South 橘 枳 and South 橘 爲枳. Summary of The State of AI Reports for the past five years. Created without LLM’s help ^^

원문: https://stateofai.substack.com/p/the-state-of-state-of-ai-report

Contrary to the author’s intention, I made the entire text into a PDF and asked ChatGPT to summarize it, and they organized it as follows.

---

The 10 main topics that I’m summarizing in this article

1) Generative AI Boom: Recently, AI has greatly improved its ability to create images, text, audio, and more realistically. This has revolutionized creative content creation in many fields, including art, entertainment, and education.

2) Advances in Natural Language Processing and Image Generation: Natural Language Processing (NLP) and Image Generation Technologies have greatly advanced the way AI understands and generates human language and visual information. This has enabled more natural interactive AI and more sophisticated image generation.

3) Model-based reinforcement learning: This technology is a way for AI to self-learn and optimize decision making through its interactions with the environment. Significant progress has been made in several areas, including gaming, robotics, and resource optimization.

4) Importance of Large Language Models: Large Language Models provide complex language comprehension and generation abilities like humans. This has enabled innovative developments in natural language processing technology.

5) High model training cost: Training advanced AI models requires large computational resources, which increases their cost and impact on the environment. This is an important consideration for the sustainability of AI research and development.

6) Emergence of Unprogrammed Capabilities: Some AI systems often show capabilities that developers have not explicitly programmed in the course of training. This ’emergence’ provides a new understanding of AI’s potential and limitations.

7) Improvements in scene recognition and world modeling: AI’s ability to recognize the world is essential for understanding its surroundings, recognizing objects in complex scenes, and responding appropriately to situations. Advances in this field have greatly improved the practicality of AI.

8) AI in Science Research: AI plays an important role in accelerating basic science research, including life sciences, physics, and chemistry. This allows researchers to make new discoveries faster and solve complex problems.

9) Real World AI Applications: AI technology has been applied in various real-life fields such as autonomous vehicles, medical diagnostics, and financial services. It is promoting innovation in various aspects of society.

10) AI Safety, Regulation, and Ethics: Advances in AI technology raise safety, regulatory, and ethical issues. This is an important factor in determining the responsible use and direction of development of AI.

---

Key technology elements and products mentioned in the report represent important advancements in AI and machine learning, including:

1) Reinforcement Learning and Gaming: DeepMind led the field of reinforcement learning with outstanding performance on several game benchmarks, including Go, StarCraft II, Quake II Arena Capture the Flag, and Dota 2.

2) Generative AI and LLMs (Large Language Models): Although we didn’t call them LLMs at first, models in this field began to show impressive results in 2018. Specifically, Transformer models have emerged as the MVP of the research section by 2020.

3) Natural Language Processing (NLP): By 2021, NLP had actually become a ‘resolved’ problem. OpenAI represented data as token sequences, learning transformers that resulted in impressive results in image completion.

4) Extending to vision, audio, and 3D point clouds: Transformer models have expanded not only in the visual field but also to several other domains such as audio and 3D point clouds.

5) Model Scaling and GPT Series: OpenAI emphasized the importance of model scaling through GPT-2 and GPT-3. This served as evidence of “scaling laws for the improvement of model performance” in the field of research.

6) Image generation techniques: Following Generative Adversarial Networks (GANs), variational and diffusion models have shown superior capabilities in more stable training in image generation and modeling overall image distribution.

7) Scene Recognition and World Modeling: Early AI lacked the ability to analyze and portray scenes, but labs tried to address this problem through attempts to develop common sense world models of objects and behaviors. OpenAI’s CLIP and GPT-4V demonstrated the ability to accurately analyze image inputs.

8) Model-based reinforcement learning: Models such as Dreamer, DreamerV2, DreamerV3, and PlaNet enabled global model construction for planning and decision making as well as policy learning.

9) Self-driving technology: Model-based reinforcement learning is useful for predicting future states in the field of self-driving vehicles, providing great value for safety, efficiency, and anomaly detection. Wayve has achieved significant improvements in speed and steering.

---

The report explores important aspects and questions of AI development in many areas. This includes an in-depth analysis of how AI is developing in scientific research, real-world applications, and political contexts. In particular, the following key topics are highlighted:

1) AI Accelerates Scientific Research: AI is driving important developments in various basic scientific research fields, including biology, chemistry, and physics. These developments enable new scientific discoveries and understanding.

2) Rapid advances in real world applications: the increasing use of AI in real world applications such as autonomous vehicles, which demonstrates the practicality of the technology while presenting new challenges.

3) Increasing importance of AI in the political context: Advances in AI technology are playing an important role in political domains such as national security, defense industry, and elections, which have implications for international relations and internal political debates.

4) Growing Interest in AI Safety: The rapid development and potential of AI technology raises new questions about safety, ethics, and regulation. These issues are important factors in determining the direction of AI research and development.

5) Concern About AI and Elections: There is growing concern about the possibility of AI having a political impact through video generation and other media manipulation. This could be an important issue in future elections, requiring a political and regulatory response.

6) EU’s Leading Approach to AI Regulation: The EU is playing a leading role in AI regulation by introducing regulations to respond to AI risks early. This facilitates discussions on how policies and regulations should develop in line with rapid changes in technology.

These contents show that the development of AI has a wide range of impacts on various aspects of society. Research, policy, and regulatory efforts to maximize the positive possibilities of AI technology and minimize potential risks are important.

---

The report points out a number of concerns and critical factors related to the development of AI technology. The main concerns are as follows:

1) Model training costs: Training large parameter models can cost millions of dollars, which increases the economic burden on the industry.

2) Safety concerns about “emergence”: models are starting to show capabilities that developers have not explicitly programmed, which is an interesting development but at the same time raises safety concerns.

3) Concerns about AI and elections: There are concerns about the possibility of AI being misused in a political context, especially the impact of technologies such as video generation on elections.