Get GenAI guide

Access HaxiTAG GenAI research content, trends and predictions.

Wednesday, June 19, 2024

The Future of Large Language Models: Technological Evolution and Application Prospects from GPT-3 to Llama 3

June 19, 2024

At the 2024 Zhiyuan Conference, Meta research scientist and the author of Llama 2 and Llama 3, Dr. Thomas Scialom, delivered a keynote speech titled "The Past, Present, and Future of Large Language Models." In his presentation, he thoroughly discussed the development trajectory and future prospects of large language models. By analyzing flagship products from companies such as OpenAI, DeepMind, and Meta, Thomas delved into the technical details and significance of key technologies like Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) used in models like Llama 2. He also shared his views on the future development of large language models from the perspectives of multimodality, Agents, and robotics.

Development Trajectory of Large Language Models

Thomas began by highlighting the pivotal moments in the history of large models, reflecting on their rapid development in recent years. The emergence of GPT-3, for instance, marked a milestone indicating that AI had achieved functional utility, thereby broadening the scope and application of AI technology. The development of large language models can essentially be seen as a collection of weights based on the Transformer architecture, trained through self-supervised learning to predict the next token with minimal loss based on vast amounts of data.

Two Ways to Scale Model Size

There are two primary ways to scale the size of large language models: increasing the number of model parameters and increasing the amount of training data. In their research on GPT-3, OpenAI discovered that enlarging the model parameters significantly enhanced performance, prompting a substantial increase in model size. However, DeepMind's research highlighted the importance of training strategies and data volume, introducing the Chinchilla model, which optimizes computational resources to achieve excellent performance even with smaller parameter sizes.

Optimization of the Llama Series Models

In the training process of the Llama series models, researchers rethought how to optimize computational resources to ensure efficiency in both training and inference phases. Although Llama 2's pre-training parameter scale is similar to Llama 1, it includes more training data tokens and employs a longer context length. Additionally, Llama 2 incorporates SFT and RLHF technologies during the post-training phase, further enhancing its ability to follow instructions.

Supervised Fine-Tuning (SFT)

SFT is a method used to align models with instructions by having annotators generate content based on given prompts. Thomas's team invested significant resources to have annotators produce high-quality content, which was then used to fine-tune the model. Although costly, SFT significantly improves the model's ability to handle complex tasks.

Reinforcement Learning from Human Feedback (RLHF)

Compared to SFT, RLHF involves annotators comparing different model-generated answers and selecting the better one. This feedback is then used to train a reward model, which improves the model's accuracy. By expanding the dataset and adjusting the model size, Thomas's team continuously optimized the reward model, ultimately achieving performance that surpasses GPT-4.

Combining Human and AI Capabilities

Thomas emphasized that the real strength of humans lies in judging the quality of answers rather than creating them. Therefore, the true magic of RLHF is in combining human feedback with AI capabilities to create models that surpass human performance. The collaboration between humans and AI is crucial in this process.

The Future of Large Language Models

Thomas believes that the future of large language models lies in multimodality, integrating images, sounds, videos, and other diverse information to enhance their processing capabilities. Additionally, Agent technology and robotics research will be significant areas of future development. By combining language modeling with multimodal technologies, we can build more practical Agent systems and robotic entities.

Importance of Computational Power

Thomas stressed the critical role of computational power in AI development. As computational resources increase, AI model performance improves significantly. From the ImageNet competition to AlphaGo's conquest of Go, AI technology has made rapid strides. In the future, as computational resources continue to expand, the AI field is poised to witness more unexpected breakthroughs.

Through Thomas's insightful speech, we not only gained a comprehensive understanding of the development trajectory and future direction of large language models but also recognized the pivotal role of technological innovation and computational resources in advancing AI. The research and application of large language models will continue to have profound impacts across technological, commercial, and social domains.

TAGS

Meta research scientist, Thomas Scialom, Llama 2 model, Llama 3 advancements, GPT-3 development, Transformer architecture, Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), Chinchilla model optimization, Multimodal AI future, Agent technology in AI, Robotics in AI development, Computational power in AI, AI model scaling, AI performance breakthroughs, DeepMind research, OpenAI innovations, AI training strategies, AI application prospects, Zhiyuan Conference 2024, Future of large language models.