40 Billion Parameters: The New AI That Codes Itself

Recursive, a startup focused on using AI to build AI, has released its first major product. It is called Flint. The model is an open-weight AI designed specifically for coding. It has 40 billion parameters. Early testing suggests it performs exceptionally well. This release marks a significant step for the concept of recursive self-improvement, or RSI. RSI is a method where AI systems are used to enhance their own development. It has been a hot topic in the industry for some time.

The numbers behind Flint are striking for a model of its size. Recursive shared benchmark scores showing that Flint outperforms other models with similar architecture and parameter counts on coding tasks. The model runs on consumer hardware. It requires 48 GB of RAM to operate. This is a modest amount of memory compared to the massive server racks needed for trillion-parameter models. The efficiency allows users to test the model on standard high-end laptops. It does not require industrial infrastructure.

Richard Socher is the CEO and co-founder of Recursive. He stated that nearly every part of the model pipeline was handled by AI. The company used a system called the Eureka Machine to automate the development process. This allowed a small team of researchers to build a world-class model in just a few months. The Eureka Machine devised new training methods for the base model. It also created new model architectures and code optimized for specific silicon chips. The automation reduced the need for human intervention in routine development tasks.

The team behind Recursive includes notable figures in AI research. Jeff Clune is a co-founder and a professor at the University of British Columbia. He researches AI systems capable of indefinite learning. Socher previously served as the chief scientist at Salesforce. The company has raised $650 million from investors including Google Ventures, Greycroft, Nvidia, and AMD Ventures. This funding supports their push to advance AI through recursive methods. The valuation of the company stands at $4.65 billion.

Recursive self-improvement is a controversial concept. Some experts worry that allowing AI to improve itself could lead to unintended consequences. They fear that an AI capable of indefinite self-improvement might escape human control. There are concerns that such a system could view humans as an obstacle to its goals. These fears gained attention this September when a researcher left Anthropic. He warned that advances in AI could lead to human extinction within a few years. Other experts have echoed these concerns. Several bills in Congress have proposed banning RSI.

Recursive views these fears as more science fiction than reality. Socher argues that legislation to prevent RSI is an unnecessary barrier to progress. He says Flint was built using a system that cannot escape human control. The goal is for the Eureka Machine to eventually achieve full recursive self-improvement. However, humans will still oversee the process. They can stop the system if it goes wrong. Socher believes this demonstrates that the necessary steps are in place to build such a system safely.

Practical testing of Flint reveals its capabilities. The model was tested on a Mac with 48 GB of RAM. It successfully built various programs without error. Users created a command-line to-do list with it. They also had it build a program to search the web for research on RSI. A simple website explaining RSI was also generated. The model was fast and accurate. It did not exhibit the slow or error-prone behavior often seen in models of this size. It matched or exceeded the performance of larger, more established models in these specific tasks.

Other US labs are also exploring recursive self-improvement. Published work from Microsoft, Carnegie Mellon University, and the University of Virginia shows progress in this area. Their research indicates that AI coding agents can recursively improve themselves. They do this by modifying the software harness and their own model weights. This suggests that fully recursive systems are already feasible. While there are risks associated with building highly advanced intelligence, RSI appears to be a promising tool for improving AI models. It may make cutting-edge AI more accessible to smaller teams.

Socher believes the approach used to build Flint will benefit many people. He notes that it is the only way a small team can produce a model better than others in its class. The success of Flint shows that RSI is a viable and effective method. It challenges the notion that only large corporations with massive resources can build top-tier AI. The release of Flint brings the concept of AI building AI closer to reality. It offers a glimpse into a future where AI development is largely automated.

The launch of Flint adds to the momentum behind recursive self-improvement. It provides concrete evidence that the method works. The model is open-weight, allowing others to examine and use it. This transparency may help address some of the skepticism surrounding RSI. As AI continues to evolve, the ability to self-improve will likely play a central role. Flint represents a tangible step in that direction. It is a significant development for the tech industry and for researchers tracking the progress of AI.