TL;DR:
- Google DeepMind employed a large language model to solve a complex mathematical problem, marking a significant breakthrough.
- FunSearch, the innovative tool, showcases the potential of large language models to uncover new scientific insights.
- It combines Codey, a specialized language model, with filtering systems to propose and refine solutions to math problems.
- FunSearch successfully tackled the cap set problem, a challenging mathematical puzzle, and surpassed human-designed solutions.
- The tool’s versatility extends to solving various mathematical challenges, including the bin packing problem.
- Mathematicians see potential in this approach for leveraging the power of large language models in research.
Main AI News:
In a groundbreaking achievement, Google DeepMind harnessed the potential of a substantial language model to tackle an age-old mathematical conundrum. This momentous feat, documented in a recent issue of Nature, marks a pivotal moment in the realm of pure mathematics. It signifies the maiden instance in which a large language model has been utilized to unravel a longstanding scientific enigma, delivering verifiable and invaluable insights hitherto undiscovered. “It’s not in the training data—it wasn’t even known,” asserts coauthor Pushmeet Kohli, who serves as the Vice President of Research at Google DeepMind.
The prevailing perception regarding large language models is that they excel at conjuring information rather than generating novel facts. However, Google DeepMind’s innovative tool, christened FunSearch (not to be confused with actual amusement, but rather an abbreviation for its quest to uncover mathematical functions), challenges this notion. It convincingly demonstrates that, under specific conditions and with judicious selection, these models can indeed make genuine discoveries, thus shedding light on a novel path in their utilization.
FunSearch marks a continuation of DeepMind’s trailblazing ventures into the realms of fundamental mathematics and computer science through the application of artificial intelligence. Initially, AlphaTensor made an astonishing breakthrough by enhancing a crucial computation central to diverse code types, thereby outpacing a record that had stood for half a century. Subsequently, AlphaDev paved the way for optimizing algorithms that are utilized trillions of times daily. However, these accomplishments were achieved without relying on large language models. Both AlphaTensor and AlphaDev operated on the foundation of DeepMind’s game-playing AI, AlphaZero, treating mathematical problems as if they were puzzles akin to Go or chess. Nevertheless, this approach constrained their scope of applicability, as Bernardino Romera-Paredes, a DeepMind researcher involved in both AlphaTensor and FunSearch, points out, “AlphaTensor excels in matrix multiplication but little else.”
FunSearch adopts an alternative strategy. It amalgamates a formidable language model known as Codey, a specialized variant of Google’s PaLM 2 fine-tuned for computer code, with complementary systems designed to filter out incorrect or nonsensical answers while retaining promising ones. Alhussein Fawzi, a research scientist at Google DeepMind, concedes, “To be very honest with you, we have hypotheses, but we don’t know exactly why this works.” The journey commences with researchers outlining a problem in Python, a widely-used programming language, deliberately omitting instructions on how to resolve it. FunSearch steps in at this juncture, enlisting Codey to complete the missing segments, essentially offering code suggestions to solve the problem.
A second algorithm evaluates and scores Codey’s proposals. Even if the suggestions are not yet flawless, the best among them are preserved and returned to Codey, which endeavors to refine the program. Kohli elucidates, “Many will be nonsensical, some will be sensible, and a few will be truly inspired. You take those truly inspired ones and you say, ‘Okay, take these ones and repeat.‘” Following millions of suggestions and numerous iterations, which spanned several days, FunSearch succeeded in generating a correct solution to the cap set problem, a problem involving the determination of the largest feasible size for a specific type of set. Analogously, envision plotting dots on graph paper, and the cap set problem becomes the challenge of strategically positioning dots without ever forming a straight line between three of them.
While the cap set problem may appear niche, it holds profound importance within the mathematical community. Mathematicians have yet to reach a consensus on its resolution, let alone identify the actual solution. Moreover, it bears relevance to matrix multiplication, the computational task that AlphaTensor previously optimized. Terence Tao, a renowned mathematician at the University of California, Los Angeles, hails the cap set problem as “perhaps my favorite open question,” as stated in a blog post from 2007. Tao expresses intrigue in FunSearch’s capabilities, asserting, “This is a promising paradigm. It is an interesting way to leverage the power of large language models.”
One significant advantage of FunSearch over AlphaTensor is its potential applicability to a wide array of problems. This stems from its capacity to generate code—a set of instructions for producing solutions—rather than furnishing the solutions themselves. Different problems necessitate distinct sets of code. Additionally, FunSearch’s outcomes are more comprehensible. Fawzi explains that a code-based recipe often presents a clearer solution than the esoteric mathematical answers it generates.
In a test of its versatility, the researchers deployed FunSearch to tackle another formidable mathematical challenge: the bin packing problem, which entails optimizing the allocation of items into as few containers as possible. This problem holds great significance in computer science, spanning applications from data center management to e-commerce. Remarkably, FunSearch unveiled a solution that surpassed those devised by human counterparts in terms of speed.
As mathematicians grapple with the intricate balance of harnessing the capabilities of large language models while mitigating their inherent limitations, Terence Tao reflects, “This certainly indicates one possible way forward.” FunSearch’s remarkable capacity to explore uncharted mathematical territories hints at a promising avenue for future exploration and innovation.
Conclusion:
Google DeepMind’s FunSearch represents a game-changing development in the intersection of artificial intelligence and mathematics. This breakthrough not only showcases the potential of large language models in solving complex problems but also offers a versatile tool for various applications. In the business world, this innovation could lead to more efficient problem-solving and optimization processes, potentially impacting industries like data management and e-commerce. It underscores the evolving landscape of AI-driven solutions and their potential to revolutionize traditional approaches to problem-solving.
