TL;DR:
- AI-driven weather forecasting surpasses traditional methods.
- ECMWF leads the way with experimental AI forecasts.
- Tech giants like Google DeepMind and Huawei compete for supremacy.
- AI models learn from historical data for rapid, accurate predictions.
- Ensemble forecasting and climate modeling are the next frontiers.
- AI complements traditional forecasts but raises transparency concerns.
Main AI News:
In the realm of weather forecasting, artificial intelligence (AI) has ushered in a transformation akin to a “quiet revolution.” Over the past few decades, meteorological predictions have seen steady improvement, with today’s 6-day forecasts rivaling the accuracy of 3-day forecasts from 30 years ago. Gone are the days of sudden, unanticipated severe weather events or heatwaves catching people off guard. This remarkable progress has undoubtedly saved lives and resources, but it has come at a substantial cost – the deployment of energy-intensive supercomputers running around the clock to generate a handful of daily forecasts.
Enter artificial intelligence, the catalyst behind the latest revolution in numerical weather prediction. In a matter of minutes, utilizing affordable desktop computers, AI systems trained specifically for this purpose can now produce 10-day forecasts that match or even surpass the quality of traditional models. This paradigm shift has even won over the world’s leading weather authority, the European Centre for Medium-Range Weather Forecasts (ECMWF), which recently initiated its experimental AI forecasting program. These cutting-edge algorithms have the potential to enhance forecast frequency and liberate computational resources for addressing other complex meteorological challenges.
“It’s very, very exciting to know we can generate global predictions that are skillful, really cheaply,” exclaims Maria Molina, a dedicated research meteorologist at the University of Maryland with a focus on AI applications.
The race to develop the most proficient AI models in the weather prediction arena has attracted tech giants such as Google DeepMind and Huawei. Google DeepMind’s GraphCast model, showcased in Science, and Huawei’s Pangu-Weather, featured in Nature, have emerged as frontrunners. Google’s short-term AI weather model, which delivers 24-hour forecasts, has demonstrated astonishing accuracy, surpassing the capabilities of most weather agencies. This rapid progress in a field once deemed implausible has been described by Aditya Grover, an AI researcher at the University of California, Los Angeles, as a technological achievement with all the necessary components in place.
Traditional weather models initiate their forecasts by inputting a snapshot of current conditions derived from satellite data, weather stations, and buoys into a grid-based computer model that subdivides the atmosphere into countless grids. This snapshot is then projected forward in time by applying the principles of fluid dynamics to each grid, a computationally intensive process that can take hours to complete on supercomputers boasting a million processors. Consequently, weather agencies typically update their forecasts only four times daily.
In sharp contrast, the new wave of AI models bypasses the costly task of solving complex equations, opting instead for “deep learning.” These models discern patterns in the natural evolution of the atmosphere after being trained on a 40-year dataset comprising ECMWF “reanalysis” data, which combines observations with short-term model predictions, offering a comprehensive historical weather overview. Given the same initial atmospheric snapshot formed by a blend of observations and modeling, GraphCast, for instance, outperforms ECMWF’s forecast for up to 10 days in 90% of verification targets, including hurricane tracks and extreme temperatures. Remarkably, despite the lengthy training process involving 32 computers over four weeks, the resulting algorithm operates efficiently in less than a minute on a standard desktop computer. As Rémi Lam, lead author of the GraphCast study, asserts, “It is fast, accurate, and useful.”
Moreover, these achievements extend to real-world scenarios. Earlier this year, ECMWF researchers subjected Pangu to a rigorous test, feeding it only the observations utilized in their operational weather model. These observations provide a more limited atmospheric perspective compared to the reanalysis snapshots employed to evaluate GraphCast. Surprisingly, Pangu’s forecast skills closely matched those of ECMWF’s primary model, albeit with slightly less precision in predicting rainfall and finer-scale features. “It was an even playing field,” remarks Zied Ben Bouallègue, who led the analysis, revealing the remarkable prowess of AI in meteorological prediction.
The pace of these advancements has been nothing short of astonishing. A pivotal moment occurred in 2020 when Stephan Rasp, now at Google, spearheaded the creation of WeatherBench. This innovative platform made ECMWF’s reanalysis data accessible and set a benchmark for measuring forecasting accuracy, sparking competition among researchers. In 2022, Ryan Keisler, a physicist currently affiliated with KoBold Metals, a mineral exploration firm, published a preprint detailing a simplistic model that exhibited substantial forecasting skill for 6-day predictions. Keisler’s sentiment reflects the inevitability of success given the wealth of historical data available for learning purposes. “Given how much historical data there was to learn from, it just had to work at some level,” he surmises.
The next phase involves producing ensemble forecasts, an innovation aimed at capturing forecast uncertainty by running models multiple times to generate a spectrum of potential outcomes. AI researchers can follow the traditional approach of subtly adjusting initial conditions before each model run, or they can adapt AI generative techniques, which have proven transformative in text and image generation, to dynamically modify conditions. According to Stephan Rasp, who has been at the forefront of these developments, virtually every research group is actively exploring these avenues. Ensemble forecasts hold promise in enhancing AI models’ ability to predict extreme events, such as powerful hurricanes, which currently tend to be underestimated in terms of intensity.
To advance further, AI models may gradually detach from their reliance on reanalysis data, which inherently carry the biases of traditional models. Instead, they could glean insights directly from the vast troves of raw observational data amassed by weather agencies, as Google’s short-term weather model already does, drawing from weather station, radar, and satellite data.
The potential applications of these AI models extend beyond weather forecasting. Christopher Bretherton, an atmospheric scientist at the Allen Institute for AI, underscores their capacity to contribute to high-resolution climate models developed for exascale computers, the latest in ultrafast computing technology. Although AI models cannot independently project long-term climate changes, as they rely on 40 years of data insufficient to capture complex climate trends influenced by various factors such as clouds, gases, and aerosols, they can serve as invaluable tools once climate models yield sufficient data for AI training. These AI-driven emulators can significantly accelerate the pace of climate research, running simulations a hundred times faster than conventional models.
While traditional forecasts are unlikely to disappear anytime soon, AI is rapidly approaching a stage where it can complement existing methods effectively. Concerns about the opacity of AI systems, often referred to as “black boxes” due to the difficulty of understanding their decision-making processes, may slow down adoption. However, Matthew Chantry, who oversees ECMWF’s AI endeavors, argues that such concerns are overstated, highlighting the intricate complexity of traditional models, which also exhibit a degree of opaqueness in their functioning.
Ultimately, the preference for traditional or AI-based forecasts may come down to user preferences. As Aditya Grover aptly poses, “If you’re a farmer in the field, would you care about the more accurate forecast or the one you can write down with physical equations?” The answer to this question will shape the future landscape of weather prediction, where AI has firmly established itself as a powerful force to be reckoned with.
Conclusion:
The integration of artificial intelligence into weather forecasting represents a transformative shift that offers increased accuracy and efficiency. Market-wise, this advancement may lead to the coexistence of AI and traditional methods, with user adoption depending on the perceived transparency of AI systems. Furthermore, AI’s potential to accelerate climate research suggests long-term opportunities for innovation and growth in this sector.
