Triggered by an earthquake, sudden seafloor uplift can displace vast volumes of water, creating a tsunami that imperils lives and leaves behind widespread destruction. The earlier that residents of vulnerable coastal communities receive warning for the impending event, the faster their response and evacuation. Time and accuracy are paramount.
Predictive tsunami propagation models are only as good as the data sources, physics models, and computational resources powering them. Many current approaches rely on matching data to precomputed seismic scenarios rather than solving a high- fidelity model as measurements are simultaneously captured in the near field of the earthquake. Some models are too simple to produce accurate forecasts, while others are too slow to solve the necessary physics calculations at the time of the event. The ensuing warnings may be too basic, too late, or otherwise incorrect. False positives can undermine public trust, and false negatives can be devastating.
Researchers from Lawrence Livermore; the University of Texas at Austin’s (UT) Oden Institute; and the University of California, San Diego’s Scripps Institution of Oceanography (Scripps) joined forces to improve the state of the art. The team developed a tsunami prediction model that uses pressure sensors and 3D acoustic–gravity wave simulations to infer seafloor movement during an earthquake and then forecasts the subsequent tsunami waves. The resulting simulation is a digital twin—a virtual model of its physical counterpart—that can depict tsunami formation in real time with quantified uncertainties.
The team’s framework includes an innovative sequence of “offline” computations ahead of the “online” tsunami forecast. The offline phase set a record as the largest finite element simulation performed, while the online phase can run in 0.2 seconds—10 billion times faster than conventional methods.
The research won the 2025 Association for Computing Machinery’s prestigious Gordon Bell Prize awarded at the International Conference for High Performance Computing, Networking, Storage, and Analysis, also known as SC25. The Gordon Bell Prize is widely considered the top award in the field of high-performance computing (HPC). (See the box below.)
This achievement involves dedicated scientists, innovative mathematical algorithms, and one of the world’s fastest supercomputers. These collaborators with complementary expertise and converging research paths introduced a scientific framework that highlights speed and scale and offers life-saving potential.
Recurring Recognition
In 2006, the Association for Computing Machinery—the world’s largest scientific computing society—became a co-sponsor of the international Gordon Bell Prize, which was established in 1987. Named for an American pioneer in computing architecture, the annual award recognizes breakthroughs in high-performance computing (HPC), such as a peak performance achievement or a unique solution to a scalability problem.
As a leading HPC center, Lawrence Livermore has won the award several times since its first win in 1999 for high-resolution simulations of fluid turbulence in compression flows. Other Laboratory winners include a 2013 fluid dynamics simulation that reached 14.4 petaflops of sustained performance (see S&TR, July/August 2014, Evidence of a Turbulent Beginning); a 2007 simulation of 62.5 billion atoms of liquid aluminum (see S&TR, September 2010, Crossing Computational Frontiers); and a 2005 model of the solidification of molten metal (see S&TR, July/August 2006, Keeping an Eye on the Prize). Laboratory teams have also reached the finals with efforts toward parallelization of a 2.4-billion-particle plasma simulation (2009) and unprecedented accuracy of a first-principles molecular dynamics code (2016).
These decades of research have ushered in immense advances in HPC technology, and award submissions must push ever- newer frontiers. Erik Draeger, director of the Laboratory’s HPC Innovation Center, points out, “Computational scientists have reached dizzying new heights in capability and performance by rethinking algorithms to best utilize accelerator-based hardware, embracing programming abstractions to enable portability, and abandoning old paradigms that no longer make sense. HPC has never been more difficult to do well, but the gulf between the leading edge and middle of the road has never been greater.”
According to Draeger, a Gordon Bell Prize winner in 2006, Livermore’s recurring appearance in the award’s history is a “natural consequence” of enduring HPC expertise. “As remarkable as the scientists and mathematicians who contribute to these submissions are, one must also acknowledge the Laboratory’s mission and long history of excellence in supercomputing as a key factor in their success,” he says. “This is team science at its best.”
Real-World Impact
The 21st century has seen some of the deadliest tsunamis following high- magnitude earthquakes. For example, a magnitude 9.1 quake in the Indian Ocean in 2004 produced a tsunami that killed more than 225,000 people in 14 countries. At the time, no early warning system existed for the region. In 2011, a tsunami resulting from a magnitude 9.0 quake off the Japanese coast caused 20,000 deaths and notably damaged the Fukushima Daiichi nuclear power plant. Japan’s early warning system did not accurately predict the seismic intensity and tsunami risk.
Although some countries’ early warning systems have advanced in speed and accuracy over the years, many seismically active regions still lack sufficient tsunami detection and prediction capabilities. In North America, for example, several tectonic plates come together in the Cascadia subduction zone, which stretches 1,000 kilometers along the west coast. The last magnitude 8.0 or greater earthquake occurred in this region more than 300 years ago, and probability estimates indicate that another major seismic event could happen within decades. A tsunami would cover the distance from fault line to shore in a matter of minutes. “The Cascadia subduction zone doesn’t have the comprehensive sensor network now deployed in Japan,” explains Stefan Henneking, UT research associate and lead author of the team’s prize-winning paper. “We built a framework that can quantify how many sensors would be needed and where they should be placed for optimal data acquisition to minimize uncertainty for decision makers.”
Inference and Prediction
The team’s tsunami forecasting framework combines coupled acoustic–gravity wave equations with a hypothesized network of 600 seafloor pressure sensors spread throughout the Cascadia subduction zone. Spatiotemporal seafloor motion can be inferred from the pressure measurements. From that inference, wave heights can be predicted as water rushes toward the shore.
Developed at UT, this approach solves a Bayesian inverse problem, which reconstructs unknown inputs from observed outputs and treats the unknowns as random variables. The inverse solution calculates a probability distribution over possible inputs, thus quantifying uncertainty within the model. In other words, the framework works “backward” to infer the seafloor motion that most likely produced the sensor data. In contrast, a forward problem would start with seafloor motion inputs and use the wave equations to predict pressure readings and wave heights.
“Substantial uncertainty exists in characterizing earthquakes,” Henneking points out. “Our approach takes a different angle by not inferring the earthquake source but, rather, working with data from ocean-bottom pressure sensors to directly determine the earthquake-induced seafloor motion and tsunami propagation.” In addition to coupling inference with prediction, the framework provides confidence estimates for the wave height forecasts.
Making these inferences over such a large zone requires calculating seafloor velocity at many locations over time—in other words, a high- dimensional set of parameters to capture the necessary physics for accurate tsunami forecasting. To construct these parameters, the team discretized the seafloor and ocean into a 3D mesh with a spatial resolution of 300 meters and modeled a 7-minute event from the onset of seafloor motion. This scheme resulted in 2.4 million spatial points for the seafloor velocity parameter field at each of 420 time steps (1 per second), for a total of just over 1 billion parameters. Additionally, the pressure and velocity states that describe the propagating ocean acoustic and tsunami waves are captured by many billions of spatial points. The resulting high-resolution simulation provides incredible detail and complexity.
Real-Time Results
Simulation accuracy comes with a computational cost, which is a formidable obstacle in time-sensitive tsunami forecasting. For instance, among its many calculations, the framework must solve a wave propagation equation for each of the 600 pressure sensors. The research team needed a feasible way to manage computation-heavy operations before quickly solving the inverse problem. The answer is a unique decomposition of the solution process into offline (ahead-of- time) and online (real-time) phases.
Several steps in the process can be completed ahead of an actual earthquake. Since the region’s seafloor topography and water depths are known, as are the pressure sensor locations, a 3D mesh of the area can be generated along with the parameter data set. Furthermore, the relationships between seafloor motion and expected sensor recordings are known from physics models.
The framework also incorporates a few mathematical tricks. For example, the acoustic–gravity wave propagation model is a linear time-invariant system, which means the relationship between inputs and outputs is constant over time. Henneking explains, “The ground shakes at time t, and in the seconds that follow, the pressure waves respond to that shaking. The same response would be observed if the system were to shift in time.” This invariance means the motion- to-sensors mapping can be computed in advance to reduce the number of wave equation solutions required.
Moreover, once precomputed, this mapping can be performed quickly with a solver that applies a fast Fourier transform to the wave equation over time. This transformation leverages mathematical structure to reuse certain calculations and is much faster than trying to solve the full wave equations in real time.
These advanced mathematical algorithms prepare major portions of the tsunami digital twin for a real event. Thanks to the heavy-duty offline phase, the online phase accomplishes in a fraction of a second what would otherwise take an HPC cluster 50 years to compute with a single- point model. “Real-time prediction with full-physics models is out of the question with conventional methods,” adds Henneking. “However, with our framework, we don’t need to compute the underlying wave propagation model when an earthquake occurs—the computation is already done.” Decision makers and emergency responders in seismically active coastal areas can take advantage of the framework’s real-time tsunami predictions on a laptop or, for a large number of forecast locations, a small computing cluster. When an earthquake occurs, pressure data acquired by the seafloor sensors can be rapidly assimilated using the precomputed information, enabling accurate real-time tsunami forecasting.
Application Engine
Alongside UT’s custom algorithms, the tsunami framework relies on the open- source MFEM (Modular Finite Element Methods) software library developed at Livermore. MFEM provides powerful discretization algorithms for large-scale scientific applications. Breaking down a simulation into separate mathematical elements allows them to be calculated across a supercomputer’s processors, improving computational throughput. Key MFEM innovations, such as finite element partial assembly and matrix- free algorithms, help application codes reduce memory usage while maximizing efficiency.
Using MFEM, the research team discretized the acoustic–gravity wave equations, solved the wave propagation calculations on the 3D high-resolution ocean mesh, and ran the simulation on HPC systems with graphics processing unit (GPU)–based architectures, including the Laboratory’s exascale El Capitan supercomputer. MFEM tackled the most computationally demanding steps of the framework’s offline phase. These capabilities arose from multiple investments in MFEM’s development over its two-decade history. The project has been continuously supported by the Department of Energy’s (DOE’s) Office of Science, the National Nuclear Security Administration’s Advanced Simulation and Computing Program, and Livermore’s Laboratory Directed Research and Development Program.
Long before the tsunami research, Livermore’s MFEM team prepared the software for a new generation of HPC systems as part of DOE’s Exascale Computing Project (ECP). (See S&TR, February 2021, The Exascale Software Portfolio.) During ECP’s 8-year tenure, the MFEM team reworked the discretization algorithms to support the GPU paradigm. DOE’s three exascale systems were not yet built when ECP kicked off, and much of the MFEM work involved software portability to existing and future types of GPUs. As ECP concluded, El Capitan—capable of more than 2 quintillion operations per second— was deployed, at the time topping the list of the world’s most powerful HPC systems. (See S&TR, December 2024, Introducing El Capitan.)
The MFEM library is a proven asset for many of Livermore’s national security applications, and prospective collaborators are attracted to its robustness. The software has been open source for 16 years, which means MFEM can reach new users as well as benefit from outside contributions (see S&TR, January/February 2018, Ambassadors of Code). “We have the building blocks to describe many physics processes including the wave propagation model needed in the tsunami application,” states Livermore computational mathematician and MFEM project lead Tzanio Kolev. “The team and expertise we’ve gathered over the years made it easy to join the tsunami forecasting project.” (See the box below.)
Team Cascadia
Research worthy of the Gordon Bell Prize can take years to coalesce. Stefan Henneking and Omar Ghattas began working on the tsunami forecasting problem in 2021, bringing in University of Texas at Austin (UT) colleagues Sreeram Venkat and Milinda Fernando. The group experimented with new mathematical methods and various problem sizes, eventually scaling their nascent framework to run on petascale supercomputers capable of 1 quadrillion operations per second.
Meanwhile, development of Livermore’s MFEM (Modular Finite Element Methods) software had been progressing toward exascale computing portability for years. Henneking contributed to this effort as a Laboratory intern—an experience that introduced him to MFEM’s capabilities and the Livermore team including John Camier, Veselin Dobrev, and Tzanio Kolev from the Laboratory’s Center for Applied Scientific Computing. MFEM became the discretization engine underpinning the tsunami prediction framework.
In 2024, the UT group connected with seismologist Alice-Agnes Gabriel from the University of California, San Diego’s Scripps Institution of Oceanography. She provided earthquake models to complement the tsunami framework, and the team turned their attention to the Cascadia subduction zone. Henneking explains, “We had this zone in mind because of its proximity to the shoreline, providing very little time to warn residents in the event of a tsunami. With Alice’s contribution, we transitioned to realistic earthquake scenarios based on dynamic rupture models for Cascadia.”
These efforts converged at the 2024 International Conference for High Performance Computing, Networking, Storage, and Analysis (SC24), where Henneking ran into Kolev and described efforts to scale the tsunami early warning model. Kolev suggested using El Capitan, which had just been benchmarked as the world’s fastest HPC system. Like many of Livermore’s supercomputers, the newly sited El Capitan was temporarily available for unclassified projects before it began supporting stockpile modernization work.
The MFEM experts optimized the way their software executed operations on El Capitan’s processors, which enabled the offline phase of the tsunami framework to compute simulations with tens of trillions of unknowns. The record-breaking simulation run took place in the early hours of March 2, 2025—scaling to 55.5 trillion degrees of freedom on the full El Capitan system. Calling themselves Team Cascadia, the eight researchers submitted their paper to the Gordon Bell Prize committee a month later.
In addition to the Gordon Bell Prize, the team received two industry accolades in 2025: HPCwire’s Reader’s Choice Award for “Best Use of HPC in Physical Sciences” and Hyperion Research’s HPC Innovation Excellence award. A video of the digital twin was also featured in the SC25 Art of HPC exhibit, which showcased creative visualizations.
Breaking It Down
Physics and math go hand-in-hand. Finite elements provide numerical approximations of solutions to partial differential equations (PDEs), which are fundamental to simulating physical phenomena that vary in space and time. These techniques subdivide physical domains into computational meshes, which are collections of small shapes called mesh elements. This discretization allows continuous physical fields to be approximated and computed in parallel rather than serially. Livermore’s MFEM software delivers the infrastructure and advanced algorithms for these operations, so application codes can focus instead on underlying physics and problem parameters.
Basic, or low-order, finite elements are computationally cheaper per element, but more of them are usually needed to achieve a given accuracy. High-order finite elements can depict curved and other complex geometries with better accuracy but are more computationally expensive. Although MFEM supports arbitrary orders of finite elements, the software especially shines at high orders on unstructured meshes composed of different element shapes such as triangles, quadrilaterals, tetrahedra, and hexahedra.
Unstructured meshes allow irregularity and flexibility within a simulation. The seafloor does not lie on, and cannot be mapped to, a perfect grid, so the tsunami framework leverages an unstructured, hexahedral mesh of contours and depth measurements. In such large-scale scientific simulations, the mesh is a game changer. Distributing the computation of mesh elements across a supercomputer’s nodes and computing them simultaneously make finer meshes and more detailed models practical, which in turn produce results with higher fidelity to the physics being simulated.
Scaling It Up
The tsunami framework showed promise early. The UT and Scripps researchers demonstrated model computations on a mesh with 9.4 billion elements on the petascale Perlmutter supercomputer, housed at the National Energy Research Scientific Computing Center. To solve the acoustic–gravity wave propagation model at higher resolution for big subduction zones (such as Cascadia) in the offline phase, they needed to scale the finite element mesh to an even larger size—a challenge, but not an impossibility, for the Laboratory.
MFEM’s high-order finite element algorithms are designed for modern supercomputers, keeping processors busy while reducing unnecessary data movement. When a simulation runs, the PDEs are discretized on a mesh, equations are solved in parallel, and each iteration updates the overall solution. MFEM manages these tasks regardless of computing architecture.
Crucially, the type of processor dictates how such operations are performed. On HPC systems with central processing units (CPUs), a small number of cores in each CPU tackle large chunks of work, whereas on GPU-accelerated systems the same work is divided into smaller chunks that are processed by thousands of cores. Leveraging a GPU’s simpler cores for parallelization provides significant simulation speedups, but porting complex scientific applications from CPUs to GPUs is often a massive undertaking. MFEM eases this transition by incorporating GPU support into its algorithms—an effort that began with ECP’s Center for Efficient Exascale Discretizations, which Kolev led.
Although MFEM already supported GPUs when the tsunami collaboration began, the team sought additional optimization opportunities, especially because it was eyeing the Laboratory’s El Capitan supercomputer. “To scale the application to a bigger problem size and to squeeze as much performance out of the machine, we explored ways for the finite element computations to go faster and use less memory,” says Kolev.
First, the MFEM team optimized how the software handles kernels, which are specialized functions that process data-intensive tasks across GPUs. Computational mathematician Veselin Dobrev explains, “We looked closely at each kernel to see if we could enhance an aspect of its execution pipeline or combine kernels. We discovered some operations could be improved by more carefully managing data allocations or eliminating certain operations.”
The second optimization effort targeted how MFEM manages application data in memory. For example, data from meshing and other computations can impede performance if transferred inefficiently between different types of memory or moved unnecessarily. El Capitan’s unified memory architecture lets CPUs and GPUs share the same physical memory within devices called accelerated processing units, giving the team flexibility in how memory is allocated and used across compute units. Furthermore, MFEM took on memory tasks originally performed by the application, reducing runtime. Computational mathemetician John Camier adds, “We were able to remove some of the objects that normally remain in memory.”
Stress Test
With MFEM primed for GPU processing, the team wanted to leverage exascale computing power as well. The researchers took advantage of a window of opportunity in early 2025, before El Capitan was restricted to stockpile modernization and other national security projects, and prepared practice runs on Livermore’s petascale system Tuolumne. This effort would demonstrate El Capitan’s capabilities, analogous to testing a new racecar engine on an open road. Kolev remembers, “We never know what to expect with a new machine. We were some of the first users on El Capitan. Livermore Computing’s system administrators supported this work tremendously, even at night and on the weekends.”
Scaling the simulation to the full system took many iterations. “We usually run on a small number of GPUs at first, then increase the numbers and see the scaling get worse. The system incurs a performance penalty as GPUs increase,” explains Dobrev. If just one GPU failed, the whole simulation had to be restarted. Camier recalls, “El Capitan was new and at the edge of HPC capabilities. A drop in performance could happen at any number of GPUs.” Run after run, the team watched the scaling metrics creep upward. Their first simulation run specified 340 GPUs, then doubled in each subsequent run until reaching 43,520 GPUs, or a 128-fold increase over the first run.
The GPU scaling trajectory does not mean the simulation itself runs 128 times faster. The performance results were still impressive, with a 100.9-times speedup or 79 percent parallel efficiency over the smallest run. (Reaching 100 percent is highly unlikely because of the computational costs associated with inter-processor communication, load balancing, and other factors.)
Mesh Test
To address the limitation of problem size, the team pushed MFEM’s functionality to an unprecedented scale. The GPU optimizations enabled finer mesh resolution for more realistic simulation and efficient memory storage for 3D hexahedral meshes. A problem’s size and quality are measured by its mesh size and finite element degrees of freedom (DOFs). Within the elements of a mesh, physical variables such as velocity and pressure are represented by DOFs. The more DOFs, the higher the spatial resolution and detail.
The simulation’s strong scaling run involved 434 billion DOFs, but this was not the largest possible problem size. “Four hundred billion DOFs is one of the largest finite element problems the mathematics community has tried to solve. With the new MFEM kernel and memory optimizations, we can perform 100,000 time steps on El Capitan in less than five minutes, which is unprecedented, mind-boggling performance,” states Kolev.
Incrementally increasing the GPU allocation again, the team investigated how large the mesh could scale with the full complement of 43,520 GPUs. It ultimately generated an unstructured mesh with 55.5 trillion DOFs, which is the largest known finite element simulation of its kind. Camier says, “All of the work carried out since the beginning of the MFEM project culminated here.”
Beyond Cascadia
In pursuit of disaster preparedness, the team’s research advances many fields—from seismology and oceanography to HPC and computational mathematics. Collaboration paved the way. “We show that our tsunami framework works as intended for a problem of this scale, and it can actually make accurate, high-fidelity forecasts for tsunamis in very short order,” Henneking states. “No one team member could have done this on their own.”
The UT and Scripps researchers are refining the framework’s predictions by modeling 175 sensors instead of 600—doing more with less—as well as analyzing alternate sensor placements so decision makers can have flexibility when launching a sensor network and early warning system. Additional exploration in progress includes extending and applying these methods to other regions, and the team is currently modeling the 2011 Tōhoku, Japan, earthquake that resulted in the Fukushima disaster.
Beyond tsunami forecasting, Henneking says, “The framework is broadly applicable to several problems of interest to DOE, many of which are also impactful to society.” For example, the wave propagation solution could be adapted to problems that involve electromagnetics, acoustics, subsurface hazards, or other types of autonomous systems.
As for MFEM, the Livermore group is already incorporating the GPU optimizations into their source code so other applications can take advantage of them. Kolev notes, “The Cascadia collaboration allowed us to identify and implement enhancements that will directly benefit the performance of the Laboratory’s MFEM-based proprietary codes running on El Capitan.”
—Holly Auten
For further information contact Tzanio Kolev (925) 423-9797 (kolev1 [at] llnl.gov (kolev1[at]llnl[dot]gov)).
