WHEN Dawn, Lawrence Livermore’s latest supercomputer, was installed in 2009, scientists at the Institute for Scientific Computing Research (ISCR) were among the first to put it to work. In doing so, they demonstrated an innovative parallelization strategy for simulating at unprecedented resolution and scale the hot, dense plasmas that will occur during fast ignition, the implosion–explosion process to ignite a fusion reaction, at the National Ignition Facility. Their strategy helped solve one of the most difficult scaling problems in numerical simulation: the efficient parallel calculation of long-range interactions. Long-range forces (such as electrostatic or gravitational) are relevant to a variety of modeling scenarios. Previous efforts to develop a fully scalable solution to this complex calculation have failed on less powerful machines.
ISCR’s mission is to assemble and maintain multidisciplinary teams of researchers who work with Livermore project specialists in designing computer applications that address program needs. The institute also collaborates with students and other guests to advance computational capabilities. (See the box below.) Teams choose test-bed projects to push the envelope of application performance on today’s supercomputers. Through this work, ISCR is creating a scientific computing capability that supports the Laboratory’s missions and is developing the technology and expertise needed to effectively utilize next-generation computers, which are expected to expand from thousands to millions of processors.
Computers keep getting larger and more powerful, and Livermore excels at maximizing their potential. “Big computers do not arrive as turnkey devices,” says ISCR Director Fred Streitz, who also leads the modeling and simulations group in Livermore’s Physical and Life Sciences Directorate. “To use these machines to their utmost, exploring more complex problems with ever-increasing resolution, scientists must rethink old approaches and devise new algorithms.”
Streitz is quick to credit his team of computational specialists for the institute’s success. He lauds team leader Jim Glosli and team members Bor Chan, Milo Dorr, Erik Draeger, Jean-Luc Fattebert, Liam Krauss, David Richards, Tom Spelce, and Michael Surh for their ingenuity and dedication in completing the fast-ignition simulation. “Computers run 24/7,” says Streitz, “and the researchers operating these machines sometimes must do nearly the same.”
Glosli, Richards, and Streitz are veterans of a previous campaign, which ran for six continuous weeks and used the entire BlueGene/L supercomputer—more than 200,000 processors. In that groundbreaking simulation, they modeled up to 62.5 billion atoms of liquid aluminum flowing across liquid copper. For their effort, they were honored with the 2007 Gordon Bell Prize and with the Laboratory Director’s Science and Technology Award in 2008.
Between them, ISCR team members have earned eight Gordon Bell prizes over the past five years. Named for C. Gordon Bell, one of the founders of supercomputing, this prize annually recognizes outstanding achievement in high-performance computing, with an emphasis on rewarding innovative science applications. Bell established the prize in 1987 to encourage the further development of parallel processing, the computer design philosophy that has driven high-performance computing since the 1980s.
A New Supercomputing Era
Sequoia will process calculations designed to build more accurate physical models of nuclear weapon detonations and will strengthen predictive capabilities by running very large suites of complex simulations. This work is a cornerstone of the National Nuclear Security Administration’s Stockpile Stewardship Program to ensure the safety, security, and effectiveness of the U.S. nuclear weapons stockpile without underground testing.
Improving Traffic Flow on the Processors
Impossible to some, but not for the ISCR team. In analyzing the problem, the researchers noted that the Coulomb solver has two main computational pieces that, although largely independent, have different scaling behaviors. This observation led them to develop a heterogeneous decomposition process so that scientists can flexibly map, or “tune,” the computational pieces to subsets of the hardware. The tunable approach provided excellent scaling of the Coulomb problem to thousands of processors. As a result, computational experts can for the first time incorporate long-range physics into extremely large-scale simulations.
Think of driving down a winding, two-lane road. No matter how fast you want to drive, the slowest car determines the speed for all cars in your lane. Computer scientists faced this problem in extending sequential applications to calculate complex interactions. Solving that problem led to parallel computing, which is like adding more lanes to the highway.
Even with today’s massively parallel technology, the analogy still applies. “Using a single strategy to parallelize a simulation limits overall scalability because the least scalable component still holds back the entire code,” says Glosli. “In our analogy, it’s like having a truck in every lane.” By assigning parts of a calculation to different processors—in essence, limiting trucks to just a few lanes—the team developed an approach for parallelization that allows multiple force terms to be computed concurrently. The overall calculation now scales effectively across hundreds of thousands of processors, maximizing the power of Dawn. The team was recognized as a 2009 Gordon Bell Prize finalist for developing this strategy.
“The heterogeneous decomposition of the computational problem and optimal mapping to hardware has far-reaching implications for scientific computing,” notes Glosli. “It likely will affect the way future computer codes are developed for massively parallel environments.” The flexibility of this approach allows more complicated models to be developed, and the technique can be applied on current and next-generation machines. The team has also developed methods to include shorter-range physical processes, such as radiation, recombination, ionization, and fusion, in the code. Says Streitz, “Our goal for this project is to deliver a comprehensive simulation tool for computing correlations and transport properties in burning plasma.”
Key Words: Dawn, Gordon Bell Prize, Institute for Scientific Computing Research (ISCR), National Ignition Facility, plasma simulation, Sequoia, Summer Visitor Program.
For further information contact Fred Streitz (925) 423-3236 (firstname.lastname@example.org).
Lawrence Livermore National Laboratory
Privacy & Legal Notice | UCRL-TR-52000-10-9 | September 9, 2010