View the LLNL home Back to the S&TR home Subscribe to Our magazine Send us your comments Browse through our index






Privacy &
Legal Notice

May 2002

The Laboratory
in the News

Commentary by
Jeff Wadsworth

Building a Virtual Telescope

A New Understanding of Soft Materials

At Livermore, Audacious Physics Has Thrived for
50 years





Building a Virtual Telescope

The Egyptian god Djehuty was the guide to heaven, earth, and the netherworld;
lord of calculation, wisdom, and judgment; and protector of knowledge, mathematics,
and science. His image is seen in many hieroglyphic tablets.


IN the Egyptian pantheon, Djehuty was the guide to heaven, earth, and the netherworld; lord of calculation, wisdom, and judgment; and protector of knowledge, mathematics, and science. It seemed appropriate, then, for Lawrence Livermore astrophysicists David Dearborn and Peter Eggleton to take his name for their breakthrough three-dimensional code that simulates the evolution and structure of stars.
The physical processes of stars have long been of interest to Livermore researchers because understanding the prime mechanism of stellar energy— thermonuclear fusion—is part of the Laboratory’s national security mission. “Stars are high-energy-density ovens,” says Dearborn. “Several Laboratory programs are interested in the properties of stars, and many Livermore physicists have backgrounds in astrophysics.”
Dearborn points out that stars provide the standards of reference for measuring the size, age, chemical composition, and evolution of the universe. Stars have also been used as physics laboratories that strengthen our understanding of complex physical processes. For example, they have been used to better understand the properties of hot plasmas as well as fundamental particles such as neutrinos. Stars have also been used to suggest the properties of exotic particles such as axions, which have been proposed to explain why the universe contains more matter than antimatter.
Eggleton notes that scientific knowledge of stars may appear to be mature, but in fact, much of what we know about stars—especially the way they generate energy and how they evolve from a dust cloud to a supernova or red giant—may well be significantly incomplete. “We need to improve our knowledge about stars,” he says.
The reason for the imperfect understanding is that many stellar processes are complex, three-dimensional phenomena that have been modeled only in coarse approximation using one-dimensional computer codes. For example, the transport of energy through a star by convection from its superhot core is a three-dimensional process, which limits the value of one-dimensional calculations, even for perfectly spherical stars. (See the box below.) Although a one-dimensional convection simulation could be inaccurate by only 10 percent at any moment in time, such “small” errors can easily accumulate over time. The result might be a final discrepancy of 100 to 200 percent in some properties calculated for such stellar objects as Cepheids, which are large, pulsating stars often used to calculate the distance scale of the local universe.

Probing the Interiors of Stars

Stars, unlike planets, produce their own energy and do so by thermonuclear fusion. Much of the complexity underlying the computer code Djehuty, Livermore’s three-dimensional code for star structure and evolution, is its realistic simulation of fusion, which converts hydrogen nuclei into helium ions. The process is often called hydrogen burning and is responsible for a star’s luminosity.
Fusion reactions occur in the core, the innermost part of the star. In a star about the size of our Sun, the hydrogen fuel is eventually consumed after billions of years. The core slowly starts to collapse to become a white dwarf while the envelope expands to become a red giant. Our Sun will reach this stage in about 5 billion years.
In contrast, the core of a star larger than the Sun is driven by a complex carbon–nitrogen–oxygen cycle that converts hydrogen to helium. In these massive stars’ cores, hot gases rise toward the surface, and cool gases fall back in a circulatory pattern known as convection. After depleting its hydrogen—and subsequently its helium, carbon, and oxygen—the contracting core of a massive star becomes unstable and implodes while the other layers explode as a supernova. The imploding core may first become a neutron star and, later, a pulsar or black hole.
The cores of stars are turbulent in a manner analogous to a boiling kettle, says Livermore astrophysicist Peter Eggleton. Driven by enormous heat, the material in a core takes about a month to completely circulate (our Sun accomplishes it in about two weeks). “One-dimensional simulations give you an average of what’s going on in the kettle instead of telling you what’s happening on a second-to-second basis, so we are forced to make some bold assumptions.” Eggleton also says that one-dimensional codes cannot model time-dependent convection in such events as helium flashes, which occur in the late stages of a red giant star.
One of the long-standing issues of astrophysics has been determining the correct convective core size of stars. Astronomical observations have suggested that the convection region is larger than has been assumed since the 19th century. Astronomers call the situation convective-core overshoot, meaning that the core probably extends beyond the long-accepted boundary.
Determining the exact size of the convective core is of more than passing interest. If the core is indeed larger than has been assumed, then stars could be much older than has been believed, which has profound implications
for how the universe evolved and its real age. “The

igniting star

When low-mass stars such as our Sun become red giants, they grow a helium core. Eventually the helium core ignites and begins burning to carbon and oxygen. The ignition begins in a shell that initially expands and drives a weak shock into and out of the star. The image shows the velocity contours of the expanding shell in a cutaway segment of a star in which ignition is beginning. The red areas represent the highest velocity, corresponding to the rapidly expanding shell both in front and in back (barely visible).

modeling of convection is one of the weakest points in our understanding of stellar structure and evolution,” says Livermore astrophysicist David Dearborn.
The issue over the size of the convection region is serving as a way to verify and validate the accuracy of Djehuty. The code development team made convective core overshoot a priority in part because the fusion process occurs during the earliest and simplest phase of stellar evolution—during what is called the main sequence. The main sequence is shown on a Hertzsprung–Russell diagram, which plots stars’ temperatures versus their brightness, thereby showing their evolution.

simulations of core of a star

Two simulations taken about 8 minutes apart show the changes inside the core of a star four times the mass of our Sun. Colors represent relative velocity (increasing from blue to yellow), and the arrows show the direction of convective currents.

“Observations assure us that our best one-dimensional approximations of convection are flawed,” says Eggleton. “With Djehuty, we have a three-dimensional code with accurate physics to determine what exactly happens in the core. There are big rivers flowing in stars’ cores, and we want to follow them.”
One simulation modeled a star early in its evolution, prior to its joining the main sequence. As expected, it did not show any convection motions from thermonuclear fusion. Another simulation studied a massive star that had just reached the main sequence and so witnessed the onset of convective motion from fusion. A third simulation looked at a red giant, a very old star that possesses a large core of helium. The helium eventually ignites in what is called a helium flash.
The simulations suggested that a star’s convective core indeed exceeds its classical boundary. Additional computationally intense simulations, each requiring a month of supercomputer time, will be done this year to model a star’s convective core at key stages in its lifetime.

diagram of star temperature vs. brightness

The Hertzsprung–Russell diagram plots the temperatures of stars versus their brightness and is useful for plotting their evolution. This diagram follows a star with six times the mass of our Sun. The star spends most of its lifetime in the main sequence, characterized by producing fusion in its inner core. Djehuty simulations are modeling stars in every phase of their evolution.

Need for 3D Codes
Convection is only one of many stellar phenomena that require a three-dimensional simulation code for accurate modeling. Other complex phenomena that astrophysicists have long desired to simulate include the evolution of elements created in a star, the preexplosion structure of supernovas, and the physics of binary stars, which comprise nearly half of the visible mass of the universe.
Dearborn says that developing a three-dimensional code to realistically model stars is challenging for even the most accomplished teams of computer scientists and astrophysicists. Before Djehuty, three-dimensional stellar models were limited to about 1 million zones. (Computer simulations divide an object into numerous small cells, or zones, whose behavior is governed by sets of physics equations. The totality of the zones, or cells, is called a mesh.) The million zones represent only modest segments of a star. Moreover, the simplified models did not incorporate all the physics pertinent to a star’s core where nuclear energy is produced, and they did not simulate gravity in a realistic manner. “While the earlier codes are important starts toward improving our understanding, it is clear that the solutions to some problems necessitate whole-star modeling,” Eggleton says.
The advent of massively parallel computing, wherein computers have hundreds and even thousands of processors, and Livermore’s participation in the National Nuclear Security Administration’s Stockpile Stewardship Program—to assure the safety and reliability of the nation’s nuclear stockpile—led Livermore scientists to gain expertise in supercomputers and parallel codes. Along with astrophysicist Kem Cook, Dearborn and Eggleton saw that Livermore was becoming a uniquely qualified institution to move the calculation of stellar properties to a higher level of understanding. In particular, they saw that one element of stockpile stewardship, which uses massively parallel computing techniques to simulate the performance of nuclear warheads and bombs in a program called Advanced Simulation and Computing (ASCI), would be pertinent to their quest for a whole-star, three-dimensional model.
Dearborn and Eggleton’s vision was to take advantage of Livermore’s expertise in ASCI computations, code and algorithm development for massively parallel computers, astrophysics, high-energy-density physical data and processes, and experience in interdisciplinary coordination to attack the fundamental questions of stellar structure and evolution.

four views of simulated convection in a star Increasingly magnified sections of a star with four times the mass of the Sun can be seen in these Djehuty simulations. Here, (a) and (c) are the same as (b) and (d), respectively, but show the location of mesh zones. A closeup of the star’s convective core is shown in (e). Colors represent relative velocity (increasing from blue to yellow). The bulk of motion lies in the core, where convection currents driven by carbon–nitrogen–oxygen burning occur. The areas of convection appear to extend beyond what one-dimensional models depict, but Djehuty’s models are consistent with recent astronomical observations. (f) A two-dimensional slice of a Djehuty three-dimensional simulation depicting convection currents deep inside the core. The arrows signify the directions of the currents.
two views of simulated convection in a star

A Laboratory-Wide Team
In 1999, Dearborn and Eggleton assembled a team to develop Djehuty as a three-year Strategic Initiative under Laboratory Directed Research and Development funding. The collaboration has included John Castor, Steven Murray, and Grant Bazan from the Defense and Nuclear Technologies Directorate; Kem Cook from the Physics and Advanced Technologies Directorate; Don Dossa and Peter Eltgroth from the Computation Directorate’s Center for Applied Scientific Computing; and several other contributors. “Collaboration from throughout the Laboratory has been essential in this project,” says Dearborn.
The team designed Djehuty to operate on massively parallel machines with the best available physical data about stars and with algorithms tailored specifically for the massively parallel environment. Notes Dearborn, “There’s been tremendous work at the Laboratory in developing parallel codes and learning how to do calculations in a manner that won’t bog down the machines.” The code development process involved assembling and reconfiguring a number of Livermore codes that already existed, many of them parts of unclassified software belonging to the ASCI program, and optimizing them for astrophysical simulations.
Djehuty also takes advantage of the Laboratory’s significant knowledge about opacity (a measure of the distance photons at a particular frequency travel through a particular material) and equations of state (the relationship between a material’s pressure, temperature, and volume). Opacity and equation of state are two key pieces of data that are used in stockpile stewardship work for studying matter under extreme conditions. In that respect, says Dearborn, developing Djehuty is well aligned with Livermore’s programmatic interests that focus on understanding high-temperature physics and performing numerical simulations of complex physical reactions.
The code currently features accurate representations of different elements’ equations of state, opacities, radiative diffusion transport (how photons are absorbed and reemitted when they interact with atoms and electrons in a star’s interior), and nuclear reaction network (fusion reaction rates and abundance of species formed). Finally, Djehuty features a gravity package for spherical stars, a provision that is being improved significantly so it will be possible to simulate a host of aspherical stellar objects.

some members of Djehuty team

Some postdoctoral scientists and the project leaders on the Djehuty development team. From left, Rob Cavallo, Stefan Keller, team leaders Peter Eggleton and David Dearborn, and Sylvain Turcotte.

The First Simulation
The team’s early strategy was to test the code’s accuracy and achieve some optimization of it. In September 2000, using the 680-gigaops (billion calculations per second) TeraCluster 2000 (TC2K) parallel supercomputer at Livermore, the team successfully executed a three-dimensional simulation of a star. This was the first three-dimensional simulation of an entire star, but it ran on just one of TC2K’s 512 processors, using only some of the code’s physics on a modest mesh containing approximately 400,000 cubic zones. “Our first models were too small to accurately represent a star’s structure, but they were sufficient to study different zone mesh structures and to optimize the physics equations we were using,” says Dearborn.
Satisfied with the early simulations on one processor, the team then modified the code to run in a massively parallel computing environment. “It’s a big transition going from one to many processors because we need at least 10 million zones to model an entire star,” says Dearborn. Fortunately, he says, Livermore has invested significant resources to figure out how to break up a complex physics problem, such as following fusion reactions in time, for efficient processing by hundreds and even thousands of processors.
Generating and monitoring large three-dimensional meshes containing millions of zones is a huge task. To aid computing, the Djehuty team constructs a mesh sphere of seven blocks: one in the center and six surrounding it. The outlying six are distorted at their outer edges to make them spherical. Each block contains at least 1 million zones. Each zone represents thousands of kilometers on a side, and several thousand zones are assigned to a processor. All the processors must communicate efficiently with each other simultaneously. The key to Djehuty’s simulation power is its ability to access many processors to efficiently compute the physics in each of the millions of zones. “We’re fortunate to have so many people who can develop a code like this,” says Dearborn.

The team has run simulations on increasing numbers of processors on the TC2K. Several simulations, using 128 processors and 56-million-zone meshes, were some of the largest astrophysics calculations ever performed; they generated close to a terabyte (trillion bytes) of data. The team has also begun to perform simulations on Livermore’s ASCI Frost, the unclassified portion of ASCI White, currently the world’s most powerful supercomputer. Simulations on ASCI Frost have used 128 of that machine’s processors to evolve stars with 60-million-zone meshes.
With the code running satisfactorily in a massively parallel environment, Dearborn and Eggleton focused on resolving a long-standing controversy in astrophysics. That controversy surrounds the discrepancy between the results from one-dimensional stellar models and data gained from astronomical observations concerning the size of the convection region inside a star. (See the box above.) This region is where hot plumes of gas rise and fall. The team has simulated the cores of several stars, ranging from young stars before the onset of fusion reactions to old stars about half the age of the universe. Eggleton says that one-dimensional computer models are especially incomplete in simulating late stellar evolution, which is often characterized by deep mixing of gases and sudden pulses of energy.

Virtual Telescope at Work
Eggleton compares Djehuty to a kind of virtual telescope that can take snapshots during a star’s lifetime of several billion years and examine in detail the star’s structure and the various physical processes at play. “There is no comparable three-dimensional code, although there have been heroic efforts to develop one,” he says. As a result of the early simulations, the Livermore team anticipates being able to accurately model in three dimensions, for the first time, a host of important stellar objects. For example, Djehuty will be vital to understanding supernovas, the brightest objects in the universe, and about which much is unknown, as well as Cepheids.
Dearborn predicts that Djehuty will provide an important link between theory and observation that will further our knowledge of stellar structure and evolution. Livermore’s Stefan Keller is conducting a number of observational studies to verify the Dhejuty simulations. One study uses a certain population of Cepheids to observationally determine the relationship between mass and luminosity that is dependent on the original amount of mixing in the star’s convective core. Preliminary results indicate that these Cepheids are considerably more luminous than predicted by standard one-dimensional models, a result suggesting a larger degree of mixing than was previously thought. Djehuty simulations appear to confirm the observations.
In another study, astrophysicist Rob Cavallo is observing variations in the surface abundances of some elements in evolved red giant stars. The variations are caused by some form of nonconvective mixing process, which can only be determined with the use of a fully three-dimentional code such as Djehuty.
The team is also working to improve the code and better interpret its output. One goal is improving the accuracy of opacities. “There are a range of problems where a star’s behavior depends on the opacity of material whose composition is rapidly changing,” says Dearborn. The team plans to attack those problems by permitting the code to generate opacity levels using OPAL, a database of stellar opacity that was developed at Livermore several years ago. (See S&TR, April 1999, Duplicating the Plasmas of Distant Stars.)
Another task is improving the techniques to better visualize and thereby understand the vast amounts of data generated by Djehuty. Analysis and visualization are the key for turning huge numerical simulations into scientific understanding, says Dearborn, and at present, “We must improve our ability to analyze three-dimensional structures. With longer, larger, and more realistic simulations, we must develop better tools to analyze our simulations to extract the greatest amount of information. We can’t eyeball 10 million zones in three dimensions. We must have ways for a computer to look for irregularities and flag them.”
Recently, the team began using MeshTV, a program that was designed at Livermore to visualize data for three-dimensional meshes. MeshTV can display an animation of data changing over time and permit a user to rotate, zoom, or pan an object while a movie assembled from the data is playing. (See S&TR, October 2000, The Many Faces of Carbon Dioxide.)

A Continual Code Development
Djehuty development will never be finished, although it will eventually become much less a development code and more a production code ready for use. The team continues to enhance Djehuty’s physics and refine its algorithms. Development is also under way to permit simulation of rapidly rotating stars and, in particular, binaries. Binary stars revolve around a common center of gravity and sometimes exchange some of their mass or even merge into one star. Often, one binary is distorted by the gravitational pull of the other, and the result is seen in varying brightness.
“Simulating binaries has become our main physics priority,” says Dearborn. “We want to see how mass comes off one star and is absorbed by the other.” One-dimensional codes don’t work for binaries because when two stars interact, the problem is three-dimensional.
Binary simulations require a more accurate means to simulate gravity, one that automatically changes to reflect a star’s size, shape, and internal physics. Once this enhanced gravity treatment is incorporated into Djehuty, the code will be able to represent binaries as well as stellar objects that are not perfectly spherical. “Once work on binaries begins,” says Dearborn, “we will enter completely new territory because calculations so far have been very crude.”
The Livermore effort to revolutionize stellar evolution and modeling calculations has been well received at two international conferences. The enthusiasm generated by this work has led to two proposals to the National Aeronautics and Space Administration from U.S. academic researchers interested in collaborating with the Djehuty team on binary star evolution. Other researchers have proposed using the code to study white dwarfs, the phase of stellar evolution that occurs late in stars’ lifetimes, depending upon their starting masses. Dearborn and Eggleton have also received inquiries about the possibility of modifying the code to run simulations of large planets and brown dwarfs.
Several postdoctoral scientists and university students have joined the Djehuty development team. With a user manual recently completed, the team is seeking university collaborators, both graduate students and visiting scientists, who would visit for several months at a time and join in astrophysical research that can be done nowhere else.
Dearborn and Eggleton hope to see a user facility established at the Livermore branch of the University of California’s Institute of Geophysics and Planetary Physics (IGPP). The Livermore IGPP currently collaborates with all UC campuses, more than thirty U.S. universities, and more than twenty international universities. “Djehuty is a unique institutional asset for attracting astronomers and physicists interested in stars and what can be learned from them,” says Eggleton.

—Arnie Heller

Key Words: Advanced Simulation and Computing (ASCI), ASCI Frost, ASCI White, binary stars, brown dwarfs, Cepheids, convective core, Djehuty, helium flash, Hertzsprung–Russell diagram, Institute of Geophysics and Planetary Physics (IGPP), Mesh TV, stellar evolution, supernovas, TeraCluster 2000 (TC2K), white dwarfs.

For further information contact David S. Dearborn (925) 422-7219 (



Printer-friendly version

Back | S&TR Home | LLNL Home | Help | Phone Book | Comments
Site designed and maintained by Kitty Tinsley

Lawrence Livermore National Laboratory
Operated by the University of California for the U.S. Department of Energy

UCRL-52000-02-5 | May 28, 2002