LAWRENCE Livermore scientists using the world's most powerful supercomputers are turning to another computational engine-the human visual system-to make sense of the huge volumes of data produced by the giant computers. Increasingly, advanced three-dimensional visualization tools are seen as key to gaining greater insight from the reams of computer data that can easily overwhelm scientists.|
Developing the new generation of visualization and data management tools is the task of a program called VIEWS (Visual Interactive Environment for Weapons Simulation) that is part of the Laboratory's Computation Directorate. Formed in 1999 by combining several related activities, VIEWS integrates hardware (storage, networking, and visualization) and software (data exploration and management) to help interpret the supercomputer data generated in the Department of Energy's Accelerated Strategic Computing Initiative (ASCI) program. VIEWS researchers also explore innovative visualization approaches such as stereo display goggles and advanced human- computer interfaces.
"See and understand" is the mantra of the 40 VIEWS computer scientists and electronics engineers, according to computer scientist and program leader Terri Quinn. The program's new visualization tools, she says, together open a larger, clearer window into ASCI data that promotes greater understanding by relying on the human eye to discern subtle changes among colors and patterns. In this way, scientists can compare the results of different codes, compare code results to experimental results, and spot errors in new codes.
ASCI was established to develop computer tools to help validate the safety, reliability, and performance of the nation's nuclear stockpile in the absence of nuclear testing. The massively parallel ASCI supercomputers employ thousands of processors that work in unison to generate terabytes (trillions of bytes) of raw data. Advanced visualization techniques allow scientists to track physical parameters such as temperature, pressure, and velocity shake by shake (a shake is 10 nanoseconds) as they follow the detonation of a nuclear device. Seeing events proceed in three dimensions is necessary because component aging and manufacturing variations introduce asymmetries whose effects on performance must be understood.
The VIEWS effort includes collaborations with a number of University of California (UC) campuses, the University of Utah, and the Supercomputer Centers at the University of Illinois and UC-San Diego, as well as colleagues at other DOE national laboratories. Much of the research is conducted at the Laboratory's Center for Applied Scientific Computing, which maintains strong links to the nation's academic community.
Quinn says the work demands close relationships with physicists from the Laboratory's Defense and Nuclear Technologies Directorate, who develop and use ASCI codes. Says Quinn: "Working with 3-D simulation data is new to everyone, and we don't know how best to go about it. We speculate on how best to view 3-D data sets and then we develop solutions. We share our solutions with the physicists to obtain their reactions." She also notes that the group must "run as fast as it can" to meet physicists' demands for faster and easier ways to access the terabytes of ASCI data.
The hardware backbone supporting the advanced visualization tools includes an expanding network of graphics workstations, two visualization servers (called TidalWave and EdgeWater), a separate video delivery network using optical fibers, and accompanying storage systems. All elements are connected to ASCI computers via the Laboratory's classified network of high-speed optical fibers.
Numbers Are Transformed|
The visualization servers free the ASCI supercomputers to do what they do best: sheer number crunching. The servers use dedicated microprocessors, "graphics pipes" (sophisticated graphics cards that support high-end graphics techniques), and several terabytes of disk storage to transform reams of numbers from ASCI runs into graphical objects. The servers are capable of redrawing complex objects interactively many times every second and delivering the resulting imagery to a growing number of scientists' offices and to a 15-projector display wall.
Quinn says that visualization techniques (2-D simulations, graphs and charts, and low-resolution 3-D views) that were developed for earlier generations of computers can't handle the scale and complexity of ASCI calculations. For example, the first-ever 3-D simulation of a nuclear weapon's primary (the first stage of a hydrogen bomb) was completed last November using Livermore's Blue Pacific supercomputer. The simulation used a mesh composed of tens of millions of cells, hundreds of times more than a comparable 2-D simulation. The simulation ran a total of 492 hours on 1,000 processors and used 640,000 megabytes of memory in producing 6 million megabytes of data contained in 50,000 graphics files.
"With 3-D we appeal to the mind's ability to see patterns," says Livermore physicist Jim Rathkopf. He explains that stockpile stewardship involves "a lot of physics happening simultaneously." A major advantage of 3-D visualization is the ability to display several physical parameters, such as pressure, temperature, and density. New methods of visualizing those parameters are being devised to indicate the direction of fluid flow in time and space.
Livermore physicist Steve Langer notes, "People are still learning how to explore the 3-D nature of their data. We're finding out what kinds of visual information are useful and what kinds aren't." Three-dimensional visualization is particularly useful to Langer because he simulates the compression of tiny fusion capsules by the National Ignition Facility, now under construction at Lawrence Livermore as a key stockpile stewardship asset. Advanced visualization permits Langer to follow a simulated imploding capsule in time to determine how spherical it will remain as it is compressed.
Quinn says that terascale data visualization makes unprecedented demands on display resolution and leads to a quest for ever-more pixels. "If you put lots of pixels on a screen, you can more easily pick out small anomalies in performance that could turn out to be critical," she says. "Having more pixels and a method to display them-without choking the rest of the system-is important."|
Pixel is short for picture element, a single point in a computer display of a graphic image. (On color monitors, each pixel is composed of three dots-red, blue, and green-that appear to converge into one.) Computer monitors divide the screen into thousands (or millions) of pixels, arranged in rows and columns. The quality of a display largely depends on its resolution, that is, the number of pixels. A resolution of 1,024 by 768 has nearly 800,000 pixels, a resolution of 1,280 by 1,024 has about 1.3 million pixels, and a resolution of 1,600 by 1,200 has nearly 2 million pixels.
The more pixels, the greater the picture detail, and often, the greater the insight on the part of the user. In response to the need for greater resolution, VIEWS staff have begun an effort to develop "personal" displays that have far more pixels than common computer monitors. They have also built one of the largest interactive displays ever, called the Assessment Theater.
A Wall Projects an Eyeful|
The Assessment Theater is the most striking example of Livermore's new visualization world. Here, scientists follow computer-generated ASCI movies of unprecedented image quality on a 5-meter-wide by 2.5-meter-tall projection screen called a power wall. First installed in December 1998, it was upgraded substantially in 1999.
The power wall currently uses 15 state-of-the-art projectors arranged in a 5 by 3 array to create a large-scale display with great detail and clarity. Each projector is driven by a visualization server to generate an image with a 1,280 by 1,024 resolution, similar to that of a high-end monitor. The overall display has a total resolution of 6,400 by 3,072, or nearly 20 million pixels, giving it a resolution 15 times better than that of a typical desktop display.
The projectors must be kept in sync, particularly in edge-matching each projected image to its neighbor without overlapping pixels or creating separation lines between the tiled projections. Thankfully, the liquid-crystal-display projectors have a single lens that can be adjusted for exact pixel positioning without moving the projector body. Once again, the human eye comes into play because, says computer scientist Randy Frank, "The human eye is particularly adept at detecting image edges and color gradients." The projector's color output, which varies as the projectors' xenon lamps age, must also be properly calibrated. Livermore experts are developing a computerized system to automatically adjust each projector's color output.
Electronics engineer Bob Howe, who leads the deployment of new VIEWS systems, says the power wall is an essential tool for answering the basic question, "What on earth can we learn from all of this data?" He says that by using the power wall, a large group of scientists can collectively view simulations (at rates up to 20 frames per second) and then stop or zoom in on specific features. In this way, many scientists can work together and share their interpretations and insights.
The power wall is often the only way to view a simulation when very fine detail needs to be scrutinized. Simulations are based on meshes of millions of individual cells, and a power wall permits the clear visualization of individual cells. On a typical desktop monitor, individual cells can be visualized, but it's easy to lose sight of what is happening in the tens of thousands of surrounding cells. "Details we see on a power wall simply get lost on a typical computer monitor. To see the context, we need larger displays," says Livermore physicist Tom Adams.
Although scientists are still gaining experience in using the power wall, they are pleased with the results thus far. One physicist had spent several days searching in vain on his desktop monitor to locate an error he knew resided in his newly written code. It didn't take more than a few minutes viewing the simulation on the power wall when he jumped out of his seat and pointed to a section of the display containing a deformed cell. "That's it!" was his exhilarated response.
More Tools for Scientists|
Another power wall, a smaller 4 by 2 array, will be housed in the Visualization Work Center, now under construction and scheduled to debut by late December. While the Assessment Theater's single large room is suitable for team visualizations and for formal presentations, the Visualization Work Center is designed to allow small teams and individual users to work simultaneously. For example, a team of scientists could divide the wall into two 2 by 2 displays to simultaneously compare two simulations. The center will have several rooms with advanced visualization equipment, including arrays of flat-panel displays connected by high-speed optical fibers to the visualization servers. "We want to make advanced visualization tools ever more accessible to scientists," says Rathkopf.
Smaller aggregations of 17-inch, high-resolution (1,280 by 1,024), flat-panel displays in 2 by 1 and even 2 by 2 configurations are being installed in the offices of ASCI scientists. Connected to the visualization servers, the displays provide a new level of resolution for an individual office.
Users have the choice of using the flat-panel displays on a desk or hanging them on the wall. They can choose to have the panels tiled as a mini power wall or have each monitor assigned to different software windows. Still other scientists are opting for more traditional cathode-ray-tube monitors that measure 64 centimeters diagonally and have a resolution of 1,920 by 1,600 (3 million pixels).
Despite the introduction of these new monitors, VIEWS staff are not satisfied with current display technologies. They are working with the Laboratory's Information Science and Technology Program to encourage companies to produce better monitors. The Digital Display Integration project, headed by electronics engineer Norm Bardsley, is evaluating a new generation of flat-panel displays that exhibit more than 5 million full-color pixels. Bardsley says flat-panel displays, with their inherent sharpness, have the best potential for achieving ever higher levels of resolution.
The display project is also working to speed the transition from the traditional analog interface between computer and monitor to the new digital interface standard. Digital interfaces make far more efficient use of flat-panel displays. In like manner, Bardsley and colleagues are working to implement digital connections that will greatly speed data transfer for ASCI applications using the newest optical-fiber technology.
Software Advances to Keep Pace|
Trying to keep pace with the explosive growth of new visualization hardware are software tools that allow scientists to more easily store, retrieve, and search ASCI data. Many of the tools are developed by Livermore's Scientific Data Management (SDM) project. "We want scientists to more efficiently use the power of ASCI machines and quickly zero in on data they want to retrieve for further analysis," says computer scientist and group leader Celeste Matarazzo.
The group's work focuses on the creation and use of metadata, which are data describing the contents or relevance of a computer file. (Matarazzo compares metadata to sticky notes on the outside of a file folder.) In this case, metadata ranges from a scientist's brief notes on a particular simulation to system-level documentation on the size, type, and creation date of a set of data.
SDM project members developed SimTracker, a secure Web-based metadata tool that keeps track of files used in a simulation and summarizes the results. SimTracker automatically generates summaries that provide a quick overview and index to archived simulation results. The summaries provide convenient access to the data and analysis tools, graphical snapshots, links to files, and assorted annotations. SimTracker is used not only at Livermore but also at Sandia and Los Alamos national laboratories and has drawn interest from the Department of Defense and the larger scientific community. The group has also developed Enotes, a metadata editing and browsing tool that works like an online notebook, as a companion to SimTracker.
The workhorse software for visualizing and analyzing ASCI data is a program called MeshTV. The program visualizes data for 3-D meshes ranging in size from hundreds of thousands to hundreds of millions of cells. It can present results from a host of displays varying from a power wall to a desktop monitor. It handles many different mesh types, provides many different ways of viewing the data, and is virtually independent of the hardware being used. Mesh cells are all independent, and one cell can contain multiple types of materials for realistic simulations.
MeshTV provides data comparison through multiple side-by-side graphics windows, overlaying of images, and simultaneous display of data from different files or different data from the same file. It can play an animation of the user's data changing over time. A user can rotate, zoom, or pan an object while the movie is playing.
The program was originally developed in the early 1990s by Livermore computer scientists Jeff Long and Eric Brugger to help scientists analyze data produced by codes much less sophisticated than today's. It was modified to take advantage of the power of ASCI's massively parallel supercomputers. With this modification, typically hundreds to a thousand of the more than 5,000 processors in an ASCI supercomputer work together to display changes in a mesh. The program has also been enhanced to run on a power wall.
New Browser for Terrascale Data|
A VIEWS visualization project headed by Randy Frank is developing even newer ways to explore the volumes of ASCI data. One important tool, the Terascale Browser, made its debut in July. The program can be used on the desktop as well as on a power wall. "No commercial visualization software exists that can handle terabytes of information," says Frank. "Some give up, while others can take hours to redraw."
The Terascale Browser rapidly investigates large data sets by providing an easy-to-navigate overview of a simulation. It is not intended to be a full-featured visualization tool, but rather a system to allow scientists to explore an entire terabyte data set to find areas of interest that they can then pursue in greater detail with full-featured tools like MeshTV. The software creates new images in less than one second by making use of specific features of ASCI hardware. In particular, it uses what Frank terms "terascale technologies," which include speculative techniques that make assumptions about what the user might want to see next.
The group has also produced a software program called X-movie for playing ASCI animations at 20 frames per second (similar to video) on the nearly 20-million-pixel power wall. "Displaying images 20 frames per second is not hard to do if your pixel count is small," says Quinn. "Displaying 20 frames per second when you have 20 million pixels is quite an accomplishment." The program permits users to zoom in and out of selected areas and move forward and backward in time, in addition to providing VCR-like controls; it is ideal for power-wall presentations.
Research in Goggles and Gloves|
Frank's group tests new commercial visualization hardware, from power walls to stereo display goggles, in a laboratory near the newly arrived ASCI White supercomputer (see S&TR, June 2000, New Day Dawns in Supercomputing). The group is currently exploring the effectiveness of goggles that give the illusion of 3-D objects popping out of a computer monitor. The goggles have liquid crystal shutter lenses. In conjunction with specialized software, the goggles alternate between left- and right-eye-specific images on a monitor 120 times a second. With each eye seeing a different perspective, on-screen objects appear to have real depth and texture. "Although this technology is not new, applying it to large scientific simulations is still under investigation. The resolution of these immersive-type displays is still lower than what we would like," says Quinn.
A new generation of projectors might also allow stereoscopic eyewear to be used with a power wall. Livermore experts are also testing the combination of the goggles and "data gloves." The gloves, electronically attached to a workstation, allow the user to reach out and seemingly move and examine the 3-D images.
In a related research effort, Livermore computer scientists are exploring new ways to allow users to interact with their data in a more intuitive manner. For example, power-wall users must currently enter commands using a workstation and a mouse. One promising technique would allow users to stand directly in front of a power wall while a camera tracks and translates their hand gestures into computer commands. A sweep of a hand, for example, might be a signal to play an animation, while a hand held up might be a stop command.
The group is also keeping a watchful eye on the rapid evolution of personal computer technology, particularly graphics cards for PCs. One idea is to use networks of hundreds of PCs to achieve the same performance as a vastly more expensive visualization server. The Laboratory is funding a research group at Stanford University to determine if such an idea makes both practical and economic sense "We're taking advantage of the tremendous growth in the PC game industry and using the same advanced graphics cards game that players use," says Quinn.
Quinn says that both VIEWS developers of new visualization tools and early users among ASCI scientists are true pioneers in the new world of visualization. Both groups have much to learn about how best to present, store, and analyze ASCI data. However, the many successes to date show that Lawrence Livermore researchers are on the right track as they help to assure that the nation's nuclear stockpile remains safe, secure, and reliable.
Key Words: Accelerated Strategic Computing Initiative (ASCI), Assessment Theater, MeshTV, metadata, power wall, SimTracker, Terascale Browser, Visual Interactive Environment for Weapon Simulation (VIEWS), Visualization Work Center.
For more information contact Terri Quinn (925) 423-2385 (firstname.lastname@example.org).
ABOUT THE SCIENTIST
TERRI QUINN is the assistant department head for Scientific Computing and Communications in Livermore's Computation Directorate. She leads the Visual Interactive Environment for Weapons Simulation (VIEWS) program, which is responsible
for research and development, deployment, and operations of scientific visualization and data management systems for the high-performance computers developed by the Accelerated Strategic Computing Initiative. She has been a computer scientist and software engineer at Lawrence Livermore for 15 years, working in the Nuclear Weapons Program, Treaty Verification Program, and Yucca Mountain Program. She received a B.A. in mathematics from the University of California at Irvine in 1977, and an M.S. in engineering and applied science from the University of California at Davis in 1984. She was a nuclear engineer in the Naval Sea Systems Command, Division of Nuclear Reactors, in the period between her undergraduate and graduate education.