IN the field of high-performance computing (HPC) and advanced simulations, Lawrence Livermore researchers have gained a worldwide reputation for success, especially in calculations showing how matter responds to extreme pressures and temperatures. Now, researchers have applied their expertise to a new type of simulation that aims to realistically mimic a beating human heart. The results could contribute to advancements in human health in much the same way that Livermore’s computational work for stockpile stewardship helps ensure the safety, security, and reliability of U.S. nuclear weapons.
The new simulations are made possible by a highly scalable code, called Cardioid, that replicates the electrophysiology of the human heart. Developed by Laboratory scientists working with colleagues at the IBM T. J. Watson Research Center in New York, the code accurately simulates the activation of each heart muscle cell and the cell-to-cell electric coupling.
On every heartbeat, electric signals normally traverse the entire heart in an orderly manner, resulting in a coordinated contraction that efficiently pumps blood throughout the body. However, these signals can become disorganized and cause an arrhythmia, a dysfunctional mechanical response that disrupts the heart’s pumping process and can reduce blood flow throughout the body. Without medical intervention, a serious arrhythmia can lead to sudden death and accounts for about 325,000 deaths every year in the U.S.
The groundbreaking heart simulations were developed and performed on Lawrence Livermore’s Sequoia supercomputer, a BlueGene/Q system designed to achieve 20 quintillion floating-point operations per second (20 petaflops). The machine, which was built by IBM, has 98,304 nodes, each with 16 central processing units, or cores. When the full system is in operation, more than 1.5 million cores are available to execute calculations in parallel. Cardioid assigns roughly 3,800 heart cells to a node, for a total of about 370 million cells. The code is highly scalable, meaning it is written so that its performance increases in proportion to the number of cores applied to a problem.
Success through Long-Term Partnership
During the months required to “shake down” Sequoia, while IBM and Livermore scientists installed and tested the machine in the process of bringing nodes online, managers in the Laboratory’s Computation Directorate made the system available for unclassified science calculations. “Our heart code work has been a great opportunity to demonstrate Sequoia’s power with an application that most people consider important to society, in this case, cardiac modeling,” says computational scientist Art Mirin. He notes that when the shake-down testing is complete, Sequoia will be dedicated to classified simulations in support of the nation’s Stockpile Stewardship Program.
An Extended Look at Cardiac Health
Richards, a computational physicist who wrote the largest share of the code, notes that Cardioid is essentially an extensive reworking of a code IBM scientists developed a few years ago for machines with only 2,000 nodes. “Livermore is one of the few places with the expertise to get past a lot of potential barriers that developers would likely encounter in adapting an existing code to run on the world’s most powerful supercomputer,” he says.
Richards explains that although the Livermore team was not experienced at simulating the human heart, many HPC techniques are “agnostic” to the specific problem at hand. That is, writing different types of codes for parallel supercomputers requires similar development tasks, no matter the phenomena being modeled.
In working on the code, IBM computational biologists contributed their expertise in cardiology, while Laboratory scientists provided support in computational science, especially parallel algorithms. “The Cardioid effort became an interdisciplinary problem involving both computer science and physics,” says Richards. “Because I was trained in the physical sciences, I could ask meaningful questions about heart function and understand how to apply the answers in a complex calculation.”
Cardioid allows simulation at roughly the spatial resolution of a heart cell, which is about 0.1 millimeters long. It thus provides researchers with a level of detail that was impractical with early codes. High-fidelity simulation at the organ level requires a three-dimensional discrete model of the human heart.
To achieve this resolution, the IBM scientists combined two-dimensional cross-sectional images from the Visible Human Project®, a detailed dataset from the National Library of Medicine. The team also developed software to reconstruct the anatomy of a torso so that an electrocardiogram from a typical body surface could be simulated. When combined with these components, Cardioid offers a multiscale simulation capability that spans from subcellular mechanisms up to clinical signals collected from actual patients.
Simulating Thousands of Heartbeats
Extended cardiac simulations are critical when investigating how specific medications affect heart rate. Many drugs disrupt heart rhythm. In fact, even those designed to prevent arrhythmias can be harmful to some patients. In most cases, however, researchers do not fully understand the exact mechanisms producing these negative side effects. With Cardioid, scientists can examine heart function as an anti-arrhythmia medication is absorbed into the bloodstream and its concentration changes. “Observing the full range of effects produced by a particular drug takes many hours,” says Mirin. “With Cardioid, heart simulations over this timeframe are now possible for the first time.”
The Cardioid simulation has been named as a finalist in the 2012 Gordon Bell Prize competition, which annually recognizes the most important advances in HPC applications. The Livermore–IBM team hopes the code will grow into a product that is widely adopted by medical centers, pharmaceutical companies, and medical device firms, helping them better understand the mechanisms that can lead to heart ailments and the potential drug interactions that may occur during treatment. One intriguing idea is to merge a Cardioid simulation with a patient’s clinical data—electrocardiograms, magnetic resonance imaging, and computed tomography scans, for example—to better quantify treatment options for each individual.
A Boost to Economic Competitiveness
“HPCIC is about industry teaming with some of the world’s foremost practitioners of simulation and visualization,” says Fred Streitz, the center’s director. A computational physicist, Streitz led two of the six Livermore teams awarded the Gordon Bell Prize for groundbreaking simulations. (See S&TR, July/August 2006, Keeping An Eye on the Prize; September 2010, Quickly Identifying Viable Pathogens from the Environment.)
At HPCIC, industrial partners can access Livermore’s supercomputing resources and technical expertise in an open collaboration area with office space, classrooms, and networked conference rooms. The center hosts conferences, workshops, and training events to encourage HPC development and innovation in an environment that protects intellectual property and promotes collaboration. The current computing system at HPCIC provides industrial partnerships with 300 trillion (tera) flops of computing power. In the near future, a 5-petaflops “mini-Sequoia” machine, called Vulcan, will be available.
Enhancing the Code
“Congestive heart failure is a complex and multifaceted disease,” says Jeremy Rice, a biomedical engineer at IBM and a Cardioid collaborator. “An accurate electromechanical heart model could be the key to developing effective new therapies.” The team also wants to incorporate physiological systems such as coronary blood vessels that feed heart tissue to create a more comprehensive model with even wider applicability.
“HPC can be used for so many applications beyond national security,” says Richards. “Through our collaborations, we want to demonstrate the impact it can have on a broad section of society.”
Streitz adds that HPC involves much more than performing the same simulations in a shorter time. “It’s about doing something in a new way that otherwise would have been impossible.”
Key Words: arrhythmia, Cardioid code, congestive heart failure, Gordon Bell Prize, high-performance computing (HPC), High Performance Computing Innovation Center (HPCIC), Livermore Valley Open Campus (LVOC), Sequoia, Vulcan.
For further information contact Fred Streitz (925) 423-3236 (firstname.lastname@example.org).
Lawrence Livermore National Laboratory
Privacy & Legal Notice | UCRL-TR-52000-12-9 | September 1, 2012