Lawrence Livermore National Laboratory

Back to top

Back to top

Hundreds of tall cylinders arranged pair-wise in rows, photographed from an elevated vantage point

The ACES in Our Hand

The Adaptive Computing Environment and Simulations (ACES) project will advance fissile materials production models and reduce risk of nuclear proliferation.

Uranium enrichment is central to providing fuel to nuclear reactors, even those intended only for power generation. With minor modifications, however, this process can be altered to yield highly enriched uranium (HEU) for use in nuclear weapons. The world’s need for nuclear fuel coexists with an ever-present danger—that a nonnuclear weapons nation-state possessing enrichment technology could produce weapons-grade fissile material to develop a nuclear arsenal or supply radiological materials to others.

Nuclear nonproliferation—preventing the spread of nuclear weapons—is part of Livermore’s national security mission. To this end, the Laboratory provides scientific and technological solutions and advice to governing bodies—including the National Nuclear Security Administration (NNSA) and the International Atomic Energy Agency—to identify and counter emerging threats.

The Adaptive Computing Environment and Simulations (ACES) project will augment and modernize the Laboratory’s ability to serve this mission through three thrusts. First, its researchers will develop new computer models and simulations of fissile materials enrichment using the gas centrifuge-based method. “In nonproliferation work, analysts often integrate computational modeling into their assessments,” says Stefan Hau-Riege, associate division leader in Applied Physics and leader of the ACES project. “The idea is to provide them with a modern capability to model fuel enrichment integrating data science and machine learning.” In the second thrust, researchers will create a new computational infrastructure to support and sustain this modeling capability. Finally, ACES will recruit and develop a trained workforce to carry out nonproliferation work that requires a detailed understanding of centrifuge enrichment technology. “ACES will capture knowledge from subject matter experts, improve the modeling and the computer environment to sustain the current expertise, and bring in new expertise,” says Eddy Banks, deputy division leader for the Global Security Computing Applications Division (GS-CAD).

Improved Enrichment Simulations

The task of estimating enrichment yield derives from the fundamental chemical and physical properties of uranium isotopes.Two isotopes of the element uranium are commonly found on Earth, differing only in the number of neutrons in their nuclei. Each house 92 protons in its nucleus, but Uranium 235 (235U) has 143 neutrons versus the 146 neutrons of Uranium 238 (238U). Nuclear fission—the release of energy from splitting atomic nuclei—provides heat to a nuclear reactor, transforming liquid water to steam that spins turbines to generate electricity. In theory, both isotopes can be split to release thermal energy, but in practice, only 235U can sustain a nuclear chain reaction in a reactor or a nuclear device. Naturally occurring uranium ore contains little 235U, a mere 0.7 percent by mass. Because the dominant isotope, 238U, does not contribute to nuclear fission, natural uranium must be purified or “enriched” to increase its percentage of 235U. Most nuclear power reactors utilize low-enriched uranium (LEU), which contains between 3 to 5 percent 235U.

Colorful flow chart that starts from the left and proceeds clockwise, with some strands diverging
This nuclear fuel cycle infographic depicts the process of converting uranium ore concentrate into uranium hexafluoride (UF6 ), enriching UF6 concentrations of uranium 235 (235U), and then fabricating uranium oxide (UO2) or mixed-oxide (MOX) metal alloys for nuclear reactor fuel rods that are eventually reprocessed or disposed.

HEU, the type of uranium used in weapons, can be obtained through multiple methods; however, gas centrifuges are overwhelmingly used to enrich uranium fuel. After mining and milling, uranium ore is eventually converted to a gaseous state, uranium hexafluoride (UF6), and passed through a cascade of rapidly spinning cylindrical centrifuges that separate the isotopes. The enriched UF6 gas is eventually converted back into solid uranium for the next step in the nuclear fuel cycle—conversion to fuel rods for reactor use.

A transparent cylinder with a red bottom face is filled with small spheres traveling in different directions, marked by arrows

Inside a centrifuge or separator, centripetal force generated by rotation pushes heavier uranium 238 (238U)-containing gas outwards, leaving lighter gas molecules containing 235U closer to the centrifuge’s center. Streams of gas slightly enriched with either 235U or 238U are then drawn out of opposite ends of the centrifuge. Passing UF6 through one centrifuge only purifies the gas by a fraction of a percent, so this gas must travel through hundreds of centrifuges before it reaches a concentration high enough to be utilized for reactor fuel.

“Once a country has enrichment technology, how can we ensure that it performs only low-level enrichment and does not divert material to make weapons?” asks Kyle Chand, senior analyst in the Operations and Analytics group for GS-CAD. Most—but not all—of the world’s nations are signatories to the Treaty on the Non-Proliferation of Nuclear Weapons. Nonsignatories and extranational groups present areas of security concern. Through ACES, nonproliferation experts can simulate fissile materials enrichment at purported LEU enrichment facilities to understand how they might alter their processes to produce HEU.

Although the United States previously conducted research with centrifuge-based materials enrichment, which spurred small private-sector capacity, the nation did not rely on this technique to produce most of its fissile material stockpile. The lack of domestic experience with the process at industrial scale concerns nonproliferation experts, who assess the magnitude, efficiency, and effectiveness of other countries’ enriched uranium production processes based on their technological configurations. “We understand how the industrial processes work, but we have uncertainty over what goes into the models,” says Chand. “We need to incorporate uncertainty quantification within software models and integrate them with data sciences. The ACES infrastructure will provide a modern, state-of-the-art infrastructure for data science not yet available within the NNSA complex.”

Another major benefit of ACES is the experience its simulations will provide for increasing industrial capacity. “The U.S. doesn’t have an enrichment industry now,” says Chand. “We have a research and development program, but it’s not at the industrial level yet. ACES will allow us to gain experience if we bring up an industrial capacity.” Computer modeling of enrichment cascades can provide nonproliferation analysts with signatures that characterize enrichment facility performance as well as system configuration and reconfiguration performance. At the Laboratory, ACES researchers will expand the capabilities of an existing set of models, applying data science and machine learning to model centrifuge enrichment at the cascade and facility scale. The models will also be revamped to incorporate uncertainty quantification—methods that assess how accurately the models reflect known results in operating enrichment plants.

A second, complementary project at Oak Ridge National Laboratory (ORNL) focuses on modeling centrifuge enrichment at the component level and the gas dynamics of enrichment. Lawrence Livermore and ORNL will collaborate to develop an integrated modeling and simulation capability. To be effective, this integrated capability requires a computational environment that can handle diverse data and structures, function on a variety of computer systems, and adapt to the needs of projects with different objectives and users.

A series of large, dark-colored boxes (computer banks) in a dimly-lit room
The high-performance computing hardware shown here will help researchers model centrifuge–based enrichment technology for the ACES project. (Photo by Garry McLeod.)

Advanced Computational Environment

Prior to ACES, nonproliferation analysts lacked a computing environment that efficiently modeled real-world uranium enrichment processes. Producing HEU requires a concert of material inputs, technologies, supplies, concentrations of experts at facilities, and other signatures that analysts can glean from public data. Analysts then generate models with a range of initial assumptions that can predict what they might later confirm. Previously, creating models was the task of programmers since analysts generally lacked the necessary coding skills. New functionality provided by ACES will circumvent the laborious process of changing model parameters and rerunning of simulations, yielding information the analysts need.

ACES pioneers a new type of computing environment that has been used in the private sector, but which has yet to be applied to NNSA’s nuclear nonproliferation efforts. “Adaptive computing environments are unlike those in high-performance computing (HPC),” says Hau-Riege. “These adaptive environments consist of a variety of nodes, and they lack the ultrafast networking used in HPC because they don’t need them. For this application, we require nodes with different connected features.” Node types can range from standalone desktops to cloud-based computers with different operating systems, and visualization and data-transfer capabilities. Porting a model—the translation of one programming language or protocol to another—is time-consuming, and usually necessitates rewriting code.

Prior to ACES, the Laboratory’s efforts in computer modeling for nonproliferation were small scale and informal. “When I started at the Laboratory, we had servers and people running software that would affect other people’s applications,” says Livermore computer scientist Ted Stirm. “We also lacked adequate personnel. Now we have funding and resources to architect a system that will provide stable support to multiple groups and projects.”

3 diagrams are present. In the top left image, colored polygons representing
As part of the ACES project, programmers will be able to containerize data and software (left). Containerized modules are independent of a computer’s operating system, so the software can be plugged into different systems without rewriting the code. ACES will also allow different types of computer systems to connect within the ACES cluster (right).

The ACES computing infrastructure will leverage containers, which encapsulate software applications so that they can be deployed on a variety of systems, notebooks that allows users to create interactive documents, and Kubernetes, which maintains and manages ACES’s containers. The nonproliferation world has a diverse digital workload utilizing web servers, machine learning with graphics processing units (GPUs), and web interfaces. HPC is not geared toward that kind of model. Traditional HPC systems are designed for performing tightly coupled, large-scale, and complex modeling tasks, but they are ill-suited for running services or providing the flexible compute options necessary for mixed workloads. A heterogeneous cluster with different node types allows the user to run whatever they need by leveraging containers, which package the code with the run-time environment. “We are not replacing HPC—we want to complement it by leveraging cloud-native, industry-standard technology so the applications developed for ACES can easily run on any cloud environment, with minimal porting. Kubernetes gives us that capability,” says Stirm. The cloud environment makes software exchange possible. If the analysts need a different kind of computer system to solve a problem, they can move an application. “We can ship containers. We can build at Livermore, ship to Oak Ridge and then run on their systems. Also, the containers talk to each other—the cascade simulation tool can talk to a facility simulation tool or call up a centrifuge tool,” says Hau-Riege. ACES will also incorporate Jupyter Notebooks to help nonproliferation analysts run simulations themselves, analyze and document results, produce graphical plots, and share work with colleagues.

A chart with arrows that link together people and rectangular areas
A container-based, open-source software system used to deploy, maintain, scale up, and manage software, Kubernetes’s architecture (above) allows users to “containerize” executable code so that it can be transferred from one operating system to another.

For those on the programming side, ACES provides a virtual test bed for comparing model results to real-world data from enrichment operations to validate that the models and simulations are providing reliable, explainable results. “We’ll be coupling these models into a framework and collecting them into a virtual test bed capability,” says Banks. Programmers can create a test bed for a particular facility configuration, incorporating specific models and tools to simulate the configuration. Users can model processes at different scales, from individual centrifuges and cascades to entire facilities and even combinations of facilities. The test bed will fully manage simulations, provide information about its parameters and progressions, and capture the data generated by the simulations. Programmers will use test beds in the development environment, during the build-out of ACES infrastructure to test models and other software, as well as in the operational environment for nonproliferation projects.

Three images arranged in columns. The left image contains 9 funnel-shaped objects comprised of small pixel-like boxes. The middle image shows one of these funnel-shaped models in detail. The third image features two mathematical plots: on top, the plot shows a dark and light patch side-by-side on the left-hand side; on the bottom, a different plot shows the bright spot towards the ceiling of the plot and the dark spot towards the bottom.
The Adaptive Computing Environment and Simulations (ACES) project will allow users to simulate the centrifuge environment of fissile materials at different scales from an entire enrichment plant to cascades of centrifuges down to a single component.

Hiring and Sustaining Workforce

NNSA has committed to the ACES project to establish state-of-the-art computational solutions and grow and sustain an expert workforce. Prior to the Nonproliferation Stewardship Program (NSP), the federal government had no formal effort or dedicated funding to maintain expertise in the nonproliferation modeling and simulation subfield. Expertise in uranium enrichment has been dwindling with the aging of prior generations, and has not been replaced—formal university training in the field does not exist and a lack of awareness of the critical importance of this technology hampers recruitment.

ACES provides the opportunity for NSP to recruit and train dedicated new talent. “The project will bring a stable workforce and development environment,” explains Chand. “We don’t learn about uranium enrichment in college, so having a technical environment to train people is essential to developing new hires.” This advanced computing hardware and software environment provides the infrastructure necessary to attract and retain people. “ACES has a cutting-edge program integrating data science into nonproliferation. This is new, it’s making inroads at the Laboratory, and will provide an interesting environment to address problems in nonproliferation requiring data science,” says Chand.

“ACES is an exciting project for computer scientists coming to the Laboratory,” says Hau-Riege. “They can deploy software in Kubernetes and containerize it without worrying about how to move it from one system to another.” Beyond computer scientists, ACES is bringing in diverse specialties to support the next generation of nonproliferation analysis including physicists, computational modeling experts, data scientists with experience in machine learning and data-driven models, full-stack software developers, web developers, and nonproliferation analysts to interface with the Laboratory’s customers in government agencies.

Support for R&D

When complete, ACES will provide a set of modernized codes that incorporate better understanding of uncertainty quantification, a virtual enrichment test bed, a robust and portable computational environment that can be leveraged for other national security programs, and a revitalized workforce with the diverse expertise necessary to lead the next generation of nonproliferation stewardship. With this sustained computing infrastructure in the making, the Laboratory and NNSA are considering expanding ACES’s range of applications. “This capability can be applied to other contexts including energy security, cybersecurity, biosecurity, and other areas within Livermore’s Global Security Principal Directorate,” explains Scot Olivier, program leader for Nonproliferation Research and Development at the Laboratory. “Researchers in Global Security are also exploring ways to scale and implement this infrastructure for tackling a variety of national security problems. Through ACES, the nation will have a solid scientific and technical basis for understanding and assessing nuclear proliferation threats posed by fissile fuel enrichment in the nuclear fuel cycle and a capability to provide decision-makers with the knowledge necessary to meet those threats.”

—Allan Chen

Key Words: Adaptive Computing Environment and Simulations (ACES) project, container, data science, fissile material, gas-centrifuge enrichment, highly enriched uranium (HEU), high-performance computing (HPC), International Atomic Energy Agency, Jupyter Notebook, Kubernetes, low-enriched uranium (LEU), machine learning, Nonproliferation Stewardship Program (NSP), Treaty on the Non-Proliferation of Nuclear Weapons.

For further information contact Stefan Hau-Riege (925) 422-5892 (