S&TR | January/February 2018

Contents

The Laboratory in the News

The High Value of Open-Source Software

Commentary by Bruce Hendrickson

Ambassadors of Code

Livermore's culture of open-source software development strengthens Laboratory programs with an engaged user community.

Microbiome Research Takes Flight

A Livermore-developed biodetection system helps researchers survey the microbial population of the International Space Station and evaluate pathogenic risks to the crew.

Modeling Seismoacoustic Waves of an Explosive Nature

An innovative computer model simulates mechanical waves both underground and in the atmosphere, offering new insight into earthquakes and terrorist blasts.

A Solid Hydrogen-Storage Solution

Laboratory researchers lead efforts to store hydrogen in solid materials for advancing fuel-cell technologies.

Awards

Patents

Staff

About S&TR

The Laboratory in the News

Colloidal Nanoparticles with Dynamic Color

Lawrence Livermore researchers have developed a technique to change the color of assembled nanoparticles with an electrical stimulant. The team used an electrophoretic deposition process to build amorphous photonic material (APM) structures with nanoparticles having cores of iron oxide and shells of silica. Instantaneous color changes are possible and fully reversible with noticeable differences between transmitted and reflected colors. The research is featured on the cover of the April 3, 2017, edition of Advanced Optical Materials.

The team used nanoparticles to improve color contrast and expand color schemes by combining pigmentary color (from inherent properties) and structural color (from particle assemblies) and by varying nanoparticle concentrations, shell thicknesses, and external electric stimuli. The resulting colors are dynamically tunable because the structure of the nanoparticles and their interparticle distances are highly affected by the electric field. Jinkyu Han, lead author of the paper, says, “The assemblies of the nanoparticles can not only imitate interesting colors observed in living organisms, but can also be applied in electronic paper displays and colored-reflective photonic displays.”

A nanoparticle arrangement that is not perfectly ordered or crystalline forms the APM structures, resulting in colors that do not change with the viewing angle. “The angle independence of the observed colors from the assemblies is quite a unique and interesting property of our system and is ideal for display applications,” says Han. The team’s technique can be applied to electronic devices, such as digital signs, mobile phones, electronic billboards, and e-readers.

Contact: Jinkyu Han (925) 423-2517 (han10@llnl.gov).

Two Pathways to Polymeric Molecule Crystallization

In research appearing in the April 17, 2017, edition of Nature Materials, scientists from Lawrence Livermore and Pacific Northwest National Laboratory (PNNL) investigated crystallization pathways for polymeric molecules. A better understanding of these pathways can lead to improvements in pharmaceutical development and energy technologies that depend upon complex molecular crystals. In the study, the scientists compared the beginning crystallization of a peptoid, a simple biomimetic polymer, to that of a slightly altered version.

The one-step crystallization process is well understood—simple molecules assemble by attaching together one molecule after another. However, experiments suggest that complex molecules require a two-step crystallization process, wherein the molecules first form a disordered group and then rearrange into a crystal. “The findings address an ongoing debate about crystallization pathways,” says materials scientist Jim De Yoreo, a former Livermore employee who is now affiliated with PNNL and the University of Washington. “They imply one can control the various stages of materials assembly by carefully choosing the structure of the starting molecules.”

Using Livermore’s new ultrafast atomic force microscope to characterize the crystallization process, Laboratory scientists Aleksandr Noy and Yuliang Zhang found that the simpler peptoid followed the one-step process while the other proceeded in two steps. “We were not expecting that such a minor change would make the peptoids behave this way,” says De Yoreo. “The results are leading us to think about the system in a new way, which we believe will lead to more predictive control over the design and assembly of biomimetic materials.”

Contact: Aleksandr Noy (925) 423-3396 (noy1@llnl.gov).

CASTing New Limits on the Search for Dark Matter

Livermore researchers, leading an international collaboration on the CERN Axion Solar Telescope (CAST, see image below), presented new results on the properties of axions in a paper published in the May 1, 2017, edition of Nature Physics. Axions are hypothetical particles that could constitute some or all of the universe’s mysterious and abundant dark matter. Lawrence Livermore has been an active collaborator in CAST since 2005, primarily funded through support from the Laboratory Directed Research and Development Program.

The CAST experiment searches for axions using a helioscope, which tracks the Sun as it moves across the sky. The CAST helioscope has operated since 2003 and follows the movement of the Sun for 90 minutes at dawn and dusk over several months each year. Any solar axions entering CAST’s 10-meter-long, 50-ton magnet would be converted by its strong magnetic field into x-ray photons. The x-ray telescope developed by Livermore researchers focuses the photons into a small spot, greatly enhancing the instrument’s sensitivity.

During the latest data-gathering campaign, from 2013 to 2015, CAST demonstrated a factor-of-three improvement in signal-to-noise ratio. This improvement was made possible, in large part, through more enhanced detection systems with lower background levels. Livermore’s Mike Pivovaroff, a member of the CAST collaboration and an author of the paper, says, “This work is ground-breaking science with implications for elucidating the nature of dark matter, solving a long-standing particle physics problem (hinting at physics beyond the Standard Model) and constraining cosmological models.” The results are also important to the development of the International Axion Observatory (IAXO), a proposed successor to CAST in which Livermore will have a major leadership role.

Contact: Julia Vogel (925) 424-4815 (vogel9@llnl.gov).

Commentary: The High Value of Open-Source Software

Bruce Hendrickson

Associate Director for Computation

FROM defense technology and energy solutions to basic science research, software plays a critical role in Lawrence Livermore’s mission activities. Our large collection of software is a precious Laboratory asset, one that benefits both Lawrence Livermore and other national laboratories, and in many cases, the public at large.

For decades, the Laboratory has provided a portion of its software to the public as open source. Although it may seem counterintuitive for the Laboratory to release software to external users at no cost, our investment in open-source software (OSS) is an essential element of the institution’s work. Livermore’s OSS manages installation, authentication, storage, performance, analytics, and other key high-performance computing (HPC) tasks. The Laboratory also develops large-scale application codes for external use. The feature article, Ambassadors of Code, describes several projects that illustrate the breadth of capabilities provided by our open-source portfolio.

Lawrence Livermore’s active participation within the open-source community creates a symbiotic relationship in which staff can collaborate with external users, including other government agencies, to produce cost-effective solutions to complex problems and bolster software to its full potential. For instance, HPC centers, such as those at Department of Energy (DOE) laboratories, can work together to increase parallel performance of a multiphysics code. Through the National Nuclear Security Administration’s Advanced Simulation and Computing Program, we are developing open-source tools to ensure portability of HPC codes to next-generation architectures. Progress on high-profile, multi-institutional initiatives, such as DOE’s Exascale Computing Project, is made possible through shared commitment to open-source technology. The Laboratory abounds with examples of how OSS helps us prepare for future needs and decrease development and maintenance costs.

Engaging with the open-source community further enables the Laboratory to stay at the forefront of HPC advances. External users and developers provide feedback on software functionality, uncover bugs, and contribute to enhancements. With their assistance, we improve software quality and accelerate development. High-quality software helps build our brand and reputation within the broader scientific community and HPC industry, leading to new collaborations and business opportunities. The open-source model thus adds tremendous value to the Laboratory’s computational capabilities, which helps it remain competitive and successful.

In addition, Livermore scientists frequently use tools already available in the wider software community to save time and effort. A scientist can analyze a data set with an open-source natural language processing program instead of building one from scratch. Developers may also leverage an open-source programming language to rewrite older software for modern use cases. This approach often serves Livermore’s national security mission. For example, a Laboratory physics team is adapting visual processing OSS for analyzing images captured from Cold War–era atmospheric nuclear test films. (See S&TR, October/November 2017, Preserving the Past to Protect the Future.) Data from these films help validate simulations of a weapon’s energy yield, which supports stockpile stewardship.

OSS also aids in recruiting the Laboratory’s next generation of computer scientists and engineers. When an outside user community works on Livermore OSS, a hiring pipeline emerges. One possible scenario involves graduate students using the software as part of projects at their universities. Later, when they become summer interns at the Laboratory, their mentors could evaluate their potential as future employees through their contributions to OSS software and user collaborations.

Employee retention is equally important. Embracing OSS empowers Laboratory staff, providing an avenue by which they can connect with user communities to work on software they are passionate about. In doing so, developers grow their professional reputations and networks. The Laboratory encourages this exploration and innovation as part of career development. Many Livermore scientists have received prestigious recognition—including R&D 100 awards—for their open-source tools.

As we begin a new year, Livermore’s open-source development continues apace. Users all over the world are exploring the Laboratory’s OSS and reaching out to exchange ideas. Such engagement creates relationships that can inspire advances, new collaborations, and even development of commercial products. The Laboratory’s Innovation and Partnerships Office provides crucial guidance and support for these opportunities, strengthening the Laboratory’s reputation and capabilities. Ultimately, OSS increases our mindshare in the scientific community. The resulting technological growth positions Livermore for solving tomorrow’s computational challenges.

Ambassadors of Code

The Laboratory’s long history of developing and supporting open-source software has led to thriving user communities and international collaborations.

LAWRENCE Livermore develops and enhances numerical simulation codes to support basic science research, advance next-generation computer science, improve simulation and modeling capabilities, and meet the growing demands of high-performance computing (HPC) systems. Although some of this software is classified or controlled, much of the code developed at Livermore can benefit a range of users and applications outside the Laboratory. An increasingly vital form of technology transfer occurs through the release of open-source software (OSS), wherein the source code is made freely available to the public for inspection, modification, and enhancement.

For decades, the Laboratory has made software developed for programmatic work publicly available as open source. In 2002, the Department of Energy (DOE) issued a software release policy recommending that code produced for its programs be made open source unless exceptions could be justified. Several years later, in 2016, the U.S. government established the Federal Source Code Policy, mandating that code developed by or for government institutions be made available to other federal agencies. The policy further directs government agencies to provide at least 20 percent of their code to the public as OSS. In recent years, Laboratory scientists and engineers have organized unclassified software repositories on mainstream open-source hosting services, such as GitHub, and built active communities with external collaborators. Today, more than 350 software packages developed at Livermore are available to federal, industry, academic, and other public users.

Livermore’s OSS addresses several key HPC and non-HPC needs—compilers, package managers, data analytics, visualization tools, input/output benchmarking, data storage, parallel performance, workload management, and math and physics codes—across multiple operating systems and programming languages. Supported by diverse funding streams, all directorates participate in developing valuable software resources. The Laboratory’s presence—and reputation—in the open-source community spans online repositories, conferences, publications, and social media.

Strength in User Numbers

Releasing software as open source is a common industry practice. Microsoft and Google, for example, make portions of their software widely available, and all major web browsers and front-end web development languages are built on open-source technologies. Scientists who incorporate OSS into their disciplines have begun to reference and include source codes in scientific papers to encourage reproducibility. The Laboratory’s open-source strategy has several advantages that help Livermore keep pace with rapidly growing technological and market needs.

According to Livermore computer engineer Ian Lee, the Federal Source Code Policy acknowledges the value of OSS both among and beyond national laboratories. He explains, “Government agencies and others are trying to solve the same problems. The open-source mandate allows agencies and contractors to align and make the most of external resources. We can share projects and avoid redundancies. In particular, the specialized HPC community benefits from information sharing. Livermore computer scientist Todd Gamblin says, “There aren’t many sites that conduct large-scale HPC work, and we need to develop common HPC infrastructure with collaborators rather than perpetuate siloed efforts.”

Building a community around an open-source project enables users to provide valuable feedback, which can result in useful contributions such as new features and bug fixes. In cybersecurity, for example, open-source encryption software is viewed by developers and researchers as more trustworthy than closed-source (proprietary or licensed) software because the former can be scrutinized by the larger user community. Indeed, Laboratory scientists often leverage externally developed software for internal projects and programs (see External Software Enhances Internal Programs below). In addition, computational mathematician Tzanio Kolev reasons, “When you know other people will review your work, you impose a higher standard for yourself.” The feedback process improves industry standards and Livermore’s computing capabilities. Open-source exposure can also prompt discovery of new applications and commercialization potential for a software program. For instance, the Laboratory-developed DYNA3D code used for simulating the mechanical behavior of collision events was released as OSS in 1978. A more advanced commercial code, LS-DYNA, was built on its predecessor’s success. LS-DYNA became the leading commercially licensed product for collision event simulation (see S&TR, June 2017, Ready, Set, Innovate! Entrepreneurship Flourishes at the Laboratory).

For developers, OSS participation is evidence of professional talent and productivity. “Just as a scientist can publish research papers, a developer can demonstrate his or her work through a software portfolio,” explains Lee. Kolev adds, “The Laboratory’s encouragement of OSS is amazing. Intellectual freedom is crucial to scientists.” In addition, the open-source community gives developers a means of honing skills and provides an avenue for the Laboratory to evaluate potential hires through their contributions to Livermore and other open-source code.

Livermore’s open-source software (OSS) is predominantly released to the public on the GitHub hosting platform. GitHub tracks user actions (top right) such as the number of Watchers (followers who receive change notifications), Stars (signifying both bookmarking and appreciation), and Forks (copies created for independent development) for each software project.

Aiming for Maximum Exposure

Although Livermore has been producing OSS for many years, until recently users had no central location for accessing the Laboratory’s code. Software repositories were posted to websites on the llnl.gov domain, personal websites, and numerous external hosting platforms such as SourceForge; Bitbucket; or GitHub, today’s most popular source code hosting service in the world. To make Livermore OSS more easily accessible, Lee and colleagues launched a website (software.llnl.gov) in 2015. He says, “This portal offers a full corpus of what the Laboratory develops as open source by providing a centralized collection of pointers to externally hosted repositories.”

Laboratory developers create a repository for each open-source project predominantly via the Livermore organization’s main page on GitHub (github.com/llnl), from which Livermore’s portal dynamically pulls the information. In both locations, users can find software by name or key words. “Together, these resources facilitate the management, support, and discovery of our open-source repositories,” says Lee.

On GitHub, Laboratory developers publish updates to source code and associated documentation while interacting with user communities. These users can download the software, suggest features and enhancements, report bugs, leave comments or questions, and communicate directly with Livermore developers. The repositories record each code change (known as a commit), version history, and contributor activity. Among Livermore’s most actively developed and widely used software repositories are ROSE, MFEM (Modular Finite Element Methods), ZFS on Linux, and Spack.

New versions of software include a combination of enhancements, optimizations, and bug fixes. Developers releasing code to the open-source community rely on hosting platforms such as GitHub to help track and assess internal and external contributions to these new versions. (Image courtesy of www.atlassian.com. Licensed with CC BY 2.5 AU.)

A More Sophisticated Compiler

Software can be analyzed for debugging, performance tuning, and other optimization tasks. To conduct these analyses, developers depend on advanced software packages called compilers to translate human-readable code into machine-friendly binary instructions. Enter ROSE, Livermore’s homegrown compiler infrastructure, which remains unique among compiler solutions after nearly 25 years of development and 14 years as OSS. Project leader Daniel Quinlan notes, “Early on, we made the argument to the Department of Energy that ROSE should be OSS because the software fulfills an ongoing need. Software is produced every year, in a range of languages, for everything from refrigerators to pacemakers to cars.”

While most compilers transform source code into binary code, ROSE also generates more sophisticated source code. This source-to-source capability, combined with support for C++ and its many versions, gives researchers flexibility when optimizing simulation codes for multiple operating systems and hardware architectures. In 2009, Quinlan’s team won an R&D 100 Award for this innovative technology (see S&TR, October/November 2009, ROSE: Making Compiler Technology More Accessible). Today, as Livermore and other institutions prepare for the exascale computing era—faster, more powerful machines working on diverse architectures—scientists use ROSE to evaluate their software’s portability. “ROSE’s automated transformation helps ensure codes will keep working when ported to next-generation systems,” explains Quinlan.

The ROSE compiler infrastructure can parse both source code and binary code. Ancillary tools developed by the Livermore team with community support perform automated analysis, transformation, and optimization tasks.

Livermore initially released ROSE as OSS so collaborators at Argonne National Laboratory could use it, and the open-source approach has benefited the project’s progress and longevity. “If developers want their work to be influential, users must be able to find it, and the barrier to entry must be low. We’re always striving to make ROSE easy to use and install with good documentation,” says Quinlan. Though most code contributions are made by Livermore developers, ROSE is widely used by an external community including students, researchers at other national laboratories, and project collaborators. The project has adapted to a changing market. New team members include business manager Greg Pope, who joined the group to help scale ROSE’s capabilities.

Quinlan and Pope envision a bright future for ROSE as software requirements evolve for home automation and security, medical devices, financial transactions, power grids, and antiquated systems in need of coding revisions. “ROSE can evaluate software in these applications, and its automation in upgrading codes can remove some risk by reducing the probability of human error,” states Pope. “Our challenge is to meet these needs without degrading technology—to balance agility and discipline.”

Finite Elements, Infinite Possibilities

Downloaded from more than 80 countries at a rate of 10 downloads per day, MFEM has made an impression on a specialized field. Project leader Kolev notes, “The field of scientific software is small but relatively crowded. MFEM has become known for high performance and flexibility.” His team first released MFEM as OSS in 2010, moving the repository to GitHub in 2015.

MFEM is a discretization library for simulation codes, acting as a mathematical base layer for large-scale physics applications. Research scientists leverage MFEM’s high-order finite element meshes, spaces, and discretization algorithms as building blocks for simulating and visualizing physical phenomena. By breaking down calculations into discrete components, MFEM saves application developers time and effort. “MFEM produces accurate results quickly,” states Kolev, citing as frequent customers the Livermore teams who develop finite element codes such as BLAST (see S&TR, September 2016, Laying the Groundwork for Extreme-Scale Computing).

Since its debut as OSS, MFEM has evolved to include adaptive mesh refinement—a process that continuously fine-tunes a simulation’s grid points—and boasts efficient performance on supercomputers. Beyond Livermore, collaborators hail from national laboratories, universities, commercial companies, and startups. “Releasing MFEM as open source has enabled us to collaborate with partners in a more seamless way. We see new capabilities made possible through their contributions,” explains Kolev. “When a code is published to the user community, developers learn how others use it, making it easier to improve and uncover the code’s bugs.”

The logo for the MFEM project illustrates the high-order mesh elements and physics field representations it applies to high-performance computer simulations. This image was created with GLVis, another Livermore-developed OSS that uses MFEM to generate accurate finite-element visualizations.

Kolev leads the Center for Efficient Exascale Discretizations (CEED), a co-design center within DOE’s Exascale Computing Project (ECP). CEED is a multiyear research partnership that involves more than 30 computational scientists from 2 DOE laboratories and 5 universities. MFEM’s foundational algorithms apply to several CEED objectives, such as developing high-performance simulations using high-order methods on quadrilateral, tetrahedral, and hexahedral grids. Kolev notes, “MFEM has progressed as an open-source project. Being chosen for CEED is a testament to its maturity.” In addition to MFEM and other associated OSS, CEED’s public forum, benchmarks, and mini-apps operate as open-source repositories on GitHub, where participants can track issues and communicate easily across all projects. “Open-source capabilities provide the only way to manage such a multi-institutional effort. We share work, offer feedback, and improve applications in a productive way,” says Kolev. “The best solutions come from this approach.”

Designed to Scale

According to computer scientist Brian Behlendorf, “HPC file systems are notoriously long-term investments when developed from scratch. Even a small bug can result in data loss, which users do not forgive quickly.” In the early 2000s, Sun Microsystems, Inc., designed a file system called ZFS for the Solaris operating system and released it as OSS. ZFS’s scalable design and advanced storage features caught the attention of Behlendorf’s team at Livermore, which was tasked with scaling file systems for the Sequoia supercomputer. However, ZFS did not support Linux machines, so the team adapted it for this purpose. The resulting file system was called ZFS on Linux. “Sun had a five-year head start on development and testing. We were able to take the next step because of their critical insights with the ZFS design—and because it was open source,” explains Behlendorf.

Livermore released ZFS on Linux as OSS in 2011 and has since built ancillary tools that facilitate testing, configuration, and operating system compatibility. The file system’s capabilities have expanded to include failover protection, richer accounting and quota functionality, and performance improvements that guarantee data integrity even as computing power increases. “Everything is more challenging with large-scale data,” says Behlendorf. “We anticipated this challenge with Sequoia and are well situated for next-generation machines such as Sierra. ZFS on Linux was designed to scale.” (See S&TR, March 2017, A Center of Excellence Prepares for Sierra.)

ZFS on Linux, now OSS, was developed to create a more cost-effective, less complex, and higher performance file system for the Sequoia supercomputer.

ZFS on Linux has been a boon for companies that build data centers, institutions that store simulation or raw experimental data, and other national laboratories. “We receive contributions from many sources, even the occasional ‘drive-by’ contributor using it on a home system,” states Behlendorf. The community’s enthusiastic response has also improved Livermore’s ongoing development. He continues, “We’ve benefited tremendously from testing by our broad range of users. They often stress the software in unique ways, which can help uncover bugs.” ZFS on Linux’s high quality and powerful features have prompted mainstream Linux operating systems such as Ubuntu and Debian to include the software in their distributions.

Maintaining high-quality OSS is not easy. Users’ priorities may conflict with those of the project, and some contributors may not consider the amount of work required to support a software feature long term. Behlendorf cautions, “Keeping a higher level view of all needs is essential. Developers have to think outside of their own use case so they do not inadvertently break one feature by adding another.” Still, the challenges are worth the effort. “One strength of OSS is that the development cost is spread out. Developers can create code much faster, and of better quality, if they are willing to share their code,” he says.

Recipe for Success

Large-scale scientific applications, such as mathematical and physics codes, rely on hundreds of external software programs, or packages. Installing different package versions and configurations quickly becomes laborious—developers frequently have to rebuild packages because of bug fixes, operating system upgrades, and other circumstances. As is the case with most developers, Gamblin performed this work manually until he reached his limit in 2013. “I wanted to solve the problem rather than repeat it,” he says.

ARES, one of the Laboratory’s proprietary hydrodynamics codes, has dozens of dependencies requiring a variety of package types. Spack enables multiple versions to be built nightly across different environments, reducing the code’s deployment time on new machines from weeks to days.

Gamblin created Spack, a Python-based package manager, to install software automatically. Spack operates on a range of HPC platforms because it understands the complex dependencies among HPC software packages. Spack makes it easy to leverage others’ software packages, which allows application scientists to focus on science instead of software infrastructure. “Building software on a supercomputer can be a painful process,” explains Gamblin. With Spack, a research team can build many different versions of their code and test new configurations before deployment to a production environment.

Gamblin’s team released Spack as OSS in 2014. Each new package added to the code base makes Spack more flexible for users working with diverse platforms, programming languages, simulation frameworks, and other variables. Saving time and effort improves reproducibility and performance. “Spack is a repository of recipes for building HPC software. It lets users leverage a larger software ecosystem because they are not spending their time rediscovering how to build every package,” says Gamblin.

Spack’s 2016 average of 100 downloads per day ballooned to 400 downloads per day in 2017. Every week, the GitHub repository logs 7,000 views while the team assesses hundreds of contributions. They have adapted to this growth with cloud-based testing services and continuous integration, a process that merges multiple copies of code to detect any build issues. “One challenge is maintaining stability while updating features incrementally. However, Spack wouldn’t be as stable as it is if users didn’t find problems with it,” notes Gamblin.

Although Spack can run on personal computers or relatively small clusters, significant contributions come from users at other HPC centers. Livermore’s relationship with the community is a win–win. Gamblin explains, “External users have contributed most of Spack’s more than 2,400 packages, which Livermore could not have developed alone.” This success has led others to take notice, especially for efforts such as DOE’s ECP, which has adopted Spack to manage software releases for DOE’s entire exascale software stack. “Package managers are the glue that hold software ecosystems together because they allow developers to use each others’ software with a push of a button,” says Gamblin. “We are optimistic that Spack will be that glue for ECP and enable a thriving exascale software ecosystem.”

External Software Enhances Internal Programs

Lawrence Livermore scientists are often contributors to and customers of externally developed open-source software (OSS), including frameworks such as Python and Drupal, which underpin programmatic work. Computer engineer Ian Lee notes, “We can lead the charge in user communities where a national laboratory may not normally have a seat at the table but where our needs must be discussed.”

At the Laboratory’s National Atmospheric Release Advisory Center (NARAC), a Livermore software development team is using open-source technologies to modernize NARAC’s Linux-based central system. NARAC operations depend on a fast, reliable computer modeling system to provide real-time assessment of emergency response strategies for airborne releases of hazardous materials (see S&TR, January/February 2012, Livermore Responds to Crisis in Post-Earthquake Japan). Led by information technology and software manager John Fisher, the multiphase modernization effort includes rebuilding the central system’s graphical user interface (GUI) framework with a scalable OSS suite. “We are implementing software for the long term, so we have chosen OSS with broad community support,” says Fisher.

The central system consists of 50 data sources, 28 servers, 8 storage units, and more than 3 million lines of code. Fisher’s team explored OSS that could simplify this large, complex architecture and selected the popular Angular platform developed by Google. According to software developer Sei Jung Kim, the decision to use Angular informed subsequent decisions. “Angular recently underwent a major redesign. We pursued solutions for the visual layer that were compatible with the platform,” she says. For instance, the team turned to the PrimeNG user interface toolkit to provide a consistent, attractive, and powerful user experience.

One challenge of combining open-source tools is determining which versions to use. For example, Kim states, “Every time we update PrimeNG to the latest version, we must ensure other dependencies, such as Angular, are in sync.” In addition, the team must work through compatibility issues among open-source solutions. “When you use multiple tools, they don’t always like each other,” notes Fisher. Every OSS integrated into the system undergoes a detailed evaluation by NARAC developers.

NARAC’s new technology stack is designed to streamline development while delivering a better user experience and avoiding software obsolescence. Fisher remarks, “The maturity of our current system allows us to more easily identify the capabilities we need. Finding well-supported, well-documented solutions for niche requirements is work, but not using open-source tools makes development and maintenance more difficult and expensive in the long run.” Already the team is seeing usability improvements with several desktop applications developed from OSS. For example, an observed meteorological database GUI allows NARAC users to browse real-time global weather data such as temperature, humidity, wind speed, and air pressure. Around 10 gigabytes of data accumulate per day, but the user only wants information related to a specific event under analysis. “The GUI can bring up a selected portion of data based on user input,” explains Kim.

As the central system modernization project progresses, Fisher remains confident in the team’s approach to reducing dependency on proprietary software. “The Laboratory’s work demands sophisticated technology solutions,” he says. “With open-source tools, we are upgrading existing functionalities and adding new capabilities with better usability.”

Livermore’s open-source software (OSS) is predominantly released to the public on the GitHub hosting platform. GitHub tracks user actions (top right) such as the number of Watchers (followers who receive change notifications), Stars (signifying both bookmarking and appreciation), and Forks (copies created for independent development) for each software project.

Paying It Forward

Livermore’s OSS portfolio has grown thanks to an encouraging culture at the Laboratory. For instance, the site-wide Developer Day session gives employees across the Laboratory a chance to learn about each other’s projects, including OSS efforts. At Livermore’s seasonal “hackathons,” developers try out OSS with existing work or new projects under consideration (see S&TR, June 2013, Software Developers Get Their Hack On). This commitment, in turn, has increased Livermore’s presence and leadership in the open-source community.

Livermore’s OSS portal tracks software release trends in dynamic graphs developed by computer scientist Laura Weber. (top left) More than half of Livermore developers contribute to externally developed OSS. (top right) Three-fourths of Livermore open-source projects receive external contributions. (bottom) The first Laboratory-developed open-source repositories were created years before Git (2005) and GitHub (2008) emerged as community-based solutions.

Beyond online interaction, Laboratory scientists actively connect with the open-source community by attending industry events and presenting at conferences. “When the software we describe at these events is open source, the audience can access and examine it right away, which helps convince them that it’s worth their time to investigate further,” states Kolev. Lee presented at the 2016 GitHub Universe Conference. Gamblin gives Spack tutorials and presentations at supercomputing conferences, and Behlendorf is a fixture at OpenZFS events. Lee and Gamblin also run Twitter accounts (@LLNL_OpenSource and @spackpm) to publicize the Laboratory’s OSS news and communicate with those interested in Spack and other projects.

Livermore computer scientist Todd Gamblin delivers Spack tutorials, talks, and presentations at supercomputing conferences and other face-to-face meetings. Participating in industry events allows Laboratory scientists to connect and engage with the open-source community. (Photo by Meg Epperly.)

Livermore’s software development workforce embodies this collaborative spirit, and Lee strongly encourages sharing code as the default in cases where classification and sensitivity are not issues. “Embracing an open-source strategy means we can make code developed by one available to all. Sharing information with others working in the same technology stack and receiving their constructive feedback is quite valuable,” he advises. “Without this feedback along the way, more rework would be needed later. Code does not have to be perfect to be useful.”

—Holly Auten

Key Words: Center for Efficient Exascale Discretizations (CEED), compiler, data storage, discretization, Exascale Computing Project (ECP), Federal Source Code Policy, file system, GitHub, high-performance computing (HPC), MFEM (Modular Finite Element Methods), National Atmospheric Release Advisory Center (NARAC), open-source software (OSS), package manager, portal, repository, ROSE, source code, Spack, ZFS on Linux.

For more information contact Ian Lee (925) 423-4941 (lee1001@llnl.gov).

Microbiome Research Takes Flight

ON a flight to Mars or during an extended stay in low-Earth orbit on the International Space Station (ISS), the little things sometimes matter the most. Regardless of the stringent cleanliness protocols required prior to space flight, a sizeable population of microscopic organisms—bacteria, fungi, and viruses—ride along with the astronauts and their gear. Once in the closed system of the spacecraft, those microorganisms, or microbes, are difficult to avoid or entirely eliminate. Although more than 80 percent of microorganisms have a neutral or beneficial role in human health, the remainders could cause illness and potentially put the success of a long-term space mission at risk.

Biologist Nicholas Be explains, “Microgravity, increased radiation levels, restricted diet, and limited hygiene options in space cause our immune systems to function a little differently than on Earth. As a result, organisms that might only have an effect on immunocompromised individuals on Earth could have broader health implications in space.” The development and composition of microbial communities can also be different in space. The harsh space environment is highly selective for hardy organisms. Further, radiation and microgravity can boost the growth and mutation rates of some microbes, which may make any illness stemming from them more severe or difficult to treat.

Cataloging and characterizing the composition and evolution of the ISS crew members’ microbiome—the microbial communities in, on, and around the astronauts—is a key step to ensuring that the right prevention methods and treatment protocols are in place as NASA develops new missions to ISS and beyond. With this goal in mind, biologist Crystal Jaing is leading a team of researchers from Lawrence Livermore, the Jet Propulsion Laboratory (JPL), the NASA Ames Research Center, and Johnson Space Center in systematically investigating the ISS crew members’ microbiome and their environment. Leveraging the Laboratory’s extensive bioinformatics capabilities, including the Lawrence Livermore Microbial Detection Array (LLMDA), and high-performance computing resources, the scientists are evaluating microbiome data to determine which, if any, pathogens are of health concern during space flight.

The International Space Station (ISS) is a unique microgravity research laboratory in low-Earth orbit. Astronauts working on ISS conduct experiments in biology, physics, astronomy, and other scientific areas. (Image courtesy of NASA.)

Microbes that Matter

NASA has studied the microorganisms inhabiting the confined environments of spacecraft for decades, but Jaing’s project, called Microbial Tracking-2 (MT-2), is more ambitious in breadth and depth than previous endeavors. Past assessments have only examined subsets of the microbiome. The recently concluded MT-1 experiment, for instance, explored the types of microbes present on the surfaces and in the air of ISS over a one-year period. MT-2 goes further by studying microbial populations of the astronauts as well as those in the station environment. “We are conducting the first study that looks at the crew and environment together in a comprehensive and integrated way,” says Jaing. In addition, MT-1 cataloged only bacteria and fungi, whereas this study also considers viruses, which Jaing says “are somewhat under-studied” in space. “We can provide countermeasures once we know what astronauts are breathing in and breathing out,” adds JPL’s Kasthuri Venkateswaran. “Beyond the microbes affecting the crew, we need to know what microbes are in the environment and riding on the cargo.”

So far, four astronauts from three countries are participating in MT-2. As part of the project, each astronaut collects saliva, nasal, mouth, and skin samples several times before, during, and after their time on ISS. They also gather ISS surface and air samples from various locations. The astronauts’ missions to ISS overlap, enabling continuous monitoring of ISS over an 18-month period. Throughout that time, the researchers can explore the evolution of each astronaut’s microbiome and that of the station as a whole as well as how the two interact. For instance, do microbes living in the air of ISS eventually become part of the astronauts’ nasal microbiome? If so, do they then pose a health risk to crew members’ health by displacing existing beneficial nasal microbes?

As part of the analysis, the researchers will compare microbiome data (hygiene and health samples from the crew) with environmental data (temperature and humidity inside the station) to better understand why changes in the microbiome occur. The team suspects that individual samples from the astronauts will contain distinct collections of microbes that are likely to be affected by different environmental factors associated with space travel. Skin microbes are expected to be more susceptible to space radiation, for example, while microorganisms in the mouth may be influenced by changes in an astronaut’s diet. Data gathered from MT-2 will be shared with the scientific community through an open-access NASA database, called GeneLab, and will serve as a baseline for studies of the microbiome.

Probing Pathogens

Each of the institutions involved in MT-2 will analyze a portion of the samples. Researchers at Johnson Space Center are evaluating viruses and determining which viruses might be harmful to the crew. JPL scientists are performing microbiological analysis, using traditional culturing techniques and DNA sequencing to ascertain which bacteria and fungi are present and alive. However, less than 10 percent of microbes can be cultured. Therefore, Jaing, Be, and their Livermore colleagues are using complementary molecular detection methods to ensure the entire array of organisms represented in the samples is cataloged and studied. A primary focus is identifying virulent or antibiotic-resistant genes in the samples.

A key weapon in their analysis arsenal is LLMDA, a DNA-based detection system developed in 2008 that can rapidly and simultaneously identify all the sequenced microorganisms in a sample, even new microbes containing DNA sequences previously identified in other pathogens. (See S&TR, April/May 2013, A Faster and Cheaper Method to Detect Agents of Disease.) “The Livermore array provides approximately 50- to 100-fold greater coverage of microbes than culturing, and it can process samples in about one day,” says Jaing.

Researchers used DNA sequencing and the Livermore Metagenomic Analysis Toolkit to identify and determine the relative abundance of species found in samples from ISS and an Earth-based clean room. By performing this type of in-depth microbial analysis, researchers gain a better understanding of how microbial populations evolve in a spacecraft environment and how they may affect the crews’ health.

The array contains nearly 400,000 probes—short, synthetic, single-stranded DNA sequences—arranged in a checkerboard pattern on a small glass slide. When a fluorescently labeled fluid sample containing the genetic material from the microbes contacts the probes, squares with genetic material matching those in the samples are activated. The activated squares produce a fluorescent pattern from which the species, or sometimes even the subspecies, present in the sample can be identified. Then, with the Livermore Metagenomic Analysis Toolkit (LMAT), the researchers can dig deeper into the data. (See S&TR, October/November 2015, Two-Part Microbial Detection Enhances Bioidentification; and S&TR, December 2010, Quick Detection of Pathogens by the Thousands.) LMAT’s open-source software efficiently searches through genetic data cataloged in public repositories to more specifically identify organisms in ISS samples. By comparing gene database information with the species and genes identified in the sample, the researchers can then pinpoint genetic mutations, including those that have caused a species to become more virulent (producing a more severe illness in its hosts) or more resistant to antibiotics.

Test Run for Techniques

The first set of crew-derived and environmental samples (76 in total) for MT-2 was retrieved in July 2017 on a SpaceX capsule—the second-ever flight by a private company to reach ISS and return to Earth successfully. Analysis of these and subsequent samples is ongoing. However, Livermore and JPL scientists have provided the scientific community with a “sneak peek” of what their powerful combination of characterization and bioinformatics techniques can offer by testing their processes on previously collected environmental samples from ISS and an Earth-based clean room.

Nicholas Be (left), together with Crystal Jaing and colleagues from Lawrence Livermore and three NASA research centers, are using the Lawrence Livermore Microbial Detection Array to study microorganisms found aboard ISS. (Photo by Randy Wong.)

“It’s clear from our analysis that the ISS microbial population profile is distinct from that of the clean room,” says Be. The team found more evidence of bacteria and fungi relating to human occupation (such as those that cause staphylococcus infection and acne) in ISS environmental samples than in the clean room. The team suspects that although a larger number of people spend time in the clean room than on ISS, access to conventional cleaning materials and adherence to rigorous hygiene protocols reduces contamination in the clean room microbiome. Overall microbial diversity was also lower in the ISS relative to clean room samples, and as anticipated, the team observed more examples of resistant and virulent gene sequences in the ISS.

Having shown that their analysis techniques can achieve thorough results, the researchers have begun applying these methods to the MT-2 samples. This work will provide insight into how humans and microbes influence one another and are influenced by the crowded, complex, yet isolated spacecraft environment. NASA can also use MT-2 data to perform a risk assessment of potential pathogens, determine the level at which they could become a threat to the astronauts, and develop strategies for eliminating or treating the microbes. In the coming years, NASA and its research partners could develop capabilities that would enable crew members to perform microbial characterization while in space—for instance, on the way to Mars. All of these efforts serve the goal of ensuring that astronauts, together with a beneficial mix of microorganism populations, thrive as they continue to venture farther from home for longer periods than ever before.

—Rose Hansen

Key Words: bacteria, International Space Station (ISS), Lawrence Livermore Microbial Detection Array (LLMDA), Livermore Metagenomic Analysis Toolkit (LMAT), Microbial Tracking-2 (MT-2), microbiome, microgravity, NASA, pathogen.

For further information contact Crystal Jaing (925) 424-6574 (jaing2@llnl.gov).

Modeling Seismoacoustic Waves of an Explosive Nature

WHAT do an aboveground terrorist explosion, a volcanic eruption, and a meteor impact have in common? These events all release substantial mechanical energy that travels through the ground as seismic waves and through the atmosphere as sound. Scientists have long understood the in-ground propagation of seismic and acoustic waves, but determining the realistic propagation of acoustic waves through the atmosphere and how they interact with seismic waves has not been possible, until now.

Funded by the Laboratory Directed Research and Development (LDRD) Program, a Livermore team led by seismologist Artie Rodgers recently completed a project to develop a first-of-its-kind seismoacoustic simulation capability, known as the ElAc (Elastic and Acoustic) code. ElAc simulates seismic waves in the solid ground, acoustic waves in the atmosphere, and the energy flow between the two. The code can examine waves generated by various sources—lightning strikes, shallow earthquakes, or near-surface explosions, to name a few. ElAc has many applications in national security, including nuclear nonproliferation, nuclear forensics, and energy security.

One Code, Two Environments

In particular, Livermore scientists are studying acoustic waves in the infrasound region, at frequencies below 20 hertz (the normal limit of human hearing). In the atmosphere, these low-frequency waves can travel long distances and around obstacles, such as mountain ranges, with little dissipation. Since Earth’s seismic structure remains nearly constant, remote sensors are useful for capturing data on seismic and ground-based acoustic waves, helping scientists detect and characterize an event at standoff distances. Measuring sound waves in the atmosphere is more difficult because the environment is perpetually changing, and thus experiments to measure acoustic waves are not repeatable.

For near-surface energy sources, mechanical waves readily convert from one form to another (seismic or acoustic), especially in areas of varied topography. This energy coupling can be important to accurately represent the wavefield. However, prior to ElAc, no computational tool provided realistic modeling of both seismic and acoustic signals and their interactions. Existing codes included simple, idealized assumptions. For example, infrasound frequencies moved across a flat topography through an atmosphere represented either as a uniform, still medium with no wind, temperature variations, or pressure differences, or with simple altitude-varying properties.

The ElAc code is designed to model mechanical waves in the atmosphere and in the solid earth with realistic, heterogeneous properties and varying surface topography in three dimensions. The code’s multidisciplinary development team consisted of Rodgers, geophysical acoustician Keehoon Kim, and computational scientists Anders Petersson and Bjorn Sjogreen. ElAc builds on a three-dimensional (3D) seismic modeling code, SW4, developed by Petersson and Sjogreen. The team incorporated SW4’s solid-earth seismic wave propagation method into ElAc, giving the new code all the features needed for realistic seismic wave simulations.

(left) Conventional codes used for seismic modeling, such as Livermore’s SW4, treat the atmosphere as a vacuum when simulating seismic waves from below-surface energetic events. (right) The new ElAc (Elastic and Acoustic) code includes realistic effects of the atmosphere and can model events in the near atmosphere, on the surface, and underground. In the future, ElAc may also be used to model waves in water.

Atmospheric modeling requires the accurate representation of many factors. Kim says, “Wind speed and direction are major considerations in simulating acoustic waves in the atmosphere. For instance, if you stand downwind of a lightning strike, the thunder arrives at your position earlier than if you were standing upwind.” In addition, if a source is close to or far above the ground, the sound speed varies because of differences in air temperature. Acoustic waves can also be reflected or refracted from the boundaries between atmospheric layers.

ElAc models the ground as an anelastic solid material and the atmosphere as an inviscid compressible fluid. The code uses coupled differential equations, called linearized Euler equations, to solve problems related to fluid flow. The code also addresses the interchange of energy between the two environments. “With the ElAc code, we can model mechanical waves—acoustic and seismic—as they move from the atmosphere into the solid earth and vice versa,” says Petersson. “We configured the model so that the two environments ‘talk’ to each other, enabling energy to realistically propagate through the interface.”

Using high-performance computers, ElAc can simulate sound propagation through a fully three-dimensional, complex medium. The code can account for the scattering of sound waves from complex topography (top) and in turbulent atmospheres (bottom).

The interaction between the motions in the atmosphere and the solid ground is achieved by imposing interface conditions along the surface topography. Petersson and Sjogreen discretize the computational domain using curvilinear meshes, where the cells of the meshes are hexahedral. The team also developed new numerical methods for solving the governing equations. In particular, they adapted the “summation by parts” mathematical method. Summation by parts is a class of mathematical methods that are often applied to flow problems, including turbulence and wave propagation. These methods provide highly accurate representations of waves and other small features that travel long distances or persist for extended durations. These new techniques guarantee accuracy and stability of the numerical solution.

To demonstrate the full capabilities of ElAc, the team simulated the propagation of waves from a hypothetical meteor explosion occurring 1,000 meters above Berkeley, California. In this scenario, acoustic waves struck the solid earth and generated rapidly traveling seismic waves. The scenario included a southerly wind that varied depending on the terrain and an atmospheric temperature that decreased from 15°C at the surface to –36°C at an altitude of 8 kilometers. The computational grid had a total of 218 million points, and the 20-second-long scenario comprised more than 15,000 time steps. Time histories of the solution were recorded for two surface stations. The study revealed that the new code could be used to describe how meteorological conditions, including vertical stratification in the low atmosphere and variations in wind speed and direction, affect near-ground sound propagation during controlled explosion experiments. They also found that variations in wind speed dramatically affect local infrasound propagation, and these effects can strongly bias interpretation of infrasound signals.

ElAc was used to simulate coupled seismoacoustic wave propagation following a hypothetical meteor explosion above Berkeley, California. (top) The computational domain shows surface topography, the location of the source (green dot), and the line where vertical cross sections are plotted. Color gradient represents elevation in meters. (center) Five seconds after the explosion, acoustic waves from the source strike the solid earth and generate seismic waves, which travel faster than the acoustic waves. (bottom) Eleven seconds after the explosion, the acoustic waves spread along the surface, continuously generating seismic waves. Color gradients for acoustic and elastic waves indicate atmospheric overpressure in pascals and displacement magnitude in meters, respectively.

ElAc has also been used to compute Green’s functions—which characterize response to impulse—for a new method to estimate explosion yield. In fact, the code is key to the lifecycle plan for the National Nuclear Security Administration’s Defense Nuclear Nonproliferation Research and Development Nuclear Forensics Venture. In addition, its ability to produce coupled seismoacoustic simulations will be invaluable for nuclear forensics and explosion-monitoring activities. In the future, the code could be extended to cover longer distances by incorporating Earth’s curvature.

As part of the LDRD project, Kim has also developed a new two-dimensional (2D) acoustic wave propagation code (called AC2D) for modeling acoustic waves from explosions. As Kim notes, “The fully 3D ElAc takes into account all the physics but is expensive in terms of computational resources. The 2D version is much more time and cost effective.” AC2D also includes variations of the atmosphere over land and has uses in nuclear forensics as well as for the Source Physics Experiment (see S&TR, June 2015, Seismic Research Making Waves). “AC2D could also be used for long-range acoustic simulations supporting forensic analyses of explosions or other energetic events at long ranges, up to hundreds of kilometers or more,” says Kim.

Team Effort Yields Fruitful Results

According to Rodgers, the LDRD project is enabling new capabilities. “ElAc fills an important gap. The code not only provides an accurate model of seismic waves in the solid earth but also models the atmosphere, including realistic three-dimensional variations in pressure, temperature, and winds.” The result is an enhanced computational tool for investigating source and propagation effects in controlled experiments, providing improved models that can be validated with real-world experiments. Rodgers adds, “ElAc was a truly multidisciplinary project involving experts in applied mathematics, computer science, geophysics, and acoustics and is exemplary of the types of projects in which Livermore excels.”

—Ann Parker

Key Words: AC2D code, acoustic wave, ElAc (Elastic and Acoustic) code, explosion detection, infrasound, nuclear forensics, nuclear monitoring, seismic modeling, seismoacoustic, seismology, SW4 code.

For further information contact Keehoon Kim (925) 422-6800 (kim84@llnl.gov).

A Solid Hydrogen-Storage Solution

THE key to solving some scientific problems lies in viewing them from multiple angles, taking into account various disciplines including physics, chemistry, mathematics, and computation. At Lawrence Livermore, early-stage research to store hydrogen in solid materials, such as metal hydrides, could be a boon for advancing the hydrogen fuel economy. New results from these efforts, gleaned from this multidisciplinary approach, are reinvigorating scientists engaged in creating a technology infrastructure to produce, distribute, and store hydrogen for fuel cell–based transportation and other industrial applications. Such advances could ultimately transform the hydrogen economy and enable energy security, energy storage, and economic growth through cutting-edge emerging technologies.

One significant challenge to this work is determining how to store enough hydrogen for transportation applications in a sufficiently compact, efficient form factor. Although hydrogen-powered vehicles are on the road today, they require large, heavy, cylindrical tanks of compressed gas. “These cylindrical tanks are needed to keep stresses from building up, but they take up significant room in the car,” says Livermore materials scientist Brandon Wood. “Also, the energy penalty to compress gas into the tanks is high.”

Hydrogen storage tanks for vehicles, such as the one seen here in a Toyota Mirai, are large and cylindrical. By storing hydrogen within a solid material rather than as a pure compressed gas, more hydrogen could be contained in smaller vessels that better fit automotive interiors. (Photo courtesy of Mario Roberto Durán Ortiz. Licensed with CC BY-SA 4.0.)

To store a higher concentration of hydrogen more compactly, scientists are looking to solid materials that can either physically absorb the gas or chemically combine with it. “It’s counterintuitive,” says Wood, “but you can make hydrogen denser if you store it within a solid material rather than as a pure compressed gas. Solid-based storage would also enable development of gas tank designs that are better fitted to an automotive interior than a cylinder.” Such systems must operate at reasonable temperatures, absorb hydrogen in the time it takes to fill a typical gasoline tank, and promptly release hydrogen into the fuel cell as needed—a seemingly insurmountable feat.

Despite the challenges, scientists are not giving up. Wood and other Laboratory researchers are involved in a multi-institutional project called the Hydrogen Storage Materials—Advanced Research Consortium (HyMARC) to accelerate technology development. Funded by the Department of Energy’s (DOE’s) Fuel Cell Technologies Office within the Office of Energy Efficiency and Renewable Energy, HyMARC’s ambitious goal, says Wood, “is to provide the foundational understanding of the kinetic and thermodynamic processes that govern hydrogen storage in materials and to equip the research community with tools that enable the development of on-board solid-state hydrogen storage materials.” HyMARC partners include Lawrence Livermore, Lawrence Berkeley, and Sandia national laboratories, as well as collaborations with several universities and other institutions. While Livermore leads the theory and computational aspects of the project, Sandia spearheads activities to synthesize the materials, and Lawrence Berkeley primarily provides material characterization through chemical analysis.

Speeding Hydrogen Reactions

HyMARC’s approach to technology development is fundamentally different in organization from earlier research strategies. It encourages scientists to work more cooperatively on theory, data, synthesis, and materials characterization, and to proliferate knowledge through databases that the entire community can share. “The project is a cultural shift from how scientists typically work,” says Livermore computational materials scientist Tae Wook Heo. HyMARC is one of several consortia that constitute the DOE Energy Materials Network. Within this network, three groups specifically address the multiple challenges related to hydrogen infrastructure. By simultaneously developing new capabilities and methods and integrating them with specific projects at partner institutions, the researchers aim to more quickly solve technical problems and further emerging technologies.

In collaboration with colleagues at Sandia, researchers at Maahidol University in Thailand, and the National Institute of Standards and Technology, Wood’s team set out to understand the chemistry of a nanoconfined lithium–nitrogen–hydrogen system wherein each pore within the confined space is measured in nanometers. The research revealed the underlying chemistry and showed how eliminating intermediate species in the reaction pathway can accelerate the uptake and release of hydrogen.

Lithium nitride can combine with hydrogen to form lithium amide and lithium hydride and release hydrogen as a gas in the reverse reactions. However, the reactions are slow, in large part, scientists hypothesized, because they need to form an intermediate species before completion. Wood says, “We reasoned that if we could bypass the intermediate species by confining the active species into a medium, the reactions might cycle more quickly.”

(top) During hydrogenation, lithium amide (LiNH_2, blue) grows inward from an outer shell as lithium hydride (LiH, green) evolves from a dispersed molecular cluster to tiny nucleated crystallites that eventually coarsen. (bottom) Dehydrogenation follows the reverse pathway, with lithium nitride (Li_3N, yellow) forming the outer shell. The lithium imide intermediate (Li_2NH, red) forms an outer core between the shell and the inner core. The smaller the particle, the more likely that Li_2NH is eliminated entirely, making the dehydrogenation reaction more efficient.

The Sandia team devised a method to nanoconfine lithium nitride in a matrix of porous carbon to speed the reaction. The researchers conducted experiments and executed molecular dynamics simulations on Livermore’s high-performance computing resources to demonstrate how confining the particles to smaller sizes increased the cycling speed. They found that thermodynamics favoring the formation of the intermediate solid phases become unfavorable with smaller particle sizes because the solid interfaces that form carry an energy penalty that grows as the particle size decreases. Above a certain threshold for particle size, the intermediate phases always form. Keeping the particle size just below that threshold eliminates the intermediate phases and maximizes the speed of the reaction.

The Livermore research is the first to explain the key role of internal solid interfaces in nanoconfined metal hydrides and to provide practical guidance on how to design a more efficient hydrogen-storage medium. “This fundamental understanding is necessary to elucidate the mechanistic pathways of the reaction,” says Wood, “but it also provides practical design rules.”

Finding the Framework for Better Science

Livermore postdoctoral researcher Shinyoung Kang, who works on many of the Laboratory’s HyMARC projects, explains that one of the team’s goals is to create a development framework. She says, “We want to model realistic materials under actual operating conditions. We are modeling materials at several length scales and integrating our models with experiments.”

In the past, most material characterization was done at the macroscale, while simulations on high-performance computers were limited to the scale of several atoms. High-performance computers are now powerful enough to perform theoretical modeling at larger scales and with greater complexity using density functional theory and quantum molecular dynamics. Experimental techniques have also improved, enabling researchers to use x-ray spectroscopy and spectromicroscopy to analyze materials with unprecedented resolution and chemical accuracy. “We are in a place where we can verify our theories with experiments at different scales. This capability is a success story for the experiment–theory approach to science,” says Kang.

The Holy Grail of Hydrogen Storage

The approach taken by Wood and his team to understand the basic chemistry of potential hydrogen-storage materials helped them better evaluate the obstacles for using another promising metal hydride as a storage medium. Wood says, “Magnesium diboride (MgB_2) has been the Holy Grail of hydrogen storage. When hydrogenated, it forms magnesium borohydride. This chemical has one of the highest known theoretical capacities of any material to store hydrogen. However, its hydrogenation reaction is slow—it can take a week to undergo the process—making it unsuitable for transportation applications.”

Hydrogenation forms a mixture of lithium amide and hydride as an outer shell around a lithium nitride particle. Shown here, hydrogen molecules (pairs of grey spheres) enter a nanoparticle of lithium amide (beige and magenta spheres) and lithium nitride (yellow and magenta spheres) confined within a graphene shell. This solid can store significantly more hydrogen by volume than a tank of gaseous hydrogen. (Rendering by Alexander Tokarev.)

A research team, involving Wood and Livermore computational physicist Keith Ray as well as spectroscopists Alex Baker and Jon Lee, characterized the reactants at Lawrence Berkeley’s Advanced Light Source to uncover the reason for the material’s sluggish uptake of hydrogen. Computer simulations were performed at Livermore. Hydrogen gas, they found, reacts with MgB_2 where the material is “broken”—its irregular surfaces at edges and corners. Hydrogen preferentially reacts at these defect sites, suggesting that a faster MgB_2 storage medium would have particles possessing a rougher, more irregular structure to maximize the rate of hydrogenation and dehydrogenation. “Understanding the fundamental chemistry of the materials suggests a pathway to better engineering these materials for hydrogen storage,” says Wood. “We have begun investigating whether adding small amounts of other materials to MgB_2 increases reaction rates.”

The overall research effort at Livermore is expanding in new directions. The Laboratory is discussing a possible collaboration with NASA on hydrogen-storage materials for deep space flights, such as a mission to Mars. Livermore’s expertise in hydrogen technology and space systems is an ideal match for the work. DOE’s H2@Scale program offers another opportunity because such materials can enable energy storage and avoid curtailing intermittent renewables such as wind and solar. Through this program, Livermore is partnering with the private sector to develop technologies in support of building a distributed hydrogen infrastructure that integrates transportation with the industrial and residential sectors. With so many projects underway, clean hydrogen, a fuel possessing one of the highest energy densities per unit of weight found in nature, is poised to play a significant role in the nation’s economy.

—Allan Chen

Key Words: boron, density functional theory, fuel cell, H2@Scale program, hydrogen infrastructure, hydrogen storage, Hydrogen Storage Materials—Advanced Research Consortium (HyMARC), lithium hydride, lithium nitride, lithium–nitrogen–hydrogen system, magnesium borohydride, metal hydrides, molecular dynamics, nanoconfinement.

For further information contact Brandon Wood (925) 422-8391 (wood37@llnl.gov).

Awards

Lawrence Livermore biosafety officer Carolyn Hall was certified as a Specialist Microbiologist in Biological Safety by the American Society for Microbiology (ASM). To attain certification, applicants must hold a Ph.D. in biology, microbiology, or a related field; have four years of experience as a biosafety professional; and pass ASM’s stringent National Registry for Certified Microbiologists examination. The test covers a broad range of topics, including biological risk assessments, work practices and procedures, facility design, equipment operation and maintenance, and federal regulations. Only 44 percent of the applicants passed this year’s examination. Hall holds a Ph.D. in microbiology and immunology from Stanford University.

Workforce Magazine announced that Kellie Glaser, program manager for the Livermore Laboratory Employee Services Association, was a recipient of the 2017 Workforce Game Changers Award. This international award program selects the top human resources practitioners and strategists under the age of 40 who are dedicated to promoting the profession with innovative people-management practices. Glaser was recognized for her efforts in engaging Livermore’s workforce through programs designed to improve work–life balance.

Six Lawrence Livermore employees— Maryum Fatima Ahmed, Brent Blue, Sahar El-Etr, Kim Knight, Jack Kotovsky, and Charalynn Macedo —have received National Security Affairs (NSA) certificates from the Bush School of Government and Public Administration at Texas A&M University. The employees completed a yearlong, graduate-level program that includes courses, seminars, and other activities. The NSA certificate program is administered by the Bush School in cooperation with the Laboratory’s Office of University Relations and Science Education. Sixty-six Livermore employees have graduated from the program since its inception in 2008.

Livermore physicist Andrew MacKinnon has been awarded the John Dawson Award for Excellence in Plasma Physics Research by the American Physical Society’s (APS’s) Division of Plasma Physics. The annual award recognizes a specific outstanding achievement in plasma physics research. MacKinnon was cited by APS “for pioneering use of proton radiography to reveal new aspects of flows, instabilities, and fields in high-energy-density plasmas.”

Patents

Battery Management Systems with Thermally Integrated Fire Suppression

Todd W. Bandhauer, Joseph C. Farmer

U.S. Patent 9,704,384 B2

July 11, 2017

Modulated Method for Efficient, Narrow-Bandwidth, Laser Compton X-ray and Gamma-ray Sources

Christopher P.J. Barty

U.S. Patent 9,706,631 B2

July 11, 2017

Silicone Elastomers Capable of Large Isotropic Dimensional Change

James Lewicki, Marcus A. Worsley

U.S. Patent 9,708,451 B2

July 18, 2017

Mobile App for Chemical Detection

Gregory Klunder, Chadway R. Cooper, Joe H. Satcher, Jr., Ephraim A. Tekle

U.S. Patent 9,708,641 B2

July 18, 2017

Tricyclic Gyrase Inhibitors

Daniel Bensen, Zhiyong Chen, John Finn, Thanh To Lam, Suk Joong Lee, Xiaoming Li, Douglas W. Phillipson, Leslie William Tari, Michael Trzoss, Junhu Zhang, Felice C. Lightstone, Toan B. Nguyen, Sergio E. Wong, Paul Aristoff, Michael Jung

U.S. Patent 9,732,083 B2

August 15, 2017

Non-Contacting “Snubber Bearing” for Passive Magnetic Bearing Systems

Richard F. Post

U.S. Patent 9,739,307 B2

August 22, 2017

S&TR Staff

Scientific Editor: Eric Duoss (Jan/Feb–present)

Managing Editor: Ken Chinn

Publication Editors: Paul Kotta and Caryn Meissner

Writers: Holly Auten, Lauren Casonhua, Allan Chen, Rose Hansen, Arnie Heller, Ben Kennedy, Dan Linehan, and Ann Parker

Designers: Mark Gartland, Mary Gines, Amy Henke, and Alexandria Diaz

Proofreaders: Paul Kotta and Caryn Meissner

Print Coordinator: Diana Horne

S&TR Online: Lauren Casonhua, Rose Hansen, Lanie L. Helms, and Pam Williams

S&TR, a Director's Office publication, is produced by the Technical Information Department under the direction of the Office of Planning and Special Studies. S&TR is available on the Web at str.llnl.gov.

Printed in the United States of America.

Available from:

National Technical Information Service

U.S. Department of Commerce

5285 Port Royal Road

Springfield, Virginia 22161

UCRL-TR-52000-18-1/2

Distribution Category UC-99

January/February 2018

About S&TR

At Lawrence Livermore National Laboratory, we focus on science and technology research to ensure our nation’s security. We also apply that expertise to solve other important national problems in energy, bioscience, and the environment. Science & Technology Review is published eight times a year to communicate, to a broad audience, the Laboratory’s scientific and technological accomplishments in fulfilling its primary missions. The publication’s goal is to help readers understand these accomplishments and appreciate their value to the individual citizen, the nation, and the world. The Laboratory is operated by Lawrence Livermore National Security, LLC (LLNS), for the Department of Energy’s National Nuclear Security Administration. LLNS is a partnership involving Bechtel National, University of California, Babcock & Wilcox, Washington Division of URS Corporation, and Battelle in affiliation with Texas A&M University. More information about LLNS is available online at www.llnsllc.com. Please address any correspondence (including name and address changes) to S&TR, Mail Stop L-664, Lawrence Livermore National Laboratory, P.O. Box 808, Livermore, California 94551, or telephone (925) 423-3432. Our e-mail address is str-mail@llnl.gov. S&TR is available on the Web at str.llnl.gov.

© 2018. Lawrence Livermore National Security, LLC. All rights reserved. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. To request permission to use any material contained in this document, please submit your request in writing to Public Affairs Office, Lawrence Livermore National Laboratory, Mail Stop L-3, P.O. Box 808, Livermore, California 94551, or to our e-mail address str-mail@llnl.gov.

This document was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes.