S&TR | July/August 2018

Patterns

Contents

The Laboratory in the News

Constant Vigilance in Forensic Science

Commentary by Bruce E. Warner

The Case for Modern Forensic Science

Livermore’s Forensic Science Center guides the field toward more objective techniques and data-driven results.

X-Ray Machine Does Double Duty

An upgrade to the decades-old Flash X Ray (FXR) expands its contributions to the Laboratory’s national security mission.

A Slam Dunk Program for Aspiring Researchers

At the Laboratory’s annual Research Slam, postdoctoral researchers show their creativity and expertise while building valuable communication skills.

Interweaving Timelines for Faster Solutions

Livermore’s XBraid algorithm uses high-performance computers more effectively and helps reduce the time to solution for various applications.

Awards

Patents

Staff

About S&TR

The Laboratory in the News

Advancing Bioscience with “Brain-on-a-Chip”

Lawrence Livermore scientists and engineers have developed an in vitro “brain-on-a-chip” device for testing and predicting the long-term effects of biological and chemical agents, disease, or pharmaceuticals on the brain. The device, part of the Laboratory’s iCHIP (in vitro chip-based human investigational platform) project, simulates the central nervous system by recording neural activity from multiple brain cell types deposited and grown onto microelectrode arrays. The results were published in the November 21, 2017, edition of PLOS ONE.

To re-create the structural anatomy of the brain, researchers divided the chip into four distinct areas—an inner region further split into three subregions and an outer region representing the brain’s cortex. They deposited primary hippocampal and cortical cells onto the inner and outer regions’ electrodes, and then monitored the “bursts” of electrical potential that cells emit when communicating and observed how the cells interacted over time. The researchers also successfully performed tests with a four-cell insert to prove more cell types could be used simultaneously. “While we cannot fully recapitulate a brain outside of the body, this work is an important step in terms of increasing complexity of these devices and moving in the right direction,” said co-lead author and Laboratory engineer Dave Soscia.

Scientists say the platform is part of the Laboratory’s broader vision for countering emerging and existing threats. It allows them to study the networks formed among various regions of the brain, and obtain timely, human-relevant data without animal or human testing. With the brain-on-a-chip platform, researchers could analyze how disease spreads through the organ, model epilepsy and other neurological disorders, or examine the effects of chemical or biological exposure over several months.

Contact: David Soscia (925) 423-8949 (soscia1@llnl.gov).

Laboratory Licenses Nanolipoprotein Technology

Ann Arbor, Michigan–based EVOQ Therapeutics has licensed the Laboratory’s nanolipoprotein (NLP) technology for cancer immunotherapy, which deploys the body’s own immune system to fight cancer. Developed by Livermore biomedical researchers over the last decade with funds from the Laboratory Directed Research and Development Program, NLPs are water-soluble molecules that are 6 to 30 billionths of a meter in size and resemble high-density lipoproteins (HDL), known as “good cholesterol” in humans. EVOQ is developing a vaccine delivery platform that uses synthetic HDL.

Checkpoint inhibitors, which block normal proteins on cancer cells, are one immunotherapy approach that holds promise in the cancer treatment community. Although inhibitors have only been successful in 20 to 40 percent of cases, research using animal models has shown that when the synthetic HDL delivery is combined with the approach, complete tumor regression is exhibited in about 85 percent of colon carcinoma and melanoma cases. Ideally, synthetic peptides, small fragments of neoantigen proteins, would be incorporated into NLPs along with adjuvants (molecules that activate the patient’s immune system). NLPs would then enter the body’s lymph nodes, activating T-cells that circulate throughout the body and destroy the cancerous tumor cells.

NLPs were initially developed between 2005 and 2008 by a team of Livermore scientists led by Paul Hoeprich. In addition to the use of NLPs for transporting cancer vaccines, Lawrence Livermore scientists continue exploring other applications for the technology. These research areas include developing vaccines for influenza and chlamydia, formulating drug molecules to enhance NLP efficacy, and shuttling therapeutics across the blood–brain barrier.

Contact: Paul Hoeprich (925) 422-7363 (hoeprich2@llnl.gov).

Researchers Create Tunable, Green Detergents

Surfactants, also known as detergents, are used extensively in the cosmetics, oil, food, agriculture, healthcare, and pharmaceutical industries. However, the majority of surfactants are petrochemicals, which can have a negative impact on the environment and increase the nation’s dependence on foreign and domestic oil. With the demand for biosurfactants—those produced from microorganisms—on the rise, scientists at Lawrence Livermore have begun studying a new “tunable” kind that is environmentally friendly and can have broad industrial utility. The research appeared in the January 2, 2018, edition of PLOS ONE.

The new Laboratory surfactant is derived from the red yeast Rhodotorula and is composed of a linear carbohydrate connected to a fatty acid. “The primary challenge is that microorganisms produce biosurfactants as a complex mixture of closely related surfactants, not a pure, single type of compound,” says Matt Lyman, a Livermore biologist and lead author of the paper. “If we can harness the surfactant diversity found in nature and provide it in a pure form that is ‘tunable’ for industry needs, then it becomes a powerful technology beyond what exists today for commercial biosurfactants.” The research was conducted through a collaboration between the Laboratory’s Biosciences and Biotechnology Division and the Forensic Science Center.

The goal of the Livermore team is to use Rhodotorula as a “microbial factory” to produce a base surfactant compound at minimal cost. The team will then perform routine chemical modifications to enable it to slide up and down the hydrophilic–lipophilic balance (HLB) scale, which determines whether a surfactant has more of an affinity toward water or oil. Such a system would provide a “made-to-order” biosurfactant with a specific HLB most useful to customers and their applications.

Contact: Matt Lyman (925) 424-2039 (lyman2@llnl.gov).

Commentary: Constant Vigilance in Forensic Science

Bruce E. Warner

Associate Director for Global Security

EVERY day, Lawrence Livermore advances science and technology in service to national security. Our specialized centers and institutes pursue this mission with novel research projects, first-class facilities, and multidisciplinary staff. As the feature article, The Case for Modern Forensic Science, describes, the Laboratory’s Forensic Science Center (FSC) is a notable example of our longstanding leadership in a rapidly evolving field.

Founded in the early 1990s, the FSC brings many strengths to modern forensic science. Much of the center’s success comes from commitment to both operational and research activities. Analyzing samples for the criminal justice system, the intelligence community, and international entities drives research and development in forensic techniques and instrumentation. Such scientific improvements, in turn, enable FSC staff to process increasingly complicated samples for multiple sponsors. Recent proteomics breakthroughs are direct outgrowths of this balanced approach.

These remarkable achievements are made possible through FSC scientists’ technical excellence and the center’s advancement of analytical methods for chemical, biological, radiological, nuclear, and explosives samples. The FSC applies data-driven forensic analyses to real-world cases, routinely supporting evidentiary efforts for the U.S. Department of Justice and its Federal Bureau of Investigation.

Indeed, the FSC brings many best practices to conventional forensic science, such as using nondestructive techniques that preserve materials for further evaluation. Demonstrating its commitment to producing data of the highest quality, center operations comply with the International Organization for Standardization’s competency and calibration requirements. In addition, the center consistently receives top marks on proficiency tests that require detection of minute quantities of chemicals in highly complex samples. These practices enable FSC scientists to study the composition of a sample and understand its origin through chemical synthesis and attribution projects.

Organizations at home and abroad value the FSC’s capabilities. As an example, the U.S. Environmental Protection Agency relies on the FSC as the reference laboratory for the Environmental Response Laboratory Network, providing analysis methods, standards, and training. Moreover, the FSC is one of only two U.S. laboratories certified by the Organisation for the Prohibition of Chemical Weapons (OPCW) for analysis of environmental and biomedical samples and is one of only 22 laboratories worldwide that supports OPCW treaty verification and alleged-use investigations. In addition, FSC staff, through the Department of State’s Chemical Security Program, train other countries’ first responders to properly identify and investigate chemical threats.

Together FSC’s strengths—a comprehensive portfolio, high-quality research and development, rigorous methodologies, practical experience, and accreditations—tell a compelling, but incomplete, story. Even the best commercial laboratories lack two additional factors contributing to the center’s success. First, since the FSC is a federally funded research and development center—motivated by mission, not profit—sponsors are assured of objective results. Second, the center has access to capabilities beyond its own walls. Whether the need is expertise or equipment, the FSC can draw upon the skill and proficiency of Laboratory colleagues who are experts in a range of scientific and engineering disciplines. With this in-house support, FSC staff can collaborate with the U.S. Department of Homeland Security, the National Nuclear Security Administration, and other partners in an even broader scope of work.

As world events evolve in complexity, the nation’s problems have become increasingly challenging. The FSC’s most important assets are the outstanding people who work together as a team to solve problems. Livermore’s national security mission depends on constant vigilance, and the FSC’s dedication and technical excellence is reflective of the entire Laboratory workforce. Our ability to anticipate and adapt to changing needs makes us an asset to our sponsors, who are motivated to send their most difficult problems here.

The Case for Modern Forensic Science

Chemist Brian Mayer cleans a source component of a mass spectrometry system. The Forensic Science Center (FSC) relies on a suite of complementary instrumentation to meet proficiency requirements for multiple agencies. (Photo by George Kitrinos.)

Success in a high-stakes field does  not come easily. Last fall, Lawrence Livermore’s Forensic Science Center (FSC) completed another grueling 15-day testing period for the Organisation for the Prohibition of Chemical Weapons (OPCW), the international body that oversees compliance with the Chemical Weapons Convention treaty. With the ability to detect evidence of chemical warfare agents (CWAs) both as intact chemicals and through their degradation products, the FSC has earned its standing among OPCW-certified laboratories in the United States as one of only two that analyzes environmental samples and one of three that determines human exposure to CWAs. (See S&TR, June 2013, Ready to Respond to Chemical Weapons.)

OPCW’s proficiency tests challenge the world’s best forensic laboratories in technical skills, analytical methodologies, safety protocols, chain-of-custody procedures, and more. Test samples can be made of any material spiked with unknown chemical compounds at varying concentrations. A single mistake—failure to identify spiking compounds, false-positive results, or misreported data—can strip a laboratory of its accreditation. For the eighth consecutive test, the FSC came out on top with an “A” grade.

“These proficiency tests push the limits of our team, technologies, and technical expertise. The system we have developed over the years to enable success across these testing areas has become foundational to much of the work we execute at the FSC,” says the center’s director Brad Hart. According to Glenn Fox, former FSC director and now associate director for the Laboratory’s Physical and Life Sciences Directorate, “The vision for the FSC was ahead of its time. The Laboratory saw a need for, and invested in, its unique capabilities.”

As the scientific community questions the reliability of traditional forensic science in law enforcement investigations, the FSC provides leadership in advancing conventional methodologies through its accuracy in chemical, biological, radiological, nuclear, and explosives (CBRNE) forensic analyses and its research and development efforts in analytical techniques. Bruce Warner, Livermore’s principal associate director for the Global Security Principal Directorate, states, “The Forensic Science Center has been providing a balanced operational and research portfolio for 27 years. Government sponsors rely on the center’s deep technical capabilities for sample analysis and origin determination and for development of new techniques to address emerging threats.”

Categories of chemical warfare agents (CWAs) include nerve, blistering, and incapacitating agents. The FSC’s chemical forensic work includes characterizing the constantly evolving range of CWAs.

Inflection Point

In 2016, the U.S. President’s Council of Advisors on Science and Technology (PCAST) released a report recommending better scientific standards for forensic methods, specifically those used to compare different forms of evidence such as firearm ballistics, bitemarks, and hair, among others. The document drew on numerous papers, criminal cases, and reports from other federal agencies to capture the scope of the current techniques’ shortcomings. PCAST concluded that methods reliant on “significant human judgment” are too subjective and, therefore, not scientifically defensible in court.

The PCAST report built on an earlier study from the National Research Council regarding the state of forensic science, and the cumulative scrutiny leads to what Hart calls an inflection point in transforming this branch of the discipline. He explains, “Inadequate techniques undermine the criminal justice system and increase resistance to their use in the broader national security community. We see opportunities to improve objectivity in this area because of our focus on advancing capabilities that generate high-quality, defensible data.”

The FSC is already making strides in traditional forensic analysis with projects that expand the possibilities for human identification. Hart states, “Lawrence Livermore has a unique opportunity to influence this field. Few places can offer what we provide—extensive basic science research that informs our development of technologies and capabilities for application to real-world scenarios.”

Strict Science and Safety

One of the FSC’s key functions is advancing science in the public interest, and the center’s staff thrive on the challenge of growing its forensic capabilities. “We like to push technology in new directions. For example, we consider how next-generation versions of instruments, such as mass spectrometers, will affect what we can detect in the environment,” says Carolyn Koester, the center’s deputy director for operations. FSC colleague and chemist Brian Mayer adds, “The center joins sophisticated equipment with multidisciplinary expertise in the same physical laboratory space. This synergy is critical to our continual modernization.” (See Necessity and Invention in Mid-20th Century Film Technology below.)

Although most work done at the FSC is performed by in-house staff, the depth and breadth of Livermore’s expertise across the complex enables investigators from other areas to help conduct diverse examinations. For example, a specialist from another area of the Laboratory may participate in a specific project, or another facility may provide access to complementary equipment. Chemist Pat Grant says, “We can ‘borrow’ scientists from other Laboratory programs for an integrated evaluation, thanks to the scientific and technical diversity onsite.”

Safety is ingrained in all FSC activities. A strict acceptance procedure ensures proper triage, routing, and screening of materials while minimizing human exposure and sample contamination. Biological materials are handled in a dedicated biosafety area, and the cradle-to-grave custody process includes decontamination and proper disposal of all samples after analysis. “Inappropriate handling could result in severe consequences,” notes Koester. “Until we confirm what the specimen is, we treat all samples with the highest precautions,” says FSC deputy director Audrey Williams.

When needed, the center runs 24 hours a day, 7 days a week, and staff are dedicated to optimum and efficient turnaround. “We address real-time problems and think comprehensively across the forensic landscape—an attribute that our external partners value,” explains Hart, citing the FSC’s long-standing chemical attribution signatures program with the U.S. Department of Homeland Security. In addition, as a partner laboratory with the Federal Bureau of Investigation (FBI), the Laboratory’s FSC analyzes special nuclear materials and various chemical threats. The center’s relationship with the FBI became codified after the September 11, 2001, terrorist attacks. Grant is the principal investigator for the center’s FBI casework, which includes analysis of weapons-grade materials and other questioned samples. He notes, “When the FBI needs chemical or nuclear forensic capabilities for uncommon investigations, they often come to us.”

Forensic chemist and FSC deputy director Audrey Williams loads prepared samples into the FSC’s inductively coupled plasma–mass spectrometry system, which is commonly used to analyze a sample’s trace metal components or contaminants. Beforehand, samples undergo acid digestion to remove organic materials while generating and retaining metal ions. (Photo by George Kitrinos.)

The C in CBRNE

Chemical threats account for a significant portion of the FSC’s investigations, so research and development efforts include synthesizing small amounts—as allowed under the Chemical Weapons Convention treaty—of chemical warfare agents to understand possible production methods clandestine actors may use. For example, FSC scientists have studied multiple pathways for making sulfur mustard, also known as mustard gas. “We look for data that make a given synthesis route or the final product unique,” explains Williams. This work informs investigations by comparing a sponsor’s questioned sample to idealized materials. Furthermore, comparing samples obtained by different sponsors may reveal whether they are related.

The FSC’s chemical forensic expertise extends to chemical attribution. By building profiles or “signatures” of different CWAs, FSC staff can determine important information about a sample. A chemical signature is defined as a collection of impurities, including reaction byproducts, degradation products, and unreacted starting chemicals and reagents and their relative concentrations, detected in a final synthetic product. These chemical signatures are used to provide clues to how the sample was synthesized.

According to Williams, the FSC approaches each research question and chemical sample as a puzzle to be solved. “Our research, development, and operational missions strengthen each other because we have built a collective knowledge base,” she says. For instance, the center’s CWA detection strategies for OPCW proficiency tests underpin everyday work identifying chemical signatures and characterizing chemical threats. Williams continues, “We provide expert-level evaluation of our measurements, not just the raw data. We go beyond simply reporting that certain chemicals are present in a sample to pull all the clues together in context. Essentially, X and Y together may suggest Z is occurring.”

The B in CBRNE

The FSC’s success in biological forensic analysis springs from a robust cycle of research followed by applied proof-of-concept studies; partnerships with Livermore’s Biosciences and Biotechnology Division as well as other laboratories; and collaborative scale-up, testing, and deployment phases. Brian Souza, senior biologist and group leader for Biosecurity and Bioforensics, describes this process as unique. “Our biologists think outside the box. In addition, we have access to select agents, including dangerous pathogens,” he says. Souza’s team works closely with the Department of Defense, its Defense Threat Reduction Agency, and other agencies to advance development of biological countermeasures using Livermore’s capabilities in bioinformatics and computational biology.

Biological countermeasures neutralize or reverse physiological responses to naturally occurring and engineered pathogens. To fight antimicrobial resistance, FSC scientists are exploring the potential of viruses that attack bacteria—known variously as phages, bacteriophages, or microbial viruses. Souza’s team and collaborators are developing new methods for characterizing phage genomes. “We need to understand how viruses are organized genetically. Computer modeling of phage proteins streamlines research and development prior to experimentation,” he notes.

Biological countermeasures “go viral” with the use of phages (inset)—viruses that attack bacteria. A phage’s head contains DNA while the rest of the body is made of protein-based structures. Phages attach to receptors on the surface of a bacterium, penetrate the cell, and replicate their DNA, ultimately destroying the cell.

Phage genomes are not as predictably structured as bacterial genomes, and popular DNA-annotation tools fall short of identifying phage genes. Using Livermore’s high-performance computing resources, the team has designed a one–two punch for conducting the required analysis. First, PHANOTATE, a novel computational algorithm, identifies genes by maximizing their translatable parts—called open reading frames—within the phage’s DNA. Next, the Python-based Phage Annotation Toolkit and Evaluator (PhATE) program uses PHANOTATE’s results to predict protein structure and function encoded in the open reading frames. The combination of PHANOTATE and PhATE enables researchers to determine and model genes that when translated into proteins are used during infection and destruction of their host cell. Understanding these mechanisms of action may help researchers discover better and more effective protein-based countermeasures and antimicrobials.

Beyond national security and public health applications, research into viral DNA markers can guide further forensic endeavors. Souza explains, “Informatics tools enable us to assemble DNA sequences faster, so the methods we develop for biological studies can be shared to benefit human forensic work that uses DNA.” In addition, a microbe’s epigenetic DNA changes (those not related to DNA sequencing) and its genetic markers based on environmental exposure can inform studies of similar mechanisms in human proteins. Souza adds, “Our center-based approach to forensic science is crucial. We have several boutique capabilities that, along with investments and skilled partners, allow us to move quickly in this space.”

The R, N, and E

Before the FSC was formally established, Livermore radiochemists measured nuclear and radioactive materials resulting from nuclear explosive testing. The 1990s saw a worldwide increase in material seizures, and since then, FSC scientists have been at the forefront of nuclear forensics and attribution, helping to deter trafficking of illicit materials. (See S&TR, October/November 2014, Promoting International Security through Nuclear Forensics.) Grant remarks, “We have learned much more from comprehensive analyses of real-world interdictions and samples than from scripted exercises or fabricated test specimens.” This accumulated experience has produced a primary reference source for the nuclear forensic field. Grant, along with FSC associates Ken Moody and the late Ian Hutcheon, authored the seminal textbook, Nuclear Forensic Analysis, now in its second edition.

Different aspects of the center’s CBRNE capabilities often dovetail with surprising results. In the FSC’s first high-profile case, a controversial nuclear energy experiment at a California research laboratory caused a fatal explosion. Debris analysis revealed only natural background radioactivity, but further investigation uncovered another potential explosion initiator unrelated to the original hypothesis. The case underscores the importance of comprehensive investigation—an imperative for objective forensic analysis—and reinforces the need for access to a variety of skills and techniques.

(left) Shown here is one of two specimens obtained from a nuclear smuggling enterprise that turned out to be a scam. (right) Researchers evaluated all aspects of the container using multiple imaging techniques, such as neutron radiography. After “breaking and entering” the specimen, the team identified an assortment of packaging and filler materials, including plant matter, pharmaceutical compounds, synthetic fibers, and magnetic objects. (Photo by Pat Grant; radiograph courtesy of Harry Martz, Bill Brown, Randall Thompson, and John Rodriguez.)

When conducting an investigation, FSC scientists must consider both principal and collateral aspects of the work at hand. For explosives, such analysis means identifying impurities and additives within explosive compounds, any residues, and the composition of the primary container. For interdicted radiological or nuclear materials, a complete evaluation may also include microscopic analysis of packaging materials and fibers, biological traces left by the traffickers, and other substances such as dusts or pollens, which could provide route and storage information.

This careful approach means FSC scientists can uncover a hoax just as well as revealing a real threat. In one notable investigation, two containers advertised as smuggled nuclear materials were acquired. Despite being labeled as containing uranium and plutonium, neither object held radioactive or weapons-grade materials. FSC scientists, along with colleagues from Livermore’s Nondestructive Characterization Institute, evaluated all aspects of the containers. They began with a series of nondestructive tests, including gamma-ray spectrometry and x-ray and neutron radiography, to interrogate the specimens’ interiors for safety assessment. Ultimately, subsequent forensic analysis determined that the contents were not dangerous.

Promising Proteomic Identification

Enhancing objectivity in traditional forensic science requires maximizing the potential of tiny pieces of evidence, such as a skin cell or bone fragment. Proteins are a viable source of human identification because they can express DNA mutations and are found in many types of tissue. According to chemist Deon Anex, proteomics can augment DNA analysis in degraded samples such as human remains. He says, “Proteins are more robust in the environment than DNA.” In a recent collaboration, FSC scientists developed a technique for biological identification that uses proteins found in human hair. (See S&TR, July/August 2015, A New Role for Hair in Human Identification.) Anex and Hart are now leading an effort to expand this work to other human tissues.

Proteomic analysis of hair protein begins with an extraction process that dissolves the hair and isolates the peptide chains. Liquid chromatography–mass spectrometry (LC–MS) is used to separate the peptides and analyze their amino acid sequences.

Funded by the Laboratory Directed Research and Development Program, FSC scientists are refining identification methods to use a single hair instead of a bulk sample while also increasing the diversity of the data—that is, incorporating hair from multiple ethnic groups. The project also includes development of an extraction technique for analyzing mitochondrial DNA in hair and for identifying genetically variant peptides in bone, teeth, and skin cells from fingermarks or palmprints. “The techniques we have developed for hair can be modified to use with skin cells,” notes Anex. Furthermore, amelogenin, the protein found where a tooth’s enamel and dentin meet, contains markers for X and Y chromosomes. Thus, an individual’s sex could be determined from analysis of teeth alone, aiding in human identification.

Anex’s team draws on the FSC’s stable of sophisticated equipment to “read” a full protein sequence. Liquid chromatography–mass spectrometry (LC–MS) is an effective divide-and-conquer strategy for this work. After a sample is prepared with an enzyme that segments a protein chain into multiple peptides, LC’s high-pressure liquid flow separates thousands of peptides. Each peptide is then fragmented using MS, and the resulting mass spectrum helps determine the sequence of amino acids. Sequence variations observed in these peptide profiles can be used to identify individuals.

The FSC’s proteomics work continues to grow thanks to the Intelligence Advanced Research Projects Activity, which funds research across many scientific fields for the U.S. intelligence community. Anex and colleagues are building a database of protein mutations for individualized human DNA sequences with an eye on operational applications. In the future, crime laboratories could use a protein analysis kit that contains population-dependent reference standards for peptide markers, enabling faster evidence evaluation. Other applications include identifying disaster victims and creating genetic histories from archeological specimens. Anex says, “FSC scientists are using forensic analysis tools in ways the conventional proteomics community is not. We are excited to continue developing these methods for more specific and practical analyses.”

FSC’s LC–MS system can measure the mass of a peptide in a sample weighing less than 1 microgram with extremely high accuracy. (inset) The nanospray interface injects separated peptides into the MS component as they elute from the LC unit at a rate of 300 nanoliters per minute. (Photos by George Kitrinos.)

Top-Notch Technology

The instrument catalog at Lawrence Livermore’s Forensic Science Center (FSC) reflects the variety of highly specialized work the center’s scientists perform. These investments help ensure that forensic analysis remains focused on defensible data. According to Carolyn Koester, the FSC deputy director of operations, “The center acquires and maintains modern technology to stay at the forefront of forensic science. We have access to equipment not readily available elsewhere.” Altogether, FSC operations comply with the International Organization for Standardization’s competency and calibration requirements, collectively known as ISO 17025.

Most of the FSC’s tools are co-located in approximately 4,000 square meters of laboratory space, though others are available in nearby Lawrence Livermore facilities. “We can address a wide range of problems in a small space,” notes chemist Brian Mayer. Infrared, Raman, and nuclear magnetic resonance spectroscopies provide a wealth of chemical information without damaging samples. X-ray, gamma-ray, and alpha-particle spectrometries support identification and quantitative analyses of materials undergoing radioactive decay.

Many configurations of mass spectrometry (MS) are crucial for identifying and characterizing trace amounts of materials, chemicals, and elements. Koester states, “Measuring mass easily and accurately in complex matrices is a game changer. MS is the common denominator in much of our work.” Inductively coupled plasma–mass spectrometry (ICP–MS) has excellent sensitivity over the element range from lithium to the actinides. In ICP–MS, a high-temperature plasma converts the atoms of the elements in the sample to ions. The masses of the ions are then measured, and concentrations of elements below 1 part per billion can be detected.

While ICP–MS detects inorganic compounds, MS coupled with either gas chromatography (GC) or liquid chromatography (LC) detects organic chemicals. Sample introduction into the MS system by GC or LC allows many hundreds of compounds in a sample to be separated as they move through a column, enabling identification of each of these substances by their unique retention times within the column in conjunction with their mass spectra. GC–MS is valuable for analyzing small organic compounds, such as toxic industrial chemicals and chemical warfare agents. LC–MS is used for the detection of large molecules, including biomolecules that indicate human exposure to chemical agents and the proteins and peptides at the heart of the FSC’s proteomics work.

Ready for Anything

FSC staff have come to expect the unexpected. Grant, who served for 15 years as the center’s first deputy director, says the future of the field’s constantly changing environment is difficult to predict. “Real-world CBRNE samples and scams generally throw something new at us regardless of our experience level,” he states. So, with traditional forensic applications in mind, FSC scientists are increasing the reliability and accessibility of CBRNE forensic analyses.

One forward-looking project uses machine learning for statistical analysis of chemical attribution signatures. “Statistics can provide a wealth of chemical insight into a given sample, and machine learning offers new mechanisms for deep, objective data analysis,” explains Mayer. The team is testing what a computer program can “learn” from data on synthesis routes of the opioid 3-methylfentanyl (3MF). Certain chemical features of the drug indicate, for example, whether the chemist started with a commercially available material versus a homemade concoction. Recently published results show that the computer program can discriminate among 3MF synthesis routes by identifying the presence or absence of specific impurities in a crude reaction mixture. Eventually, FSC staff plan to help make chemical signatures data more relevant to mainstream law enforcement. In fact, Mayer’s team is making this analysis field portable by coding the program for use on a simple laptop.

FSC scientist Carlos Valdez examines the product of a chemical synthesis. (Photo by George Kitrinos.)

Outside the Laboratory, FSC scientists support the U.S. Department of State’s Chemical Security Program (CSP) with specialized CWA training for first responders in other countries. High-priority areas such as Jordan, Lebanon, and Turkey have benefitted from hands-on field experience and tabletop exercises conducted by forensic specialists. FSC staff teach participants about protection protocols, prevention measures for reducing the risk of further contamination, use of screening equipment, and evidence collection. “CSP views us as their chemical threat response and laboratory analysis experts,” says Armando Alcaraz, who leads the FSC’s CSP efforts in addition to overseeing the center’s OPCW work.

Since 2013, the center has worked with response teams all over the world, notably receiving recognition from the Iraqi Ministry of the Interior when CSP-trained responders correctly identified sulfur mustard after an attack in Kirkuk. The multiphase program also helps responders interface with their country’s forensic laboratories to establish and maintain analysis protocols. Alcaraz notes, “When new chemical threats emerge, such as improvised agents, we add them to the curriculum.”

As technology and threat spaces advance, so do the FSC’s capabilities. “The center has become more diversified as times have changed,” observes Fox. Hart agrees, “The challenges our sponsors present are constantly evolving. As a result, we must anticipate and adapt to ensure we are ready for anything that comes through the door.”

—Holly Auten

Key Words: chemical attribution; chemical, biological, radiological, nuclear, explosives (CBRNE) forensic analyses; Chemical Security Program (CSP); chemical warfare agent (CWA); DNA; forensic science; Forensic Science Center (FSC); gas chromatography–mass spectrometry (GC–MS); human identification; inductively coupled plasma–mass spectrometry (ICP–MS); liquid chromatography–mass spectrometry (LC–MS); machine learning; nuclear forensics; Organisation for the Prohibition of Chemical Weapons (OPCW); phage; Phage Annotation Toolkit and Evaluator (PhATE); proteomics.

For further information contact Brian Mayer (925) 423-1128 (mayer22@llnl.gov).

X-Ray Machine Does Double Duty

The Flash X Ray (FXR) is an important diagnostic tool for hydrodynamic testing that enables scientists to capture images of an exploding device. (Photo by George Kitrinos.)

IMAGINE an x-ray machine billions of times more powerful   than those used by dentists, a machine whose x rays can penetrate more than 30 centimeters of steel and provide a stop-motion photo of an explosion. This extreme radiography is the core capability of Lawrence Livermore’s Flash X Ray (FXR), a decades-old linear induction accelerator that continues to assist the Laboratory in fulfilling its national security mission.

First developed at Livermore in the mid-20th century, induction linear accelerators are used in applications requiring extremely high currents of charged particles in a single short pulse. With FXR, scientists can see into the heart of test objects at an exact moment after detonation, revealing data about implosion symmetry. In the past, this sophisticated x-ray machine could only capture one image per experiment. However, a recent upgrade to the system now allows two images to be taken only microseconds apart. These images will provide new understanding into the implosion process of nuclear weapons. FXR’s new double-pulse capability, which has been a long time coming, proves the adage that good things are worth the wait.

Revealing Key Details

FXR is located in the Contained Firing Facility (CFF) at the Laboratory’s remote Site 300. CFF is a modernized hydrodynamic test facility for gathering crucial data that helps scientists assess the operation of a nuclear weapon’s primary stage. (See S&TR, September 2007, Ramrods Shepherd Hydrodynamic Tests; and S&TR, June 2004, A Bang-Up Job Keeping Things Clean at the Contained Firing Facility) A primary typically contains a plutonium shell called a pit. Hydrodynamic tests are conducted to study the behavior of surrogate primary-stage materials in response to extremely high temperatures and pressures. FXR provides high-speed x-ray radiographic images of the process.

Only nonnuclear hydrodynamic experiments are carried out at CFF. These tests involve an inert (nonfissile) material wrapped in a high explosive. When the high explosive is detonated, the resulting explosive compression deforms the inert material making it denser and even melting it. Radiographs taken with FXR can capture an entire test device or concentrate on a small area. Scientists can “view” details of the experiment and gain insight into the process from detonator initiation through the implosion phase.

The Contained Firing Facility at the Laboratory’s Site 300 is home to unique experimental capabilities such as FXR, which helps weapons scientists assess the operation of nuclear weapons.

Lawrence Livermore’s FXR complements the linear induction accelerators at Los Alamos National Laboratory’s Dual-Axis Radiographic Hydrodynamic Test (DARHT) facility. Together, the instruments provide hydrodynamic capabilities to all the national laboratories in the Department of Energy’s weapons complex. FXR’s wide field-of-view images capture large features, providing context for how a weapon’s pieces interact. Scientists depend on such images to determine that no physical interference exists between the exploding device and the supporting and surrounding structures. DARHT’s two x-ray machines record narrow field-of-view, three-dimensional interior images of materials, allowing scientists to examine small features in greater detail.

Data gathered from experiments help weapons scientists verify and normalize computer models used for stockpile stewardship. Livermore’s Scott McAllister, associate program director for hydrodynamic and subcritical experiments, explains, “We combine such models with data from current experiments and past nuclear tests to verify we have confidence that our nuclear stockpile will perform. Codes are improving all the time, so diagnostic tools such as FXR must also evolve to maintain parity to the codes. We are always developing new ways to retrieve more data from each experiment. By obtaining multiple images from a single experiment, we can better test our models.”

During an FXR double-pulse experiment, pulses of electrons, separated by a few microseconds, are created in the machine’s injector and accelerated through electrically independent modules. (Blue cells indicate modules used for the first pulse, while orange cells indicate modules used for the second pulse.) When the enhanced electron beams hit the tantalum target, the electrons lose energy, emitting powerful x rays. The collimator narrows and focuses the x-ray beams onto the exploding device. Photons that pass through the device are recorded by a gamma-ray camera, creating a high-resolution x-ray image.

Two Pulses, More Data

FXR’s newest upgrade essentially divides the machine in half. For each shot, 5 of the 10 injector cells work together to fire a 1.5-megavolt pulse that enters the accelerator tube containing 44 electrically independent modules. Twenty-two modules are used for each of the two pulses. Each module fires sequentially, adding energy and acceleration to the negatively charged particles, propelling them forward until the 9-megavolt beam is traveling close to the speed of light. The electron beam then enters a drift section where magnets “steer” the beam toward the tantalum target.

As the electrons pass through the target, the electric field created by the stationary charged particles of the heavy tantalum nuclei causes the electrons to decelerate and radiate x-ray photons. These photons enter the exploding test device. Most of the photons are absorbed—how many being a function of the density of the material they encounter. The photons that reach the camera create the x-ray image. The two electron beams are spaced 1.5 to 4 microseconds apart, providing two closely spaced images of the exploding device.

Engineer Jennifer Ellsworth and her team reconfigured FXR to create this double-pulse capability, while retaining the machine’s single-pulse functionality. The biggest challenge, Ellsworth notes, was integrating the subsystems and components. “Once the upgrades were in place, we had to ensure everything worked together and the system as a whole was reliable,” she says. As an added challenge, since the facility was in use during regular, day-shift hours, Ellsworth and her team had to conduct all their upgrades during the swing shift.

This team included Livermore personnel, staff from private industry, and colleagues from Los Alamos and Sandia national laboratories. Livermore beam physicists and engineers, as well as FXR colleagues, were heavily involved in the project. CFF crew ran the shots, and employees from Livermore’s Center for Accelerator Mass Spectrometry refurbished the injector’s trigger units. Personnel from Honeywell’s Mission Support and Test Services developed a special two-frame imaging system for the project that was essential to the upgrade effort. The first successful static double-pulse shot occurred in August 2016. Approximately six months later, the team conducted their first two-pulse dynamic experiment with excellent results.

Images captured from FXR’s first “dynamic” experiment show changes to a detonator device during the first and second pulses, which occurred 7.8 and 10.8 microsconds after detonation, respectively.

Flash Forward

The recent upgrades to FXR increase the utility of the decades-old machine, enhancing its contributions to the Laboratory’s national security mission. This valuable capability helps weapons scientist to ensure the safety, security, and effectiveness of the U.S. nuclear stockpile without nuclear testing. Toward this end, FXR is one of the resources that Livermore scientists have turned to in their work on life-extension programs for the W80-4 and W78-1 nuclear warhead systems. Life-extension programs focus on the challenges associated with aging materials, older designs, and aged parts for weapons that were designed with a 20-year service life.

FXR is also being used to evaluate hardware subcomponents for Scorpius, a next-generation x-ray radiographic system at the Nevada National Security Site’s underground U1a Complex. The national laboratories use U1a to conduct subcritical experiments supporting the nation’s nuclear stockpile. Scorpius will provide radiographs from four unidirectional pulses per experiment.

Scientists are invigorated by FXR’s double-pulse capability as it demonstrates what is possible as technology advances. Researchers are using an end-to-end simulation capability for the machine and other simulation tools to improve machine performance for both single- and double-pulse operations. “Eventually, we want the machine to be able to generate two closely spaced full-energy pulses,” says Ellsworth. “To make that function a reality is a big engineering challenge. We would have to reconfigure FXR such that the whole machine can be fired twice. This reconfiguration would allow more x rays to be generated, which would provide a stronger signal and higher penetrating power. Such a capability would allow us to use the double-pulse capability on a wider range of experiments.”

Ultimately, notes McAllister, the goal is to have a machine that can take x-ray movies. Similar to the processes FXR is designed to capture, the machine itself is dynamic and evolving. Each evolution takes FXR another step forward in experimental capability. Together with validated computer models and historic nuclear test data, FXR experiments provide scientists at the national laboratories with the confidence to say, “This weapon will work.”

—Ann Parker

Key Words: Contained Firing Facility (CFF), Dual-Axis Radiographic Hydrodynamic Test (DARHT) facility, Flash X Ray (FXR), hydrodynamic test, linear induction accelerator, Site 300, Scorpius, stockpile stewardship, subcritical experiments, x-ray radiography.

For further information contact Scott McAllister (925) 422-3807 (mcallister1@llnl.gov).

A Slam Dunk Program for Aspiring Researchers

THE postdoctoral research experience is a pivotal one for many scientists. During this time, they dig into the deepest technical details of their disciplines to advance scientific understanding. One challenge postdocs face is aptly communicating their often highly technical research to nonexperts in their field. To help them overcome this challenge, two years ago, the Laboratory’s University Relations and Science Education Office, led by Annie Kersting, launched a friendly competition. Called the Research Slam, the competition invites postdocs to each give a three-minute presentation about their research for a chance to win monetary prizes.

The Research Slam is orchestrated in two rounds. Workshops, seminars, and web resources are available to help postdocs prepare their presentations. Finalists also benefit from one-on-one coaching and have the opportunity to present to judges Director William Goldstein and members of his senior management team. Kris Kulp, director of the Laboratory’s Institutional Postdoc Program, says, “The biggest benefit of the Research Slam is helping postdocs improve their communication and presentation skills. The event forces them to think about the most important aspects of their work.”

First-prize winner Ashley Campbell agrees with the tenet of the competition. “The experience pushed me to be a better communicator. As scientists we have a duty to effectively relay our findings to the world,” she says. Altogether, 13 finalists were chosen from the 49 entrants who competed in the 2017 Research Slam. The diversity and quality of their research is well represented by the four prize winners, who presented on subject matter ranging from biology and particle physics to nuclear science and biosensors.

Ashley Campbell, a postdoctoral researcher in the Laboratory’s Nuclear and Chemical Sciences Division, was the 2017 Research Slam first-place winner.

Microbial Dinner Parties

In January 2016, Campbell, a postdoctoral researcher in the Laboratory’s Nuclear and Chemical Sciences Division, visited the Luquillo Experimental Forest (LEF) in Puerto Rico—the only tropical rainforest in the territorial United States—to collect soil samples for her study into how climate change is affecting smaller scale ecosystems. (See S&TR, April/May 2016, Uncovering Dirty Secrets about Soil Carbon.) She says, “We are investigating how microbes in the soil survive in an ever-changing environment.”

Microbes living in the soil feast on organic matter and regularly expel greenhouse gases, such as carbon dioxide or methane, as byproducts of their metabolic processes. Microbes are a vital part of carbon cycles, which maintain the balance between natural carbon emissions and carbon stored in soils. However, what the microbes eat—and thus how much carbon dioxide they release—varies depending on environmental conditions.

Funded by a Department of Energy (DOE) Early Career Research grant awarded to Laboratory biologist Jennifer Pett-Ridge, Campbell and postdoctoral colleagues from Lawrence Berkeley National Laboratory and the University of California at Berkeley created an experiment to analyze whether new or old carbon sources are preferentially eaten by microbes under varying conditions. The scientists placed soil in jars under 4 different environments and monitored them for 44 days. They found that microbes prefer new carbon sources, such as just-fallen leaves, because they are easier to break down. However, in oxygen-rich environments the microbes will also eat older carbon sources because they can make the oxidative enzymes needed for digesting it. When oxygen is lacking for extended periods of time, the microbial community adapts. Campbell says, “Iron-cycling microbes can pop up in these environments. Microbes will use iron as an electron donor or acceptor (in lieu of oxygen) to help them eat.”

Microbes under 4 different soil conditions were monitored for 44 days to determine whether microbes preferentially eat new or old carbon sources. The red dots and stars indicate times when the microbes were harvested—occurring immediately before and after a redox switch, for example, when fluctuating from oxygen-deficient (anoxic, –O_2) to oxygen-rich (oxic, O_2) conditions and vice versa. (Rendering by Alex Paya.)

Looking forward, the team plans to analyze LEF soils harvested after the 2017 hurricane. Campbell says, “We want to evaluate how soils respond to hurricane conditions and how microbes change after a natural event.” Ultimately, the results of Campbell’s research may provide insight into global phenomena arising from climate change, and may also be useful for climate modeling.

The Universe and Neutrinos

Prevailing theory suggests that at the time of the universe’s birth, the Big Bang created an environment wherein matter and antimatter existed in equal quantities. However, since the two substances annihilate one another, scientists struggle to understand why the universe today consists primarily of matter. Neutrinos—tiny, nearly massless particles with no charge—may be at the heart of solving this cosmic puzzle.

Research Slam second-place winner Jason Brodsky, also a postdoc in the Laboratory’s Nuclear and Chemical Sciences Division, is involved in a multi-institutional collaboration called the neutrinoless Enriched Xenon Observatory (nEXO) to better understand the nature of the neutrino. “The nEXO project is looking to prove the existence of neutrinoless double beta decay—otherwise known as neutrino self-annihilation,” he says. “If we can prove that neutrinos can annihilate themselves, it would validate a particular theory of the universe.” (See S&TR, July/August 2016, A Detector to Study the Neutrino's Nature.)

Researchers are working on the engineering requirements for the neutrinoless Enriched Xenon Observatory detection vessel, which will include a central time projection chamber (inset) and a containment system comprising a low-radioactivity refrigerant, a cryogenic vacuum space, and a water tank equipped with photomultiplier tubes to track and reject background signals. (Rendering by Ryan Chen.)

Neutrino self-annihilation would require that the particle and its antiparticle are one and the same, defying conventions outlined in the Standard Model of physics. Brodsky and colleagues are designing a large, heavily instrumented tank filled with enriched liquid xenon-136 for detecting the decay. “The size of the container will help us have a clean center that is far from everything else in the chamber to prevent signal interference and better enable us to detect the decay process,” he says. “Today, the big focus is on the detector’s engineering requirements.”

With nEXO clearly on its way to becoming a reality, Brodsky is excited for what the future holds. He says, “I like thinking about the influence my research could have on future science and technology.”

Nuclear Safety and Human Health

Two Research Slam participants, Tony Nelson and Allison Yorita, tied for third-place honors for their work in criticality safety and bioscience, respectively. Nelson, a postdoc in the Laboratory’s Nuclear Criticality Safety Division, presented on his efforts to improve the safety of fissile material operations through improved benchmarks for intermediate-energy systems—that is, those that include a nuclear material and a small amount of a moderator, such as water or oil.

In the mid-1940s, a ball of plutonium known as the Demon Core caused two fatal accidents when it was unintentionally brought to critical mass inside a laboratory. Today, nuclear safety engineers use computer simulations to plan and test fissile material operations. Criticality engineers test the simulations’ accuracy using data from benchmark experiments wherein physical materials are brought to their criticality point. The lack of benchmarks for intermediate-energy systems is a major concern. Nelson says, “A certain amount of a fissile material by itself may not be critical, but when it is combined with another element, it can potentially start a nuclear fission chain reaction and release radiation.”

In a project funded by the DOE Nuclear Criticality Safety Program, Nelson and colleagues from Livermore and Los Alamos national laboratories, the Naval Nuclear Laboratory, and L’Institut de Radioprotection et de Sûreté Nucléaire in France designed a test bed to quickly and easily establish missing benchmarks. The design includes stacking uranium plates, polyethylene, and the material to be benchmarked (in this case, hafnium) using remote-controlled machines. Plates are gradually added to the system until it reaches its criticality point. Experimental results are then compared to those from computer simulations to validate radiation transport software. Nelson says, “Our test bed is unique because we can easily adjust the energy and swap out the material we are studying.”

To improve intermediate-energy nuclear benchmarks, Laboratory postdoc Tony Nelson has developed a test bed wherein uranium, polyethylene, and hafnium plates are stacked in a repeating pattern to slowly bring the assembly to critical mass. A polyethylene reflector surrounds the assembly.

The final Research Slam winner, Yorita, is a postdoc in the Laboratory’s Materials Engineering Division who is using her expertise to help rehabilitate victims of the nation’s opioid crisis. Although scientists understand the basic physiological response of drug addiction, they have not discovered which parts of the brain are most affected. Funded through the Laboratory Directed Research and Development Program, Yorita is developing a sensor to monitor the brain’s chemical interactions and possibly help researchers determine the cause of drug addiction. (See S&TR, October/November 2016, Peering into the Brain with Chemical Biosensors.)

Yorita’s device is designed to “listen” to chemical interactions using an enzyme known as glutamate oxidase. “In experiments, an electrode array containing glutamate oxidase is implanted into the brain of a specimen,” explains Yorita. “We then apply a voltage to the electrodes. When the enzyme layer reacts with a specific chemical, it generates hydrogen peroxide, which creates a spike in the electrical current. We are evaluating different areas of the brain and the frequencies at which these signals are being released.” In addition, Yorita is studying how to extend the life of the device in vivo via an enzyme stabilization material. The goal is to increase the device’s lifetime to one year—the amount of time it generally takes to treat opioid addiction.

Slam Ramps Up in the Summer

For Campbell, Brodsky, Nelson, and Yorita, the postoc Research Slam not only allowed them to more effectively communicate their work, it also brought visibility to their projects. “If you put effort into it, the program can help expand your capabilities,” says Campbell. Brodsky adds that strong communication skills, such as those developed through the Research Slam, are essential to advocating the necessity of basic science research.

Biosensors are fully packaged devices slightly wider than a human hair. Using membrane coatings for chemical sensing, the sensors can “listen” to chemical and electrical signals in the brain.

Entries are submitted each summer, and the next event is scheduled for October 2018. Kulp urges postdocs who have previously entered the Research Slam but did not win to re-enter the following year. In fact, Campbell participated in her first Research Slam in 2016, learned from the experience, and came back to win one year later. Says Kulp, “It’s a lot of work, but the program serves as a valuable tool for improving presentation skills and boosting career development.”

—Lauren Casonhua

Key Words: biosensor, carbon cycle, criticality safety, glutamate oxidase, intermediate-energy system, Laboratory Directed Research and Development Program, neutrino, neutrinoless Enriched Xenon Observatory (nEXO), nuclear benchmark, postdoctoral research, Research Slam.

For further information contact Kris Kulp (925) 422-6351 (kulp2@llnl.gov).

Interweaving Timelines for Faster Solutions

IN 1965, Intel cofounder Gordon Moore predicted that the number of transistors in an integrated circuit would double every year, enhancing overall processor performance by increasing clock speed—the rate at which the processors can execute instructions. Ten years later, Moore revised his prediction, reducing the rate of doubling to every two years, a trend that—rather astonishingly—held true for decades. However, by the early 2000s, the pace of advancement in clock speed had slowed as chip components approached fundamental limits in size and the upper bounds of energy usage.

The rapid rise in computing power, once made possible by the succession of ever faster, smaller, and more affordable transistors, enabled scientists to develop and run increasingly complex—and computationally demanding—simulations without lengthening the time to solution. As improvements to processor speeds wane, those who design and use such scientific applications have been faced with the specter of plateauing application performance. Fortunately, researchers continue to look beyond hardware innovations, such as chip components, to speed up simulations by finding and refining application- and algorithm-based methods for reducing solution time, most notably through increased use of parallelism. The focus on parallelism is particularly important because increases to computing power today and in the future will occur only through more, not faster, processors.

Software Offers Solutions

Parallel processing involves programming a computer system to perform certain tasks simultaneously using multiple processors, rather than sequentially. This approach is essential for efficient operations on today’s high-performance computing systems, which might have thousands or even millions of processors. For a typical scientific simulation, intended to help researchers understand how a system evolves through space and time, parallelism might involve simultaneously calculating many grid elements on a spatial grid (spatial parallelism) or many time steps in the simulation (temporal parallelism), or both.

Although spatial parallelism is ubiquitous in scientific computing, parallel time integration is still a nascent area for exploiting performance gains. Researchers began exploring the method back in the 1960s, but it only gained traction in the scientific computing community over the last decade. The reason for this latent interest, surmises Livermore computational mathematician Jacob Schroder, is that the approach is more difficult to execute than more traditional methods, and it seems counterintuitive, since to humans, time is a sequential concept.

(left) Typically, time steps in a simulation are solved sequentially, as depicted in this conceptual image that shows figures passing information from one to another in order. (right) Parallel time integration techniques use the answers from less precise versions of the problem to accelerate the calculation of finer scale versions, which allows the application to more rapidly converge to a solution of the desired accuracy.

Even now, researchers working on parallel time integration face some skepticism, especially from individuals who have not yet exhausted their opportunities for accelerating application performance through more conventional approaches. Computational mathematician Rob Falgout says, “The problem is that many people have not yet faced a bottleneck in their work, and thus they are hard to convince regarding the utility of this method. For them, the issue is down the road, but I find it difficult to imagine that they will not have to address it eventually. Scientists are always striving for greater simulation accuracy.” He suggests that since greater accuracy generally requires more computationally expensive simulations, these researchers will ultimately need to exploit every possible avenue for reducing the time to solution for their codes.

At Lawrence Livermore, home to a host of large and sophisticated applications, improvements in application speed and accuracy are driven by national and global security challenges, and thus the need for new approaches is more immediate here than at many institutions. Fortunately, with support from the Laboratory Directed Research and Development Program, Falgout, Schroder, and their Livermore colleagues have developed a promising solution—a novel software that works in tandem with existing high-performance computing applications to calculate all of a problem’s time steps simultaneously. Called XBraid, the software has decreased solution time by as much as 50 times for some types of simulations. (See S&TR, September 2016, Laying the Groundwork for Extreme-Scale Computing.)

Scalable and Nonintrusive

As the name suggests, XBraid functions by “braiding” multiple timelines of differing accuracies together for a faster solution, using multigrid methods similar to those the team has successfully applied to speed up spatial calculations. Schroder provides a simple example. If a scientist wants to predict the temperature for a given city with a high level of fidelity—say, by calculating temperature at an interval of once a second for a whole year—and approaches the problem in the standard fashion, the system would have to calculate some 31 million time steps sequentially. By incorporating XBraid, the application would instead simultaneously calculate the solution at several different levels of detail—predicting the temperature, for instance, once a day, once an hour, once a minute, and once a second.

In this example, the only timeline computed sequentially is the coarsest (least precise) one, which has 365 time steps. Those coarse-grained solutions are then fed into the even finer scale problem (the once-an-hour calculation). These solutions are in turn fed into the finer scale version of that problem, and so on, accelerating the solution process at the finest scale. Despite the unconventional approach, the accuracy of the results is the same, up to a user-defined tolerance. Schroder says, “The goal of parallel in time is not to compute a different solution than sequential calculation methods—it is simply to do it more quickly.” Further, the method is scalable, so the problem can be sped up by increasing the number of processors working on the calculation.

Although XBraid is neither the first nor the sole successful method of solving time intervals concurrently, its nonintrusive nature provides a huge advantage to those who steward and use today’s large scientific applications. XBraid was created with Livermore’s current stable of applications in mind, allowing the applications to take advantage of parallel time integration without having to be rewritten, which for a large application could easily take a dozen or more software developers 5 to 10 years to complete.

This conceptual drawing illustrates the vast, complex resources that may exist in the electrical grid of the future. Pairing the XBraid algorithm with Livermore’s GridDyn simulation software could enable faster yet accurate simulations of grid operations, which could benefit both operations and contingency planning.

Power Grids and Neural Networks

Over the past few years, the XBraid team has experimented with optimizing XBraid for various problems, many of which have never successfully incorporated such methods. For one project, the team added temporal parallelism to GridDyn, a Livermore-developed, open-source tool for simulating the electrical power grid. These simulations offer insight into how scheduled interruptions (maintenance, for instance) and unscheduled interruptions (weather, equipment malfunctions, or even an act of terrorism) might affect the supply and distribution of electricity to homes and businesses. The XBraid team focused their initial efforts on simulations involving scheduled outages of grid components. Running GridDyn on Livermore’s Quartz cluster, the team demonstrated a roughly 50-fold speed-up for a power system modeling the Western United States.

“Power grid simulations are one of the more promising application areas for parallel time integration,” says Schroder, noting that rapid and accurate simulation results are especially important as more renewable power is incorporated into the grid. “Researchers have a weather model to determine wind speed at different times of day. However, one cannot predict precisely when the wind will die,” he adds. “If an unexpected disruption occurs, such as if wind power drops sooner than expected, it creates a sudden cascade of events within the network. Fortunately, we are starting to see some success with simulating unexpected disruptions.”

XBraid has also produced favorable results in other challenging areas, such as training artificial neural networks. (See S&TR, June 2016, Deep Neural Networks Bring Patterns into Focus.) With this popular and effective method of machine learning, data (often in the form of digitized images) are fed through a network, and the resulting output is compared to the desired target, which typically classifies the data into a category. Errors between the output and the target are “back propagated” through the network, assigning blame to the parts of the network responsible for the error. As training proceeds, links in the network strengthen and weaken themselves and converge toward a configuration that minimizes overall error. These networks “learn” by sequentially processing thousands or millions of such training runs.

Sequential training for artificial neural networks often involves showing the network a series of labeled digital images to “teach” it to categorize data correctly. Livermore researchers have sped up this popular machine-learning method by exploiting its similarities with sequential time-step techniques used in more standard scientific simulations.

The idea to apply XBraid to machine learning came one day when Schroder realized the inherent similarities between the serial processing of information in neural network training and that of more traditional time-dependent simulations. By treating training runs as time steps, the team has successfully applied the method such that a 50-training-run problem can be used to help solve a 100-training-run problem, and so forth. In initial feasibility studies, the team saw a six-fold improvement in training time over 13,000 training runs. The work is still in its early stages, but Schroder is optimistic. He says, “Parallel time integration can be highly valuable for machine learning because it provides a new perspective and novel parallel capabilities to the development community.”

A Necessary Step

XBraid provides a nonintrusive, powerful, open-source solution to the bottleneck posed by performing sequential time steps for problems involving thousands or millions of time steps. Thus far, the XBraid team has demonstrated the viability of this relatively unexplored way to reduce time to solution. XBraid speeds up computations and helps applications make better use of today’s supercomputers, for which the number of processors grows with every generation. Asserts Falgout, “Our work is not just an interesting project—it is a necessary one.”

—Rose Hansen

Key Words: algorithm, artificial neural network, GridDyn, machine learning, Moore’s law, multigrid, parallelism, parallel time integration, power grid, XBraid.

For further information contact Rob Falgout (925) 422-4377 (falgout2@llnl.gov).

Awards

Victor Reis, former senior adviser in the Department of Energy’s (DOE’s) Office of the Secretary and Undersecretaries, became the third recipient of the John S. Foster Jr. Medal. Established by Lawrence Livermore National Security, LLC, and bestowed annually by the director of Lawrence Livermore, the medal recognizes an individual for exceptional leadership in scientific, technical, and engineering development and policy formulation in support of U.S. nuclear security objectives. Reis was recognized for his significant contributions to national security; his innovative leadership in science and technology; and dedication to national service, particularly for guiding the nation’s nuclear program through the end of underground testing. Reis spent more than 50 years in government and private industry in service to national security. He held leadership positions in the Office of Science and Technology Policy in the Executive Office of the President and at the Massachusetts Institute of Technology’s Lincoln Laboratory, the Defense Advanced Research Projects Agency, and the U.S. Departments of Defense (DOD) and Energy (DOE). As the DOE assistant secretary for Defense Programs, he defined his legacy as the architect and original sponsor of the National Nuclear Security Administration’s science-based Stockpile Stewardship Program. In addition to the John S. Foster Jr. Medal, Reis also has received DOE’s James Schlesinger Medal and two DOD Medals for Distinguished Public Service.

At the international SC17 conference, a High Performance Computing for Manufacturing (HPC4Mfg) project aimed at improving the operational efficiency of paper manufacturers earned an HPC Innovation Excellence Award. The project, led by Livermore researcher Will Elmer, was a collaboration with consumer goods manufacturer Procter & Gamble and required the development of a parallel program called p-fiber. The program is capable of quickly preparing the fiber geometry and meshing input needed to model thousands of paper fibers at once. As part of the HPC4Mfg effort, Elmer and team generated up to 20 million finite elements and modeled the most paper fibers in a simulation to date. At SC17, HPC Innovation Excellence awards were presented for only 10 projects out of more than 110 submissions. The award recognizes outstanding achievements enabled through HPC.

Charles Orth, a Lawrence Livermore physicist for more than 40 years, was presented with the 2017 Albert Nelson Marquis Lifetime Achievement Award by the publication Marquis Who’s Who, a directory of short biographies of notable figures. The publication bestows the award on a biographee who has demonstrated “leadership, excellence, and longevity within their industry and profession.” Orth has been involved in a broad range of research, including the negative search for quarks, high-altitude cosmic-ray spectrometry, Monte Carlo nuclear electromagnetic cascade calculations, and real-time corrected telescope images, as well as extensive work in the National Ignition Facility. Orth also was directly involved with the joint Livermore–NASA project to conceptualize the VISTA spacecraft, a rocket design that could be powered by inertial confinement fusion.

Patents

Systems and Methods for Enhancing Optical Information

Peter Thomas Setsuda DeVore, Jason T. Chou

U.S. Patent 9,857,660 B2

January 2, 2018

Filtration Device for Rapid Separation of Biological Particles from Complex Matrices

Sangil Kim, Pejman Naraghi-Arani, Megan Liou

U.S. Patent 9,861,939 B2

January 9, 2018

Additive Manufacturing of Short and Mixed Fibre-Reinforced Polymer

James Lewicki, Eric B. Duoss, Jennifer Nicole Rodriguez, Marcus A. Worsley, Michael J. King

U.S. Patent 9,862,140 B2

January 9, 2018

Encapsulated Proppants

Roger D. Aines, William L. Bourcier, Eric B. Duoss, Jeffery James Roberts, Christopher M. Spadaccini, Joshuah K. Stolaroff

U.S. Patent 9,862,880 B2

January 9, 2018

Wireless Battery Management Control and Monitoring System

James M. Zumstein, John T. Chang, Joseph C. Farmer, Jack Kovotsky, Anthony Lavietes, James Edward Trebes

U.S. Patent 9,869,726 B2

January 16, 2018

S&TR Staff

Scientific Editor: Eric Duoss (Jan/Feb–March), Kathryn Mohror (April/May–present)

Managing Editor: Ken Chinn

Publication Editors: Paul Kotta and Caryn Meissner

Writers: Holly Auten, Lauren Casonhua, Allan Chen, Rose Hansen, Arnie Heller, Ben Kennedy, Dan Linehan, and Ann Parker

Designers: Mark Gartland, Mary Gines, Amy Henke, and Alexandria Diaz

Proofreaders: Paul Kotta and Caryn Meissner

Print Coordinator: Diana Horne

S&TR Online: Lauren Casonhua, Rose Hansen, Lanie L. Helms, and Pam Williams

S&TR, a Director's Office publication, is produced by the Technical Information Department under the direction of the Office of Planning and Special Studies. S&TR is available on the Web at str.llnl.gov.

Printed in the United States of America.

Available from:

National Technical Information Service

U.S. Department of Commerce

5285 Port Royal Road

Springfield, Virginia 22161

UCRL-TR-52000-18-7

Distribution Category UC-99

July/August 2018

About S&TR

At Lawrence Livermore National Laboratory, we focus on science and technology research to ensure our nation’s security. We also apply that expertise to solve other important national problems in energy, bioscience, and the environment. Science & Technology Review is published eight times a year to communicate, to a broad audience, the Laboratory’s scientific and technological accomplishments in fulfilling its primary missions. The publication’s goal is to help readers understand these accomplishments and appreciate their value to the individual citizen, the nation, and the world. The Laboratory is operated by Lawrence Livermore National Security, LLC (LLNS), for the Department of Energy’s National Nuclear Security Administration. LLNS is a partnership involving Bechtel National, University of California, Babcock & Wilcox, Washington Division of URS Corporation, and Battelle in affiliation with Texas A&M University. More information about LLNS is available online at www.llnsllc.com. Please address any correspondence (including name and address changes) to S&TR, Mail Stop L-664, Lawrence Livermore National Laboratory, P.O. Box 808, Livermore, California 94551, or telephone (925) 423-3432. Our e-mail address is str-mail@llnl.gov. S&TR is available on the Web at str.llnl.gov.

© 2018. Lawrence Livermore National Security, LLC. All rights reserved. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. To request permission to use any material contained in this document, please submit your request in writing to Public Affairs Office, Lawrence Livermore National Laboratory, Mail Stop L-3, P.O. Box 808, Livermore, California 94551, or to our e-mail address str-mail@llnl.gov.

This document was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes.