Lawrence Livermore National Laboratory



Decorative image for opening article.
(Photo by Lanie Helms.)

Grassroots Innovation Grows Great Computational Ideas

The Computation Directorate at Lawrence Livermore provides researchers with the cutting-edge hardware and software needed to conduct potentially world-changing work in the Laboratory’s mission space. Capabilities range from web application development to some of the most powerful computers on the planet, all driven by innovation aimed at benefitting the nation. One way Computation sustains technical leadership is by innovating for itself—investing in the people and ideas that will support Livermore’s pioneering work today and tomorrow.

Harnessing the special skills of its software engineers, developers, programmers, and system administrators, Computation has adopted a novel set of programs designed to foster worthwhile concepts and see them to their logical conclusions. Developer Day, Hackathon, and Idea Days can take a programmer’s innovative idea and help her determine how to put that idea to work.

Diving into the Talent Pool

Lawrence Livermore employs close to 8,000 people. Some are dedicated to operating the world’s most powerful laser while others model complex systems on supercomputers and still others work to advance global security, to name just a few. Each effort, along with the rest of Livermore’s programs and operations, requires some degree of Computation’s support. To meet the specific needs of each Livermore organization, many Computation employees are assigned—or “matrixed”—to organizations all across the Laboratory. While allowing Computation employees to focus on their customers’ needs, this arrangement can also make it difficult for geographically scattered Computation employees to get together and share ideas. “The matrix approach is good for Livermore programs because it allows them to pull from the large pool of Labwide talent,” says Kyle Dickerson, a web application developer matrixed to the Global Security Directorate. “But a side effect is that a matrixed person might not be interacting very frequently with other people of the same specialty.”

Developer Day brings together the Laboratory’s software community to improve productivity and sustainability, amplify useful software practices, and promote shared practices. Dickerson was a member of the event’s inaugural organizing team, which also included David Beckingsale, Todd Gamblin, and Cyrus Harrison. Their plan was to jump-start communication among different programs doing similar work. Dickerson states, “The programs have a lot of common frameworks, programming languages, tools, and work environments. Many of us are also solving common problems, so if one of us solves a problem, it would be more efficient for the Laboratory if another group did not have to solve that same problem all over again.”

Development groups from various Laboratory organizations gathered together in September 2017 for the first Developer Day.
Development groups from various Laboratory organizations gathered together in September 2017 for the first Developer Day, where ideas and solutions could be shared across the Laboratory. Feedback was overwhelmingly positive, and another Developer Day is being planned for summer 2018. (Photo by Randy Wong.)

Avoiding unnecessary repetition is a large part of what made the first Developer Day a huge success in September 2017. The day began with a series of “lightning talks”—five-minute presentations that development teams from around the Laboratory gave about their work and the languages, frameworks, and platforms they use. These short introductions also helped bridge the information gap and inform attendees about other opportunities throughout the Laboratory. After the lightning talks, the organizers encouraged participants to strike up conversations during short breaks and further explore the efforts of their colleagues. Longer talks focused on hiring and onboarding the right candidates, along with “deep dives” into special programming concerns at Livermore—such as how to synchronize code and data across an air-gapped network—that cannot be solved with standard industry approaches. The feedback, Dickerson says, was overwhelmingly positive. In a 2017 survey, 100 percent of respondents said they hoped the event would be held yearly, and plans are already underway to bring Developer Day back in the summer of 2018.

Sharing ideas and collaborating can have concrete results. Dickerson remembers one attendee realizing that his next project might not take eight months as he had thought. “He said, ‘If I just talk to two groups at the Laboratory, I can probably get it done in a month instead.’ Those groups had already solved all the issues he was facing,” says Dickerson. “Developer Day is just one day of attendees’ time. If, because of it, a project takes one month instead of eight, that is a win.”

Hacking It

Big wins in productivity can lead to even bigger ideas. To this end, Computation several years ago launched Hackathon (see S&TR, June 2013, Software Developers Get Their Hack On). The event is described as “a 24-hour chance to brainstorm, foster creativity, learn new things, accomplish something fun that might not be on the top of a developer’s to-do list, build prototypes, and explore something new—even if it fails.” Gary Laguna, a computer scientist supporting Livermore’s Environmental Protection Department, sits on Hackathon’s steering committee. He explains, “What often happens is that someone at Hackathon uses the time to flesh out an idea they’ve been thinking about but never had the time to work on in their day-to-day responsibilities. Hackathon gives people a chunk of time to try out an idea and see whether it works.”

For Justin Barno, a developer in Global Security’s geophysical modeling program, interest in Hackathon began as a way of learning Spark, a new Java interface that could help solve some of his group’s data challenges. The challenges were not immediately urgent, but he knew they would be someday soon. “Whether it was Spark or another technology, we needed something,” says Barno. “Hackathon was the perfect place to see whether the technology would succeed or fail, or whether it was even useful in the first place.” Today Barno is a member of the organizing committee tasked with evolving the thrice-yearly event—and keeping the coolers full of soda. The atmosphere that allows participants to “fail fast” comes, Laguna says, from its grassroots organizational structure—or relative lack thereof. “We have a really good recipe right now,” he adds. “I would be hesitant to put too much structure into the event.”


Computation's thrice-annual Hackathon.


Computation’s thrice-annual Hackathon invites participants from all over Lawrence Livermore to work on any project that may benefit their work in the short or long term. Organizers say the 24-hour timeframe encourages people to “fail fast” and learn a lesson, then move on to the next idea. (Photo by Randy Wong.)

For Robert Blake, Yuriy Ayzman, and Shivshankar Sundaram, the spring 2017 Hackathon was an opportunity to experiment with how magnetic resonance images of the human heart are processed. Usually, doctors have to painstakingly separate the heart from the rest of the picture, pixel by pixel. However, by the end of the 24-hour Hackathon, the three had determined how to use neural networks to understand test images better and produce results that rivaled those of existing tools. In short, their Hackathon-inspired idea yielded an innovation with much promise.

The Next Step

For successes such as those achieved at Hackathon, Computation also provides a funding source that supports the most promising projects into their next stage of evolution. During a twice-yearly event called Idea Days, any Computation employee can propose an idea to a committee of colleagues and, if the idea is accepted, receive up to two full weeks of funding. Proposals are informal, short, and known only to the committee until the selected projects are published. Idea Days does not fund actual research but rather supports projects meant to benefit Computation and the Laboratory more broadly. In May 2017, one such proposal came from Ayzman, Blake, and Sundaram’s team, hoping to build on their Hackathon project and further automate cardiac image processing. The three proposed enhancing deep-learning models in a way that would greatly benefit machine-learning efforts across the Laboratory, in addition to making image processing in general more efficient and robust.

A subset of proposals are selected for “Pitch Day,” where employees explain their idea in detail—the problem to be solved, how to solve it, and the benefits expected. When a funded project is completed, the team is required to report back to the committee on the results achieved. The entire process is designed to be fluid, uncomplicated, and unrestricted so as not to impede creativity or originality. Ayzman, Blake, and Sundaram were successful with their Idea Days application. The project was so successful, in fact, that further funding was approved by Katie Lewis, head of Computation’s Applications, Simulations, and Quality Division. Today, the team is using its image-processing automation approach to improve data analysis at the National Ignition Facility, the world’s most powerful laser and the site of important experiments in stockpile stewardship.

Idea Days, Hackathons, and Developer Days together form a mechanism that allows software engineers, application developers, web designers, and a host of other Computation employees to smoothly and efficiently pioneer improvements that better enable the entire Laboratory to deliver on its missions. By thus cultivating new ideas from all aspects of the Computation Directorate, the Laboratory is equipping itself with the technological know-how that will be needed for the next generation of research—and researchers.


Cyrus Harrison, a Developer Day organizer, Livermore computer scientist, and associate division leader, explains his work in the Applications, Simulations, and Quality Division during a




Cyrus Harrison, a Developer Day organizer, Livermore computer scientist, and associate division leader, explains his work in the Applications, Simulations, and Quality Division during a “deep dive,” a talk that delves into a specific issue or endeavor. (Photo by Lanie Helms.)

“My belief is that the people who are drawn to events like these are the people who will truly be the drivers at the Laboratory in the future,” says Laguna. “They are the ones who want to do things beyond their day-to-day jobs. They want to push the envelope and try new things. That is something the Laboratory and the nation will always need.”

—Ben Kennedy

Key Words: application development, Computation Directorate, Developer Day, Hackathon, Idea Days, innovation, image processing, web development.

For further information contact Lori Diachin (925) 422-7130 (diachin2@llnl.gov).