About
The ORNL AI Seminar Series (Biweekly/Hybrid), organized by the AI Initiative, serves as a platform for researchers and engineers from diverse scientific, engineering, and national security backgrounds spanning ORNL, universities, and industry. Our main objective is to encourage collaboration with the goal of driving transformative advancements in safe, trustworthy, and energy-efficient AI research and its applications.
The seminar will be held every other Thursday from 10 am to 11 am ET. Please reach out to the organizers if you would like to recommend a spearker or give a talk.
Next Presentation
CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback
Microsoft Teams
Time: 10-11 a.m. ET, Monday, 04/25/2024
Dr. Sutanay Choudhury
Abstract
The discovery of new catalysts is essential for the design of new and more efficient chemical processes in order to transition to a sustainable future. We introduce an AI-guided computational screening framework unifying linguistic reasoning with quantum-chemistry based feedback from 3D atomistic representations. Our approach formulates catalyst discovery as an uncertain environment where an agent actively searches for highly effective catalysts via the iterative combination of large language model (LLM)-derived hypotheses and atomistic graph neural network (GNN)- derived feedback. Identified catalysts in intermediate search steps undergo structural evaluation based on spatial orientation, reaction pathways, and stability. Scoring functions based on adsorption energies and barriers steer the exploration in the LLM’s knowledge space toward energetically favorable, high-efficiency catalysts. We introduce planning methods that automatically guide the exploration without human input, providing competitive performance against expert-enumerated chemical descriptor-based implementations. By integrating language-guided reasoning with computational chemistry feedback, our work pioneers AI-accelerated, trustworthy catalyst discovery.
Bio
Dr. Sutanay Choudhury is a Chief Scientist of Data Sciences in Advanced Computing, Mathematics and Data division at Pacific Northwest National Laboratory (PNNL), and the Deputy Director of Computational and Theoretical Chemistry Institute at PNNL. His current research focuses on development and application of representation learning and reasoning methods towards solving challenging problems in computational chemistry and digital health. Dr. Choudhury’s research has been supported by US Department of Energy, Department of Homeland Security, Department of Veterans Affairs, US Department of Defense and Microsoft Research. He also developed StreamWorks, a streaming graph analytics system that received a R&D100 award for novel applications in cyber-security.
Upcoming Presentations
NIST’s Autonomous Research Laboratories for Materials Exploration and Discovery
Room: TBA
Time: 10-11 a.m. ET, Monday, 05/09/2024
Dr. A. Gilad Kusne
Abstract
Autonomous research laboratories (also known as self-driving labs) accelerate the scientific process - letting scientists fail smarter, learn faster, and spend less resources in their studies. This is achieved by placing machine learning in control of automated lab equipment, i.e., putting machine learning in the driver’s seat. The user defines their goals and the machine learning then selects, performs, and analyzes experiments in a closed loop to home in on those goals. By integrating prior knowledge into the machine learning framework, even greater accelerations can be achieved. Knowledge sources include theory, computation, databases, and human intuition. In this talk I will discuss NIST’s diverse set of autonomous labs for materials exploration and discovery. I will also discuss how integrating external knowledge boosts their performance.
Bio
A. Gilad Kusne received his B.S., M.S., and Ph.D. degrees from Carnegie Mellon University. He is a Staff Scientist with the National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, an adjunct professor with the University of Maryland, and a Fellow of the American Physical Society. His research is part of the White House’s Materials Genome Initiative at NIST, a project which aims to accelerate the discovery and optimization of advanced materials. He leads the machine learning team of an international, cross-disciplinary effort building autonomous research systems, with the goal of advancing solid state, soft, and biological materials. For these systems, machine learning performs experiment design, execution (in the lab and in silico), and analysis. For his work, he has been awarded the NIST Bronze Award (highest internal award). He is also the lead founder and organizer of the annual Machine Learning for Materials Research Bootcamp and Workshop (now on its 9thyear)—educating next generation and mid-career material scientists in machine learning.
Past Presentation
Scientific AI surrogate models for simulation and optimization
Room: Building 5700, Room F234
Time: 11:00 a.m. - 12 p.m. ET, Thursday, 03/21/2024
Dr. Raphaël Pestourie
Abstract
In this talk, I will showcase how surrogate models accelerate the evaluation of properties of PDE solutions. I will present a precise definition of the computational benefit of surrogate models and example surrogate models. We will then show how surrogate models can be combined to solve a challenging multiscale problem in optics. We will show that, through a synergistic combination of data-driven methods and direct numerical simulations, surrogate-based models present a data-efficient and physics-enhanced approach to simulating and optimizing complex systems. This approach has the benefit of being interpretable. I will also share ways forward and opportunities where I am actively seeking collaborations.
Bio
Raphaël Pestourie is an assistant professor at Georgia Tech in the School of Computational Science and Engineering. He earned his Ph.D. in Applied Mathematics Harvard University in 2020. Prior to Georgia Tech, he was a postdoctoral associate at MIT Mathematics, where he worked closely with the MIT-IBM Watson AI Lab. Raphaël’s research synergistically combines data-driven surrogate models and direct numerical simulations to accelerate simulation and optimization of complex systems for AI-driven discoveries.
Accelerating Astronomical Discovery: Synergies Between Generative AI, Large Language Models, and Data-Driven Astronomy
Room: Building 5100, Room 140
Time: 10-11 a.m. ET, Thursday, 03/25/2024
Dr. Yuan-Sen Ting
Abstract
The era of data-driven astronomy presents researchers with an unprecedented volume of data, comprising billions of images and tens of millions of spectra. This abundance of information prompts critical questions about expediting astrophysical discoveries, uncovering previously undetected celestial phenomena, and deciphering the intricate physics governing the Universe. In this presentation, we explore the intersection of artificial intelligence and astronomy, aiming to bridge existing gaps by harnessing the synergies between these domains. We highlight the transformative potential of generative AI in tackling complex, non-linear inference problems that have traditionally been considered intractable, offering an alternative to conventional Bayesian inference methods. Moreover, we introduce our work, AstroLLaMA, on developing a Large Language Model (LLM) specifically tailored for astronomical research. This initiative leverages cutting-edge advancements in LLM technology to facilitate hypothesis generation and accelerate discoveries, laying the foundation for the realization of a fully integrated AI Astronomer.
Bio
Yuan-Sen is an Associate Professor in astronomy and computer science at the Australian National University and an Associate Professor in astronomy at the Ohio State University. Yuan-Sen’s research applies machine learning to advance statistical inferences using large astronomical survey data. A Malaysian native, Yuan-Sen received his PhD in astronomy and astrophysics from Harvard University in 2017. After graduating, Yuan Sen was awarded a unique four-way joint postdoctoral fellowship from Princeton University, Carnegie Institute for Sciences, NASA Hubble and the Institute for Advanced Study at Princeton before reallocating to Australia. Yuan-Sen also serves as the co-chair of the NASA Cosmic Programs Stars Science Interest Group and leads future spectroscopic surveys as the science group leader. He is an author of more than 185 publications, many of them on topics at the frontier of astrophysics and machine learning.
ChipNeMo: Domain-Adapted LLMs for Chip Design
Virtual (via Microsoft Teams)
Time: 10-11 a.m. ET, Thursday, 04/11/2024
Robert Kirby, Mingjie Liu
Abstract
ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we instead adopt the following domain adaptation techniques: custom tokenizers, domain-adaptive continued pretraining, supervised fine-tuning (SFT) with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our results show that these domain adaptation techniques enable significant LLM performance improvements over general-purpose base models across the three evaluated applications, enabling up to 5x model size reduction with similar or better performance on a range of design tasks. Our findings also indicate that there’s still room for improvement between our current results and ideal outcomes. We believe that further investigation of domain-adapted LLM approaches will help close this gap in the future.
Bio
Robert Kirby is a Senior Research Scientist at NVIDIA working on the Applied Deep Learning Research team. He received his undergraduate degree in Computer Engineering at The University of Illinois Urbana-Champaign and has been working in various parts of the NVIDIA GPU research and design team for the past 13 years. His research focus has included both large scale language modelling as well as deep learning applications to various parts of the ASIC design flow. Recently he has been focused on studying and improving the process of applying state of the art LLMs to domain specific tasks.
Mingjie Liu is currently a Research Scientist at NVIDIA, where he actively conducts research on Electronic Design Automation. He received his PhD degree in electrical and computer engineering from the The University of Texas at Austin in 2022. His prior research includes applied machine learning for design automation and design automation for analog and mixed-signal integrated circuits. He is currently working on applying Large Language Models for chip design automation tasks.
Schedule
Please reach out if you are interested in presenting at a future event
| Date | Location | Name | Affilication | Talk |
|---|---|---|---|---|
| 03-20-2024 | Building 5700, Room F234 | Dr. Raphaël Pestourie | Georgia Institute of Technology | Scientific AI surrogate models for simulation and optimization |
| 03-25-2024 | Building 5100, Room 140 | Dr. Yuan-Sen Ting | The Australian National University | Accelerating Astronomical Discovery: Synergies Between Generative AI, Large Language Models, and Data-Driven Astronomy |
| 04-11-2024 | Teams | Mr. Robert Kirby Dr. Mingjie Liu |
Nvidia | ChipNeMo: Domain-Adapted LLMs for Chip Design |
| 04-25-2024 | Teams | Dr. Sutanay Choudhury | Pacific Northwest National Laboratory (PNNL) | CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback |
| 05-09-2024 | Hybrid | Dr. A. Gilad Kusne | NIST | NIST’s Autonomous Research Laboratories for Materials Exploration and Discovery |
| 06-06-2024 | Teams | Dr. Ishan Thakkar | University of Kentucky | TBA |
| 06-20-2024 | Teams | Dr. Kutyniok | Ludwig Maximilian University of Munich | TBA |
Organization
For questions, please contact us.
 |
 |
 |
| Jong Youl Choi HPC Data Research Scientist Computer Science and Mathematics Division ONRL |
Chen Zhang Computational Scientist Computer Science and Mathematics Division ONRL |
Prasanna Balaprakash Director of AI Programs Distinguished R&D Staff Scientist Computing and Computational Sciences Directorate, ORNL |