About
The ORNL AI Seminar Series (Biweekly/Hybrid), organized by the AI Initiative, serves as a platform for researchers and engineers from diverse scientific, engineering, and national security backgrounds spanning ORNL, universities, and industry. Our main objective is to encourage collaboration with the goal of driving transformative advancements in safe, trustworthy, and energy-efficient AI research and its applications.
The seminar will be held every other Thursday from 10 am to 11 am ET. Please reach out to the organizers if you would like to recommend a spearker or give a talk.
Next Presentation
Upcoming Presentations
Past Presentation
Scientific AI surrogate models for simulation and optimization
Room: Building 5700, Room F234
Time: 11:00 a.m. - 12 p.m. ET, Thursday, 03/21/2024
Dr. Raphaël Pestourie
Abstract
In this talk, I will showcase how surrogate models accelerate the evaluation of properties of PDE solutions. I will present a precise definition of the computational benefit of surrogate models and example surrogate models. We will then show how surrogate models can be combined to solve a challenging multiscale problem in optics. We will show that, through a synergistic combination of data-driven methods and direct numerical simulations, surrogate-based models present a data-efficient and physics-enhanced approach to simulating and optimizing complex systems. This approach has the benefit of being interpretable. I will also share ways forward and opportunities where I am actively seeking collaborations.
Bio
Raphaël Pestourie is an assistant professor at Georgia Tech in the School of Computational Science and Engineering. He earned his Ph.D. in Applied Mathematics Harvard University in 2020. Prior to Georgia Tech, he was a postdoctoral associate at MIT Mathematics, where he worked closely with the MIT-IBM Watson AI Lab. Raphaël’s research synergistically combines data-driven surrogate models and direct numerical simulations to accelerate simulation and optimization of complex systems for AI-driven discoveries.
Accelerating Astronomical Discovery: Synergies Between Generative AI, Large Language Models, and Data-Driven Astronomy
Room: Building 5100, Room 140
Time: 10-11 a.m. ET, Thursday, 03/25/2024
Dr. Yuan-Sen Ting
Abstract
The era of data-driven astronomy presents researchers with an unprecedented volume of data, comprising billions of images and tens of millions of spectra. This abundance of information prompts critical questions about expediting astrophysical discoveries, uncovering previously undetected celestial phenomena, and deciphering the intricate physics governing the Universe. In this presentation, we explore the intersection of artificial intelligence and astronomy, aiming to bridge existing gaps by harnessing the synergies between these domains. We highlight the transformative potential of generative AI in tackling complex, non-linear inference problems that have traditionally been considered intractable, offering an alternative to conventional Bayesian inference methods. Moreover, we introduce our work, AstroLLaMA, on developing a Large Language Model (LLM) specifically tailored for astronomical research. This initiative leverages cutting-edge advancements in LLM technology to facilitate hypothesis generation and accelerate discoveries, laying the foundation for the realization of a fully integrated AI Astronomer.
Bio
Yuan-Sen is an Associate Professor in astronomy and computer science at the Australian National University and an Associate Professor in astronomy at the Ohio State University. Yuan-Sen’s research applies machine learning to advance statistical inferences using large astronomical survey data. A Malaysian native, Yuan-Sen received his PhD in astronomy and astrophysics from Harvard University in 2017. After graduating, Yuan Sen was awarded a unique four-way joint postdoctoral fellowship from Princeton University, Carnegie Institute for Sciences, NASA Hubble and the Institute for Advanced Study at Princeton before reallocating to Australia. Yuan-Sen also serves as the co-chair of the NASA Cosmic Programs Stars Science Interest Group and leads future spectroscopic surveys as the science group leader. He is an author of more than 185 publications, many of them on topics at the frontier of astrophysics and machine learning.
ChipNeMo: Domain-Adapted LLMs for Chip Design
Virtual (via Microsoft Teams)
Time: 10-11 a.m. ET, Thursday, 04/11/2024
Robert Kirby, Mingjie Liu
Abstract
ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we instead adopt the following domain adaptation techniques: custom tokenizers, domain-adaptive continued pretraining, supervised fine-tuning (SFT) with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our results show that these domain adaptation techniques enable significant LLM performance improvements over general-purpose base models across the three evaluated applications, enabling up to 5x model size reduction with similar or better performance on a range of design tasks. Our findings also indicate that there’s still room for improvement between our current results and ideal outcomes. We believe that further investigation of domain-adapted LLM approaches will help close this gap in the future.
Bio
Robert Kirby is a Senior Research Scientist at NVIDIA working on the Applied Deep Learning Research team. He received his undergraduate degree in Computer Engineering at The University of Illinois Urbana-Champaign and has been working in various parts of the NVIDIA GPU research and design team for the past 13 years. His research focus has included both large scale language modelling as well as deep learning applications to various parts of the ASIC design flow. Recently he has been focused on studying and improving the process of applying state of the art LLMs to domain specific tasks.
Mingjie Liu is currently a Research Scientist at NVIDIA, where he actively conducts research on Electronic Design Automation. He received his PhD degree in electrical and computer engineering from the The University of Texas at Austin in 2022. His prior research includes applied machine learning for design automation and design automation for analog and mixed-signal integrated circuits. He is currently working on applying Large Language Models for chip design automation tasks.
CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback
Microsoft Teams
Time: 10-11 a.m. ET, Monday, 04/25/2024
Dr. Sutanay Choudhury
Abstract
The discovery of new catalysts is essential for the design of new and more efficient chemical processes in order to transition to a sustainable future. We introduce an AI-guided computational screening framework unifying linguistic reasoning with quantum-chemistry based feedback from 3D atomistic representations. Our approach formulates catalyst discovery as an uncertain environment where an agent actively searches for highly effective catalysts via the iterative combination of large language model (LLM)-derived hypotheses and atomistic graph neural network (GNN)- derived feedback. Identified catalysts in intermediate search steps undergo structural evaluation based on spatial orientation, reaction pathways, and stability. Scoring functions based on adsorption energies and barriers steer the exploration in the LLM’s knowledge space toward energetically favorable, high-efficiency catalysts. We introduce planning methods that automatically guide the exploration without human input, providing competitive performance against expert-enumerated chemical descriptor-based implementations. By integrating language-guided reasoning with computational chemistry feedback, our work pioneers AI-accelerated, trustworthy catalyst discovery.
Bio
Dr. Sutanay Choudhury is a Chief Scientist of Data Sciences in Advanced Computing, Mathematics and Data division at Pacific Northwest National Laboratory (PNNL), and the Deputy Director of Computational and Theoretical Chemistry Institute at PNNL. His current research focuses on development and application of representation learning and reasoning methods towards solving challenging problems in computational chemistry and digital health. Dr. Choudhury’s research has been supported by US Department of Energy, Department of Homeland Security, Department of Veterans Affairs, US Department of Defense and Microsoft Research. He also developed StreamWorks, a streaming graph analytics system that received a R&D100 award for novel applications in cyber-security.
NIST’s Autonomous Research Laboratories for Materials Exploration and Discovery
Room: TBA
Time: 10-11 a.m. ET, Monday, 05/09/2024
Dr. A. Gilad Kusne
Abstract
Autonomous research laboratories (also known as self-driving labs) accelerate the scientific process - letting scientists fail smarter, learn faster, and spend less resources in their studies. This is achieved by placing machine learning in control of automated lab equipment, i.e., putting machine learning in the driver’s seat. The user defines their goals and the machine learning then selects, performs, and analyzes experiments in a closed loop to home in on those goals. By integrating prior knowledge into the machine learning framework, even greater accelerations can be achieved. Knowledge sources include theory, computation, databases, and human intuition. In this talk I will discuss NIST’s diverse set of autonomous labs for materials exploration and discovery. I will also discuss how integrating external knowledge boosts their performance.
Bio
A. Gilad Kusne received his B.S., M.S., and Ph.D. degrees from Carnegie Mellon University. He is a Staff Scientist with the National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, an adjunct professor with the University of Maryland, and a Fellow of the American Physical Society. His research is part of the White House’s Materials Genome Initiative at NIST, a project which aims to accelerate the discovery and optimization of advanced materials. He leads the machine learning team of an international, cross-disciplinary effort building autonomous research systems, with the goal of advancing solid state, soft, and biological materials. For these systems, machine learning performs experiment design, execution (in the lab and in silico), and analysis. For his work, he has been awarded the NIST Bronze Award (highest internal award). He is also the lead founder and organizer of the annual Machine Learning for Materials Research Bootcamp and Workshop (now on its 9thyear)—educating next generation and mid-career material scientists in machine learning.
Towards Petascale In-Package Computing with Unconventional Technologies and Architectures
Microsoft Teams
Time: 10-11 a.m. ET, Thursday, 06/06/2024
Dr. Ishan Thakkar
Abstract
Growing performance and energy efficiency demands of AI workloads put tremendous pressure on today’s computing systems, breaking down the two longstanding technological pillars of computing evolution: Moore’s law and von Neumann computer architecture. This has jeopardized the continuous evolution of computing systems in terms of their energy efficiency and computing capabilities. Our research focuses on avoiding this undesired outcome. This talk will showcase how we strive to design fundamentally high-speed and energy-efficient in-package computing systems by employing the principles of heterogeneity, parallelism, and reconfigurability, to contribute innovative solutions that are based on unconventional, but strong, technological pillars such as Integrated Electro-Photonics, Processing-in-Memory, Neuromorphic Hardware Design and Stochastic Computing.
Bio
Dr. Ishan Thakkar is an Assistant Professor in the Department of Electrical and Computer Engineering at the University of Kentucky, Lexington, KY. He received his Ph.D. and M.S. in Electrical Engineering from Colorado State University (CSU), Fort Collins, CO. Dr. Thakkar is the recipient of the Outstanding Reviewer Award from IEEE/ACM CODES+ISSS 2022. He is the recipient of the Best Paper Award from ACM GLSVLSI 2021, a Best Paper Award Nomination from IEEE ISVLSI 2021, the Best Paper Award Honorary Selection from ACM GLSVLSI 2020, the Best Paper Award from the IEEE/ACM SLIP 2016 workshop, a Best Paper Nomination from IEEE ISQED 2016, and a Best Paper Finalist Selection from the IEEE TMSCS journal in 2015, for his research contributions. He is currently an Associate Editor for the IEEE TCVLSI Newsletter. He serves in the ACM SIGDA Executive Team as the Social Media Chair. His research broadly focuses on the design and optimization of unconventional (more-than-Moore) architectures and technologies for energy-efficient, reliable, and secure computing. More specific more-than-Moore technology interests of his include silicon photonics, optical computing, in- memory computing, stochastic/unary computing, monolithic 3D (M3D) integration, and polymer and transparent conductive oxides-based photonic devices and sensors.
Reliability of Artificial Intelligence: Chances and Challenges (canceled)
Microsoft Teams
Time: 10-11 a.m. ET, Thursday, 06/20/2024
Dr. Kutyniok
Abstract
Artificial intelligence is currently leading to one breakthrough after the other, in industry, public life, and the sciences. However, one current major drawback worldwide, in particular, in light of regulations such as the EU AI Act and the G7 Hiroshima AI Process, is the lack of reliability of such methodologies.
In this lecture, we will provide an introduction into this vibrant research area. We will discuss the role of a theoretical perspective to this highly topical research direction, and survey recent advances, for instance, concerning explainability. Finally, we will discuss fundamental limitations, which seriously affect reliability of artificial intelligence, and discuss the necessity of next generation AI computing.
Bio
Gitta Kutyniok (https://www.ai.math.lmu.de/kutyniok) currently has a Bavarian AI Chair for Mathematical Foundations of Artificial Intelligence at the Ludwig-Maximilians-Universität München, and is in addition affiliated with the DLR-German Aerospace Center and the University of Tromso. She received her Diploma in Mathematics and Computer Science as well as her Ph.D. degree from the Universität Paderborn in Germany, and her Habilitation in Mathematics in 2006 at the Justus-Liebig Universität Gießen. From 2001 to 2008 she held visiting positions at several US institutions, including Princeton University, Stanford University, Yale University, Georgia Institute of Technology, and Washington University in St. Louis. In 2008, she became a full professor of mathematics at the Universität Osnabrück, and moved to Berlin three years later, where she held an Einstein Chair in the Institute of Mathematics at the Technische Universität Berlin and a courtesy appointment in the Department of Computer Science and Engineering until 2020. In 2023, together with colleagues she founded the start-up EcoLogic Computing GmbH.
Gitta Kutyniok has received various awards for her research such as an award from the Universität Paderborn in 2003, the Research Prize of the Justus-Liebig Universität Gießen and a Heisenberg-Fellowship in 2006, and the von Kaven Prize by the DFG in 2007. She was invited as the Noether Lecturer at the ÖMG-DMV Congress in 2013, a plenary lecturer at the 8th European Congress of Mathematics (8ECM) in 2021, and the lecturer of the London Mathematical Society (LMS) Invited Lecture Series in 2022. She was also honored by invited lectures at both the International Congress of Mathematicians 2022 (ICM 2022) and the International Congress on Industrial and Applied Mathematics (ICIAM 2023). Moreover, she was elected as a member of the Berlin-Brandenburg Academy of Sciences and Humanities in 2017 and of the European Academy of Sciences in 2022, and became a SIAM Fellow in 2019 and an IEEE Fellow in 2024. She currently acts as LMU-Director of the Konrad Zuse School of Excellence in Reliable AI (relAI) in Munich, serves as Vice President-at-Large of SIAM, and is spokesperson of the DFG-Priority Program “Theoretical Foundations of Deep Learning” and of the AI-HUB@LMU, which is the interdisciplinary platform for research and teaching in AI and data science at LMU.
Gitta Kutyniok’s research work covers, in particular, the areas of applied and computational harmonic analysis, artificial intelligence,compressed sensing, deep learning, imaging sciences, inverse problems,and applications to life sciences, robotics, and telecommunication.
Empowering Graph Deep Learning at Scale
Microsoft Teams
Time: 10-11 a.m. ET, Thursday, 07/11/2024
Dr. Xiaorui Liu
Abstract
Graph deep learning such as Graph Neural Networks (GNNs) has attracted tremendous research interest and become one of the most exciting and emerging research topics in AI and scientific computing. GNNs not only facilitate knowledge discovery for numerous impactful applications but also accelerate critical scientific discoveries across various scientific domains. However, they face new complexities and unique scalability challenges as compared to traditional graph analytic or deep learning algorithms due to the end-to-end training requirements and non-iid nature of graph data.
This talk first summarizes current solutions and challenges for training GNNs at scale and then presents a promising implicit modeling perspective to significantly reduce memory, computation, and communication costs. Finally, this talk delves into the practical applications in web-scale recommender systems. The talk provides insight to revolutionize the scalability, efficiency, and effectiveness of training graph deep learning models at exascale.
Bio
Dr. Xiaorui Liu has been an Assistant Professor of Computer Science at North Carolina State University since Fall 2022. He received his Ph.D. degree in Computer Science from Michigan State University in 2022. His research interests include large-scale machine learning, trustworthy artificial intelligence, deep learning on graphs/language/vision, and AI applications. He was awarded the ACM SIGKDD Outstanding Dissertation Award (1st Runner-up) in 2023, Amazon Research Awards in 2022 and 2023, NCSU Data Science Academy Award in 2023, NCSU Faculty Research and Professional Development Award in 2023, Best Paper Honorable Mention Award at ICHI 2019, MSU Cloud Computing Fellowship in 2021 and Engineering Distinguished Fellowship in 2017. Dr. Liu has published innovative works in top-tier conferences such as NeurIPS, ICML, ICLR, KDD, AISTATS, WWW, SIGIR, ICDE, SDM, ICDM, and CIKM. He also organized and presented 9 tutorials related to his research in large-scale GNNs, distributed machine learning, graph representation learning, and trustworthy AI. More details can be found on the homepage: https://sites.google.com/ncsu.edu/xiaorui/
Causal representation learning in temporal settings
Microsoft Teams
Time: 11:00 a.m. - 12 p.m. ET, Thursday, 08/22/2024
Dr. Sara Magliacane
Abstract
Causal inference reasons about the effect of unseen interventions or external manipulations on a system. Similar to classic approaches to AI, it typically assumes that the causal variables of interest are given from the outset. However, real-world data often comprises high-dimensional, low-level observations (e.g., pixels in a video) and is thus usually not structured into such meaningful causal units. Causal representation learning aims at addressing this gap by learning high-level causal variables along with their causal relations directly from raw, unstructured data, e.g. images or videos.
In this talk I will focus on learning causal representations in temporal sequences, e.g. sequences of images. In particular I will present some of our recent work on causal representation learning in environments in which we can perform interventions or actions. I will start by presenting CITRIS (https://arxiv.org/abs/2202.03169), where we leveraged the knowledge of which variables are intervened in each timestep to learn a provably disentangled representation of the potentially multidimensional ground truth causal variables, as well as a dynamic bayesian network representing the causal relations between these variables. I will then show iCITRIS (https://arxiv.org/abs/2206.06169), an extension that allows for instantaneous effects between variables. Finally, I will focus on our most recent method, BISCUIT (https://arxiv.org/abs/2306.09643), which overcomes one of the biggest limitations of our previous methods: the need to know which variables are intervened. In BISCUIT we instead leverage actions with unknown effects on an environment. Assuming that each causal variable has exactly two distinct causal mechanisms, we prove that we can recover each ground truth variable from a sequence of images and actions up to permutation and element-wise transformations. This allows us to apply BISCUIT to realistic simulated environments for embodied AI, where we can learn a latent representation that allows us to identify and manipulate each causal variable, as well as a mapping between each high-level action and its effects on the latent causal variables.
Bio
Dr. Sara Magliacane is an Assistant Professor at the Amsterdam Machine Learning Lab, University of Amsterdam. Her research explores how causality can enhance machine learning (ML) algorithms in robustness, cross-domain generalization, and safety. She focuses on causal representation learning, causal discovery, and causality-inspired ML, investigating how causal concepts help ML and reinforcement learning adapt to new domains and nonstationarity.
Previously, Dr. Magliacane was a Research Scientist at the MIT-IBM Watson AI Lab and a postdoctoral researcher at IBM Research NY. She holds a PhD from VU Amsterdam, with internships at Google Zürich and NYC, and has a background in Computer Engineering from Politecnico di Milano, Politecnico di Torino, and the University of Trieste. Her work is recognized through various publications and esteemed conference participation, establishing her as a leading expert in causality and machine learning.
Schedule
Please reach out if you are interested in presenting at a future event
| Date | Location | Name | Affilication | Talk |
|---|---|---|---|---|
| 03-20-2024 | Building 5700, Room F234 | Dr. Raphaël Pestourie | Georgia Institute of Technology | Scientific AI surrogate models for simulation and optimization |
| 03-25-2024 | Building 5100, Room 140 | Dr. Yuan-Sen Ting | The Australian National University | Accelerating Astronomical Discovery: Synergies Between Generative AI, Large Language Models, and Data-Driven Astronomy |
| 04-11-2024 | Teams | Mr. Robert Kirby Dr. Mingjie Liu |
Nvidia | ChipNeMo: Domain-Adapted LLMs for Chip Design |
| 04-25-2024 | Teams | Dr. Sutanay Choudhury | Pacific Northwest National Laboratory (PNNL) | CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback |
| 05-09-2024 | Hybrid | Dr. A. Gilad Kusne | NIST | NIST’s Autonomous Research Laboratories for Materials Exploration and Discovery |
| 06-06-2024 | Teams | Dr. Ishan Thakkar | University of Kentucky | Towards Petascale In-Package Computing with Unconventional Technologies and Architectures |
| 06-20-2024 | Teams | Dr. Kutyniok | Ludwig Maximilian University of Munich | Reliability of Artificial Intelligence: Chances and Challenges |
| 07-11-2024 | Teams | Dr. Xiaorui Liu | North Carolina State University | Empowering Graph Deep Learning at Scale |
| 08-22-2024 | Teams | Dr. Sara Magliacane | University of Amsterdam | Causal representation learning in temporal settings |
| 10-24-2024 | Teams | Dr. Graham West | Meharry Medical College School | Echoes of Athens: Bringing Advances in AI/ML to Ancient Greek Papyrology |
| 11-07-2024 | Teams | Dr. Lav Varshney | University of Illinois Urbana-Champaign | Information Lattice Learning |
| 12-05-2024 | Teams | Fernanda Foertter | Oak Ridge National Laboratory | AI Data Readiness |
Organization
For questions, please contact us.
 |
 |
 |
 |
| Yan Liu Computational Scientist Computational Sciences & Engineering Division ORNL |
Jong Youl Choi HPC Data Research Scientist Computer Science and Mathematics Division ORNL |
Chen Zhang Computational Scientist Computer Science and Mathematics Division ORNL |
Prasanna Balaprakash Director of AI Programs Distinguished R&D Staff Scientist Computing and Computational Sciences Directorate, ORNL |