Program

Abstract
The emergence of powerful generative AI (e.g. large language / vision models) would not have been possible without recent advances in computing systems and accelerators. This talk sheds light on the important role that generative AI itself can play in designing the next generation of computing systems and hardware that in turn would fuel the next generation of AI breakthroughs. Concretely, I will discuss our work on learned optimization for hardware resource allocation and model mapping, which inspired a new and ongoing trend of policy gradient methods for solving combinatorial optimization in computer systems, a generalizable deep reinforcement learning method for chip floorplanning that saved several weeks of design cycle for Google TPUs, and an automated framework for full-stack HW/SW co-design that resulted in drastic Perf/TCO improvements in custom accelerator design. Finally, I will discuss the opportunities and challenges for future computing systems in the era of large generative models.

Bio
Azalia Mirhoseini is a Member of Technical Staff at Anthropic, and an incoming Assistant Professor of Computer Science at Stanford University. Previously, she was a Staff Research Scientist and Team Lead at Google Brain, where she co-founded the Machine Learning for Systems Team. Azalia has published more than 40 peer-reviewed papers at scientific venues such as Nature, ICML, ICLR, NeurIPS, UAI, ASPLOS, SIGMETRICS, DAC, DATE, and ICCAD. She has received a number of awards, including the MIT Technology Review 35 under 35 award, the Best Ph.D. Thesis Award at Rice, and a Gold Medal in the National Math Olympiad in Iran. Her work has been covered in various media outlets, including WIRED, CNBC, ABC News, MIT Technology Review, and IEEE Spectrum.

Abstract
Direct mind-machine teaming will help us treat brain disorders, augment the healthy brain, and shed light on how the brain as an organ gives rise to the mind. Delivering on this promise requires the design of computer systems that delicately balance the tight power, latency, and bandwidth trade-offs needed to decode brain activity, stimulate biological neurons, and control assistive devices most effectively.

This talk presents my group's design of a standardized and general computer architecture for future brain interfacing. Our design enables the treatment of several neurological disorders (most notably, epilepsy and movement disorders) and lays the groundwork for brain interfacing techniques that can help augment cognitive control and decision-making in the healthy brain. Central to our design is end-to-end hardware acceleration, from the microarchitectural to the distributed system level. Key insights are undergirded via detailed physical synthesis models and chip tape-outs in a 12nm CMOS process.

Bio
Abhishek Bhattacharjee is an Associate Professor of Computer Science at Yale University. His work on hardware optimizations for memory translation has influenced the design of TLBs in AMD CPUs, starting with the Zen 1 architecture, and in NVIDIA's GPUs, starting with the Ampere architecture. His work on software optimizations for memory translation has been shipped in the Linux OS since the 4.14 kernel. More recently, Abhishek has been building flexible and low-power architectures for brain-computer interfacing — the topic of this talk.

Abstract
ChatGPT recently became one of the fastest growing new applications in history, thanks to its intriguing text generation capabilities that are able to answer questions, write poetry, and even problem solve. Large Language Models are now being integrated in fundamental ways into products around the tech industry. The possibilities are extraordinary, but much research remains to make these systems reliable and trustworthy, as well as integrate them into applications seamlessly. Additionally, the computational challenges behind large language modeling are also quite important. Systems for training and deploying these models must be highly scalable and run at extreme efficiency, because the amount of work necessary to converge a model can be extraordinarily large. The cost of deploying these models is a barrier to their deployment and must be lowered significantly. In this talk, I'll discuss the work we have been doing at NVIDIA to optimize systems for Large Language Model training and inference, and highlight some of the challenges that remain for future work.

Bio
Bryan Catanzaro is Vice President of Applied Deep Learning Research at NVIDIA, where he leads a team of AI researchers working on chip design, audio and speech, language modeling, graphics and vision, with the goal of finding practical new ways to use AI for NVIDIA's products and workflows. DLSS, Megatron, CUDNN, Pascaline, WaveGlow and DeepSpeech are some of the projects he's helped create. Bryan received his PhD in EECS from the University of California, Berkeley.