• 08:40 – 09:00

Opening Speech – Challenges & Opportunities of LLM and Generative AI: A Hardware Perspective

Recent advancements in Large Language Models (LLMs) and Generative Artificial Intelligence (Generative AI) have presented an exciting opportunity for developers, as exemplified by ChatGPT, which can mimic human dialogue and decision-making. However, the tremendous computing power and energy consumption required by the hardware implementing these models pose a considerable cost and challenge, due to a huge number of parameters and complex neural network architecture.

This talk will explore the potential of designing more efficient hardware for deployment and inference of large language models, from an integrated circuit (IC) hardware perspective. Possible methods of AI model compression, such as quantization, pruning or knowledge distillation will be covered as well as AI chip design based on Computing-in-Memory (CIM), Deep-Learning Accelerator (DLA) and Chiplets. This talk will also discuss the hardware and software co-design methodology and tools needed for improving the design quality and productivity.

Watch on ISES TV > PPTs in ISES Docs >
Dr. Ming-Der Shieh photo

Dr. Ming-Der Shieh

CTO of Electronic and Optoelectronic System Research Laborations (EOSL)

Industrial Technology Research Institute (ITRI)

Ming-Der Shieh received the B.S. degree in electrical engineering from National Cheng Kung University, Taiwan, in 1984, the M.S. degree in electronic engineering from National Chiao Tung University, Taiwan, in 1986, and the Ph.D. degree in electrical engineering from Michigan State University, East Lansing, in 1993. From 1988 to 1989, he was an engineer at United Microelectronic Corporation, Taiwan. After receiving the Ph.D. degree, he joined the Department of Electronic Engineering, National Yunlin University of Science & Technology (NYUST). Since 2002, he has been with the Department of Electrical Engineering, National Cheng Kung University (NCKU), where he is currently a professor. His research interests include VLSI design and testing, VLSI for signal processing, and digital communication.

Dr. Shieh was the Chairman of EE Department at NYUST (1999-2002) and NCKU (2014-2017), and Dean of Miin Wu School of Computing at NCKU from 2021-2022. He also served as the Deputy General Director of Information and Communications Research Laboratories (ICL) at Industrial Technology Research Institute (ITRI) from 2010 to 2014, and was an Adjunct Research Fellow at the Office of Science and Technology, Executive Yuan, Taiwan from 2017 to 2019. In addition, he served as the Chairman of Taiwan IC Design Society (TICD) (2016-2018), Associated Editor of IEEE Transactions on Circuits and Systems – Part I (2010~2012), and General Chair/Program Chair/Technical Committee Member of several international conferences. Currently, as the Chief Technology Officer of Electronic and Optoelectronic System Research Laboratories (EOSL) at ITRI, he is responsible for strategic planning and cross-disciplinary collaboration. He is in charge of the IC research and development in EOSL, particularly in the area of AI chip design from different aspects, heterogeneous chiplet design and integration, and automotive IC designs.

View Full Profile

ITRI is a world-leading applied technology research institute with more than 6,000 outstanding employees. Its mission is to drive industrial development, create economic value, and enhance social well-being through technology R&D. Founded in 1973, it pioneered in IC development and started to nurture new tech ventures and deliver its R&D results to industries. ITRI has set up and incubated companies such as TSMC, UMC, Taiwan Mask Corp., Epistar Corp., Mirle Automation.

View Full Profile