From Hyperscale Models to Heterogeneous SoCs: Industrializing Neural Network Deployment with ONNC

ISES Docs:

Large language models (LLMs) have become an increasingly prominent feature in various computing platforms, such as data centers, smartphones, and microcontrollers. As such, their potential applications in both human-to-machine interfaces (HMIs) and machine-to-machine interfaces (M2MI) have been extensively studied and discussed. However, implementing hyper-scale LLMs on heterogeneous multicore System-on-Chips (SoCs) poses significant challenges. In this talk, we will discuss the key challenges, including issues related to hardware/software interfaces, verification, and SoC architecture exploration. At the end of this talk, we will propose a practical workflow and corresponding solutions we’ve applied in many cases. And see how our standard tool – ONNC – is used in the workflow.

Luba Tang photo

Luba Tang


Skymizer Taiwan, Inc.

Members Only

Sorry ISES Docs are exclusive to ISES Members

For access please either login to your membership account or visit our Membership page to sign up for ISES membership.