Private LLM technology: Security-layer definitions and optimal silicon solutions

The privacy and security concerns related to execution data, model parameters, and processing algorithms have assumed an increasing pivotal role in the rapid development and massive deployment of Large Language Model (LLM) technology. This paper provides an overview of private LLM by addressing its working principles, application scenarios, security requirements, training and inference algorithms, and more importantly, its silicon and chip implementation in order to bring this game-changing technology to real-world products. This paper is organized into five sections. The first section is devoted to the background and motivations for proposing private LLM technology. According to different requirements for privacy and security, in the second section we categorize the proposed private LLM technology into three application scenarios (security levels) and then present the corresponding algorithms related to training and inferences. In order to converge private LLM into optimum silicon implementation, we present the proposed Cornami solution and its comparisons with existing solutions (GPU and ASIC) in terms of power consumption, cost and processing latency in Section 3 and Section 4. In the last section, we make some conclusions and note further discussions.