Computing Architecture for Large-Language Models (LLMs) and Large Multimodal Models (LMMs)

Large-language models (LLMs) have achieved remarkable performance in many AI applications, but they require large parameter size in their models. The parameter size ranges from several billions to trillion parameters, and results in huge computation requirements on both training and inference. General speaking, LLMs increasing more parameters are to explore "Emergent Abilities" for AI models. On the other hands, LLMs with fewer parameters are to reduce computing burden to democratize generative AI applications.