ASRock Industrial Advances Memory-Efficient Edge AI

ASRock Industrial and Phison Electronics have announced a collaboration focused on improving the deployment of large language models (LLMs) in edge computing environments. By combining ASRock Industrial's edge AI platforms with Phison's AI memory expansion architecture, the companies aim to reduce memory constraints that traditionally limit local AI inference in enterprise and industrial applications.

Memory Constraints in Edge AI Deployment
As organizations increasingly deploy AI workloads closer to where data is generated, memory capacity has become a significant limiting factor. Large language models often require substantial DRAM or VRAM resources, increasing system cost and restricting deployment on compact edge devices.

The joint solution addresses this challenge by combining edge AI hardware with a memory hierarchy that extends available model capacity beyond conventional system memory. The approach is intended for industrial automation, enterprise AI, intelligent manufacturing, and on-premise AI deployments where data processing must occur locally rather than in centralized cloud environments.

AI Memory Expansion Architecture
The collaboration integrates ASRock Industrial's AI BOX-A395 platform with Phison's aiDAPTIV technology. According to the companies, the system enables a 120-billion-parameter large language model to operate locally using 64 GB of system memory supplemented by 85 GB of aiDAPTIV cache memory.

The architecture is designed to reduce conventional memory requirements by up to 50% compared with traditional deployments of similarly sized models. Rather than relying exclusively on DRAM or VRAM, the solution uses SSD-based memory expansion to store and access model data during inference operations.

By creating an additional high-speed storage layer optimized for AI workloads, the platform can support larger models while preserving system memory resources for operating systems, edge software agents, and concurrent applications.

Edge AI Platforms and Inference Performance
The implementation includes the AI BOX-A395 platform powered by AMD Ryzen AI Max+ 395 processors and the NUC Ultra 300 BOX Series based on Intel Core Ultra Series 3 processors.

The architecture leverages a Mixture of Experts (MoE) inference approach, allowing only selected model components to be activated during processing tasks. This reduces memory utilization and computational overhead compared with architectures that require full model activation for each inference request.

In configurations equipped with 128 GB of memory, model data can be dynamically transferred to SSD cache resources, enabling the system to allocate available DRAM and VRAM to additional workloads. This allows a single edge device to perform AI inference while simultaneously supporting other software processes.

AI System Selection and Deployment Support
To simplify deployment planning, ASRock Industrial will incorporate validated aiDAPTIV configurations into its AI-Pathfinder platform. The tool is designed to help customers identify suitable hardware configurations, accelerator cards, GPUs, and edge AI systems based on workload characteristics and deployment requirements.

The objective is to reduce system design complexity and provide guidance for selecting infrastructure capable of supporting specific AI models and operational environments.

Edited by Aishwarya Mambet, Induportals Editor, with AI assistance.

www.asrockind.com