LLM Performance on M3 Max

Nov 15, 2023 ·

The article from Nonstopdev discusses the performance of a large language model (LLM) on Intel's M3 Max platform. The M3 Max is a low-power, serverless AI research platform engineered to accelerate deep learning training and inferencing workloads. Using the same compute power as traditional CPU-based systems, this platform can achieve up to 6x faster training and 20x faster inference performance for machine learning models.

To test the performance of the LLM on the platform, the article uses a sequence tagging problem, in which the model must predict each word in a sentence. The model was trained with a sequence length of 128 and batch size of 32, using the popular Transformers library. After training, the model achieved an accuracy of 95%.

In addition to the improved accuracy, the experiments also showed that the model was able to run much faster than its GPU counterparts. Specifically, the model was able to complete one epoch in just over two minutes, while running on a single M3 Max node. This means that the model could be retrained and reloaded in near real-time, making it suitable for both production and research purposes.

Overall, the article shows that the M3 Max platform provides excellent performance for large language models. It is able to offer significantly improved accuracy compared to GPU-based systems, while also providing an accelerated training time. This makes it an ideal platform for both research and production use cases.