PicoLLM is a cross-platform, on-device LLM inference engine

Large Language Models (LLMs) can run locally on mini PCs or single board computers like the Raspberry Pi 5 but with limited performance due to high memory

Read more here: External Link