Experimental Analysis of LLM Inference in Resource-Constrained Edge Environments

Deploying Large-Scale Language Models (LLMs) at the edge poses reliability challenges. This paper investigates the performance and fault tolerance in local inference of the Llama-3.2-1B-Q8 model on a Raspberry Pi 4B. Through experiments varying concurrency and context size (SQuAD v2.0), we analyzed resource saturation and latency. The results show that while RAM remains stable via 8-bit quantization, the CPU undergoes early saturation. This exhaustion causes performance failures, with high latency and instability under load. We reveal a paradox of overload in short requests (128 tokens) and the collapse of predictability in long contexts (1024 tokens). The study provides empirical evidence on the behavior of LLMs in restricted environments, supporting scaling mechanisms.