LLMs on Consumer Hardware — Part 2: Prefill and the Failure of the AI PC

Local LLM deployment focuses on privacy and on-premise inference. Inference consists of two phases: compute-bound prefill for input prompts and memory bandwidth-bound generation for output tokens. While casual use masks prefill costs, large prompts reveal hardware limitations. Additionally, research using the Ollama inference engine on an RTX 4060Ti 16GB benchmarks nine open-source LLMs. This study evaluates GPU power draw, measuring mean and peak power, total energy per prompt, and energy per output token.