Is Your Phone's LLM 37x Too Slow? Check This One Compiler Flag

Chronological Source Flow

Back

AI Fusion Summary

Building llama.cpp for Android using a Gradle debug variant produces an -O0 binary, which is 37x slower than the Release version. On a Pixel 4 with Snapdragon 855 running Qwen2.5-1.5B-Instruct Q4_K_M, generation speed increased from 0.5 tok/s to 18.5 tok/s. First token latency dropped from 1863 ms to 52 ms, and prefill for 43 tokens improved from 42 s to 0.556 s. These significant performance gains resulted solely from changing build configuration lines.