I Fine-Tuned a 270M Model on My Laptop (Full Fine-Tuning, From Scratch)

A series of experiments on the Banking77 task compared three fine-tuning methods across different model sizes. Full Fine-Tuning was applied to a 270M model, LoRA to a 1.5B model, and QLoRA to a 7B model using 4-bit quantization. Despite the size differences, all three models achieved similar accuracy. The results suggest that for this specific task, the smallest model is preferable as it is cheaper and faster to serve.