Architecture Review: Give a Free-Tier Inference Endpoint an Admission Envelope, Not a Retry Storm

Free-tier inference endpoints often suffer from retry storms when clients use blind retries without idempotency keys, leading to synchronized herds and persistent 429 errors. While tokens are free, costs persist in queue wait times, retries, and attention. Effective budget decisions require knowing tokens per accepted output and attempts per output. Consequently, the primary value of a free tier is not the cost savings, but the ability to learn operational constraints before scaling workloads.