PXAI Audio Feed
01/04 03:42 dev.to

From one model to seven — what it took to make TurboQuant model-portable

TurboQuantKV cache compressionvLLMfused paged kernelsHBM trafficLlama 3.1
01/04 03:23 dev.to

Complete Guide to llm-d CNCF Sandbox — Kubernetes-Native Distributed LLM Inference

llm-dCNCF SandboxKubernetesdistributed inferenceLLMvLLM
01/04 01:53 dev.to

Deep Dive into vLLM: How PagedAttention & Continuous Batching Revolutionized LLM Inference

vLLMLLM servingGPU memoryPagedAttentionContinuous Batchinginference throughput
31/03 15:50 dev.to

LLM Inference Optimization: Techniques That Actually Reduce Latency and Cost

LLM inferenceGPU optimizationvLLMSGLangPrometheusRunpod
28/03 00:57 dev.to

I shipped Google's TurboQuant as a vLLM plugin 72 hours after the paper — here's what nobody else tested

TurboQuantvLLM pluginKV cache compression4‑bit quantizationmemory reductiontext models
26/03 23:08 dev.to

vLLM On-Demand Gateway: Zero-VRAM Standby for Local LLMs on Consumer GPUs

vLLMGPUVRAMLLMFastAPIOn‑Demand
25/03 19:34 dev.to

EVAL #008: NVIDIA Just Open-Sourced an Inference Engine. Now What?

NVIDIA DynamoAI inferencePyTorch 2.7vLLMSGLangTransformers
24/03 17:36 dev.to

Catching a vLLM Latency Spike with eBPF and an Open-Weight LLM

eBPFvLLMMiniMaxOllamaMCPopen‑source
22/03 14:19 dev.to

llama.swap Model Switcher Quickstart for OpenAI-Compatible Local LLMs

llama‑swaplocal LLMOpenAI APIproxymodel switchingvLLM
22/03 14:19 dev.to

llama.swap Model Switcher Quickstart for OpenAI-Compatible Local LLMs

llama‑swaplocal LLMOpenAI APIproxymodel switchingvLLM
Comments
Loading...
0