Archives
All the articles I've archived.
-
I Thought ASR Needed Its Own vLLM — Here’s How Speech Models Are Actually Served
Inside the ASR inference engine: model execution, batching, streaming state, parallelism, multilingual serving, open-source runtimes, and the breakthroughs that made modern speech AI practical.
-
I Nodded Along While Inference Geeks Talked GPUs — So I Wrote the Field Guide I Wish I Had
A comprehensive, from-zero guide to GPU hardware buzzwords: HBM, memory bandwidth, FP8/FP4, NVLink, H100 vs H200 vs B200 vs GB200, NVL72 racks, AMD Instinct, TPUs, Groq, Cerebras — plus what to actually run in a homelab and why datacenter GPU infra is genuinely hard.
-
Production Inference Optimization Strategy — Full Deck (All Deliverables)
The complete slide deck covering the production inference optimization strategy: architecture, benchmarks, routing, and cost analysis.
-
Architectural Specification — Production Inference Optimization Strategy
Architectural specification for a production inference optimization strategy for the agentic inference cloud.
-
Inference Optimization Mini-Benchmark: FP16 vs Q8_0 vs Q4_K_M
Hands-on benchmark of quantization levels on a 1.5B model: decode throughput, TPOT, and memory footprint compared across FP16, Q8_0, and Q4_K_M.
-
Serving the Agentic Inference Era — Optimization Strategy (Slides)
Slide deck on optimizing LLM inference for the agentic era: quantization, KV cache, batching, and bandwidth-bound decoding.
-
I Ran vLLM on a Mac Mini With No GPU — Here's Everything I Learned About Inference
A complete, beginner-friendly guide to vLLM: what inference actually is, how to build vLLM from source on an Apple Silicon Mac with no GPU, every command explained, the three errors I hit and fixed, the flags that matter, and real throughput numbers from my living room.
-
OpenCode Doesn't Have an Auto Mode. So I Built One with vLLM Semantic Router
Cursor picks the right model for you automatically. OpenCode doesn't - yet. Here's a complete, production-shaped guide to adding intelligent auto model selection to OpenCode (or any OpenAI-compatible agent) using vLLM Semantic Router and AgentGateway.
-
I Almost Built a Grafana Stack—Then AgentGateway Shipped Everything I Needed.
I Almost Built a Grafana Stack—Then AgentGateway Shipped Everything I Needed.
-
How Cassandra Compression Actually Works (Chunks, Offsets, and Reads)
How Cassandra Compression Actually Works (Chunks, Offsets, and Reads)
-
Giving AgentGateway a Semantic Brain with vLLM Semantic Router - Inside My Homelab
Giving AgentGateway a Semantic Brain with vLLM Semantic Router - Inside My Homelab
-
I Traced Personal Agent's Source Code. Inside Was Pi... And It Dreams at 3 AM.
I Traced Personal Agent's Source Code. Inside Was Pi... And It Dreams at 3 AM.
-
I Built an AI That Decides Which AI to Talk To — Running 24/7 From My Living Room
I Built an AI That Decides Which AI to Talk To — Running 24/7 From My Living Room