Tag: benchmark

All the articles with the tag "benchmark".

• Inference Optimization Mini-Benchmark: FP16 vs Q8_0 vs Q4_K_M

  18 Jul, 2026

  Hands-on benchmark of quantization levels on a 1.5B model: decode throughput, TPOT, and memory footprint compared across FP16, Q8_0, and Q4_K_M.