// TOPIC CLUSTER TELEMETRY //

#ai-ml

High-throughput local LLM serving, GPUDirect Storage over NVMe-oF, Speculative Decoding benchmarks, and CUDA acceleration.

>_ INDEXED TRANSMISSIONS IN #AI-ML

3 dispatches

Sep 27, 2026 #ai-ml

Bypassing the CPU Bottleneck: GPUDirect Storage (GDS) with NVMe-oF in AI Clusters

A deep dive into NVIDIA GPUDirect Storage (GDS) and NVMe over Fabrics: streaming massive multi-terabyte dataset shards directly into GPU HBM without CPU bounce buffers.

by Joshua Edward McLaughlin Cox Read →

Sep 27, 2026 #ai-ml

Speculative Decoding in Production: Accelerating LLM Inference Without Accuracy Loss

How speculative decoding leverages small draft models and vectorized tree verification to double generation speed on vLLM and TensorRT-LLM without sacrificing a single bit of model precision.

by Joshua Edward McLaughlin Cox Read →

Aug 28, 2026 #ai-ml

High-Throughput Local LLM Inference: vLLM, PagedAttention, and Ollama

Architecture and optimization techniques for deploying open-weight models locally with continuous batching, AWQ quantization, and GPU memory tuning.

by Joshua Edward McLaughlin Cox Read →

← View All Dispatches & Field Notes