There is a persistent and frustrating gap between benchmark-reported model capability and the actual experience of running that same model locally. The common assumption is that this gap is a function of quantisation alone: use a lower-bit weight format, accept some quality loss, move on. A detailed experimental series published on the Level1Techs forums by researcher thr3e demonstrates that this framing is far too narrow. The real picture involves a layered accumulation of precision losses that compound across the entire inference stack, from attention backend selection through KV cache quantisation, weight format, and tensor parallelism topology.
The methodology here is worth examining carefully before the results, because it is meaningfully more rigorous than the typical "I ran three prompts at temperature zero" evaluation. The author captures full-vocabulary logits in BF16 at regular intervals across a roughly 100,000-token context derived from real network automation workstreams. Critically, this is not a synthetic benchmark. The prompt contains genuine tool calls, CLI command sequences, and multi-step reasoning chains that do not appear in any public benchmark or quantisation calibration dataset. That last point matters considerably: it eliminates the possibility that any quantised model has been inadvertently optimised to score well on the evaluation set itself.
Attention Backend Selection as a Source of Divergence
The first experiment isolates a variable most practitioners never consider: the choice of attention backend during prefill. Under vLLM, three backends are available for this workload: FlashAttention 2, Flash Inference, and Triton Attention. Everything else, weights, activations, KV cache precision, driver stack, hardware, was held constant. The only change between runs was which backend processed the attention computation.
The results are striking. For the first several thousand tokens of context, all three backends produce bit-identical top-1 token predictions. Divergence emerges progressively in later context windows and, critically, is not uniformly distributed. It appears in clusters correlated with prompt content rather than scaling smoothly with sequence length. This is consistent with the known behaviour of floating-point non-associativity: the order of reduction operations in large matrix multiplications is tile-size and kernel-specific, and accumulated rounding errors interact with the particular numerical structure of the activations at any given layer position.
The practical consequence demonstrated is not abstract. A single top-1 flip during a tool call caused FlashAttention 2 to target the wrong network interface (GigabitEthernet0/1/4 instead of GigabitEthernet0/0/1.201) and subsequently execute the wrong diagnostic command. The author then let the diverged generation continue rather than snapping it back to the reference trajectory. The forked output never recovered. This is a compelling illustration of how a single-token error in a structured output context can cascade into a semantically irrecoverable failure, and it is reproducible across runs with bit-identical logit captures.
KV Cache Quantisation and the Long-Context Cliff
The second experiment keeps weights and activations at BF16 and varies only the KV cache precision across BF16, INT8, and INT4. The divergence pattern here is qualitatively different from the attention backend experiment. Rather than clustering around content-sensitive positions, token flip rates increase more systematically with context length, which is mechanistically expected: KV cache quantisation error accumulates as the cache grows, and attention over a larger set of quantised key-value pairs compounds that error.
The failure modes are instructive. With BF16 KV cache, tool calls complete correctly. With INT8, the model occasionally diverges but recovers. With INT4, a tool call fails entirely and does not recover. This gradient of degradation maps onto the intuition that INT4 KV cache, which is increasingly common in memory-constrained deployments, represents a qualitatively different operating regime rather than a modest quantitative step down from INT8.
This finding has direct relevance to the growing deployment of long-context models. Many practitioners enable aggressive KV cache quantisation specifically to extend the context length they can serve within a fixed memory budget. The data here suggests that this trade-off is not linear: the precision loss per token is small, but its effect on structured, multi-step reasoning tasks compounds in ways that simple perplexity measurements would not capture.
Weight Format Comparisons Across Five Quantisation Schemes
The five-way weight comparison is the most practically useful section of the analysis. The candidates tested against a BF16 reference are: the official Qwen FP8 checkpoint (W8A8 with dynamic activation quantisation), an INT8 W8A16 model with no calibration dataset, NVIDIA's NVFP4 mixed-precision checkpoint, and an AWQ W4A16 model calibrated on STEM and agentic data. All were run with BF16 KV cache to isolate weight precision effects.
Several findings are counterintuitive:
- The INT8 W8A16 model (TheHouseOfTheDude) outperforms the official FP8 W8A8 checkpoint in top-1 agreement with BF16, despite being produced with no calibration data. The author's explanation is plausible: W8A16 avoids activation quantisation entirely, and the GDN (Gated DeltaNet) projections are excluded from quantisation, preserving the most numerically sensitive parts of this hybrid architecture.
- The NVFP4 checkpoint performs worst of the five, reaching approximately 50% top-1 flip rate by 88,000 tokens of context. The author notes that on the test GPU (RTX PRO 6000 Blackwell), vLLM classified the hardware as lacking native FP4 support and routed through Marlin weight-only FP4 compression rather than true FP4 arithmetic. This is a significant caveat: the result measures a specific software fallback path, not necessarily the ceiling of FP4 precision on hardware with native support.
- Both the NVFP4 and AWQ W4A16 models failed to correctly complete tool calls that BF16, FP8, and INT8 all handled successfully.
The GEMM kernel diversity across these five configurations is worth emphasising. Each quantisation scheme routes through a different CUDA kernel: BF16 uses standard torch linear, FP8 routes through CutlassFp8BlockScaledMMKernel, INT8 and AWQ both use MarlinLinearKernel via CompressedTensorsWNA16, and NVFP4 splits between FlashInferFP8ScaledMMLinearKernel and MarlinNvFp4LinearKernel depending on layer type. These are not minor implementation details. Each kernel implements matrix multiplication with different tile geometries, reduction orderings, and accumulator precisions. Comparing "the same model" across these configurations is, at the CUDA level, comparing substantially different computations.
Tensor Parallelism and the NCCL Problem
A brief but important result appears in Part 2: the same model at TP1 completes a tool call correctly, at TP2 it fails, and at TP4 it succeeds again. This non-monotonic behaviour is characteristic of NCCL all-reduce operations introducing floating-point non-determinism at shard boundaries. The order in which partial results are summed across devices is not guaranteed to be consistent, and at BF16 precision the accumulated rounding differences are sufficient to flip top-1 predictions at sensitive positions.
This is a known issue in distributed training and has been studied in the context of reproducibility for scientific computing workloads. Its appearance in inference is less well-documented, and the practical implication is uncomfortable: a model that passes validation at TP1 may silently produce different outputs at TP2 or TP4, with no error signal to the operator.
Fine-Tuned and Abliterated Models Under Structured Evaluation
Part 3 extends the methodology to "abliterated" fine-tunes of Qwen3.8-27B, models that have had refusal behaviours surgically removed. The results here are more nuanced than the typical community discourse around these models. The Heretic-ARA and Huihui abliterated variants showed very low top-1 flip rates (under 1.5%) relative to the BF16 reference, with divergences concentrated at positions where the base model was already uncertain. The AEON Ultimate variant, which combines SSM conv1d repair, an Abliterix search, and a grafted MTP head, showed 5.8% top-1 flip rates and 36 structurally invalid branch futures on the longer evaluation prompt.
The methodological point here is important: the experiment measures the complete modification recipe, not abliteration as an isolated operation. The AEON result is not evidence that refusal removal degrades structured output capability in general. It is evidence that this particular combination of interventions does.
Implications for Deployment and Evaluation Practice
The cumulative picture from these experiments is that local LLM inference operates under a regime of silent, compounding numerical degradation that standard evaluation practice is poorly equipped to detect. Perplexity on a held-out set, short-context benchmarks at temperature zero, and single-run qualitative assessment all systematically underestimate the precision loss that manifests at long context, under tool-calling constraints, and across the specific CUDA kernel paths that a given hardware and software configuration selects.
Several practical conclusions follow. First, KV cache quantisation below INT8 should be treated with significant caution for agentic workloads at long context, regardless of the memory savings it enables. Second, attention backend selection is not a performance-only decision; it has measurable precision consequences that interact with prompt content in non-trivial ways. Third, NVFP4 and similar mixed-precision formats require careful validation on the specific hardware and software path that will actually execute them, since the "native" and "fallback" paths can produce qualitatively different outputs.
The work also points toward a gap in current quantisation evaluation practice. KLD reported on model cards is largely uninterpretable without full disclosure of the reference checkpoint, runtime environment, evaluation corpus, context lengths, and aggregation methodology. The author's approach of capturing full-vocabulary logits and branching at divergence points to inspect downstream consequences is considerably more informative, and a distributable version of these tools would be a genuine contribution to the field. The forthcoming results across Hopper and Blackwell GPU families at scale should make for a revealing comparison.