Qwen3.6-35B-Hybrid-3.25bpw

This is a saliency-directed hybrid-precision candidate ablation for Qwen3.6-35B-A3B. It mirrors the allocation policy of the GLM hybrid release: the 64 highest-scoring routed experts in each of Qwen's 40 MoE layers remain in NVIDIA ModelOpt NVFP4, while the other 192 experts per layer are stored as NF3g32.

The repository uses the canonical release name: Qwen3.6-35B-Hybrid-3.25bpw.

Important runtime and size notes

This is not directly loadable by stock vLLM. runtime/ contains the tested Docker compatibility loader: it verifies hybrid.index.json, materializes the NF3g32 tail into a disposable ModelOpt-NVFP4 cache, and then starts vLLM normally with CUDA graphs enabled. It is not a native fused mixed-NVFP4/NF3 vLLM kernel.

3.25bpw is the nominal routed-expert code allocation (25% four-bit and 75% three-bit codes). It is not a physical global bits-per-weight claim. NF3g32 stores an FP16 scale per 32 values, so its physical tail storage is 3.5 bits/scalar. The safetensor payload in this release is 19.0 GiB (about 20 GB on disk), including unchanged non-expert tensors and scales. It is not a 12 GB checkpoint.

Provenance and construction

• Parent checkpoint: nvidia/Qwen3.6-35B-A3B-NVFP4.
• Parent ModelOpt index SHA-256: d67403a4e9793c0ba8a136baf14b3b76ec7b32c822267978084895e07ebd8a3e.
• Expert ranking: mateowilliam/qwen3.6-35b-a3b-reap-observations at c0fbef14f8bedccc42048cfdf3edce0e58e35d70.
• Allocation manifest SHA-256: 873cba8b4d2cede25090e9b4774cbf5704b7296d8d994cec54f80b4e0ee1bed2.
• Package index SHA-256: 662f1cfa438c052938369f7e94521b054bcb648600f16be905f46679b130fbb1.

The public saliency artifact records the Qwen3.6 model family and the matching 40-layer × 256-expert geometry, but not an immutable source config revision. Accordingly, this remains an ablation candidate rather than a provenance-closed production release.

Validation performed

• A deterministic evenly-spaced sample of 240 of 23,040 NF3 tail projections compared 251,658,240 decoded values to their parent ModelOpt-NVFP4 values: MAE 0.001525, RMSE 0.001997, relative RMSE 0.18865.
• Four fixed first-token probes against the preserved parent server selected the same greedy token in 4/4 cases. The mean absolute selected-token logprob difference was 0.0852. The observed top-logprob-table symmetric divergence was 0.0687; this is explicitly not a full-vocabulary KL result.
• The supplied Docker service completed a real chat request and an automatic Qwen XML tool-call parsing request at a 200,000-token context setting, with CUDA graph capture enabled.

These are conversion and serving checks only. Full-vocabulary logits/KL, representative generation quality, a native mixed-expert runtime, and the post-release two-Spark Terminal-Bench 2.1 / GPQA Diamond gate remain open.

Docker use

Build the included runtime image from this repository root:

docker build -t qwen36-nf3-hybrid:local -f runtime/Dockerfile runtime

HYBRID_MODEL_DIR="$PWD" \
HYBRID_CACHE_DIR=/path/to/persistent-materialization-cache \
docker compose -f runtime/compose.yml up -d

The service uses port 8012 and exposes the model ID Qwen3.6-35B-Hybrid-3.25bpw. Its default configuration is a 200,000-token window, FP8 KV cache, Qwen reasoning parsing, Qwen XML tool parsing, automatic tool choice, prefix caching, chunked prefill, async scheduling, and CUDA graphs. It does not request eager execution or disable CUDA graphs.

The Docker base image is the DGX Spark vLLM runtime used to validate this release. Replace it with an equivalent vLLM environment only after verifying ModelOpt-NVFP4 support, Qwen hybrid-attention support, and the compatibility materializer.

Files

• base-*.safetensors: unchanged parent tensors and the selected NVFP4 expert tensors.
• nf3-tail-*.safetensors: custom NF3g32 codes, scales, and input scales for the remaining expert projections.
• hybrid.index.json: package schema, tensor maps, and tail records.
• runtime/: Docker compatibility loader and its source conversion utilities.

The custom NF3g32 storage format is documented in runtime/scripts/nf3g32.py. It is a release-local format, not an interoperable safetensors quantization standard.