SpatialAV2AV — Weights

Full fine-tuned transformer weights for spatial (binaural / stereo) audio-video editing on top of LTX-2.3 (22B). The model learns source (pre-edit sound field + video) + camera-trajectory instruction → edit (target sound field + video), producing real 2-channel stereo audio (L≠R) whose spatial image follows the camera motion.

Trained on the BingoG/LTX dataset (source→edit pairs, same clip rendered under different camera trajectories via Binamix HRTF).

Repository layout

Each checkpoint lives in its own step_XXXXX/ folder with an INFO.md recording the training run, step, loss, format, and inference usage.

Folder	Step	Notes
step_04000/	4000 / 20000	Latest & recommended — best tested checkpoint (run 2026.06.19-17.33.11)

⚠️ Important: the file is torch-pickle, NOT safetensors

The weight file is named model_weights_step_04000.safetensors but it is actually a torch.save (zip/pickle) file, not a real safetensors container. This is a known quirk of the trainer (it uses accelerator.save, which writes torch format under a .safetensors name).

• ❌ safetensors.torch.load_file(...) → fails with HeaderTooLarge.
• ✅ Load with torch.load(...) and inject into the base LTX-2.3 transformer via load_state_dict(..., strict=False) (verified missing=0 unexpected=0).

The project's SpatialAV2AV_inference.py --full_weights <file> path already does this correctly.

Usage (inference)

# base model + text encoder are the stock LTX-2.3 assets (not in this repo)
python packages/ltx-trainer/scripts/SpatialAV2AV_inference.py \
  --checkpoint  <LTX-2.3/ltx-2.3-22b-dev.safetensors> \
  --full_weights step_04000/model_weights_step_04000.safetensors \
  --text_encoder_path <gemma-3-12b-it-...> \
  --meta_json <...>+pan_right.json \
  --output out_pan_right.mp4 \
  --num_frames 113 --min_size 288 --steps 20 --guidance_scale 4.0 --seed 42
# prints "双声道 L≠R? True" on success

Download just one checkpoint folder:

from huggingface_hub import snapshot_download
d = snapshot_download("BingoG/SpatialAV2AV", allow_patterns="step_04000/*")