AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8

AX Unlimited-OCR 3B MoE — MLX (MXFP8 + affine INT8 head)

This is an AutomatosX sensitivity-guided MLX conversion of baidu/Unlimited-OCR. It was created from the upstream BF16 weights. The sahilchachra/unlimited-ocr-mxfp8-mlx checkpoint was used only as a comparison reference; these are not repackaged Sahil weights.

Primary inference engine: AX Engine (model_family=unlimited_ocr) — use this for Apple Silicon serving and native MXFP8 / dual-vision loads.

Alternative runtime: mlx-vlm (weight-compatible; not the recommended production path for AutomatosX packs).

Hybrid precision (name is shorthand): MXFP8 (group size 32) on eligible language linears, affine INT8 on the LM head, BF16 on the vision encoder and other protected groups. The Hub “1B params” style size reflects active MoE capacity; the marketing name refers to the upstream 3B MoE family.

Release

• Version tag: v0.2.0 (immutable Hub git tag; pin this in production)
• Hub pin: revision="v0.2.0" or the commit SHA after publish
• Repo: AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8
• Runtime: AX Engine
• Release notes: Sensitivity-guided MXFP8 with calibrated affine INT8 LM head; primary inference via AX Engine; mlx-vlm alternative only

Immutable upstream source revision: ee63731b6461c8afcdcc7b15352e7d2ffecc2ead.
Reference checkpoint revision: 55b8031a6c867de675279d9604e38cc94b9882a4.

What was optimized

Each supported module group was quantized independently and evaluated for OCR CER, digit CER, and table structure. Eligible language linear layers use MXFP8 (8 data bits, group size 32). A joint calibration then selected affine INT8 (group size 32) for the LM head because BF16 missed the throughput floor and MXFP8 missed the CER ceiling. OCR-critical or sensitive groups remain BF16. Retained groups: vision_encoder, vision_projector.

The checkpoint declares model_type=unlimited-ocr, processor_class=UnlimitedOCRHFProcessor, and a sliding-window size of 128. Prompts should be raw Unlimited-OCR strings such as <image>document parsing. (not multi-turn chat formatting).

AX Engine maps this pack to native family unlimited_ocr (dual vision SAM-ViT-B + CLIP-L, SWA MoE language tower, MXFP8 dense/expert packs, R-SWA prefill). Prefer the shipped model-manifest.json for AX-ready loads.

AutomatosX validation

These are local release measurements, not upstream paper benchmark claims and not OmniDocBench / olmOCR-Bench scores. Absolute CER on this set is not a production accuracy bar; gates check parity with BF16 and packaging integrity.

The content-addressed held-out evaluation set had 12 samples (12 with digits, 3 with CJK, and 3 with tables), digest 7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987.

Checkpoint	Mean CER	Digit CER	CJK CER	Table score	Decode tok/s
Upstream BF16	0.205212	0.113310	0.333333	1.000000	—
Community MXFP8 reference	0.907381	0.406511	1.000000	1.000000	35.07
AutomatosX candidate	0.216331	0.111939	0.341270	1.000000	35.14

Caveats

• Small local set; some samples are synthetic. Do not treat mean CER as field accuracy.
• The community reference may use a different model_type / load path; it is a throughput and packaging baseline, not an identical-recipe peer.
• Prefer AX Engine for inference. mlx-vlm is an alternative load path only.
• Document helpers (e.g. AX-OCR PDF/profiles) may still call mlx-vlm under the hood; pin the same Hub revision when comparing stacks.

Weight integrity

Weight size: 3.582 GiB
Aggregate weight SHA-256: 5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549

Per-file weight SHA-256:

• model.safetensors: d9fe11fc8d2e333000be6a0e288681231efa69282c844f7775ca40f52db02f0c

Release gates

• ✅ weights_are_distinct — actual 5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549, limit different from 439c6acfe5e277537dfe3368b94a145a6b0da4c39e4bd00582977d030f292ad1
• ✅ weight_size_gb — actual 3.5815872186794877, limit 4.5
• ✅ native_model_metadata — actual {'architecture': True, 'model_type': True, 'mxfp8': True, 'sliding_window': True, 'processor_class': True, 'sft_format': True, 'precision_map': True, 'quantization_summary': True, 'protected_modules_preserved': True}, limit True
• ✅ candidate_source_provenance — actual {'model': 'Unlimited-OCR', 'revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead'}, limit {'model': ['baidu/Unlimited-OCR', 'Unlimited-OCR'], 'revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead'}
• ✅ evaluation_coverage — actual {'samples': 12, 'digit': 12, 'cjk': 3, 'table': 3}, limit all counts > 0
• ✅ held_out_evaluation_dataset — actual {'passed': True, 'calibration_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'evaluation_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'overlapping_image_sha256': [], 'overlapping_ground_truth_sha256': []}, limit distinct dataset digests and no shared image or ground-truth hashes
• ✅ accuracy_aggregates_recomputed — actual [True, True, True], limit [True, True, True]
• ✅ same_evaluation_samples — actual {'counts': [12, 12, 12], 'file_counts': [12, 12, 12]}, limit {'count': 12, 'files': ['0001.png', '0002.png', '0003.png', '0004.png', '0005.png', '0006.png', '0007.png', '0008.png', '0009.png', '0010.png', '0011.png', '0012.png']}
• ✅ model_identities — actual {'bf16': 'Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'reference_performance': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate_performance': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'rswa': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8'}, limit {'bf16': 'Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8'}
• ✅ immutable_model_revisions — actual {'bf16_accuracy': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_accuracy': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_accuracy': None, 'reference_performance': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_performance': None}, limit {'bf16_accuracy': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_accuracy': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_accuracy': None, 'reference_performance': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_performance': None}
• ✅ same_accuracy_recipe — actual {'prompt': ['<image>document parsing.', '<image>document parsing.', '<image>document parsing.'], 'max_tokens': [1024, 1024, 1024], 'profile': ['accurate', 'accurate', 'accurate'], 'generation_settings': [{'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}, {'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}, {'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}]}, limit identical official MLX OCR recipe
• ✅ candidate_cer_vs_bf16 — actual 0.011118653228923858, limit 0.015
• ✅ candidate_cer_vs_reference — actual -0.691050362380767, limit 0.005
• ✅ candidate_digit_cer_vs_bf16 — actual -0.0013706317359285375, limit 0.01
• ✅ candidate_table_score_vs_bf16 — actual 0.0, limit 0.01
• ✅ performance_aggregates_recomputed — actual {'reference': True, 'candidate': True}, limit {'reference': True, 'candidate': True}
• ✅ same_performance_setup — actual {'image_path': ['test_invoice.png', 'test_invoice.png'], 'prompt': ['<image>document parsing.', '<image>document parsing.'], 'max_tokens': [256, 256], 'num_warmup': [1, 1], 'num_runs': [3, 3], 'system': [{'platform': 'macOS-26.5.2-arm64-arm-64bit-Mach-O', 'processor': 'arm', 'python_version': '3.14.6', 'machine': 'arm64', 'mlx_version': '0.32.0', 'mlx_vlm_version': '0.6.6', 'chip': 'Apple M3 Max', 'total_memory_gb': 128.0}, {'platform': 'macOS-26.5.2-arm64-arm-64bit-Mach-O', 'processor': 'arm', 'python_version': '3.14.6', 'machine': 'arm64', 'mlx_version': '0.32.0', 'mlx_vlm_version': '0.6.6', 'chip': 'Apple M3 Max', 'total_memory_gb': 128.0}]}, limit identical setup with at least three complete runs
• ✅ candidate_tps_vs_reference — actual 1.001955642681865, limit 0.9
• ✅ rswa_8k_bounded — actual {'pass_conditions': {'cache_bounded': True, 'tps_stable': True, '8k_test_passed': True}, 'tokens': 8192, 'repetition_rate': 0.20496894409937888}, limit {'min_tokens': 8192, 'max_repetition_rate': 0.25}
• ✅ provenance_matches_release — actual {'source': 'baidu/Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'target': 'AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'source_config': {'path': 'config.json', 'sha256': '27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9'}, 'source_revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_revision': '55b8031a6c867de675279d9604e38cc94b9882a4', 'dataset_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'calibration_dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb'}, limit {'source': 'baidu/Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'target': 'AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'source_config_sha256': '27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9', 'source_revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_revision': '55b8031a6c867de675279d9604e38cc94b9882a4', 'dataset_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'calibration_dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb'}
• ✅ calibration_recomputed — actual {'selected': {'label': 'affine8-head', 'precision': 'affine8'}, 'models': ['AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-affine8', 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-bfloat16', 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-mxfp8'], 'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'input_artifacts': {'calibration_baseline_accuracy': {'filename': 'calibration_baseline_accuracy.json', 'size': 5680, 'sha256': 'ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6'}, 'calibration_reference_performance': {'filename': 'calibration_reference_performance.json', 'size': 2271, 'sha256': '98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4'}, 'calibration_bfloat16_accuracy': {'filename': 'calibration_bfloat16_accuracy.json', 'size': 5678, 'sha256': '5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0'}, 'calibration_bfloat16_performance': {'filename': 'calibration_bfloat16_performance.json', 'size': 2244, 'sha256': 'dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0'}, 'calibration_mxfp8_accuracy': {'filename': 'calibration_mxfp8_accuracy.json', 'size': 5689, 'sha256': '77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26'}, 'calibration_mxfp8_performance': {'filename': 'calibration_mxfp8_performance.json', 'size': 2252, 'sha256': 'e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b'}, 'calibration_affine8_accuracy': {'filename': 'calibration_affine8_accuracy.json', 'size': 5657, 'sha256': 'c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19'}, 'calibration_affine8_performance': {'filename': 'calibration_affine8_performance.json', 'size': 2247, 'sha256': 'f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f'}}}, limit content-addressed inputs and fastest passing experiment
• ✅ sensitivity_matches_calibration_dataset — actual {'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'baseline_metrics_match': True}, limit {'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'baseline_metrics_match': True}
• ✅ candidate_precision_map_matches_evidence — actual 91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75, limit 91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75
• ✅ precision_map_reproducible — actual c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a, limit c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a

Full per-sample evidence, sensitivity results, precision rules, PRD, ADR, and technical specification are included under release/, quantization/, and docs/.

Usage

Requires Apple Silicon.

Primary — AX Engine

Use AX Engine as the primary inference engine for this pack (native unlimited_ocr: dual vision + SWA MoE + MXFP8 + R-SWA).

Quantization/release measurements for these weights used: mlx-vlm 0.6.6, mlx 0.32.0, python 3.14.6.

Install

# Homebrew (recommended on macOS)
brew tap defai-digital/ax-engine
brew install defai-digital/ax-engine/ax-engine
ax-engine doctor

# or Python wheel
python3 -m pip install --upgrade "ax-engine[download]>=6.11.0,<7"

Download and prepare

# Managed download (writes model-manifest.json when needed)
ax-engine download AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8

# Pin a release tag (recommended for production)
hf download AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \
  --revision v0.2.0 \
  --local-dir ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8

# Ensure AX native manifest is present/valid
ax-engine-bench generate-manifest ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --validate

This repository ships model-manifest.json so downloads are AX-ready without a manual convert step when the file is present.

Serve

ax-engine serve ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --port 31418
# or, after managed download of the Hub id:
# ax-engine serve AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --download --port 31418

OpenAI-compatible routes are exposed on the local server. See the AX Engine docs for install, doctor checks, and API details.

Alternative — mlx-vlm

mlx-vlm can load these weights as a secondary / alternative path. Prefer AX Engine for AutomatosX deployments.

pip install 'mlx-vlm>=0.6.4'
python -m mlx_vlm.generate \
  --model AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \
  --revision v0.2.0 \
  --image document.png \
  --prompt '<image>document parsing.' \
  --max-tokens 4096

Optional — AX-OCR document toolkit

AX-OCR adds deskew, PDF, and generation profiles on top of the same Hub weights. It is a toolkit, not a replacement for AX Engine as the primary runtime.

pip install 'ax-ocr[mlx]'
# OCRPipeline / ax-ocr CLI: model_path=AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8, revision=v0.2.0

This is quantization/conversion work, not foundation-model retraining or domain fine-tuning.