zhaochenyang20/sglang-omni-arabic-tts-smoke

SGLang Omni Arabic TTS Smoke Set

This dataset is a fixed 50-sentence Arabic target-text subset derived from the Google FLEURS ar_eg test split. It is intended for lightweight, ASR-based intelligibility checks when integrating Arabic TTS models into SGLang Omni.

It is not a standard Arabic TTS benchmark and does not measure naturalness, prosody, speaker similarity, or human preference.

Source

• Dataset: google/fleurs
• Config: ar_eg
• Split: test
• Source revision: ab93cf03f9d0cd083c853fad065a6377067408aa
• Source file: ar_eg/test-00000-of-00001.parquet
• License: CC BY 4.0

Selection

Rows are scanned in source order and retained when:

• the transcription contains 6-20 whitespace-delimited words;
• at least 90% of alphabetic characters are Arabic;
• the text contains no digits;
• the normalized whitespace form has not already appeared.

The first 50 matching rows are stored. Digit-containing rows are excluded because ASR may verbalize digits while the written reference retains numeric characters.

Schema

Field	Type	Description
sample_id	string	Stable ID derived from the source row index
target_text	string	Arabic TTS target text
word_count	int64	Whitespace-delimited word count
source_row_index	int64	Row index in the pinned FLEURS Parquet file
source_id	string	Original FLEURS row ID

The dataset contains text and provenance only. Reference audio and generated model outputs are intentionally not included.

Reproduction

The materialization code lives in SGLang Omni:

python -m benchmarks.audar_tts.prepare_fleurs_dataset \
  --output-dir /tmp/sglang-omni-arabic-tts-smoke \
  --samples 50

manifest.json records the source revision, selection contract, output path, and Parquet SHA-256.

Evaluation

The initial SGLang Omni Audar-TTS smoke run synthesizes these targets with a pinned official Audar reference, transcribes each generated WAV directly to Arabic with Qwen3-ASR-1.7B, and computes WER, CER, BLEU, and chrF++ without translation.