langgz commited on
Commit
f20d841
·
verified ·
1 Parent(s): f05f591

Fix MLT model card quickstarts and benchmark scope

Browse files

Point all quickstarts at the MLT checkpoint, use deterministic decoding, select the Hugging Face hub explicitly, and clarify that the existing benchmark tables are family-level rather than MLT-specific.

Files changed (2) hide show
  1. README.md +30 -20
  2. README_zh.md +30 -20
README.md CHANGED
@@ -57,11 +57,11 @@ This model is part of the **FunASR** ecosystem — one industrial-grade open-sou
57
  </div>
58
 
59
 
60
- # Fun-ASR
61
 
62
  「[简体中文](README_zh.md)」|「English」
63
 
64
- Fun-ASR is an end-to-end speech recognition large model launched by Tongyi Lab. It is trained on tens of millions of hours of real speech data, possessing powerful contextual understanding capabilities and industry adaptability. It supports low-latency real-time transcription and covers 31 languages. It excels in vertical domains such as education and finance, accurately recognizing professional terminology and industry expressions, effectively addressing challenges like "hallucination" generation and language confusion, achieving "clear hearing, understanding meaning, and accurate writing."
65
 
66
  <div align="center">
67
  <img src="images/funasr-v2.png">
@@ -77,10 +77,7 @@ Fun-ASR is an end-to-end speech recognition large model launched by Tongyi Lab.
77
 
78
  </h4>
79
 
80
- Model Repository: [modelscope](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512), [huggingface](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512)
81
-
82
- Online Experience:
83
- [ModelScope Community Space](https://modelscope.cn/studios/FunAudioLLM/Fun-ASR-Nano), [huggingface space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano)
84
 
85
  </div>
86
 
@@ -93,19 +90,17 @@ Online Experience:
93
 
94
  # What's New 🔥
95
 
96
- - 2025/12: [Fun-ASR-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) is an end-to-end speech recognition large model trained on tens of millions of hours real speech data. It supports low-latency real-time transcription and covers 31 languages.
97
  - 2024/7: [FunASR](https://github.com/modelscope/FunASR) is a fundamental speech recognition toolkit that offers a variety of features, including speech recognition (ASR), Voice Activity Detection (VAD), Punctuation Restoration, Language Models, Speaker Verification, Speaker Diarization and multi-talker ASR.
98
 
99
  # Core Features 🎯
100
 
101
- **Fun-ASR** focuses on high-precision speech recognition, multi-language support, and industry customization capabilities
102
 
103
- - **Far-field High-noise Recognition:** Deeply optimized for far-distance sound pickup and high-noise scenarios (such as conference rooms, in-vehicle environments, industrial sites, etc.), improving recognition accuracy to **93%**.
104
- - **Chinese Dialects and Regional Accents:**
105
- - Supports **7 major dialects**: Wu, Cantonese, Min, Hakka, Gan, Xiang, Jin
106
- - Covers **26 regional accents**: including Henan, Shaanxi, Hubei, Sichuan, Chongqing, Yunnan, Guizhou, Guangdong, Guangxi and more than 20 other regions
107
- - **Multi-language Free Speech:** Supports recognition of **31 languages**, with focused optimization on East and Southeast Asian languages, supporting free language switching and mixed recognition.
108
- - **Music Background Lyric Recognition:** Enhanced speech recognition performance under music background interference, supporting accurate recognition of lyric content in songs.
109
 
110
  # Environment Setup 🐍
111
 
@@ -134,9 +129,10 @@ from funasr import AutoModel
134
 
135
 
136
  def main():
137
- model_dir = "FunAudioLLM/Fun-ASR-Nano-2512"
138
  model = AutoModel(
139
  model=model_dir,
 
140
  trust_remote_code=True,
141
  remote_code="./model.py",
142
  device="cuda:0",
@@ -148,26 +144,33 @@ def main():
148
  cache={},
149
  batch_size=1,
150
  hotwords=["开放时间"],
151
- # 中文、英文、日文 for Fun-ASR-Nano-2512
152
  # 中文、英文、粤语、日文、韩文、越南语、印尼语、泰语、马来语、菲律宾语、阿拉伯语、
153
  # 印地语、保加利亚语、克罗地亚语、捷克语、��麦语、荷兰语、爱沙尼亚语、芬兰语、希腊语、
154
  # 匈牙利语、爱尔兰语、拉脱维亚语、立陶宛语、马耳他语、波兰语、葡萄牙语、罗马尼亚语、
155
  # 斯洛伐克语、斯洛文尼亚语、瑞典语 for Fun-ASR-MLT-Nano-2512
156
  language="中文",
157
  itn=True, # or False
 
158
  )
159
  text = res[0]["text"]
160
  print(text)
161
 
162
  model = AutoModel(
163
  model=model_dir,
 
164
  trust_remote_code=True,
165
  vad_model="fsmn-vad",
166
  vad_kwargs={"max_single_segment_time": 30000},
167
  remote_code="./model.py",
168
  device="cuda:0",
169
  )
170
- res = model.generate(input=[wav_path], cache={}, batch_size=1)
 
 
 
 
 
 
171
  text = res[0]["text"]
172
  print(text)
173
 
@@ -183,11 +186,16 @@ from model import FunASRNano
183
 
184
 
185
  def main():
186
- model_dir = "FunAudioLLM/Fun-ASR-Nano-2512"
187
- m, kwargs = FunASRNano.from_pretrained(model=model_dir, device="cuda:0")
 
 
 
 
188
  m.eval()
189
 
190
  wav_path = f"{kwargs['model_path']}/example/zh.mp3"
 
191
  res = m.inference(data_in=[wav_path], **kwargs)
192
  text = res[0][0]["text"]
193
  print(text)
@@ -208,7 +216,9 @@ if __name__ == "__main__":
208
 
209
  # Performance 📝
210
 
211
- We evaluated Fun-ASR against other state-of-the-art models on open-source benchmarks, Chinese dialect datasets, and industry-specific test sets. The results demonstrate that Fun-ASR achieves superior performance across various scenarios.
 
 
212
 
213
  ### 1. Open-Source Dataset Performance (WER %)
214
 
 
57
  </div>
58
 
59
 
60
+ # Fun-ASR-MLT-Nano-2512
61
 
62
  「[简体中文](README_zh.md)」|「English」
63
 
64
+ Fun-ASR-MLT-Nano-2512 is the 800M-parameter multilingual checkpoint in the Fun-ASR family. It is trained on hundreds of thousands of hours of multilingual speech and supports 31 languages, with emphasis on East and Southeast Asian languages. For the Chinese, English, and Japanese checkpoint trained on tens of millions of hours, use [Fun-ASR-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512).
65
 
66
  <div align="center">
67
  <img src="images/funasr-v2.png">
 
77
 
78
  </h4>
79
 
80
+ Model Repository: [modelscope](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512), [huggingface](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512)
 
 
 
81
 
82
  </div>
83
 
 
90
 
91
  # What's New 🔥
92
 
93
+ - 2025/12: [Fun-ASR-MLT-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) is the multilingual 800M checkpoint, trained on hundreds of thousands of hours of speech and covering 31 languages.
94
  - 2024/7: [FunASR](https://github.com/modelscope/FunASR) is a fundamental speech recognition toolkit that offers a variety of features, including speech recognition (ASR), Voice Activity Detection (VAD), Punctuation Restoration, Language Models, Speaker Verification, Speaker Diarization and multi-talker ASR.
95
 
96
  # Core Features 🎯
97
 
98
+ **This repository hosts the MLT checkpoint.** Its supported-language scope differs from the standard Fun-ASR-Nano checkpoint.
99
 
100
+ - **31 languages:** Covers the languages listed in the model table above, with focused optimization on East and Southeast Asian languages.
101
+ - **Compact multilingual model:** Uses approximately 800M parameters.
102
+ - **Language control:** Accepts the language names shown in the inference example and supports multilingual recognition.
103
+ - **Context controls:** The FunASR inference API exposes hotwords and inverse text normalization (ITN).
 
 
104
 
105
  # Environment Setup 🐍
106
 
 
129
 
130
 
131
  def main():
132
+ model_dir = "FunAudioLLM/Fun-ASR-MLT-Nano-2512"
133
  model = AutoModel(
134
  model=model_dir,
135
+ hub="hf",
136
  trust_remote_code=True,
137
  remote_code="./model.py",
138
  device="cuda:0",
 
144
  cache={},
145
  batch_size=1,
146
  hotwords=["开放时间"],
 
147
  # 中文、英文、粤语、日文、韩文、越南语、印尼语、泰语、马来语、菲律宾语、阿拉伯语、
148
  # 印地语、保加利亚语、克罗地亚语、捷克语、��麦语、荷兰语、爱沙尼亚语、芬兰语、希腊语、
149
  # 匈牙利语、爱尔兰语、拉脱维亚语、立陶宛语、马耳他语、波兰语、葡萄牙语、罗马尼亚语、
150
  # 斯洛伐克语、斯洛文尼亚语、瑞典语 for Fun-ASR-MLT-Nano-2512
151
  language="中文",
152
  itn=True, # or False
153
+ llm_kwargs={"do_sample": False},
154
  )
155
  text = res[0]["text"]
156
  print(text)
157
 
158
  model = AutoModel(
159
  model=model_dir,
160
+ hub="hf",
161
  trust_remote_code=True,
162
  vad_model="fsmn-vad",
163
  vad_kwargs={"max_single_segment_time": 30000},
164
  remote_code="./model.py",
165
  device="cuda:0",
166
  )
167
+ res = model.generate(
168
+ input=[wav_path],
169
+ cache={},
170
+ batch_size=1,
171
+ language="中文",
172
+ llm_kwargs={"do_sample": False},
173
+ )
174
  text = res[0]["text"]
175
  print(text)
176
 
 
186
 
187
 
188
  def main():
189
+ model_dir = "FunAudioLLM/Fun-ASR-MLT-Nano-2512"
190
+ m, kwargs = FunASRNano.from_pretrained(
191
+ model=model_dir,
192
+ hub="hf",
193
+ device="cuda:0",
194
+ )
195
  m.eval()
196
 
197
  wav_path = f"{kwargs['model_path']}/example/zh.mp3"
198
+ kwargs["llm_kwargs"] = {"do_sample": False}
199
  res = m.inference(data_in=[wav_path], **kwargs)
200
  text = res[0][0]["text"]
201
  print(text)
 
216
 
217
  # Performance 📝
218
 
219
+ > **Benchmark scope:** The tables below are Fun-ASR family results reproduced from the project report. They do not contain a column identified as `Fun-ASR-MLT-Nano-2512`, so they must not be interpreted as checkpoint-specific results for this MLT model. MLT per-language results will be added when a reproducible evaluation is published.
220
+
221
+ The family-level comparison covers open-source benchmarks, Chinese dialect datasets, and industry-specific test sets.
222
 
223
  ### 1. Open-Source Dataset Performance (WER %)
224
 
README_zh.md CHANGED
@@ -1,8 +1,8 @@
1
- # Fun-ASR
2
 
3
  「简体中文」|「[English](README.md)」
4
 
5
- Fun-ASR 是通义实验室推出端到端音识别大模型,基于数万小时真实语音数据训练而成具备强大的上下文理解能力与行业适应性,支持低延迟实时听写,并且覆盖 31 个语。在教育、金融等垂直领域表现出色,能准确识别专业术与行业表达有效应对"幻觉"生成和语种混淆等挑战,实现"听得清懂其意写得准"
6
 
7
  <div align="center">
8
  <img src="images/funasr-v2.png">
@@ -18,10 +18,7 @@ Fun-ASR 是通义实验室推出的端到端语音识别大模型,是基于数
18
 
19
  </h4>
20
 
21
- 模型仓库:[modelscope](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512),[huggingface](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512)
22
-
23
- 在线体验:
24
- [魔搭社区创空间](https://modelscope.cn/studios/FunAudioLLM/Fun-ASR-Nano),[huggingface space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano)
25
 
26
  </div>
27
 
@@ -34,19 +31,17 @@ Fun-ASR 是通义实验室推出的端到端语音识别大模型,是基于数
34
 
35
  # 最新动态 🔥
36
 
37
- - 2025/12: [Fun-ASR-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) 是一款基于数万小时真实语音数据训练的端到端语音识别大模型。它支持低延迟实时转写并涵盖 31 种语言识别功能
38
  - 2024/7: [FunASR](https://github.com/modelscope/FunASR) 是一款功能全面的语音识别基础工具包,集成了多项核心功能,包括自动语音识别(ASR)、语音活动检测(VAD)、标点恢复、语言模型、说话人验证、说话人日志记录以及多说话人语音识别。
39
 
40
  # 核心特性 🎯
41
 
42
- **Fun-ASR** 专注于高精度语音识别、多语言支持和行业定制化能力
43
 
44
- - **远场高噪声识别:** 针对远距离拾音及高噪声场景(如会议室、车载环境、工业现场等)进行深度优化,识别准确率提升至 **93%**
45
- - **中文方与地方口音:**
46
- - 支持 **7 大方言**:吴、粤、闽语、客家话、赣语、湘语、晋语
47
- - 覆盖 **26 个地区口音**:包括河南、陕西、湖北、四川、重庆、云南、贵州、广东、广西等 20 多个地区
48
- - **多语言自由说:** 支持 **31 种语言**识别,重点优化东亚与东南亚语种,支持语种自由切换和混合识别。
49
- - **音乐背景歌词识别:** 强化在音乐背景干扰下的语音识别性能,支持对歌曲中歌词内容的精准识别。
50
 
51
  # 环境安装 🐍
52
 
@@ -75,9 +70,10 @@ from funasr import AutoModel
75
 
76
 
77
  def main():
78
- model_dir = "FunAudioLLM/Fun-ASR-Nano-2512"
79
  model = AutoModel(
80
  model=model_dir,
 
81
  trust_remote_code=True,
82
  remote_code="./model.py",
83
  device="cuda:0",
@@ -89,26 +85,33 @@ def main():
89
  cache={},
90
  batch_size=1,
91
  hotwords=["开放时间"],
92
- # 中文、英文、日文 for Fun-ASR-Nano-2512
93
  # 中文、英文、粤语、日文、韩文、越南语、印尼语、泰语、马来语、菲律宾语、阿拉伯语、
94
  # 印地语、保加利亚语、克罗地亚语、捷克语、丹麦语、荷兰语、爱沙尼亚语、芬兰语、希腊语、
95
  # 匈牙利语、爱尔兰语、拉脱维亚语、立陶宛语、马耳他语、波兰语、葡萄牙语、罗马尼亚语、
96
  # 斯洛伐克语、斯洛文尼亚语、瑞典语 for Fun-ASR-MLT-Nano-2512
97
  language="中文",
98
  itn=True, # or False
 
99
  )
100
  text = res[0]["text"]
101
  print(text)
102
 
103
  model = AutoModel(
104
  model=model_dir,
 
105
  trust_remote_code=True,
106
  vad_model="fsmn-vad",
107
  vad_kwargs={"max_single_segment_time": 30000},
108
  remote_code="./model.py",
109
  device="cuda:0",
110
  )
111
- res = model.generate(input=[wav_path], cache={}, batch_size=1)
 
 
 
 
 
 
112
  text = res[0]["text"]
113
  print(text)
114
 
@@ -124,11 +127,16 @@ from model import FunASRNano
124
 
125
 
126
  def main():
127
- model_dir = "FunAudioLLM/Fun-ASR-Nano-2512"
128
- m, kwargs = FunASRNano.from_pretrained(model=model_dir, device="cuda:0")
 
 
 
 
129
  m.eval()
130
 
131
  wav_path = f"{kwargs['model_path']}/example/zh.mp3"
 
132
  res = m.inference(data_in=[wav_path], **kwargs)
133
  text = res[0][0]["text"]
134
  print(text)
@@ -149,7 +157,9 @@ if __name__ == "__main__":
149
 
150
  # 性能评测 📝
151
 
152
- 我们在开源基准数据集、中文方言试集和工业测试集上,比较了 Fun-ASR 他模型的多语言语音识别性能。Fun-ASR 模型均具有明显优势
 
 
153
 
154
  ### 1. 开源数据集性能 (WER %)
155
 
 
1
+ # Fun-ASR-MLT-Nano-2512
2
 
3
  「简体中文」|「[English](README.md)」
4
 
5
+ Fun-ASR-MLT-Nano-2512 Fun-ASR 系列中约 8 亿参数模型,基于数万小时言语音训练,支持 31 种语并重点优化东亚与东南亚语种。基于数千万小时数据训练面向中文英文和日文的版本请使用 [Fun-ASR-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512)
6
 
7
  <div align="center">
8
  <img src="images/funasr-v2.png">
 
18
 
19
  </h4>
20
 
21
+ 模型仓库:[modelscope](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512),[huggingface](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512)
 
 
 
22
 
23
  </div>
24
 
 
31
 
32
  # 最新动态 🔥
33
 
34
+ - 2025/12: [Fun-ASR-MLT-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) 是约 8 亿参数的多语言模型,基于数万小时语音训练,盖 31 种语言。
35
  - 2024/7: [FunASR](https://github.com/modelscope/FunASR) 是一款功能全面的语音识别基础工具包,集成了多项核心功能,包括自动语音识别(ASR)、语音活动检测(VAD)、标点恢复、语言模型、说话人验证、说话人日志记录以及多说话人语音识别。
36
 
37
  # 核心特性 🎯
38
 
39
+ **本仓库提供 MLT 多语言模型。** 它与标准版 Fun-ASR-Nano 的支持范围不同。
40
 
41
+ - **31 种语言:** 覆盖上方模型表列出的语言,并重点优化东亚与东南亚语种
42
+ - **紧凑多语模型:** 参数量约 8 亿。
43
+ - **语种控制:** 接受推理示例中列出的种名称,并支持多言识别。
44
+ - **上下文控制:** FunASR 推理接口支持热词和逆文本标准化(ITN)。
 
 
45
 
46
  # 环境安装 🐍
47
 
 
70
 
71
 
72
  def main():
73
+ model_dir = "FunAudioLLM/Fun-ASR-MLT-Nano-2512"
74
  model = AutoModel(
75
  model=model_dir,
76
+ hub="hf",
77
  trust_remote_code=True,
78
  remote_code="./model.py",
79
  device="cuda:0",
 
85
  cache={},
86
  batch_size=1,
87
  hotwords=["开放时间"],
 
88
  # 中文、英文、粤语、日文、韩文、越南语、印尼语、泰语、马来语、菲律宾语、阿拉伯语、
89
  # 印地语、保加利亚语、克罗地亚语、捷克语、丹麦语、荷兰语、爱沙尼亚语、芬兰语、希腊语、
90
  # 匈牙利语、爱尔兰语、拉脱维亚语、立陶宛语、马耳他语、波兰语、葡萄牙语、罗马尼亚语、
91
  # 斯洛伐克语、斯洛文尼亚语、瑞典语 for Fun-ASR-MLT-Nano-2512
92
  language="中文",
93
  itn=True, # or False
94
+ llm_kwargs={"do_sample": False},
95
  )
96
  text = res[0]["text"]
97
  print(text)
98
 
99
  model = AutoModel(
100
  model=model_dir,
101
+ hub="hf",
102
  trust_remote_code=True,
103
  vad_model="fsmn-vad",
104
  vad_kwargs={"max_single_segment_time": 30000},
105
  remote_code="./model.py",
106
  device="cuda:0",
107
  )
108
+ res = model.generate(
109
+ input=[wav_path],
110
+ cache={},
111
+ batch_size=1,
112
+ language="中文",
113
+ llm_kwargs={"do_sample": False},
114
+ )
115
  text = res[0]["text"]
116
  print(text)
117
 
 
127
 
128
 
129
  def main():
130
+ model_dir = "FunAudioLLM/Fun-ASR-MLT-Nano-2512"
131
+ m, kwargs = FunASRNano.from_pretrained(
132
+ model=model_dir,
133
+ hub="hf",
134
+ device="cuda:0",
135
+ )
136
  m.eval()
137
 
138
  wav_path = f"{kwargs['model_path']}/example/zh.mp3"
139
+ kwargs["llm_kwargs"] = {"do_sample": False}
140
  res = m.inference(data_in=[wav_path], **kwargs)
141
  text = res[0][0]["text"]
142
  print(text)
 
157
 
158
  # 性能评测 📝
159
 
160
+ > **评范围说明:** 下表复用项目报告中的 Fun-ASR 系列结果,中没有名为 `Fun-ASR-MLT-Nano-2512`独立列,因此不能视为本 MLT checkpoint 的专属评测结果。待发布可复现的 MLT 分语言评测后再补充对应数据。
161
+
162
+ 以下系列级对比覆盖开源基准、中文方言数据集和工业测试集。
163
 
164
  ### 1. 开源数据集性能 (WER %)
165