msluszniak commited on
Commit
7ef2d7d
·
verified ·
1 Parent(s): 28277b0

Pin Core ML fp16 to CPU_AND_NE (6.0x on s, 2.8x on x)

Browse files

Pin compute_unit=CPU_AND_NE for the Core ML fp16 builds.

ExecuTorch defaults to ComputeUnit.ALL, which here splits the graph across GPU
and ANE and pays the transition cost. Pinning the ANE removes the GPU from
consideration. Measured on an iPhone 16 (ET 1.4.1, 10 warmup + 100 timed runs,
median of two interleaved repetitions, published build vs this one):

fast_sam s 52.45 ms -> 8.70 ms 6.0x
fast_sam x 108.85 ms -> 38.90 ms 2.8x

This is NOT a general rule and was not applied elsewhere. The same panel found
ALL already optimal for efficientnet, ssdlite, fcn, selfie and yolo26; CPU_AND_GPU
worse than ALL almost everywhere; lraspp identical at 5.90 vs 5.90 ms once
measured properly; and deeplab unchanged (0.98-1.06x).

Weights, contract and precision are unchanged; only the compile specs differ.

x/coreml/fast_sam_x_coreml_fp16.pte CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9c6ee538cdb343017f06d69b371dc27fd11a23e933fe948c5cb31f5804ce2c59
3
- size 145309935
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41414fdae45dba718616718ce7ebd63797dc3f27dc971713f51457cd8346abad
3
+ size 145311073