Events

[Arm Create 2026] Developers put the Arm AI Model Optimizer to work

|

November 16, 2026

|

9

min read

[Arm Create 2026] Developers put the Arm AI Model Optimizer to work

On October 19, 2026, developers gathered at The Tech Interactive in San Jose for a hands-on Arm Create session led by Arm and Nota AI. The session gave attendees a chance to explore the Arm AI Model Optimizer in practice. Over a 45-minute exercise, participants used their laptops to submit optimization experiments for an image-classification model targeting an Arm neural processing unit (NPU), then compared accuracy and estimated performance. Because the workflow came down to a few commands and a configuration file, everyone, including developers new to model optimization, completed the task.

The Nota AI hands-on session at Arm Create 2026. Attendees sit at each table, working through the task on their laptops.

Arm Create 2026: A Developer Hands-on Event During PyTorch Conference Week

Arm Create was a developer event held to coincide with PyTorch Conference. It centered on practical AI development across cloud to edge to physical AI. Attendees could explore technologies including Gemma, LiteRT, PyTorch, and ExecuTorch through hands-on sessions and demonstrations.

Speakers from Google, Renesas, and Nota AI joined Arm as event partners and participated in the opening panel. Representing Nota AI was Steven Kim, CEO of Nota AI America, who discussed the challenges and opportunities developers face as AI models evolve. He emphasized the growing importance of hardware-aware optimization in developing efficient AI applications.

The opening panel at Arm Create 2026. Steven Kim, CEO of Nota AI America, speaks into a microphone while speakers from Google DeepMind, Google Cloud, Renesas, and Arm sit alongside him.

Arm discussed the Early Access Program for Arm AI Model Optimizer, part of Arm AI Portal.


Arm AI Model Optimizer: An Optimization Tool from Arm, Built on NetsPresso®

Arm AI Model Optimizer is built on Nota AI's NetsPresso® technology and helps developers optimize AI models for supported Arm hardware. It applies graph optimization, quantization, and mixed precision, and enables developers to compare latency, accuracy, and memory use across optimization configurations to find the right balance for their applications. It is currently available through the Early Access Program on Arm AI Portal. For more details, see Arm's introductory blog post.

Arm’s launch blog highlights three examples of model optimization across different workloads and hardware targets.

Arm AI Model Optimizer: before and after

Whisper Basevivo X300 Pro

Decoding latency

Original
35.63 ms
Optimized
↓ 52%17.17 ms

Word error rate 4.85% → 4.85%

MobileSAMRaspberry Pi 5

Inference latency

Original
2,070.63 ms
Optimized
↓ 23%1,595.46 ms

mIoU 0.7309 → 0.7301

WideResNet-50Raspberry Pi 5

Top-1 accuracy

Original
80.82%
Optimized
−0.12pp80.70%

Inference speed 5.9x faster
Memory 74% lower

  • Original
  • Arm AI Model Optimizer (optimized)
Figure 1. Optimization examples from Arm's launch blog. For WideResNet-50, the bar shows accuracy, while inference speed and memory improvements appear separately.

The examples cover speech recognition on a smartphone and image segmentation and classification on a Raspberry Pi 5. In each case, latency decreased while accuracy changed little. The Nota AI session gave developers a chance to work through a similar optimization process themselves.


The 90-Minute Session

Nota AI split the 90-minute "Hands-on AI model optimization for Arm" session into a 40-minute introduction, a 45-minute guided exercise, and a five-minute results review.

Nota AI CTO Tae-Ho Kim opened by explaining why the tool is needed and how to use it. He explained that AI models evolve quickly, while the operations supported by deployed hardware remain fixed. Optimization helps developers adapt models to those hardware constraints. He then walked through the workflow using the command-line interface (CLI): creating an experiment, running the optimization, and comparing results.

Hands-on session attendees work through the Arm AI Model Optimizer task, each on their own laptop.

Over the next 45 minutes, attendees followed the tutorial, using their laptops to submit optimization jobs and review the results. Nota AI team members circulated between tables, answering questions and helping attendees troubleshoot issues as they worked through the exercise.

The final five minutes focused on a large language model (LLM) optimization result. The hands-on exercise targeted an image-classification model for an edge NPU, but the demonstration showed how the same optimization workflow applies to an LLM targeting an Arm-based cloud CPU. The team submitted the longer-running LLM job at the start of the session and reviewed the results at the end.


The Attendees' Optimization Results: An Image-Classification Model for an Edge NPU

The exercise demonstrated how developers can configure, run, and evaluate model optimizations for a specific hardware target.

An attendee running an optimization on a laptop during the hands-on session.

The Task: Optimize WideResNet-50 for the Alif E8 Board

The exercise followed a workflow that developers can apply to their own AI projects. The task was to optimize WideResNet-50, the image-classification model also shown in Figure 1, for the Alif Ensemble E8 DevKit. This board combines an Arm Cortex-M55 CPU and an Arm Ethos-U85 NPU, and the optimization uses two stages: graph optimization followed by 8-bit integer (INT8) quantization.

The ten-step tutorial covered installing the CLI, creating and running an experiment, reviewing reports, visualizing the model graph, and exporting a deployment-ready .pte model file for the target device.

experiment_00 — zsh
experiment_00 % np experiments detail
[DONE ] [CLI] Generated: run.detail.yaml
# run.detail.yaml edits (2)
# granularity: CHANNEL → TENSOR
# calibration: PERCENTILE → HISTOGRAM
experiment_00 % np validate
[DONE ] [CLI] Validation Passed. No issues found.
experiment_00 % np run --profile-method mlia --wait
[INFO ] [Workflow] Graph Optimize completed (3m 33s)
[INFO ] [Workflow] Graph Quantize completed (6m 17s)
[INFO ] [Workflow] Evaluate completed (50.4s)
[INFO ] [Workflow] Run completed (16m 37s)
[DONE ] [CLI] Run completed successfully.
Figure 2. Two configuration changes, followed by validation and an optimization run.

Before running, attendees adjusted two settings in the configuration file.

In the configuration file (run.detail.yaml), they changed weight granularity from per-channel (CHANNEL) to per-tensor (TENSOR) to improve execution efficiency on the Ethos-U85. They also changed the activation calibration method from PERCENTILE to HISTOGRAM to help preserve accuracy. The HISTOGRAM method uses the distribution of activation values to select a range that minimizes quantization error. After editing the values, they used np validate to check the configuration before running the experiment. Once the run completed, an np report command provided a side-by-side comparison of the original model and the results at each optimization stage.

The Result: 26.6 Percent Lower Latency, Accuracy Held

Arm MLIA estimated a 26.6 percent reduction in inference latency, from 477.14 ms to 350.24 ms. Top-1 accuracy changed from 73.98 percent to 73.47 percent, a difference of 0.51 percentage points.

The optimized model was exported as a 42.71MB .pte file targeting the Ethos-U85. The latency result was estimated without using a physical board. The accuracy numbers differ from those for the WideResNet-50 in Figure 1 because the target hardware and evaluation data are different.

experiment_00 — zsh
experiment_00 % np report
Model Latency(ms) Top-1 Acc Memory(MB) PTE Size(MB)
input_model 477.14 73.98 2.01 n/a
01_graph_optimize 477.14 73.98 2.01 n/a
02_graph_quantize 350.24 73.47 1.96 42.71
Figure 3. Results from np report (key columns only, rearranged). Latency and memory are Arm MLIA estimates; Top-1 accuracy is based on 162 evaluation images.

In Figure 3, the graph optimization stage leaves the reported metrics unchanged while restructuring operations for the target hardware. The changes in estimated latency and measured accuracy appear after quantization.

Without the tool, achieving the same result requires several separate steps. You export the model to the format supported by ExecuTorch (a runtime that runs PyTorch models on devices), configure the quantization settings and calibration data, and compile it with the Vela compiler (for the Ethos-U NPU). Then estimate latency, evaluate accuracy, and compare the results separately. Each step requires different tools and settings, but attendees completed this process with just a few commands and a single configuration file.


The Last 5 Minutes: Revealing the LLM Optimization Result for a Cloud CPU

The job submitted at the start of the session was to optimize Qwen2.5-0.5B-Instruct for AWS Graviton4, an Arm-based cloud CPU. The model and hardware target were different, but the workflow of creating a project, running optimization, and comparing results was the same as in the hands-on exercise.

The runtime was ExecuTorch and the backend was XNNPACK, which accelerates supported CPU operations. Unlike the image-classification exercise, this run added activation-aware weight quantization (AWQ) before graph optimization and quantization. AWQ helps preserve model accuracy by accounting for the importance of the different weights during quantization.

MetricBaselineAfter optimizationChange
PTE size (MB)2,5221,538−39.0%
XNNPACK delegation rate (%)38.054.5+16.5pp
OpenBookQA acc_norm (%)34.834.6−0.2pp
Table 1. Qwen2.5-0.5B-Instruct optimization results on AWS Graviton4. Accuracy was evaluated internally by Nota AI.

After optimization, the model's file size dropped by 39.0 percent, while the XNNPACK delegation rate rose from 38.0 percent to 54.5 percent. That means that a greater share of operations used the XNNPACK CPU acceleration backend. OpenBookQA accuracy changed only slightly, from 34.8 percent to 34.6 percent, in Nota AI’s internal evaluation.

The team also compared response speed using seven identical prompts. On average, the optimized model produced its first token 1.8 times faster and generated responses 2.3 times faster than the original in this demonstration.

A side-by-side demo of the original and the optimized Qwen2.5-0.5B-Instruct answering the same seven prompts.

Questions from the Floor: How Can I Use This in a Real Project?

After completing the guided exercise, attendees could freely test other models. Some went beyond the basic task to optimize ResNet-18 for a Raspberry Pi 5, while others ran Qwen2.5 on AWS Graviton4.

A Nota AI team member and an attendee discuss the hands-on progress at a table.

As the hands-on session continued, attendees asked questions about the range of supported AI models and hardware, performance comparisons before and after optimization, how to fit the tool into an existing development process, and how to evaluate it in their own environment. These were practical questions: rather than asking "Will this actually work?", attendees asked "How can I apply this to my project?" After completing the optimization themselves, attendees' interest shifted to applying the tool in their own projects.


To Try It Yourself: Start with Early Access

Arm AI Portal helps developers and coding agents discover pre-optimized AI models alongside deployment-ready code and workflows to get AI running faster on Arm. If you need to optimize your own model, Arm AI Model Optimizer is available through the portal’s Early Access Program.

This session used preset models and data, but developers can also use their own data for calibration and evaluation. Supported data types include images, speech, and text. Beyond the presets, developers can bring in supported Hugging Face models, including Qwen, Gemma, and Llama, into the optimization workflow.

Arm AI Model Optimizer currently offers about 50 preset models. There are five reference evaluation targets: Alif Ensemble E8 DevKit, Raspberry Pi 5, an Apple M4 MacBook, vivo X300 Pro, and AWS Graviton4. For step-by-step setup instructions and a first optimization experiment, follow the quick-start guide.

A Nota AI presenter guides attendees through the task in front of the Hands-on slide on screen.

Attendees used Arm AI Model Optimizer to optimize models from edge NPUs to cloud CPUs with a single tool. Arm and Nota AI plan to add more runtimes, backends, and supported hardware over time, using feedback from Early Access Program participants to guide further development.

To explore model optimization for your application, apply for the Arm AI Model Optimizer Early Access Program.

‍

Related