MLPerf / Inference

What is ML Perf

The primary tool for MLPerf benchmarks is the MLPerf suite itself, which includes both training and inference benchmarks. These benchmarks are available for a variety of platforms, including cloud, edge, and mobile environments. You can access the MLPerf benchmarks through their official GitHub repository or the MLCommons website, which provides detailed instructions on how to run the benchmarks and interpret the results.

Why we need it

The myriad combinations of ML hardware and ML software make assessing MLsystem performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria.

Test System Spec

Installation

1. Install cm language

Ubuntu, Debian

2. Prerequisites and installation

Basic Python Libraries

Results

Resnet-50 | imagenet-val dataset | Onnx model
cmr "run mlperf inference generate-run-cmds _submission" --quiet --submitter="MLCommons" --hw_name=default --model=resnet50 --implementation=reference --backend=onnxruntime --device=cpu --scenario=Offline --adr.compiler.tags=gcc  --target_qps=1 --category=edge --division=open

I got 74.258 QPS (= latency in ms 13.47) for resnet50 model with imagenet validation set.

We can find the MLPerf benchmark in ML Commons Inference table, and the result of AMD Ryzen 9 7950X 16-Core Processor was around 7.8 latency in ms.

retinanet | openimage dataset | onnx model
cmr "run mlperf inference generate-run-cmds _submission" --quiet --submitter="MLCommons" --hw_name=default --model=retinanet --implementation=reference --backend=onnxruntime --device=cpu --scenario=Offline --adr.compiler.tags=gcc  --target_qps=1 --category=edge --division=open

I got 1.193 QPS (= 518.13 latency in ms) and it was not that bad compared to AMD 7950X performance, 454 latency in ms.

Errors

val_map.txt not found error

Leave a Reply


Create your own website, free on WordPress.com.Create your own website.
Start free