← All benchmark runs
Gazebo Profiling — 2026-09-01 (server and GUI)
📄 Server findings report (PDF)
(Markdown source) ·
📄 GUI findings report (PDF)
(Markdown source, GUI section below)
Profiling of the headless Gazebo server (gz-sim-main) on
gz-sim PR #3447 (ECM on EnTT) with gz-transport PR #868 + #842
(Zenoh 1.8.0 with shared memory, GZ_TRANSPORT_IMPLEMENTATION=zenoh); every other
Gazebo library at the tip of main on 2026-09-01.
CPU: Intel Core Ultra 9 285HX (24 cores, hybrid) · GPU: NVIDIA RTX PRO 3000 (driver 580.159.03, EGL headless)
· Kernel 6.17.0-35 · perf 6.17.13 · DART 6.13.2 · OgreNext 2.3.1
Build: RelWithDebInfo, -fno-omit-frame-pointer, ENABLE_PROFILER=OFF.
Sampled with perf -e task-clock -F 997 --call-graph dwarf; all worlds at RTF=0;
rendering sensor worlds run with --headless-rendering and one subscriber per sensor topic.
Click any flamegraph to open it interactively (click frames to zoom, Ctrl+F to search).
Runtime Flamegraphs (30 s steady state, all threads)
RTF and steps/s are the steady state rate between two /world/*/stats snapshots taken 10 s and
30 s after launch (steady/results_unpinned.tsv); a single snapshot would include the hundreds of thousands of
empty iterations a free running server executes before its entities are created. "Cores busy" is CPU time / 30 s during
the perf capture.
| World | Stress axis | RTF (steady) | Steps/s (steady) | Cores busy | Flamegraph | Hotspots |
|---|
| 3k_shapes_static | 3000 static models: framework + broadphase | 0.24 | 239 | 1.00 | Open | txt |
| 3k_shapes | 3000 dynamic models: DART solver | 0.014 | 14 | 1.00 | Open | txt |
| sensors | non rendering sensors: per step overhead | 50.6 | 50,644 | 1.00 | Open | txt |
| jetty | complex scene: ODE broadphase (4 ms step) | 1.98 | 495 | 1.00 | Open | txt |
| gpu_lidar_sensor | one GPU lidar, 1 subscriber | 50.2 | 50,156 | 1.27 | Open | txt |
| sensors_demo | six rendering sensors, 8 subscribers | 5.3 | 5,335 | 1.40 | Open | txt |
| moving_robots_and_sensors | PR #3846: 2 vehicles, arm, 10 sensors, 12 subscribers | 3.0 | 3,015 | 1.49 | Open | txt |
Per Thread Flamegraphs
Share of process CPU per thread. The simulation thread is always saturated; rendering sensors run on
a second thread; Zenoh transport threads (tx-0, rx-*, SHM watchdogs) stay below 1.2%.
Loading Flamegraphs (startup + 1 iteration)
Wall clock is the median of three unprofiled runs of gz-sim-main -s -r --iterations 1
(*_wallclock_unprofiled.txt); the perf wrapped run in *_wallclock.txt is 5 to 7x slower
because of DWARF stack copying.
| World | Wall clock | Flamegraph |
|---|
| 3k_shapes_static | 2.05 s | Open |
| 3k_shapes | 2.11 s | Open |
| sensors | 0.82 s | Open |
| jetty | 1.55 s | Open |
| gpu_lidar_sensor | 1.22 s | Open |
| sensors_demo | 1.19 s | Open |
| moving_robots_and_sensors | 1.22 s | Open |
Cache Counters (perf stat, 10 s, unprofiled)
| World | IPC | L1d miss | LLC load miss | Raw |
|---|
| 3k_shapes_static | 0.71 | 8.79% | 15.7% | txt |
| 3k_shapes | 1.61 | 1.15% | 54.3% | txt |
| sensors | 2.71 | 0.10% | 6.2% | txt |
| jetty | 2.11 | 1.90% | 0.5% | txt |
| gpu_lidar_sensor | 2.59 | 0.27% | 6.6% | txt |
| sensors_demo | 2.65 | 0.73% | 39.0% | txt |
| moving_robots_and_sensors | 2.47 | 0.79% | 37.0% | txt |
Priority 1 validated: per step bookkeeping around dartsim WorldForwardStep
The server report's Priority 1 was implemented in two parts: gz-sim returns the cached step output by reference instead of copying it twice per step
(merged upstream as gz-sim #3850 on 2026-09-01), and dartsim skips the NaN check and the ChangedWorldPoses pass for static models
(gz-physics branch caguero/dartsim_static_bookkeeping). Steady state step rate
(two stats snapshots 20 s apart, median of two interleaved runs pinned to one P core), same worlds and build:
| World | Before | gz-sim part only | Both PRs | Change |
|---|
| 3k_shapes_static | 254 steps/s | 261 steps/s | 462 steps/s | 1.82x |
| sensors | 49,729 steps/s | 51,558 steps/s | 51,982 steps/s | +4.5% |
| 3k_shapes | 14.6 steps/s | 14.6 steps/s | 14.8 steps/s | +1.7% |
| jetty | 500 steps/s | 502 steps/s | 507 steps/s | +1.4% |
| moving_robots_and_sensors | 1,833 steps/s | 1,853 steps/s | 1,840 steps/s | within noise |
| sensors_demo (unpinned, 3 runs each) | 5,288 to 5,509 steps/s | — | 5,005 to 5,352 steps/s | medians 5,335 vs 5,227, −2%, within the spread of this GPU bound world |
| 3k_shapes_static flamegraphs | Open |
|---|
| Before (baseline build, same day) | Open |
After both PRs (DART World::step is 90% of the process, was 63%) | Open |
| Differential (blue = removed, red = grew in share) | Open |
| Raw steady state data (pinned interleaved runs, unpinned runs) | tsv / tsv |
GUI Process Profiling
📄 GUI findings report (PDF)
(Markdown source)
Same workspace and date; perf attached to the GUI client (gz-sim-gui-client) with the
simulation playing (gz sim -r), rendering on the NVIDIA GPU via PRIME offload, server and GUI on Zenoh with SHM.
For moving_robots_and_sensors the twelve sensor subscribers and the vehicle/arm commands are applied as in the server run.
GUI Runtime Flamegraphs (30 s, all GUI threads)
| World | Scene | GUI cores | Render thread | Qt main | Zenoh threads | Flamegraph | Subsystems |
|---|
| shapes | idle baseline, six primitives | 0.15 | 66% | 10% | 2.5% | Open | txt |
| 3k_shapes_static | 3000 visuals, nothing moves | 1.04 | 89% | 6% | 0.6% | Open | txt |
| 3k_shapes_dynamic | 3000 visuals, all moving | 0.81 | 88% | 7% | 0.6% | Open | txt |
| jetty | complex meshes and textures | 0.49 | 75% | 10% | 5.3% | Open | txt |
| moving_robots_and_sensors | PR #3846: small moving scene, 2.4 kHz state stream | 0.44 | 27% | 34% | 25.7% | Open | txt |
GUI Per Thread Flamegraphs
GUI Loading Flamegraphs (GUI launch, 20 s window)
| World | Flamegraph |
|---|
| 3k_shapes_static | Open |
| jetty | Open |
| moving_robots_and_sensors | Open |
Server State Stream Consumed by the GUI
5 s of gz topic -e -t /world/<name>/state with the world playing. The mixed world publishes on every
one time change (JointVelocityCmd written each step by DiffDrive / AckermannSteering).
| World | Messages | Entities / msg | Payload / msg | Sample |
|---|
| 3k_shapes_static | 47 Hz | 13 | 0.2 KB | sample |
| 3k_shapes (dynamic) | 11 Hz | 504 | 28 KB | sample |
| moving_robots_and_sensors | 2,443 Hz | 24 | 0.9 KB | sample |
GUI Cache Counters (perf stat, 10 s)
| World | IPC | L1d miss | LLC load miss | Raw |
|---|
| 3k_shapes_static | 0.69 | 2.36% | 69.7% | txt |
| 3k_shapes | 0.91 | 2.19% | 43.0% | txt |
| moving_robots_and_sensors | 1.88 | 1.04% | 1.7% | txt |
Unified Summary (gz_summary.sh)
Generated with FlameGraph and Linux perf.
Scripts and worlds: github.com/caguero/gz-profiling.
