Debug Isaac Sim slow scene load times

[andrewjong]

AirStack Version
0.18.0 / 0.19.0-alpha.0

Describe the bug
Isaac Sim takes several minutes each time to load, this is too slow.

To Reproduce
Steps to reproduce the behavior:

1. airstack up
2. wait for isaac sim to load (often 5 minutes)
3. be frustrated

Expected behavior
should launch super fast

Host Compute (please complete the following information):

• OS: Ubuntu 24
• GPU: RTX 5090
• CPU: core i9

Additional context

Likely causes, ranked
CUDA ComputeCache not actually being written (72K is suspect → kernels JIT-recompiled every launch)
add_colliders recursive Python over Full Warehouse — each Apply() can trigger USD recomposition; could easily be tens of seconds and is pure CPU on the script's main thread
Nucleus staleness checks / partial cache misses — cache/main may have stale entries; per-asset omni.client round-trips to airlab-nucleus.andrew.cmu.edu add latency, especially off-LAN/VPN
RTX shader / MDL compilation on first material encounter (shader cache is populating per the dir listing, but a new GPU/driver invalidates it)
Extension cold-load — 11 extensions enabled _immediate; first time after a Kit version bump rebuilds OGN cache (you can see this in ogn_generated/1.79.2/)
headless=False — runs full UI viewport (with Full Warehouse that's a lot of GPU work before you can interact)
Volume-mounted Pegasus extension (docker-compose.yaml:70) — if your host disk is HDD, many small-file reads can stall startup
wait_for_stage timeout=10s might be too short — Full Warehouse routinely takes 20–60s to fully load over Nucleus, so subsequent add_colliders runs while assets are still streaming, which compounds the recomposition cost