[Important] Isaac Lab Development Update

[rthaker01]

We're excited to share what's been happening across Isaac Lab and where we're headed next.

Our focus is on making Isaac Lab lighter weight and more modular. We want to give you flexibility in choosing your physics backend, visualizer, and renderer without requiring the full dependency stack when you don't need it. This also means expanding GPU compatibility: by decoupling from RTX rendering requirements, you will be able to run Isaac Lab on a mix of the latest NVIDIA compute GPUs (ex: L40s/H100/H200/B200), unlocking more powerful compute for your training workloads.

A key part of this effort is integrating https://github.com/newton-physics/newton, a GPU-accelerated physics simulation engine built on NVIDIA Warp. It integrates MuJoCo Warp as its primary backend and supports rigid body dynamics, cloth simulation, MPM for granular and deformable materials, inverse kinematics, and sensor simulation. With Newton, you can run Isaac Lab with significantly reduced dependencies, making it easier to deploy across different environments and platforms.

To support this direction, we're freezing the main branch and continuing development in a new develop branch. This branch will contain larger refactoring efforts to support both PhysX and Newton physics backends, as well as multi-renderer support.

In the meantime, we're providing two feature branches for early adopters:

Feature Branches

feature/isaacsim-6-0 - Isaac Sim 6.0 OSS Support

• Updated render settings for RT2 ([Isaac Sim 6.0] Updates render settings for RT2 #4142)
• Albedo annotator for faster diffuse albedo / depth rendering ([Isaac Sim 6.0] Adds albedo annotator for faster diffuse albedo / depth rendering #4158)
• Relaxed numpy dependency to >2 ([Isaac Sim 6.0] Relaxes numpy dependency to >2 #4081)

feature/newton - Isaac Lab - Newton Beta 2 branch

• Multi-Renderer Support - Pluggable visualizers and renderers allow you to swap backends without code changes ([Newton] Add Multi-Renderer Support #4115)
• Rerun Visualizer Integration - New visualization option alongside traditional workflows (Fixes Rerun visualizer and updates to newton beta 0.2.0 #4184)
• Optional Isaac Sim Dependency - Run headless training workloads without requiring a full Isaac Sim installation ([Newton] Decouples OV to enable OV-optional code paths #4165, [Newton] Fixes optional IsaacSim install #4193)
• Native Cloner Implementation - More flexible environment cloning with reduced external dependencies ([NEWTON]Replaces isaacsim cloner with more flexible native cloner implementation #4128)
• ARM Architecture Support - Expanded platform compatibility ([Newton] Updates dependencies to support ARM #4251)
• Contact Sensor Refactorization - Improved modularity and performance ([Newton] Contact Sensor Refactorization #4245)
• Graphed Workflows - New execution patterns for environment setup ([Newton] Graphed Workflows #3657)

Isaac Lab Newton branch currently supports classic RL and flat terrain locomotion workflows. To try it out, check out the feature/newton branch and refer to the https://isaac-sim.github.io/IsaacLab/main/source/experimental-features/newton-physics-integration/index.html.

---

Recent Updates on Main

Since the 2.3.1 release, we've continued landing improvements on the main branch:

Robots & Environments

• OpenArm environments (Adds OpenArm environments #4089)
• MJCF spawner - MuJoCo XML format support (Adds MJCF spawner #1672)
• Bin packing demo showcasing RigidObjectCollection (Adds bin packing demo that showcasing RigidObjectCollection advanced use case #3801)
• Arena G1 locomanipulation retargeters (Adds Arena G1 locomanipulation retargeters #4140)

Sensors & Data Collection

• Multi-Mesh RayCaster - New MultiMeshRayCaster and MultiMeshRayCasterCamera supporting raycasting against primitive shapes, arbitrary meshes, and dynamic objects like robot links and articulated bodies with memory-efficient caching (Adds Raycaster with tracking for dynamic meshes #3298)
• Friction force reporting in ContactSensor (Adds friction force reporting to ContactSensor #3563)
• Automatic IMU transform discovery (Adds automatic transform discovery for imu sensors to a viable parent body. #3864)
• Record-at-close functionality for RecorderManager (Adds record at close functionality to the RecorderManager #2826)
• Replay success validation using task termination (Adds support for validating replay success using the task’s termination success. #4170)

RL & Training

• Vectorized environments for pick-and-place demo (Supports vectorized environments for pick and place demo #3996)
• state_dependent_std parameter exposed in rsl_rl config (Exposes state_dependent_std parameter in rl_cfg for rsl_rl. #4091)
• ROS inference params for Reach policy (Adds all params for Reach policy ROS inference #4209)
• Updated RL libraries performance comparison (Updates RL libraries training performance comparison  #4109)

Bug Fixes

• Fixed noisy velocities around joint limits (Fixes noisy velocities around limits #3989)
• Fixed mesh converter collision approximation attribute (Fixes mesh converter not setting mesh collision approximation attribute #4082)
• Fixed advanced indexing shape mismatch when resetting prev_action (Fixes advanced indexing shape mismatch when resetting prev_action isaaclab.envs.mdp.actions.JointPositionToLimitsAction #3865)
• Prevented randomizing mass to zero or negative values (Prevents randomizing mass to zero or less #4060)

Architecture

• Replaced IsaacSim prim_utils with IsaacLab prim_utils (Replaces IsaacSim prim_utils with IsaacLab prim_utils #3924)

---

Contributors

Thank you to all our core contributors: @kellyguo11, @ooctipus, @AntoineRichard, @matthewtrepte, @pascal-roth, @rwiltz, @Mayankm96, @peterd-NV, @JerryJiehanWang, @KyleM73, @ashwinvkNV, @bmccann-bdai, @jtigue-bdai, @ozhanozen, @JinnnK, and many more!

[momo-van]

For those interested in where core physics, collision, sensing, and control capabilities are heading, the Newton repo just published a Beta release update along with the roadmap to general availability. It’s a good snapshot of how the building blocks are maturing and how they’re intended to integrate with downstream robotics workflows like Isaac Lab. Please visit Newton Beta announcement and Roadmap to 1.0 Release