Developer’s Guide

Developer’s Guide#

For development, we suggest using Microsoft Visual Studio Code (VSCode). This is also suggested by NVIDIA Omniverse and there exists tutorials on how to debug Omniverse extensions using VSCode.

Setting up Visual Studio Code#

The following is only applicable for Isaac Sim installed via the Omniverse Launcher. The Isaac Lab repository includes the VSCode settings to easily allow setting up your development environment. These are included in the .vscode directory and include the following files:

.vscode
├── tools
│   ├── launch.template.json
│   ├── settings.template.json
│   └── setup_vscode.py
├── extensions.json
├── launch.json  # <- this is generated by setup_vscode.py
├── settings.json  # <- this is generated by setup_vscode.py
└── tasks.json

To setup the IDE, please follow these instructions:

Open the Isaac Lab directory on Visual Studio Code IDE
Run VSCode Tasks, by pressing Ctrl+Shift+P, selecting Tasks: Run Task and running the setup_python_env in the drop down menu.

If everything executes correctly, it should create the following files:

.vscode/launch.json: Contains the launch configurations for debugging python code.
.vscode/settings.json: Contains the settings for the python interpreter and the python environment.

For more information on VSCode support for Omniverse, please refer to the following links:

Configuring the python interpreter#

In the provided configuration, we set the default python interpreter to use the python executable provided by Omniverse. This is specified in the .vscode/settings.json file:

{
   "python.defaultInterpreterPath": "${workspaceFolder}/_isaac_sim/python.sh",
}

If you want to use a different python interpreter (for instance, from your conda environment), you need to change the python interpreter used by selecting and activating the python interpreter of your choice in the bottom left corner of VSCode, or opening the command palette (Ctrl+Shift+P) and selecting Python: Select Interpreter.

For more information on how to set python interpreter for VSCode, please refer to the VSCode documentation.

Repository organization#

The Isaac Lab repository is structured as follows:

IsaacLab
├── .vscode
├── .flake8
├── LICENSE
├── isaaclab.sh
├── pyproject.toml
├── README.md
├── docs
├── source
│   ├── extensions
│   │   ├── omni.isaac.lab
│   │   ├── omni.isaac.lab_assets
│   │   └── omni.isaac.lab_tasks
│   ├── standalone
│   │   ├── demos
│   │   ├── environments
│   │   ├── tools
│   │   ├── tutorials
│   │   └── workflows
└── VERSION

The source directory contains the source code for all Isaac Lab extensions and standalone applications. The two are the different development workflows supported in Isaac Sim. These are described in the following sections.

Extensions#

Extensions are the recommended way to develop applications in Isaac Sim. They are modularized packages that formulate the Omniverse ecosystem. Each extension provides a set of functionalities that can be used by other extensions or standalone applications. A folder is recognized as an extension if it contains an extension.toml file in the config directory. More information on extensions can be found in the Omniverse documentation.

Isaac Lab in itself provides extensions for robot learning. These are written into the source/extensions directory. Each extension is written as a python package and follows the following structure:

<extension-name>
├── config
│   └── extension.toml
├── docs
│   ├── CHANGELOG.md
│   └── README.md
├── <extension-name>
│   ├── __init__.py
│   ├── ....
│   └── scripts
├── setup.py
└── tests

The config/extension.toml file contains the metadata of the extension. This includes the name, version, description, dependencies, etc. This information is used by Omniverse to load the extension. The docs directory contains the documentation for the extension with more detailed information about the extension and a CHANGELOG file that contains the changes made to the extension in each version.

The <extension-name> directory contains the main python package for the extension. It may also contains the scripts directory for keeping python-based applications that are loaded into Omniverse when then extension is enabled using the Extension Manager.

More specifically, when an extension is enabled, the python module specified in the config/extension.toml file is loaded and scripts that contains children of the omni.ext.IExt class are executed.

import omni.ext

class MyExt(omni.ext.IExt):
   """My extension application."""

   def on_startup(self, ext_id):
      """Called when the extension is loaded."""
      pass

   def on_shutdown(self):
      """Called when the extension is unloaded.

      It releases all references to the extension and cleans up any resources.
      """
      pass

While loading extensions into Omniverse happens automatically, using the python package in standalone applications requires additional steps. To simplify the build process and avoiding the need to understand the premake build system used by Omniverse, we directly use the setuptools python package to build the python module provided by the extensions. This is done by the setup.py file in the extension directory.

Note

The setup.py file is not required for extensions that are only loaded into Omniverse using the Extension Manager.

Lastly, the tests directory contains the unit tests for the extension. These are written using the unittest framework. It is important to note that Omniverse also provides a similar testing framework. However, it requires going through the build process and does not support testing of the python module in standalone applications.

Extension Dependency Management#

Certain extensions may have dependencies which require installation of additional packages before the extension can be used. While Python dependencies are handled by the setuptools package and specified in the setup.py file, non-Python dependencies such as ROS packages or apt packages are not handled by setuptools. To handle these dependencies, we have created an additional setup procedure described in the next section.

There are two types of dependencies that can be specified in the extension.toml file under the isaac_lab_settings section:

apt_deps: A list of apt packages that need to be installed. These are installed using the apt package manager.
ros_ws: The path to the ROS workspace that contains the ROS packages. These are installed using the rosdep dependency manager.

As an example, the following extension.toml file specifies the dependencies for the extension:

[isaac_lab_settings]
# apt dependencies
apt_deps = ["libboost-all-dev"]

# ROS workspace
# note: if this path is relative, it is relative to the extension directory's root
ros_ws = "/home/user/catkin_ws"

These dependencies are installed using the install_deps.py script provided in the tools directory. To install all dependencies for all extensions, run the following command:

# execute from the root of the repository
# the script expects the type of dependencies to install and the path to the extensions directory
# available types are: 'apt', 'rosdep' and 'all'
python tools/install_deps.py all ${ISAACLAB_PATH}/source/extensions

Note

Currently, this script is automatically executed during the build process of the Dockerfile.base and Dockerfile.ros2. This ensures that all the ‘apt’ and ‘rosdep’ dependencies are installed before building the extensions respectively.

Standalone applications#

In a typical Omniverse workflow, the simulator is launched first, after which the extensions are enabled that load the python module and run the python application. While this is a recommended workflow, it is not always possible to use this workflow. For example, for robot learning, it is essential to have complete control over simulation stepping and all the other functionalities instead of asynchronously waiting for the simulator to step. In such cases, it is necessary to write a standalone application that launches the simulator using AppLauncher and allows complete control over the simulation through the SimulationContext class.

"""Launch Isaac Sim Simulator first."""

from omni.isaac.lab.app import AppLauncher

# launch omniverse app
app_launcher = AppLauncher(headless=False)
simulation_app = app_launcher.app


"""Rest everything follows."""

from omni.isaac.lab.sim import SimulationContext

if __name__ == "__main__":
   # get simulation context
   simulation_context = SimulationContext()
   # reset and play simulation
   simulation_context.reset()
   # step simulation
   simulation_context.step()
   # stop simulation
   simulation_context.stop()

   # close the simulation
   simulation_app.close()

The source/standalone directory contains various standalone applications designed using the extensions provided by Isaac Lab. These applications are written in python and are structured as follows:

demos: Contains various demo applications that showcase the core framework omni.isaac.lab.
environments: Contains applications for running environments defined in omni.isaac.lab_tasks with different agents. These include a random policy, zero-action policy, teleoperation or scripted state machines.
tools: Contains applications for using the tools provided by the framework. These include converting assets, generating datasets, etc.
tutorials: Contains step-by-step tutorials for using the APIs provided by the framework.
workflows: Contains applications for using environments with various learning-based frameworks. These include different reinforcement learning or imitation learning libraries.