Skip to main content

Lesson 3.4: Integrating Isaac Sim, Isaac ROS, and Nav2 for Autonomous Systems

Overview​

This lesson covers the integration of NVIDIA Isaac Sim, Isaac ROS, and Nav2 to create complete autonomous robot systems. You'll learn to combine simulation, perception, and navigation for complex robotic behaviors.

Learning Objectives​

By the end of this lesson, you should be able to:

  • Integrate Isaac Sim, Isaac ROS, and Nav2 in a unified pipeline
  • Configure perception and navigation for humanoid robots in simulation
  • Implement autonomous behaviors combining VSLAM and path planning
  • Test and validate integrated robotic systems in simulation

Introduction to System Integration​

Combining Isaac Sim, Isaac ROS, and Nav2 creates a powerful pipeline:

  • Isaac Sim: Provides photorealistic simulation and synthetic data
  • Isaac ROS: Offers hardware-accelerated perception and processing
  • Nav2: Provides robust navigation and path planning

Integration Architecture​

�����������������    �����������������    �����������������
   Isaac Sim           Isaac ROS              Nav2         
   (Simulation)  ����  (Perception)   ����  (Navigation)   
                                                           
 " Photorealistic     " VSLAM              " Path Planning 
 " Physics            " Object Det.        " Local Planning
 " Sensors            " AI Inference       " Recovery      
�����������������    �����������������    �����������������
                                                       
         �                       �                       �
   Robot in Simulation    Processed Perception    Navigation Commands
                                                       
         �����������������������<�����������������������
                                 �
                           Autonomous Behavior

Setting Up the Integrated Pipeline​

Isaac Sim Configuration for Integration​

# config/isaac_sim_integrated_config.yaml
physics:
  dt: 0.008  # Physics timestep (120 Hz)
  substeps: 1
  gravity: [0.0, 0.0, -9.81]

renderer:
  resolution: [1280, 720]
  msaa_samples: 4

simulation:
  rendering_interval: 1  # Render every frame
  headless: false  # Enable UI for debugging

sensors:
  camera:
    width: 640
    height: 480
    fov: 90  # Field of view in degrees
    clipping_range: [0.1, 100.0]

  lidar:
    samples: 720  # Points per revolution
    channels: 16  # Vertical channels
    max_range: 25.0
    min_range: 0.1
    rotation_frequency: 10.0  # Hz
    points_per_second: 480000

robot:
  name: "humanoid_robot"
  urdf_path: "/path/to/humanoid.urdf"
  initial_position: [0.0, 0.0, 0.5]  # Start slightly above ground
  initial_orientation: [0.0, 0.0, 0.0, 1.0]  # No rotation (identity quaternion)

Isaac ROS Pipeline Configuration​

# config/isaac_ros_pipeline.yaml
isaac_ros_visual_slam:
  ros__parameters:
    enable_rectified_pose: true
    rectified_frame_id: "camera_aligned_rect"
    enable_occupancy_map: true
    occupancy_map_resolution: 0.05
    occupancy_map_size_x: 20.0
    occupancy_map_size_y: 20.0
    enable_localization_n_mapping: true
    enable_loop_closure: true
    enable_image_transfer: true

isaac_ros_stereo_rectification:
  ros__parameters:
    left_topic: "/left_cam/image_raw"
    right_topic: "/right_cam/image_raw"
    left_camera_info_topic: "/left_cam/camera_info"
    right_camera_info_topic: "/right_cam/camera_info"
    left_rect_topic: "/left_cam/image_rect"
    right_rect_topic: "/right_cam/image_rect"
    left_rect_camera_info_topic: "/left_cam/camera_info_rect"
    right_rect_camera_info_topic: "/right_cam/camera_info_rect"

isaac_ros_detect_net:
  ros__parameters:
    input_topic: "/camera/color/image_rect_color"
    output_topic: "/detections"
    model_name: "ssd_mobilenet_v2_coco"
    confidence_threshold: 0.7
    max_batch_size: 1
    input_tensor_names: ["input"]
    output_tensor_names: ["scores", "boxes"]
    input_binding_names: ["input"]
    output_binding_names: ["scores", "boxes"]
    engine_cache_path: "/tmp/trt_cache/detect_net.plan"
    trt_precision: "FP16"
# config/integrated_nav2_config.yaml
bt_navigator:
  ros__parameters:
    use_sim_time: true
    global_frame: map
    robot_base_frame: base_link
    odom_topic: /odom
    bt_xml_filename: "navigate_w_replanning_and_recovery.xml"
    default_server_timeout: 20
    enable_groot_monitoring: True
    enable_logging: True
    enable_scenario_introspection: True
    enable_tf_timeout: True
    global_frame_to_planner_frame_transforms: ["map", "odom"]
    robot_base_frame_to_carrot_frame_transforms: ["base_link", "base_link"]

controller_server:
  ros__parameters:
    use_sim_time: true
    controller_frequency: 10.0  # Slower for humanoid stability
    min_x_velocity_threshold: 0.05
    min_y_velocity_threshold: 0.5
    min_theta_velocity_threshold: 0.001
    progress_checker_plugin: "progress_checker"
    goal_checker_plugin: "goal_checker"
    controller_plugins: ["HumanoidMppiController"]

    # Humanoid-specific MPPI Controller
    HumanoidMppiController:
      plugin: "nav2_mppi_controller::MppiController"
      time_steps: 20
      control_horizon: 10
      time_delta: 0.1
      discretization: 0.1
      # Cost weights for humanoid-specific navigation
      cost_obstacles: 3.0
      cost_goal_dist: 1.0
      cost_path_align: 0.5
      cost_goal_angle: 0.2
      cost_balance: 4.0
      cost_foot_placement: 3.0

local_costmap:
  local_costmap:
    ros__parameters:
      update_frequency: 5.0
      publish_frequency: 2.0
      global_frame: odom
      robot_base_frame: base_link
      use_sim_time: true
      rolling_window: true
      width: 8  # Larger window for humanoid awareness
      height: 8
      resolution: 0.05
      robot_radius: 0.4  # Larger radius for humanoid
      plugins: ["voxel_layer", "inflation_layer"]
      inflation_layer:
        plugin: "nav2_costmap_2d::InflationLayer"
        cost_scaling_factor: 5.0
        inflation_radius: 1.0  # Larger inflation for humanoid safety