Stereolabs has integrated the ZED SDK and ZED X stereo cameras into NVIDIA Isaac Sim and added support for NVIDIA Isaac ROS to accelerate development of space-aware autonomous robots

Stereolabs, a pioneer in vision-based sensing technology, is making its ZED SDK and ZED X stereo cameras accessible in NVIDIA Isaac Sim, a powerful simulation application for developing, testing and managing AI-based robots. Additionally, these cameras are now compatible with NVIDIA Isaac ROS.

Within the NVIDIA Isaac Sim platform, developers can now harness the power of Stereolabs' ZED X stereo cameras virtually and get full access to ZED SDK, the most advanced stereovision software stack to build space-aware applications.

There are many advantages to using the new integration to virtually design and test autonomous robots:

Hardware: Users can now get access to the exact 3D model of the ZED X Camera to test and validate the number of cameras required, camera positioning, resolution, frame rate and field-of-view.
Software: Developers have access to the complete ZED SDK library, including depth sensing, localization and object detection and can therefore simulate the end-to-end software stack. They can take full advantage of advanced simulation tools like NVIDIA Isaac Sim and get massive gains in terms of reproducibility, scalability, complex scenario testing and data collection. This can lead to major improvements in overall software performance.
Simulations are increasingly becoming a new standard for robot development. And as demand for autonomous robots skyrockets in light of labor shortages and supply chain challenges, simulation platforms like NVIDIA Isaac Sim have enabled developers to design, develop and test any hardware stack in a faster and safer way — saving significant time and resources by iterating frequently without the time-consuming setup that might be required in a physical test.

"Simulation is the future of robotics engineering," said Cecile Schmollgruber, CEO of Stereolabs. "By using Stereolab cameras integrated with NVIDIA Isaac Sim, developers and engineers will be able to experiment with stereovision in a virtual environment for the first time and build space-aware applications before ever attaching a real camera to a real robot. This capability will empower developers and engineers to design, develop and test autonomous robots in much faster cycles to save massively on resources and obtain the best results in the real world."

The ZED SDK and ZED X cameras are compatible and integrated with NVIDIA Jetson modules and have also been made compatible with NVIDIA Isaac ROS. Isaac ROS is a collection of hardware-accelerated, high-performance, low-latency packages that deliver NVIDIA perception technologies to the ROS developer community. Packages like Isaac ROS nvblox will also work easily with ZED cameras.

The integration will enable developers to:

Safely test robots in real-world conditions: Developers can test ZED SDK-based tasks and applications in a virtual environment and simulate different real-world scenarios, such as navigating obstacles in a warehouse.
Use synthetic data generation to train AI: As an alternative to physical cameras, developers can send synthetic streaming frames from Isaac Sim to the Stereolabs SDK to train machine learning models and improve the performance of AI vision-based robots.
Streamline ROS integration: The integration streamlines the development process for ROS users, because the Isaac Sim extension is compatible with Stereolabs' ROS 1/2 wrapper.
Increase cost efficiencies: Isaac Sim eliminates the need for physical cameras, reducing hardware costs and making it more cost-effective to develop and test robotic systems.
Stereolabs' ZED SDK and ZED X stereo cameras are now available within the sensors catalog on NVIDIA Isaac Sim. To learn more, visit developer.nvidia.com/isaac-sim.

To see how the ZED SDK and ZED X stereo cameras will operate within the virtual environment in NVIDIA Isaac Sim platform and how the spatial mapping, depth, and point clouds outputs compare with those from a physical environment, read our blog post.

To learn more about Stereolabs and its vision-based perception technology, visit stereolabs.com.

About Stereolabs

Stereolabs is the leading provider of 3D depth and motion sensing solutions based on stereo vision and artificial intelligence. From robots to drones and cars, Stereolabs empowers machines with the ability to see and understand. We enable developers across multiple industries to create truly intelligent machines and applications that improve our daily lives.