FieldAI Develops World’s First Risk-bounded Foundation Models for Robots to Safely Operate in Complex Unstructured Field Environments, Leveraging NVIDIA Isaac Platform

FieldAI continues to redefine the boundaries of how mobile robots can be used to operate autonomously in highly variable industrial environments.

‍

FieldAI’s Field Foundation Models™️ (FFMs) enable various robot embodiments to venture into environments that few are equipped to handle: hazardous, offroad, and potentially harsh industrial settings without reliance on prior maps, GPS, or cloud communication.

‍

FieldAI’s platform-agnostic FFMs are specifically designed to enable safe deployability of foundation models via a tight integration of probabilistic methods and data-driven techniques. Combining simulation data with real-world data in FFM pushes the boundaries of the reliability solutions, and has led to field-proven performance and deployment in hundreds of sites.

Augmenting Massive Real-world Datasets with Synthetic Data

‍

NVIDIA Isaac Lab, a simulation framework for robot learning, plays an important role by augmenting FieldAI’s already massive datasets from real-world customer workflows with synthetically generated data to further improve the generalizability of the FFM models. FieldAI leverages Isaac Lab with thousands of parallel environment instances, each configured with different mission objectives, to create truly multitask and multi-purpose foundation models for robotic systems.

Human Interaction Task

‍

^{Application Example Video: Using NVIDIA Isaac Lab to Develop Large Language Models (LLMs) for Instructing and Training Humanoid Robots}

‍

‍

‍

For learning-based modules like VLMs / LLMs, integrating human feedback is crucial. FieldAI’s parallel testing frameworks developed and integrated into NVIDIA Isaac Lab enable gathering of language inputs across different instances, refining and improving the models based on human feedback and interactions.

Reinforcement Learning of Locomotion Policies

‍

FieldAI’s FFM focuses on high-level mission intelligence, enabling capabilities such as autonomous risk-aware routing, progress tracking, object tracking, and precise data capture for progress monitoring during facilities exploration. It uniquely interfaces with the lower-level locomotion stack without descending to actuator control. In cases where such locomotion policies need effective training, Isaac Lab offers reinforcement learning capabilities that play an integral part in FieldAI’s simulation stack. Running multiple instances in parallel enables efficient training of locomotion policies. These diverse scenarios contribute to developing robust and adaptable locomotion policies that excel across different environments.

‍

^{Supporting diverse robots: FieldAI’s autonomy applies to many robot embodiments, including quadrupeds, humanoids, passenger rovers, flying and ground vehicles, with their locomotion policies effectively trained using NVIDIA Isaac Sim.}

Simulation-based Parallelized Unit Testing

‍

Each major development of FieldAI’s stack is rigorously tested for edge cases and specific functionality testing through continuous integration and parallelized in-simulation unit testing in Isaac Lab. This step is part of FieldAI’s larger suite of regression tests over thousands of canonical environments, ensuring compatibility with previous versions, identifying potential code issues (e.g., performance bottlenecks) early, and maintaining consistent and reliable performance of the stack throughout the development process.

‍

^{Ensuring Robust Performance: FieldAI’s stack undergoes rigorous continuous integration and in-simulation unit testing in Isaac Lab, part of a comprehensive regression testing strategy to maintain reliability and performance across all deployments.}

‍

Moreover, differentiating between suites of existing policies can be challenging since they perform similarly in certain scenarios but differently in others. The subtle differences in behavior usually emerge under edge cases. FieldAI’s extensive testing framework enables the capture of these nuances, allowing assignment of higher-quality metrics to stack updates. By evaluating how each policy handles these edge cases, Field AI makes more informed decisions about improvements and optimizations.

‍

Each version of FieldAI’s stack is tested for hundreds of hours before it is deployed across a diverse fleet of robots. By adopting this testing strategy, FieldAI ensures that any changes to the stack do not adversely affect overall performance, thereby maintaining the system's reliability and effectiveness.

Integration Testing

‍

Beyond simulation-assisted unit testing, it is vital to evaluate how changes across different components of FieldAI’s stack affect the entire system. For example, FieldAI simulates real-time data flows, sensor inputs, and communication protocols to verify their integration across the system. This involves testing the interactions between modules to ensure seamless integration and functionality, which is done efficiently in parallel in Isaac Lab.

Continuous Improvement and Future Integrations

‍

As FieldAI continues to deploy robotics AI systems in industrial job sites around the world, those learnings are captured and incorporated into future updates and iterations of these AI models, ensuring FFMs continue to improve as Field AI’s customer list continues to expand. As FieldAI grows, the company plans to continue using NVIDIA Isaac Lab and NVIDIA Isaac Sim tools as integral parts of its development and testing stack.