Technology Blog
July 29, 2024
Field AI Develops World’s First Risk-bounded Foundation Models for Robots to Safely Operate in Complex Unstructured Field Environments, Leveraging NVIDIA Isaac Platform
NVIDIA Isaac platform and Isaac Lab simulation framework help advance Field AI’s revolutionary autonomy models
Field AI continues to redefine the boundaries of how mobile robots can be used to operate autonomously in highly variable industrial environments. Field AI’s Field Foundation Models™️ (FFMs) enable various robot embodiments to venture into environments that few are equipped to handle: hazardous, offroad, and potentially harsh industrial settings without reliance on prior maps, GPS, or cloud communication.
Field AI’s platform-agnostic FFMs are specifically designed to enable safe deployability of foundation models via a tight integration of probabilistic methods and data-driven techniques. Combining simulation data with real-world data in FFM pushes the boundaries of the reliability solutions, and has led to field-proven performance and deployment in hundreds of sites.
Augmenting Massive Real-world Datasets with Synthetic Data
NVIDIA Isaac Lab, a simulation framework for robot learning, plays an important role by augmenting Field AI’s already massive datasets from real-world customer workflows with synthetically generated data to further improve the generalizability of the FFM models. Field AI leverages Isaac Lab with thousands of parallel environment instances, each configured with different mission objectives, to create truly multitask and multi-purpose foundation models for robotic systems.
Human Interaction Tasks
Application Example Video: Using NVIDIA Isaac Lab to Develop Large Language Models (LLMs) for Instructing and Training Humanoid Robots
For learning-based modules like VLMs / LLMs, integrating human feedback is crucial. Field AI’s parallel testing frameworks developed and integrated into NVIDIA Isaac Lab enable gathering of language inputs across different instances, refining and improving the models based on human feedback and interactions.
Reinforcement Learning of Locomotion Policies
Supporting diverse robots: Field AI’s autonomy applies to many robot embodiments, including quadrupeds, humanoids, passenger rovers, flying and ground vehicles, with their locomotion policies effectively trained using NVIDIA Isaac Sim.
Field AI’s FFM focuses on high-level mission intelligence, enabling capabilities such as autonomous risk-aware routing, progress tracking, object tracking, and precise data capture for progress monitoring during facilities exploration. It uniquely interfaces with the lower-level locomotion stack without descending to actuator control. In cases where such locomotion policies need effective training, Isaac Lab offers reinforcement learning capabilities that play an integral part in Field AI’s simulation stack. Running multiple instances in parallel enables efficient training of locomotion policies. These diverse scenarios contribute to developing robust and adaptable locomotion policies that excel across different environments.
Simulation-based Parallelized Unit Testing
Each major development of Field AI’s stack is rigorously tested for edge cases and specific functionality testing through continuous integration and parallelized in-simulation unit testing in Isaac Lab. This step is part of Field AI’s larger suite of regression tests over thousands of canonical environments, ensuring compatibility with previous versions, identifying potential code issues (e.g., performance bottlenecks) early, and maintaining consistent and reliable performance of the stack throughout the development process.
Ensuring Robust Performance: Field AI’s stack undergoes rigorous continuous integration and in-simulation unit testing in Isaac Lab, part of a comprehensive regression testing strategy to maintain reliability and performance across all deployments
Moreover, differentiating between suites of existing policies can be challenging since they perform similarly in certain scenarios but differently in others. The subtle differences in behavior usually emerge under edge cases. Field AI’s extensive testing framework enables the capture of these nuances, allowing assignment of higher-quality metrics to stack updates. By evaluating how each policy handles these edge cases, Field AI makes more informed decisions about improvements and optimizations.
Each version of Field AI’s stack is tested for hundreds of hours before it is deployed across a diverse fleet of robots. By adopting this testing strategy, Field AI ensures that any changes to the stack do not adversely affect overall performance, thereby maintaining the system's reliability and effectiveness.
Integration Testing
Beyond simulation-assisted unit testing, it is vital to evaluate how changes across different components of Field AI’s stack affect the entire system. For example, Field AI simulates real-time data flows, sensor inputs, and communication protocols to verify their integration across the system. This involves testing the interactions between modules to ensure seamless integration and functionality, which is done efficiently in parallel in Isaac Lab.
Continuous Improvement and Future Integrations
As Field AI continues to deploy robotics AI systems in industrial job sites around the world, those learnings are captured and incorporated into future updates and iterations of these AI models, ensuring FFMs continue to improve as Field AI’s customer list continues to expand. As Field AI grows, the company plans to continue using NVIDIA Isaac Lab and NVIDIA Isaac Sim tools as integral parts of its development and testing stack.