Intelligent Navigation of Autonomous Maritime Robots (Work in Progress)

Brief: While reinforcement learning offers potential for continual learning and adaptability in complex scenarios, its application to real-world robotics faces significant challenges. Unlike in simulations, physical platforms struggle to collect a diverse corpus of training data due to critical safety risks and the inherent constraints of operating within a dynamic and partially observable environment. Our work draws inspiration from the human capability to fuse and exploit multiple sensing modalities, construct comprehensive models of how the world operates, and then leverage those models to adeptly navigate in challenging and often unpredictable environments. Here is an overview of how unmanned vehicles (ground, air, and surface) can exploit a world model constructed through multimodal perception, to learn near-optimal policies for guidance and control. A key aspect of the approach is learning from imagination in which the world model is used to simulate future imagined trajectories, enabling it to anticipate potential risks before encountering them in the real world. Our ongoing work and long-term vision is to evolve the traditional sense-plan-act framework into a more intuitive and cognitively inspired sense-imagine-act model. Dr. Elena Shrestha's presentation of our research.

Role: Graduate Research Assistant, Field Robotics Group

Autonomous Maritime Navigation Challenges:
- Varying weather conditions, unpredictable currents, rough seas, and strong winds
- Difficult to collecting realistic training data
- Noisy sensor measurements
- Model uncertainty (dynamics)
- Reduced performance and reliability
- Real-time decision-making is essential
- The limitations of onboard hardware
- Complex, unstructured environments
- Stochastic nature of environment
- Localization and mapping
- Power (battery) efficiency management

Goal: This project aims to overcome these obstacles, enabling the successful deployment and operation of USVs in novel and unpredictable environments, using our MBRL model.

Why Model-based Reinforcement Learning (MBRL)?
- Better Adaptation: Learns an internal model of the environment, helping it adapt to new policies and conditions, unlike model-free RL, which relies only on previous experience.
- Faster Learning: Learns efficiently with fewer interactions than model-free RL, saving time and reducing the need for costly or risky data collection.
- Clearer Decisions: Tracks important details about the environment, making it easier to understand and improve the agent’s behavior.
- Stronger Sensor Integration: Combines inputs from multiple sensors (like LiDAR and cameras), giving a more accurate view of surroundings.
- Safer High-Speed Control: Plans actions with an awareness of dynamics, balancing speed with safety, ideal for real-time sudden scenarios.

Related Work in MBRL for USV Path Planning

Our MBRL for USV Path Planning
Method:
- Adaptive Model Predictive Control (MPC): Dynamically adjusts to environmental changes using probabilistic models.
- Online Learning: Incorporates new data in real-time to improve model accuracy and robustness.
- Efficiency-Focused Design: Optimizes control actions to minimize computation while maintaining high control stability.
Key Results:
- Fast adaptation in dynamic conditions with reduced model drift.
- High reliability in trajectory tracking and position-keeping under varying disturbances.

Hardware/System Design

My Contribution: Working on “Intelligent Navigation of Autonomous Maritime Robots” towards a publication as the second author

- Developed a URDF model for the Heron USV in ROS2 and Gazebo Garden to create a realistic simulation environment for testing navigation algorithms and collecting more training data.

- Designed and tested an object avoidance algorithm in Python and C++, demonstrating simulation success and contributing to optimizing the USV’s control systems for benchmarking autonomous navigation.

- Conducted real-world navigation tests under varying wave and obstacle conditions to collect sensor data (LiDAR, camera, IMU, GPS, odometry) for multimodal data fusion. Implemented frame alignment and timestamp synchronization to improve Hector SLAM accuracy and performance analysis in RViz.

- Labeled and processed a 2,230-image dataset to train a U-Net segmentation model for applying the Lucid Dreamer model on the USV, which enhances the USV’s perception capabilities and enables improved trajectory planning and navigation in dynamic environments.

- Prepared the Heron USV for deployment by configuring battery, electrical, and mechanical systems, maintaining its functionality for every test. Designed and 3D-printed custom sensor mounts to integrate and calibrate sensors accurately, ensuring synchronized data acquisition.

Difficulties
- Heron broke down sometimes, causing a few delays in testing and collecting data
- Vehicle’s dimension (1.3m x 0.94m, 29kg)
- Only be able to test at the MHL
- MHL’s tight schedule during weekdays (8am - 4pm, not opening on weekend)
- Setup for every wave test requires quite amount of time (1-1.30 hours)

Summary
- This project aims to develop an adaptive, real-time control system for autonomous maritime navigation on the Heron USV.
- Using Model-Based Reinforcement Learning (MBRL), the project addresses key challenges such as variable environmental conditions, sensor noise, and model uncertainty.
- Through innovations in adaptive model predictive control, real-time online learning, and sensor fusion, the system aims to enable reliable USV navigation under unpredictable conditions while minimizing computational load.
- Real-world testing with wave data supports the system’s stability, demonstrating effective path-following and obstacle avoidance in both calm and minor wave disturbance.

Future Work
- Improve the model to work on higher wave disturbance.
- Config the camera capture frequency to 30 FPS.
- Fix camera’s URDF configuration in the code for future need.
- Update the dimension/measurement of each sensor location to be more accurate for better result.

[GitHub][Publication][Slide]

Skills: ROS, ROS2, Rviz, Gazebo, PyTorch, TensorFlow, U-Net, OpenCV, Python, C++, Linux, Bash/Shell Scripting, Git, Docker, Microcontroller, SolidWorks, 3D printing
Contributors' Acknowledgement: Prof. Katherine A. Skinner, Dr. Elena Shrestha, Kathryn Wakevainen, Jingyu Song, Surya Singh