Massive collection of multidimensional satellite-based 3D geographic information systems (GIS) is now a reality. Yet the correlation of such data with local information, especially in unconstrained environments such as in the wild, is a poorly addressed challenge so far.
Our project aims to provide a scalable MultiGIS high-quality data collection platform through the use of a quadrupedal robot that can autonomously perform long-distance missions in challenging environments. Our project will also develop general-purpose and automatic tools for multi-sensor data storage, data management, data consolidation, and data labelling based on Artificial Intelligence (AI) / Machine Learning (ML) algorithms and will look into solutions for efficient data use and dissemination.
This project will lay the foundations for ubiquitous robots in the wild as well as for a MultiGIS database for unstructured environments that is to be an order of magnitude larger than any of the currently existing ones.
The ROBOSAT activities are grouped in five work packages (WPs):
In Romania, ROBOSAT is co-funded by the National Executive Agency for Higher Education, Research, Development and Innovation Funding – UEFISCDI and the CHIST-ERA program.
Activity 1.1: Analysis of ethical issues related to AI usage and correlated with the project implementation - part 1: The analysis of the ethical issues related to exploring the environment with an autonomous robot and collecting a large set of information such as images, videos, coordinates, etc. revealed the importance of the following elements: (a) confidentiality and protection of geospatial data; (b) robotic autonomy and decision-making responsibility; (c) bias in labeling and data fusion algorithms; (d) environmental impact and the ethics of interacting with nature; (e) the ethics of disseminating and reusing collected data.
Activity 1.2: Identifying hardware solutions for the robotic platform: Mobile robots that operate autonomously in complex and unstructured environments require an ansamble of multimodal sensors capable of capturing diverse and complementary information. Designing such a suite of sensors involves multiple critical design decisions, such as sensor selection, component placement, power limitations, computing requirements, communication, synchronization, and calibration.
In terms of positioning, GNSS receivers typically accept single-antenna or dual-antenna configurations; the latter not only provides improved positioning accuracy but also direction estimation. The choice of antennas and receiver capabilities dictates which satellite constellations can be used. In terms of robustness in real environments and on rough terrain, the sensor system must be designed to be robust. It must withstand occasional drops and impacts during operation in various weather conditions throughout all seasons and must be dust and water resistant, in accordance with the IP65 standard. In terms of energy autonomy, the aim is for the sensor suite to consume no more than 300 W so that autonomy of several hours can be achieved by powering it from a medium-capacity battery. In terms of sensors and perception systems, it is important to equip the robot with perception sensors which have to optimize weight, performance and placement.
Activity 1.3: Analysis of suitable simulation environments for robotic platforms: During this stage, several simulation environments for robotic platforms were identified and analyzed. Each simulator analyzed has specific strengths that make it suitable for certain types of applications. Gazebo is preferred in academic and industrial projects involving ROS. Webots is excellent for beginners and prototyping. CoppeliaSim is ideal for complex industrial simulations and multi-robot scenarios. Isaac Sim is suitable for advanced AI applications and simulations with high graphics requirements. The Construct is a good choice for learning, training, and rapid testing in ROS environments without complicated configurations.
Activity 1.4: Widespread scientific dissemination of results (website, social media): As part of this activity, the ROBOSAT project website was developed and can be viewed at the following address: https://citst.ro/projects/robosat. The project is also present on LinkedIn: https://www.linkedin.com/showcase/108599174/admin/dashboard/.
Activity 2.1: Formulation of ethical principles underlying the use of AI for the project implementation: The use of robots in natural environments raises issues related to confidentiality, algorithmic bias, wildlife protection, and ecological impact, requiring the anonymization of sensitive data, validation of automated decisions, emergency shutdown mechanisms, and continuous monitoring. The project requires minimising the environmental footprint, explainability of AI decisions, complete traceability and permanent human control, with clearly defined responsibilities. At the same time, it emphasizes the need to prevent bias through data diversity, as well as the implementation of audit procedures and continuous improvement. Overall, Robosat's ethics aim at the responsible use of technology to ensure human safety, environmental protection, and fair management of collected data.
Activity 2.2: Development of the robotic platform architecture: The architecture of the ROBOSAT project was developed so that the robot can navigate autonomously on natural terrain and adapt its route according to the conditions of the surrounding environment. From a navigation perspective, the system is organized around three major perception components, each contributing to route decision-making: obstacle detection, vegetation type classification, and vegetation change identification.
Activity 2.3: Development of the database required for the project: The architecture of the ROBOSAT project was developed so that the robot can navigate autonomously on natural terrain and adapt its route according to the conditions of the surrounding environment. From a navigation perspective, the system is organized around three major perception components, each contributing to route decision-making: obstacle detection, vegetation type classification, and vegetation change identification.
Activity 2.4: Real data collection - part 1: As part of this activity, the main data sets were identified that can be used to build artificial intelligence models to identify rocks, vegetation, bushes, or vegetation changes. One identified dataset is GrandTour. The environments covered in the dataset are varied to reflect real-world challenges: indoor/outdoor, day/night, different weather conditions, natural terrain (forests, mountains), urban (train stations, university campuses). Other existing datasets identified are: The Great Outdoors (GO) — "Off-Road Multi-Modal Dataset" (RGB images, LiDAR, GPS positions), RELLIS-3D (Texas A&M) (LiDAR data, RGB images, multiple semantic annotations (grass, rock, bush, logs), GPS locations), SFU Mountain Dataset (Burnaby Mountain, Canada) (stereo color and monochrome images, LiDAR data, GPS positions, collected from trails in different conditions (seasons, night, rain, snow)), Freiburg / Freiburg Forest (DeepScene / Freiburg Forest Dataset) (RGB-D stereo images, class annotations: trail, vegetation, obstacle, etc.)), DiTer / DiTer++ (Diverse Terrain & Multi-Modal dataset) (RGB images, depth/point cloud, GPS positions, collected on diverse terrain (slopes, gravel, grass)). Positioning measurements were collected using the GNSSlogger application on multiple Android devices in order to improve the positioning of the ANYmal robot through algorithms developed by the partner.
Activity 2.5: Widespread scientific dissemination of results (website, social media):
IT Centre for Science and Technology
Av. Radu Beller, No. 25, Sect. 1, Bucharest, Romania