SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery

Authors:
Jian Song^1,2, Hongruixuan Chen¹, Weihao Xuan^1,2, Junshi Xia², Naoto Yokoya^1,2

¹ The University of Tokyo
² RIKEN AIP

Conference: Neural Information Processing Systems (Spotlight), 2024

For more details, please refer to our paper and visit our GitHub repository.

---

Overview

TL;DR:
SynRS3D is a comprehensive synthetic remote sensing dataset designed to improve global 3D semantic understanding from monocular high-resolution imagery. It includes data for three key tasks:

• Height estimation
• Land cover mapping
• Building change detection

Additionally, we introduce RS3DAda, a novel multi-task domain adaptation method to enhance performance across these tasks. Learn more about RS3DAda in our repository.

Dataset Structure

The dataset consists of 17 folders and includes a total of 69,667 images at a resolution of 512x512. After downloading and extracting the files, ensure the directory structure follows this format:

${DATASET_ROOT}  # Example: /home/username/project/SynRS3D/data/grid_g05_mid_v1
├── opt           # RGB images (.tif), also used as post-event images for building change detection
├── pre_opt       # RGB images (.tif), used as pre-event images for building change detection
├── gt_nDSM       # Normalized Digital Surface Model (nDSM) images (.tif)
├── gt_ss_mask    # Land cover mapping labels (.tif)
├── gt_cd_mask    # Building change detection masks (.tif, 0 = no change, 255 = change area)
└── train.txt     # List of training data filenames

Class Mapping for gt_ss_mask

The land cover mapping labels (gt_ss_mask) are mapped to the following categories:

• Bareland: 1
• Rangeland: 2
• Developed Space: 3
• Road: 4
• Trees: 5
• Water: 6
• Agriculture land: 7
• Buildings: 8

Image Breakdown by Folder

The dataset is organized into grid-like and irregular terrain. It includes a range of ground sampling distances (GSDs) and variations in building heights. The folder naming convention indicates these characteristics:

• grid = grid-like terrain
• terrain = irregular terrain
• g005, g05, g1 = GSD ranges (0.05m–0.3m, 0.3m–0.6m, and 0.6m–1m, respectively)
• low, mid, high = building height variations

The dataset includes the following image counts:

• 1,430 images – terrain_g05_mid_v1
• 10,000 images – grid_g05_mid_v2
• 2,354 images – terrain_g05_low_v1
• 3,707 images – terrain_g05_high_v1
• 880 images – terrain_g005_mid_v1
• 2,127 images – terrain_g005_low_v1
• 11,325 images – grid_g005_mid_v2
• 1,212 images – terrain_g005_high_v1
• 348 images – terrain_g1_mid_v1
• 4,285 images – terrain_g1_low_v1
• 904 images – terrain_g1_high_v1
• 3,000 images – grid_g005_mid_v1
• 2,997 images – grid_g005_low_v1
• 4,000 images – grid_g005_high_v1
• 7,000 images – grid_g05_mid_v1
• 7,098 images – grid_g05_low_v1
• 7,000 images – grid_g05_high_v1

---

Citation

If you find SynRS3D useful in your research, please consider citing:

@article{song2024synrs3d,
title={SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery},
author={Song, Jian and Chen, Hongruixuan and Xuan, Weihao and Xia, Junshi and Yokoya, Naoto},
journal={arXiv preprint arXiv:2406.18151},
year={2024}
}

---

Contact

For any questions or feedback, feel free to reach out via email: song@ms.k.u-tokyo.ac.jp.

Enjoy using SynRS3D!