Upload data to dataset

1. Upload Options

Upload via browser

If you have less than 500 2D images to work with, feel free to drag and drop them. If you are working with much more images, heavy videos, or point clouds, we strongly recommend using the option "Upload via S3".

Upload via S3

To upload via S3, you need to provide these credentials:

• Access key

• Secret access key

• S3 URI: Path to folder or file. For example:

  • Folder: s3://client-linh/upload/2d/

  • File: s3://client-linh/upload/2d/upload.zip

To find more info on uploading data to your S3 bucket, please visit here.

2. Data folder format on S3

2D Image

Individual mode

upload
│
└───batch2
    │   image3.png
    │   image4.png

The images should be under a folder.

In the case above, to upload both image3.png, image4.png, you may upload the folder upload/batch2 via S3.

Sequence mode

# Bucket: client-linh
upload
│
└───sequence1
│   │   image1.jpg
│   │   image2.jpg
│   |   ...
|
└───sequence2
    │   image3.png
    │   image4.png

Above is the format to upload sequences of 2D images. Each child folder (sequence1 and sequence2) represents a sequence containing a set of images.

In this example, sequence1 is a sequence containing image1.jpg, image2.jpg and so on, and sequence2 is another sequence containing image3.png and image4.png. Then to upload these 2 sequences, upload the folder upload.

3D Point Cloud

Individual mode

# Bucket: client-linh
upload
│
└───lidar
│   │   image1.bin
│   │   image2.pcd
|   |   image3.pcd
│   |   ...
|
└───camera_front
│   │   000001.png
│   │   000002.png
│
└───camera_back
│   │   000003.jpg
│
└───manifest.json

Files in the uploaded folder (in this case the folder named upload) should be structured in this example format.

Files and folders explanation:

• The folder names here like lidar, camera_front are just examples. You can specify the names of your choice.

• There is a compulsory file named manifest.json, which contains the mapping between lidar and context images. The manifest.json file should follow this format:

[
    {
        "lidar": "lidar/image1.bin",
        "camera": {
            "camera_front/000001.png": "front",
            "camera_back/000003.jpg": "back"
        }
    },
    {
        "lidar": "lidar/image2.pcd",
        "camera": {
            "camera_front/000002.png": "front"
        }
    },
    {
        "lidar": "lidar/image3.pcd",
        "camera": {}
    },
    ...
]

• manifest.json contains a list of lidar items. Within each item, lidar key corresponds to the relative path of the lidar file, and the camera key corresponds to the mapping between context image's relative path and the camera ID (camera ID will be used when projecting from 3D to 2D).

  • For instance, in this case, there are 2 context images camera_front/000001.png and camera_back/000003.jpg being mapped with a lidar file lidar/image1.bin. Context image camera_front/000001.png has camera ID as front.

Sequence mode

# Bucket: client-linh
upload
│
└───sequence1
│   │
│   └───lidar
│   │   │   image1.bin
│   │   │   image2.pcd
|   |   |   image3.pcd
│   │   |   ...
│   |
│   └───camera_front
│   │   │   000001.png
│   │   │   000002.png
│   │
│   └───camera_back
│   │   │   000003.jpg
│   │
│   └───manifest.json
|
└───sequence2
    │
    └───pointcloud
    │   │   image3.bin
    │   │   image4.pcd
    │   |   ...
    |
    └───camera_front
    │   │   000004.png
    │   │   000005.png
    │
    └───camera_back
    │   │   000006.png
    │
    └───manifest.json

    

For sequence mode, each child folder represents a sequence containing a set of point clouds and their corresponding context images.

In this case, there are 2 sequences named sequence1 and sequence2. Inside each sequence, the format follows the "Individual mode" (as specified above).

For instance, the manifest.json in sequence1 will be as below. Please note that the order of items inside the list is the order of the lidar frames within the sequence.

[
    {
        "lidar": "lidar/image1.bin",
        "camera": {
            "camera_front/000001.png": "front",
            "camera_back/000003.jpg": "back"
        }
    },
    {
        "lidar": "lidar/image2.pcd",
        "camera": {
            "camera_front/000002.png": "front"
        }
    },
    {
        "lidar": "lidar/image3.pcd",
        "camera": {}
    },
    ...
]