Resources: Zooniverse: AI4Mars

Scanning the Surface

NASA scientists are developing software (SPOCv2) that can be used to analyze Mars rover (Curiosity, Perseverance) images so that the autonomous vehicles can safely navigate the red planet. First however, the SPOCv2 artificial intelligence (AI) software must be trained. To do this, the software is given many carefully labeled images of the Martian terrain to test on. Neither NASA scientists nor computers can do this labeling work alone; the amount of training data needed is just too vast, and there are not yet any computers that can recognize and block off these terrain patterns as accurately as the human eye. This is where your work as a citizen scientist comes in. In this lab you will classify images of Martian terrain types that will train AI software to safely navigate the Mars rovers.


Image 1

The three terrain types you will identify are sand, soil, and bedrock. These are seen above from left to right: first two examples of soil, then one of sand, then finally two of bedrock. Sand appears like sand at the beach, fine and powdery and sometimes in rippled sand dune structure. The rover can easily get stuck in sand, leaving deep tire tracks in it, and needs to try to avoid it. Soil is a compacted mix of dirt and pebble. The rovers can move easily over soil, leaving light tracks in their wake. Bedrock is generally flat and smooth and appears patchy with frequent gaps between the main smooth rocks. The Mars rovers usually have success traversing bedrock as well as soil. Large rocks may also may an appearance, but these are very uncommon.

How to Use Zooniverse: AI4Mars

Visit the webpage for Zooniverse: AI4Mars. When you arrive at the webpage, click 'INTERMEDIATE: Color Curiosity MastCam Images' under 'Get started v'. You will first be prompted to review a short tutorial. Read through this. You will then be brought to the main Classify page where on the left of your screen you will see an image from a Mars rover and on the right of your screen you will see a statement that says 'Select points to draw polygons around different types of the surface.' (see below).

AI4Mars Image 2

Click on a feature that you see present in the image ('Sand', 'Consolidated soil', 'Bedrock', or perhaps 'Big rocks'. Then use the application's image-interactive ability to click to draw verticies of polygons until you've encompassed the feature you identified. If you selected Sand', green lines will be drawn when making your shape, if you selected 'Consolidated soil', teal lines will be drawn, if you selected 'Bedrock', yellow lines will be drawn, etc. When making your shapes, close up a polygon by either double-clicking or clicking on the first point you started on. When drawing a shape, an 'x' sign will appear that, if you click on it, will allow you to erase a shape you're working on if you've made a mistake. Classify all features and then click 'Done' in the bottom right (blue star above). If at any point in time you need more guidance than what is given on this lab webpage, you can click 'ABOUT', 'TUTORIAL', and 'FIELD GUIDE' (yellow stars above) to read/revisit more information about this Zooniverse project.

Classify the Images to Produce Training Materials for Artificial Intelligence Mars Navigation Software

Although all the examples you’ve seen above show patches of terrain that take up the whole image, more often the images you see will be a mix of terrain types. Mark each type of terrain you see in an image as shown below before clicking ‘Done’. Note how in the last two images, some portions of the image have been left blank. If you are unsure of the classification of part of an image (or of an entire image), this is totally fine, just mark what you can (if anything) and click ‘Done’ and move on.

AI4Mars Image 3

Here are some main ideas to keep in mind while classifying. We’ll discuss the bullet points not introduced yet in more detail further below:

  • Block out the sand, soil, and bedrock you see in each image
  • Don’t overlap your blocks
  • You do not need to block out the entire picture
  • When in doubt, just leave regions (or the whole image) unmarked and submit with ‘Done’
  • Any features likely smaller than the rover do not need to be classified
  • You should leave landscapes at a distance unlabeled

An Important Note on Perpsective

The perspective of the rover is important when classifying these images. In general, when marking features to avoid hazards, you do not need to mark any features smaller than the size of the rover, because if features in question are smaller than the size of the rover, the rover can just drive right over without issue. But how big is the rover relative to your image? This depends on the perspective of the image. Sometimes the camera on the rover is pointed up and includes a view of the horizon (see all image below except the top right image). Here, the rover is small compared to the image, especially the landscapes at a distance (for this reason, you should leave landscapes in the background (toward the top of these images where the horizon is visible) unlabeled. But in the foreground, considering labeling the ground in more, smaller patches may be important if you see variation in the landscape. Sometimes the camera on the rover is pointed down at the ground (see sixth image below, the top right image). Here, the rover is large compared to the image, and considering the ground in fewer, larger patches is okay. (Related to this, big rocks are also a terrain option, but these are extremely rare, and for the reason of perspective, it’s usually hard to know when a rock is a big rock. If you feel certain you see some big rocks, you can label them, but if not, just leave that region of the image blank.)

AI4Mars Image 4

Making Tracks and Taking Selfies

You might see some unusual details in your images (examples below). If the rover drove over the terrain before taking an image of it, tire tracks may be seen. Sometimes the rover captures part of (or a lot of) itself in the images it takes. If the rover is fully blocking the view (or if there is something really wrong with the image, as seen at the bottom right) the image can be left blank before submission; otherwise label the rest of the image as normal.

AI4Mars Image 5

A Final Word

Some images will be a bit tricky; do not stress and just do the best you can. Remember you do not have to mark the whole image (and frequently shouldn’t), and when in doubt, just leave it blank and click ‘Done’. Maybe you have an image where you only see the distant horizon, or maybe you seem to be looking up a hill of some type (see below). It’s okay to move on. Trust your work and enjoy examining an alien world.

AI4Mars Image 6