Resources: Zooniverse: AI4Mars

Scanning the Surface

NASA scientists are developing software (SPOCv2) that can be used to analyze Mars rover (Curiosity, Perseverance) images so that the autonomous vehicles can safely navigate the red planet. First however, the SPOCv2 artificial intelligence (AI) software must be trained. To do this, the software is given many carefully labeled images of the Martian terrain to test on. Neither NASA scientists nor computers can do this labeling work alone; the amount of training data needed is just too vast, and there are not yet any computers that can recognize and block off these terrain patterns as accurately as the human eye. This is where your work as a citizen scientist comes in. In this lab you will classify images of Martian terrain types that will train AI software to safely navigate the Mars rovers.


Image 1

The six terrain types you will identify are bedrock, float rock(s), sand, pebbles, hill/peak, and vein. Pebbles, sand, and bedrock are seen above from left to right: first two examples of pebbles, then one of sand, then finally two of bedrock. Bedrock is generally flat and smooth and appears patchy with frequent gaps between the main smooth rocks. The rocks are firmly attached to the ground. Though generally flat, sections of bedrock can be elevated. The Mars rovers usually have success traversing bedrock. Float rocks are rocks on the ground not firmly attached to the ground. You could pick them up; though they might need a bit of digging out and might be heavy, picking up these rocks would not be impossible. Sand appears like sand at the beach, fine and powdery and sometimes in rippled sand dune structure. The rover can easily get stuck in sand, leaving deep tire tracks in it, and needs to try to avoid it. The pebbles terrain is a compacted mix of pebbles and finer dirt, appearing almost like a soil. The rovers can move easily over pebbles, as with bedrock, leaving light tracks in their wake. Hill/peak is any pointed feature that is not a rock or a sand dune, but a larger part of the landscape. Veins, or streaks across rocks of a contrasting color, may also may an appearance, but these are very uncommon.

How to Use Zooniverse: AI4Mars

Visit the webpage for Zooniverse: AI4Mars. When you arrive at the webpage, click either rounded rectangle under 'Get started v'. You will first be prompted to review a short tutorial. Read through this. You will then be brought to the main Classify page where on the left of your screen you will see an image from a Mars rover and on the right of your screen you will see a statement that says 'Select points to draw polygons around geological features.' (see below).

AI4Mars Image 2

Click on a feature that you see present in the image ('bedrock', 'float rock(s)', 'sand', 'pebbles', 'hill/peak' or perhaps 'vein'. Then use the application's image-interactive ability to click to draw verticies of polygons until you've encompassed the feature you identified. If you selected 'bedrock', green lines will be drawn when making your shape, if you selected 'float rock(s)', teal lines will be drawn, if you selected 'sand', yellow lines will be drawn, etc. When making your shapes, close up a polygon by either double-clicking or clicking on the first point you started on. When drawing a shape, an 'x' sign will appear that, if you click on it, will allow you to erase a shape you're working on if you've made a mistake. Classify all features and then click 'Done' in the bottom right (blue star above). If at any point in time you need more guidance than what is given on this lab webpage, you can click 'ABOUT', 'TUTORIAL', and 'FIELD GUIDE' (yellow stars above) to read/revisit more information about this Zooniverse project.

Classify the Images to Produce Training Materials for Artificial Intelligence Mars Navigation Software

Although all the examples you’ve seen above show patches of terrain that take up the whole image, more often the images you see will be a mix of terrain types. Mark each type of terrain you see in an image as shown below before clicking ‘Done’. Note how in the last two images, some portions of the image have been left blank. If you are unsure of the classification of part of an image (or of an entire image), this is totally fine, just mark what you can (if anything) and click ‘Done’ and move on. (Note that in the images below, green is sand instead of bedrock, teal is pebbles instead of float rock(s), and yellow is bedrock instead of sand.)

AI4Mars Image 3


Bedrock, float rock(s), and sand require further classification into subcategories. Bedrock and flat rock(s) are usually layered or massive (meaning one mass and not layered, not necessarily large). The can also be holey or conglomerate (made up of smaller rocks), though these subcategories are rare. Sand can be ordinary sand, wavy ripples, or larger dune-like structures. When you select that sone of your regions are bedrock, flat rock(s), and sand, you will need to classify further and then click 'OK' before beginning to draw another polygon.

Main Ideas

Here are some main ideas to keep in mind while classifying. We'll discuss the bullet points not introduced yet in more detail further below:

  • Block out the bedrock, float rock(s), sand, pebbles, and hill/peaks (and veins, if any) you see in each image
  • Don’t overlap your blocks
  • You do not need to block out the entire picture
  • When in doubt, just leave regions (or the whole image) unmarked and submit with ‘Done’
  • Any features likely smaller than the rover do not need to be classified
  • You should leave landscapes at a distance unlabeled, unless they are a hill/peak

An Important Note on Perspective

The perspective of the rover is important when classifying these images. In general, when marking features to avoid hazards, you do not need to mark any features smaller than the size of the rover, because if features in question are smaller than the size of the rover, the rover can just drive right over without issue. But how big is the rover relative to your image? This depends on the perspective of the image. Sometimes the camera on the rover is pointed up and includes a view of the horizon (see all image below except the top right image). Here, the rover is small compared to the image, especially the landscapes at a distance (for this reason, you should leave landscapes in the background (toward the top of these images where the horizon is visible) unlabeled. But in the foreground, considering labeling the ground in more, smaller patches may be important if you see variation in the landscape. Sometimes the camera on the rover is pointed down at the ground (see sixth image below, the top right image). Here, the rover is large compared to the image, and considering the ground in fewer, larger patches is okay. (Note that in the images below, green is sand instead of bedrock, teal is pebbles instead of float rock(s), and yellow is bedrock instead of sand.)

AI4Mars Image 4

Making Tracks and Taking Selfies

You might see some unusual details in your images (examples below). If the rover drove over the terrain before taking an image of it, tire tracks may be seen. Sometimes the rover captures part of (or a lot of) itself in the images it takes. If the rover is fully blocking the view (or if there is something really wrong with the image, as seen at the bottom right) the image can be left blank before submission; otherwise label the rest of the image as normal. (Note that in the images below, green is sand instead of bedrock, teal is pebbles instead of float rock(s), and yellow is bedrock instead of sand.)

AI4Mars Image 5

A Final Word

Some images will be a bit tricky; do not stress and just do the best you can. Remember you do not have to mark the whole image (and frequently shouldn’t), and when in doubt, just leave it blank and click ‘Done’. Maybe you have an image where you only see the distant horizon, or maybe you seem to be looking up a hill of some type (see below). It’s okay to move on. Trust your work and enjoy examining an alien world.

AI4Mars Image 6