Working with QA Bands and Bitmasks in Google Earth Engine

Most optical satellite imagery products come with one or more QA-bands that allows the user to assess quality of each pixel and extract pixels that meet their requirements. The most common application for QA-bands is to extract information about cloudy pixels and mask them. But the QA bands contain a wealth of other information that can help you remove low quality data from your analysis. Typically the information contained in QA bands is stored as Bitwise Flags. In this post, I will cover basic concepts related to Bitwise operations and how to extract and mask with specific quality indicators using Bitmasks.

For this post, we will work with MOD11A1.006 Terra Land Surface Temperature and Emissivity Daily Global 1km dataset. But the concepts and code snippet can be applied to any other dataset easily. Looking at the bands metadata, we see that the LST_Day_1km band is accompanied by a QC_Day band containing LST Quality Indicators.

Understanding QA flags

The QC_day band contains 8-bits of information. That means the pixel values can range from 0 to 255. There are 4 different indicators stored in this band, with each having 4 possible values. Each combination of these values result in a different 8-bit number that is stored in the output. We can learn about about how these QA bits can be interpreted using a concrete example. Let’s say, you added the QC_Day band to the map and inspected a pixel. The pixel has a value of 145. What does it mean? Well, the value 145 is the result of each QA flag being set to a specific value.

Tip: Use this handy decimal to binary converter to know the binary representation of a number and vice-versa.

The figure below breaks down the value into bit pairs and shows how they can be interpreted.

Understanding BitMasks

Let’s say you want to query the QA band for all pixels where Bits 4-5 are set to a specific value. You will need to extract the information stored in Bits 4-5 while ignoring all other bits. This can be achieved using a Bitmask. Simply put, a bitmask will retain information in the bits of interest and set all other values to 0.

Bitmasks can be created using the left-shift (<<) and right-shift (>>) operators. The GEE API provides leftShift() and rightShift() functions for ee.Image() and ee.Number(). Once a bitmask is created, it can be applied on the input image using the bitwiseAnd() function. Below is a GEE API function that extracts the specific bits.

// Helper function to extract the values from specific bits
// The input parameter can be a ee.Number() or ee.Image()
// Code adapted from https://gis.stackexchange.com/a/349401/5160
var bitwiseExtract = function(input, fromBit, toBit) {
  var maskSize = ee.Number(1).add(toBit).subtract(fromBit)
  var mask = ee.Number(1).leftShift(maskSize).subtract(1)
  return input.rightShift(fromBit).bitwiseAnd(mask)
}

To understand this code, take a look at the figure below. We will continue the example from the previous section and try to see how this algorithm works by answering the question: What is the value of bits 4 and 5 in the number 145?

Masking Low Quality Data

Let’s now put all of this together and see how we can mask low-quality data from the MODIS LST images. Our goal is to use the QC_day band and extract all pixels from the LST_Day_1km image where

  • QA Flag (Bits 0-1) is 0 or 1 (LST produced of both good and other quality)
  • Data Quality Flag (Bits 2-3) is 0 (Good data quality)
  • LST Error Flag (Bits 6-7) is 0 (Average LST error ≤ 1K)
var modisLST = ee.ImageCollection("MODIS/006/MOD11A1")
var lsib = ee.FeatureCollection("USDOS/LSIB_SIMPLE/2017")
var australia = lsib.filter(ee.Filter.eq('country_na', 'Australia'))
var geometry = australia.geometry()
var terra = modisLST
  .filter(ee.Filter.date('2001-01-01', '2010-01-01'))
  .select('LST_Day_1km','QC_Day');
  
// Get a single image for testing
var image = ee.Image(terra.first())
var lstDay = image.select('LST_Day_1km')
var qcDay = image.select('QC_Day')
// Let's extract all pixels from the input image where
// Bits 0-1 <= 1 (LST produced of both good and other quality)
// Bits 2-3 = 0 (Good data quality)
// Bits 4-5 Ignore, any value is ok
// Bits 6-7 = 0 (Average LST error ≤ 1K)
var qaMask = bitwiseExtract(qcDay, 0, 1).lte(1)
var dataQualityMask = bitwiseExtract(qcDay, 2, 3).eq(0)
var lstErrorMask = bitwiseExtract(qcDay, 6, 7).eq(0)
var mask = qaMask.and(dataQualityMask).and(lstErrorMask)
var lstDayMasked = lstDay.updateMask(mask)  
var visParams = {min:13000, max:16000, palette: ['green', 'yellow', 'red']}
Map.addLayer(lstDay.clip(geometry), visParams, 'Original LST Image');
Map.addLayer(lstDayMasked.clip(geometry), visParams, 'LST Masked');
Original vs. Masked Image

You can also wrap this code in a function and map() it over a collection, to mask all images with the same criteria. Here’s the full script showing how to do it with additional comments.

If you are new to Earth Engine and want to master it, check out my course End-to-End Google Earth Engine.

18 Comments

Leave a Comment

  1. Thank for easy and detail explanation. I have one query that what is meaning for

    var cloudBitMask = 1 << 10,
    var cirrusBitMask = 1 << 11 and
    bitwiseAnd;

    in below link :-

    https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2?hl=en

    I understood that bit 10 evaluates the presence of clouds (0: absence, 1: presence), while bit 11 evaluates the presence of cirrus (0: absence, 1: presence) in Sentinel-2 image. is bitwiseAnd same as bitwiseExtract in your above article ?

  2. Thank for easy and detail explanation. I have one query that what is meaning for

    var cloudBitMask = 1 << 10,
    var cirrusBitMask = 1 << 11 and
    bitwiseAnd;

    in below link:-

    https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2?hl=en

    I understood that bit 10 evaluates the presence of clouds (0: absence, 1: presence), while bit 11 evaluates the presence of cirrus (0: absence, 1: presence) in Sentinel-2 image. is bitwiseAnd same as bitwiseExtract in your above article ?

    • << is the left shift operator. I have explained what left shift does in the blog post. bitWiseAnd is a function that is used in the function I wrote (bitwiseExtract) to extract specific bits.

  3. If I understood correctly can I define bitwiseExtract alternatively as below :-

    var bitwiseExtract = function(input, fromBit) {input.bitwiseAnd (3 << fromBit)} or

    var bitwiseExtract = function(input, fromBit) {input.bitwiseAnd (ee.Number(3).leftShift(fromBit))}

  4. I was a user of these kind of bands in the past, but I really didn’t understand them. This blog is absolutely awesome. It is totally understandable. Great work!

  5. Thank you for this article. I found it incredibly helpful. Nice and straightforward with great examples and steps.

  6. Me fue de mucha ayuda su explicacion. Soy nueva en EE y estoy usando un script que funciono bien hasta noviembre-2022 y no se porque las imagenes LST minimas las esta enmascarando mal. Los bitmask lo tengo escritos de otra forma y no comprendo que debo colocar en ee.number y pow.

    //FILTROS CALIDAD NOCHE
    var bestQAn = function(image) {
    // Selecciona las bandas de calidad
    var QAn = image.select(‘QC_Night’);
    var bitMask = ee.Number(2).pow(2).int();
    var bitMask2 = ee.Number(2).pow(3).int();
    var bitMask3 = ee.Number(2).pow(6).int();
    var bitMask4 = ee.Number(2).pow(7).int();
    var mask= QAn.bitwiseAnd(bitMask).eq(0)
    .and(QAn.bitwiseAnd(bitMask2).eq(0))
    .and(QAn.bitwiseAnd(bitMask3).eq(0))
    .and(QAn.bitwiseAnd(bitMask4).eq(0));
    // Devuelve la imagen enmascarada
    return image.updateMask(mask)
    .copyProperties(image, [“system:time_start”]);
    };

    este es el codigo completo
    https://code.earthengine.google.com/baa10260f942ee2edb9ea0dda58d154c

    Agradeceria mucho la ayuda

  7. Hi Ujawal,
    Thanks for this. I tried adaptating this code to get an average of MODIS NDVI at 1km for a10 year period. However, I’m getting an error in the ‘bitmask’ part of the code that says:
    Required argument (right) missing to function: Number.add(left, right)
    Adds the first value to the second.
    Args:
    left (Number): The left-hand value.
    right (Number): The right-hand value.

    Could you please tell what I’m doing wrong?
    Here is my code:
    https://code.earthengine.google.com/c7b9961bad34f75f68499e895f9bb1ba
    Thanks in advance,
    Senan

    • You are missing a parameter when calling the function. bitwiseExtract takes 3 parameters and you are missing the toBit parameter

      var adjacentcloud = bitwiseExtract(qcDay, 8).eq(0)

      Should be

      var adjacentcloud = bitwiseExtract(qcDay, 8, 8).eq(0)

  8. Thank you so much! Landsat document did not do justice in explaining this important concept, but this was clear and concise. Now I can happily implement it in Python 🙂

    • Great. Do share your Python implementation if you can. It is in my TODO list to implement proper bit-masking for Landsat and Sentinel-2 using XArray. All other implementations I have found are pretty bad and usually have a list of hardcoded pixel values instead of properly extracting the relevant bits.

Leave a Reply to Silvana Carina BolziCancel reply