Near Real-Time versus Standard Products

MODIS Standard Processing vs Near Real-Time Processing Cloud Top Temperature
MODIS Cloud top tempurature over the Midwest. Thin clouds over Lake Superior show sensitivity to GDAS ancillary data.

Science quality, or higher-level “standard” data products are an internally consistent, well-calibrated record of the Earth’s geophysical properties to support science. They are made available within 8-40 hours of satellite observation. Standard products corresponding to the Land, Atmosphere Near real-time Capability for EOS (LANCE) near real-time (NRT) products are archived as follows:

Comparisons between the NRT and standard products

AIRS NRT

AIRS NRT

In order to generate data products within 3 hours of observation time, a number of changes have been made to standard processing approach to expedite the availability of input data sets.

The Atmospheric Infrared Sounder (AIRS) NRT data products are identical to the AIRS routine products except for the following. The AIRS NRT products are produced:

  • The NRT granules are produced without previous or subsequent granules if those granules are not available within 5 minutes (previous and subsequent granules are used for calibration and are generally present in 5 minutes).
  • Predictive ephemeris/attitude data are used (in contrast, definitive ephemeris/attitude data are used for processing standard products).
  • Nominally, a forecast surface pressure is used; if this is unavailable, a surface climatology is then used.
  • No ice cloud properties retrievals are performed (ice cloud properties are a new retrieval in the Version 6 support product AIRX2SUP).

These differences have the following implications for the AIRS NRT Level 1B and Level 2 products.

Level 1B - The atmospheric surface pressure is not used in Level 1B processing, the AIRS Level 1b radiance data produced in the NRT system are nearly identical to the routinely processed AIRS data. There may be small differences in the geolocation (i.e., longitude and latitude). Also, since the NRT system does not wait as long for the previous or subsequent granule data to be present before processing there may be small differences in the radiances of NRT data that were processed without the previous or subsequent granules. This can happen when AIRS is entering or leaving the range of a downlink station (usually are 17 times/day). The reason for the differences is because the standard product uses the space view offset from the previous and subsequent granules for calibration purposes.

Level 2 - The AIRS Level 2 NRT retrieval products can differ from the routine products because of small differences in assumed surface pressure and/or differences in the radiances. Although it is rare, if the forecast surface pressure is not available when the NRT data are produced the NRT algorithm assumes a surface pressure based on a Digital Elevation Map (DEM) rather than the more accurate forecast surface pressure. The differences in the assumed surface pressure tend to be small (~ 10 mbar) near the equator but can become larger (~ 70 mbar) near synoptic weather events. The pressure differences can also lead to differences in the retrieved temperature and water vapor.

For more information, please read the following document, AIRS Near Real Time (NRT) data products: May 10, 2010, describing the difference between AIRS NRT and Standard Products.

Back to top

AMSR2 NRT

AMSR2 NRT

The LANCE system provides access to near real-time data (less than 3 hours from observation). Users, operational agencies and researchers utilize these products for a wide range of purposes (weather forecasting to monitoring natural hazards). These users often need data much sooner than routine science processing allows and are willing to trade science quality for timely access. Science quality products should be used for latency independent research and applications.

The AMSR SIPS plans to start producing initial AMSR2 standard science quality data products in late 2015 and they will be available from NSIDC DAAC.

Note: Please be aware that the LANCE AMSR2 NRT products are generated using modified versions of the AMSR-E standard product algorithms, which have not been updated to reflect the differences between AMSR-E and AMSR2 calibrations. All LANCE AMSR2 data should be used with the understanding that these are preliminary data products.

Back to top

MISR NRT

MISR NRT

Differences between MISR Level 1 NRT Radiances and Level 1 Standard Products

MISR Level 1 NRT radiance products contain the same content as the standard products. However, changes were made to the code to allow the L1 processing to deal with the session-based MISR L0 data and the Terra attitude data that goes along with this L0 data.

Level 1 Near Real-Time

The MISR NRT Level 1 data products consist of radiance measurements organized in 10-50 minute sessions representing a portion of a single MISR orbit. Measurements are obtained throughout the 380 km MISR swath and projected onto a Space-Oblique Mercator (SOM) map grid. The Ellipsoid-projected and Terrain-projected top-of-atmosphere (TOA) radiance products provide measurements respectively resampled onto the WGS84 reference ellipsoid and onto the topography of a digital elevation model. Spatial sampling is 275 m in all bands of MISR’s nadir camera, and in the red band of the other eight (off-nadir) cameras. The remaining channels are sampled at 1.1 km.

Level 2 Near Real-Time

MISR Level 2 NRT Products are geophysical measurements derived in near real time from Level 1 NRT products. MISR Level 2 NRT Cloud Motion Vector (CMV) datasets are distributed as BUFR and HDF format files within 3 hours of satellite overflight. Product data is organized in 10-50 minute sessions representing a portion of a single MISR orbit. Parameters are projected onto a SOM map grid and defined throughout the 380 km wide MISR swath. At present, available Level 2 NRT products include two formats of the MISR Level 2 CMV product, comprising height-resolved cloud motions representative of the wind field. The CMV product is analogous to the MISR Level 3 CMV product offered in standard production.

MISR BUFR format files have been verified to be compatible with the ECMWF BUFRDC 000400 library and the NCEP BUFRLIB v10-2-3 library.

Differences between MISR Level 2 NRT CMV and Level 3 Standard Products

MISR Level 2 NRT CMV products are intended to contain nearly equivalent science information as L3 MISR CMV product files produced with monthly and seasonal file granularity. The cloud-tracking and reconstruction algorithms of each are equivalent, except for the following details:

  • Whereas the L3 CMV product includes in the product only wind speeds up to speeds of 50 ms-1, the Level 2 NRT CMV product includes wind speeds up to 100 ms-1. Wind speeds greater than 50 ms-1 were originally excluded from the L3 CMV product because they can only be obtained for wind orientations aligned with the ground track of the satellite. The potential sampling bias associated with this limitation is less relevant for rapid response applications.
  • For data sessions in which fewer than 1000 valid retrievals are obtained prior to quality filtering, all retrievals will be omitted on the basis that insufficient data is available to perform necessary checks on the accuracy of camera pointing information.

Level 2 NRT CMV products provide a nominal 75% of the coverage provided by Level 3 CMV products.

Level 3 CMV product datasets are distributed as NetCDF format files comprising data collected over the course of a month, season, or year.

Level 2 NRT CMVs are derived from data sessions typically comprising less than the full orbit employed by standard process.

Level 2 NRT CMV product datasets are generated by a separate implementation of the same algorithms used for standard products that yields nearly equivalent results. Differences are due to the nature of NRT processing and fall within the expected uncertainty of the retrieval.

For more information about Level 2 CMVs, please read section 2 of the following document, JPL D-74995 Data Product Specification for the MISR Cloud Motion Vector Product.

Additional information about the difference between the MISR NRT and standard products is available in the Data Quality Summary for L2 CMV NRT Product and L3 CMV Product.

Back to top

MLS NRT

MLS NRT

In order to generate data products within 3 hours of observation time, a number of changes have been made to standard processing approach to expedite the availability of input data sets.

For detailed information on MLS NRT data product quality, please see the user guide.

The MLS NRT algorithm uses a simplified fast forward model to meet NRT data latency requirements and are therefore not as accurate as the retrievals that constitute the standard MLS products. Nevertheless the results are scientifically useful in selected regions of the earth's atmosphere provided that the data are screened according to the recommendations in the MLS NRT user guide.

The NRT data are typically available within 3 hours of observation and are broken into files containing about 15 minutes of data. The 7 most recent days of MLS NRT data are kept online for users to download. After this time users should download the standard, full production Level-2 data products.

Back to top

MODIS NRT

MODIS NRT

In order to generate data products within 3 hours of observation time, a number of changes have been made to standard processing approach to expedite the availability of input data sets.

Near Real-Time Approach

First, the entrained Terra attitude and ephemeris data are used as opposed to the flight data (FD) product for standard processing. Second, the rate-buffered ground based attitude data (GBAD) Aqua attitude and ephemeris data are used as opposed to the 24-hour EOS Mission Operations System (EMOS) product for standard processing. Third, certain Level 2 (L2) codes have modified production rules to relax the requirements for ancillary data products.

The production rules used for Cloud Mask/Profiles and L2 Clouds were developed by the University of Wisconsin for their near real-time production. The production rules for L2 Aerosols were used in the standard processing for near real-time support of the Arctic Research of the Composition of the Troposphere from Aircraft and Satellites (ARCTAS) campaign. The L2 Fire, Snow, and Sea Ice codes do not make use of ancillary data products and these codes are identical to the ones used in standard operations. A special version of the L2 Land Surface Reflectance for near real-time use was developed by the Principal Investigator (PI); this code uses the National Oceanic and Atmospheric Administration (NOAA) Global Forecast System (GFS) ancillary product as opposed to Global Data Assimilation System (GDAS) used in the standard processing version.

The near real-time Project Generation Executive (PGE) codes were extensively science tested and all products were compared to the baseline products generated by the standard processing. This comparison was made by the atmosphere and land Quality Assessment (QA) (Land Data Operational Products Evaluation [LDOPE]) staff and by the PIs. All PGEs have been validated for use in LANCE-MODIS by the PIs. This process will be repeated as new versions of the PGEs become available and are considered for inclusion in LANCE-MODIS.

Side by Side Comparison

Two examples of side-by-side comparisons of the standard and near real-time products are shown here. The first shows a Land Surface Reflectance granule over the Midwest. There appears to be no difference between the products. However, under close examination the near real-time view shows slightly more haze West of the Great Lakes.

MODIS Standard Processing vs Near Real-Time Processing Land Surface Reflectance
In this side-by-side comparison of a standard and near-real time Land Surface Reflectance granule over the Midwest there appears to be no difference between the products. However, under close examination the near-real time view shows slightly more haze West of the Great Lakes.

In contrast, the second side-by-side comparison is for Cloud Top Temperature for the same granule and there are very obvious differences in the region West of the Great Lakes. This is as a result of the sensitivity of this product to the GDAS ancillary data.

MODIS Standard Processing vs Near Real-Time Processing Cloud Top Temperature
MODIS Cloud top tempurature over the Midwest. Thin clouds over Lake Superior show sensitivity to GDAS ancillary data.

Back to top

OMI NRT

OMI NRT

In order to generate data products within 3 hours of observation time, a number of changes have been made to standard processing approach to expedite the availability of input data sets.

The quality of OMI NRT Level 2 products compares favorably to the standard products. There appears to be some variances at high latitudes with high solar zenith angles. In the following table we display the maximum differences for a single day and averaged over a week.

Product Variable Daily Maximum Percentage Difference Weekly Average Maximum Percentage Difference
OMTO3 Total Ozone Column 2.64% 1.40%
OMCLDRR Cloud Fraction 6.02% 1.42%
Cloud Pressure 2.82% 0.67%
OMAERUV AOD (388 nm) 5.95% 2.31%

Back to top

EOSDIS manages discipline-specific Distributed Active Archive Centers (DAACs). Each DAAC processes, archives and distributes EOS science data based on discipline. They serve the user community by providing science quality products from past and present EOS missions. Other services include:

  • Help with obtaining and selecting data
  • Data-handling and visualization tools
  • Notification of data-related news
  • Technical support and referrals

Last Updated: Aug 17, 2017 at 8:34 AM EDT