Hyrax Appendix B: Hyrax WCS Service

Hyrax Data Server Installation and Configuration Guide

Appendices

Appendix B: Hyrax WCS Service

Hyrax includes an optional WCS-2 service (specifically version 2.0.1) that can be used to access all of the geo-referenced data available to the server that meet the requirements of the WCS 2 specification. This appendix describes the the kinds of data that meet these requirements along with the configuration process of the bundled WCS service.

11.B.1. Theory of Operation

The WCS utilizes a DAP server (e.g., Hyrax) to supply both coverage metadata and binary data in response to WCS client requests. In this operational model, each DAP dataset is considered a (potential) WCS coverage and the variables within a dataset are (potential) WCS coverage Field entities.

The WCS service attempts to dynamically map DAP datasets to WCS coverages so that the data provider need not learn all of the details of the WCS specifications. All the data provider will need to provide is a simple template for each set of related datasets and coverages. The amount of detail required in a template is a function of the metadata available within a specific datatset. The template, called a DynamicService definition, must provide the domain coordinate details (Latitdue, Longitude, etc.) for a group of coverages and, depending on available metadata, may also need to provide the field/variable definitions. The template uses a regular expression to create the association between the DynamicServicedefinition and files in the DAP server.

WCS Definitions
yellow triangle icon Hack definitions. Look at the OGC abstract documents for a more comprehensive set of definitions.
  • Coverage: A mapping between a domain and a range. This might sound familiar if you remember your first algebra class, because it is the definition of a function. A coverage is a special case, because the domain is often limited to a specific geographic area defined by a range of latitude and longitude values.
  • Domain Coordinate: In a coverage, a variable that provides the values for the coverage’s domain. For example, the variable that provides the latitude values.
  • Spatial Reference System (SRS): The SRS’s axes define the domain of the coverage, which are typically latitude and longitude. The SRS also provides geo-referencing information that enables analysis tools to account for irregularities in the Earth’s geoid. Coordinate Reference System (CRS) is synonymous with this term.
11.B.2. WCS Versions Supported

The Open Geospatial Consortium (OGC) has developed the Web Coverage Service (WCS) as an open specification, and there is a suite of standards documents that describe different aspects of the service. Hyrax supports several of these standards beyond the basic WCS 2.0 core specification.

Light Bulb Icon The Open Geospatial Consortium has many documents that describe the concept of a coverage and the different features of WCS. The suite of specifications that describe WCS can be found on their website.

The WCS service bundled with Hyrax 1.14 supports the following WCS specifications:

  • WCS Core Interface Core, version 2.0.1
  • Coverage Implementation Schema (CIS), version 1.0.1
  • Range Subsetting, version 1.0.0
  • KVP Protocol Binding, version 1.0.1
  • GeoTIFF Coverage Encoding Profile, version 1.0.1
  • JPEG2000 Coverage Encoding Profile, version 1.0.0
  • CF-netCDF 3.0 encoding using GML Coverage Application Schema, version 2.0

We have partial implementations for:

  • XML/POST protocol Binding Extension, version 1.0.0
  • XML/SOAP Protocol Binding Extension, version 1.0.0
  • Scaling Extension, version 1.0.0
  • CRS Extension, version 1.0.0

If you are interested in the Earth Observation Application Profile, version 1.0.0, contact us.

11.B.3. Candidate Datasets

In order for the WCS service to work with a dataset served using DAP, that dataset must contain one or more coverages. The dataset variables must meet the WCS requirements for both structure and metadata. To qualify as a coverage, a variable in a dataset must meet the following criteria:

  • The variable must have an associated Spatial Reference System (SRS) that describes the organization of latitude and longitude for the variable.
  • The variable must be a numeric array of at least two dimentsions.
  • The variable’s right-most dimensions must be axes defined by the SRS (i.e., longitude and latitude), and they must match the SRS’s axis' order.
  • Other dimensions of the variable must be 'to the left' of the dimensions defined by the SRS.
  • The range of the coverage comprises the values of the variable. These values must have an associated unit of measure.
  • Describe how variable shape affects whether a variable can be a coverage. Or say whatever is correct.
Information Icon In practice, Hyrax is often used with data that have global extent, which corresponds to the SRS WGS84(aka EPSG 4326), and the current version of the WCS service only supports this SRS.
Light Bulb Icon We’re interested in adding support for WCS 2.1, but we’d like to gague the interest of potential users before we commit any development resources. The standard metadata for WCS 2.0 is limited to representing two-dimensional data, so variables in a dataset with three or more dimensions cannot be completely described by the CoverageDescription response. WCS 2.1, on the other hand, can represent domains with more than two dimensions. Please contact us if you are interested by sending a note to support@opendap.org.
11.B.4. WCS Installation

The WCS 2 service comes bundled as part of Hyrax-1.14.0 and newer. See the Hyrax download and installation guide included earlier in this document to get Hyrax installed and running, and then return to this guide for WCS configuration information.

Assuming that you have Hyrax installed and running on your local system, you should be able to quickly verify the WCS service is available by pointing your browser at the default WCS endpoint: http://localhost:8080/opendap/wcs. This link should return a browser renderable HTML page of the Capabilities document with a conspicuously empty Contentssection.

Hyrax 40

11.B.5. Configuration

Because WCS requires certain metadata to work (whereas DAP can function with nothing more than a variable’s name and type), our service provides a way to use WCS with DAP datasets that natively lack the required WCS metadata. We do this by creating mappings (DynamicService instances) between collections of DAP datasets that have similar domain coordinates and a WCS service for the resulting Coverages. These relationships are expressed in the wcs_service.xmlconfiguration file, a simple XML document.

wcs_service.xml

<WcsService>
    <WcsCatalog className="opendap.wcs.v2_0.DynamicServiceCatalog">
        <DynamicService                                              *1                                            
                prefix="M2SDNXSLV"                                   *2                                 
                name="MERRA-2 M2SDNXSLV WCS Service"                 *3               
                pathMatch="^/testbed-13/M2SDNXSLV\.5\.12\.4/.*$"     *4   
                srs="urn:ogc:def:crs:EPSG::4326" >                   *5                 
            <DomainCoordinate
                name="time"
                dapID="time"
                size="1"
                units="Days since 1900-01-01T00:00:00.000Z"
                min="690"
                max="690" />
            <DomainCoordinate
                name="latitude"
                dapID="lat"
                size="361"
                units="deg"
                min="-90"
                max="90" />
            <DomainCoordinate
                name="longitude"
                dapID="lon"
                size="576"
                units="deg"
                min="-180"
                max="180" />
        </DynamicService>
    </WcsCatalog>
</WcsService>

1The DynamicService creates a WCS by creating a link between DAP datasets matching the regex and the WCS meta information provided in the DynamicService definition.

2prefix: This is a simple string used by the WcsCatalog implementation to distinguish each DynamicService. Choosing a value that is in some way related to the collection being serviced can be helpful to people if there are problems later.

3name: A human readable and meaningful name that will be used by the server when it creates a link to the service in the viewers page.

4pathMatch: The value of pathMatch contains a regular expression that the server uses to determine which DAP datasets will be associated with this DynamicService.

5srs: The srs attribute defines the expected SRS for the coverages associated with this DynamicService. The SRS defines the axis labels, order, units and minimum number of domain coordinate dimensions and will be used for any dataset that does not contain an explicit SRS definition. Currently only urn:ogc:def:crs:EPSG::4326 is supported.

Information Icon Currently the only supported SRS is urn:ogc:def:crs:EPSG::4326
Using pathMatch

The pathMatch attribute is used to assign a WCS DynamicService definition to some subset (or possibly all) of the Datasets available through the Hyrax server. This is accomplished by applying the regular expression contained in the value of the pathMatch attribute to the local name (aka local url, path part of url, etc. ) of a candidate dataset.

For example in this URL

http://test.opendap.org:8080/opendap/data/nc/fnoc1.nc

The DAP service endpoint is:

http://test.opendap.org:8080/opendap/

And the local name is:

/data/nc/fnoc1.nc

So for this dataset, the string /data/nc/fnoc1.nc would be compared to the pathMatch regex when determing if a DynamicService endpoint should be advertised in the viewers page for the dataset.

In the previous example the pathMatch attribute is set like this:

pathMatch="^/testbed-13/M2SDNXSLV\.5\.12\.4/.*$"

This value tells the server to assocaiate this WCS definition with any DAP dataset whose local path name on the server matches the regular expression ^/testbed-13/M2SDNXSLV\.5\.12\.4/.*$, which can be read as, "Anything that starts with /testbed-13/M2SDNXSLV.5.12.4/."

Regular expressions are very flexible and it is possible to use them to specify a number of things at a time.

pathMatch Regular Expression Example 1

Consider the following pathMatch regular expression:

pathMatch="^.*coads.*\.nc$"

This will match any dataset path that contains the word "coads" and that ends with ".nc".

pathMatch Regular Expression Example 2

Consider the following pathMatch regular expression:

pathMatch="^/gesdisc/(M2IMNXINT|M2TMNXCHM|M2SDNXSLV|M2I1NXASM|M2TMNPMST)\.5\.12\.4/.*$"

This will match any dataset whose name begins with the following:

  • /gesdisc/M2IMNXINT.5.12.4/
  • /gesdisc/M2TMNXCHM.5.12.4/
  • /gesdisc/M2SDNXSLV.5.12.4/
  • /gesdisc/M2I1NXASM.5.12.4/
  • /gesdisc/M2TMNPMST.5.12.4/

The pathMatch feature allows a DynamicService definition to be associated with a sort of "virtual collection" of datasets on the server, which may be related merely by the fact that their coverage representations are similar.

Domain Coordinate Definitions

The Hyrax WCS relies on the DynamicService definition to be responsible for identifying the specific variables in the DAP datasets that are to be used for the geo-referenced domain coordinates of the coverage. The domain coordinates must appear in the order that they appear in the dimensions of the DAP dataset. They must also match the order of axes represented in the SRS.

yellow triangle icon If there is an unresolvable conflict, the DAP dataset cannot be served as a Coverage until a suitable SRS can be identified.

Many DAP datasets have variables with more than two dimensions, and in general WCS 2.0 only supports 2D data. However, latitude, longitude, and time are frequently seen as domain coordinates in scientific data. These can be utilized in the WCS as long as the inner most (last) two dimensions are in agreement with the SRS.

Information Icon In the WCS data model time is not considered a "domain coordinate," and is therefore not represented in the SRS. Yet it does accomodate transmitting the time domain to the client and subsetting the time domain in the manner of latitude and longitude. The result is that 3D datasets with time, latitude, and longitude fit easily into the WCS model.

In the server we treat time like any other coordinate dimension, so if there is a time dimension on the data, it needs to appear in the set of DomainCoordinate definitions for the service.

Let’s consider the DomainCoordinate definitions from the example above:

<DomainCoordinate
    name="time"
    dapID="time"
    size="1"
    units="Days since 1900-01-01T00:00:00.000Z"
    min="690"
    max="690" />
<DomainCoordinate
    name="latitude"
    dapID="lat"
    size="361"
    units="deg"
    min="-90"
    max="90" />
<DomainCoordinate
    name="longitude"
    dapID="lon"
    size="576"
    units="deg"
    min="-180"
    max="180" />

In our friend EPSG:4326, we know that the axis order is latitude,longitude and that’s the order in the example. There is also an additional time coordinate which comes prior to the defintions for latitude and longitude.

Consider the latitude DomainCoordinate:

<DomainCoordinate name="latitude" dapID="lat" size="361" units="deg" min="-90.0" max="90.0"/>

This tells the service that the coordinate axis named latitude is bound to the DAP variable lat, that a default value for size is 361 elements, the default units are degrees ("deg"), the default minimum value is -90.0 and the default maximum value is 90.0. What this means is that when the DynamicService processes a DAP dataset into a coverage, it will check the dataset’s metadata for this type of information. If any of these values can be determined from the dataset metadata, then that value is used; otherwise the values expressed in the DomainCoordinate definition are used.

Longitude and time are handled in the same way as latitude.

Providing Field defintions

Many DAP datasets either lack the metadata for determining which variables will make suitable coverages or the information may not be in an expected form or location. In order to enable these datasets to be exposed via WCS, Hyrax allows the definition of a field in the DynamicService element.

Information Icon WCS Field names have limitations on the kinds of characters they can contain. Specifically, these field names must be NCNAMEs, which means that they cannot contain special symbols such as @, $, %, &, /, +, most punctuation, spaces, tabs, newlines or parentheses. Furthermore, they cannot begin with a digit, dot (.) or minus (-), although those can appear later in the name. Because DAP variables do not have such a limitation, you may have to provide a new name for the variable.

In the following DynamicService definition, each variable in the dataset is exposed as a WCS field and basic information required by WCS is provided.

A DynamicService definition with field elements

<DynamicService
        prefix="coads"
        name="COADS WCS Service"
        pathMatch="^.*coads.*\.nc$"
        srs="urn:ogc:def:crs:EPSG::4326">
    <DomainCoordinate
        name="time"
        dapID="TIME"
        size="12"
        units="hour since 0000-01-01 00:00:00"
        min="366.0"
        max="8401.335"/>
    <DomainCoordinate
        name="latitude"
        dapID="COADSY"
        size="90"
        units="deg"
        min="-90"
        max="90" />
    <DomainCoordinate
        name="longitude"
        dapID="COADSX"
        size="180"
        units="deg"
        min="-180"
        max="180" />
    <field
        name="SST"
        dapID="SST"
        description="SEA SURFACE TEMPERATURE"
        units="Deg C"
        min="-9.99999979e+33"
        max="9.99999979e+33"/>
    <field
        name="AIRT"
        dapID="AIRT"
        description="AIR TEMPERATURE"
        units="DEG C"
        min="-9.99999979e+33"
        max="9.99999979e+33"/>
    <field
        name="UWND"
        dapID="UWND"
        description="ZONAL WIND"
        units="M/S"
        min="-9.99999979e+33"
        max="9.99999979e+33"/>
    <field
        name="VWND"                       *1                    
        dapID="VWND"                      *2                   
        description="MERIDIONAL WIND"     *3  
        units="M/S"                       *4                    
        min="-9.99999979e+33"             *5          
        max="9.99999979e+33"/>            *6         
</DynamicService>

1name - The name of the WCS Field to associate with the DAP variable. This value must be an NCNAME as described above.

2dapID - The name of the DAP variable that will provide the data for the Field

3description - A human readable description of the variable

4units - The units of the values returned

5min - The minimum value

6max - The maximum value

11.B.6. Remote DAP Server Example

TBD

Last Updated: Sep 24, 2019 at 4:05 PM EDT