Online Dissemination Help Page

Introduction

Welcome to the ESA SMOS Online Dissemination Service. Additional issues or questions not addressed in the current page can be directly referred to smos.dissemination.support@esa.int.

Supported Browsers

The following Web Browsers are supported by this system: Internet Explorer, Firefox, Google Chrome, Safari. It is recommended that a recent version is used.

Available Data

The available free and open data are organised into different collections. Files in the native Earth Explorer (EE) Ground Segment File Format Standard have the extension “.zip”, those in netCDF format have the extension “.nc”.

A description of the individual SMOS data products and a link to the relevant Product Format Specifications can be found here. The structure of the netCDF files is described here in Section 7.

SMOS Level 1 and Level 2 Science data

This collection contains the Level 1B/1C and the Level 2 operational and reprocessed Science data products belonging to the latest processing baseline:

Current products coming from the operational pipeline in the SMOS Data Processing Ground Segment have File Class "OPER". Reprocessed data is tagged as "REPR".

Please note that Level 0 and Level 1A products and auxiliary data files are only available to CalVal users, and only via FTP.

SMOS AUX ECMWF data

This collection contains the Auxiliary ECMWF product belonging to the latest processing baseline:

  • AUX_ECMWF_: ECMWF data on the ISEA 4-9 DGG corresponding to SMOS half-orbit, in EE format.

SMOS NRT data

This collection contains the L1 and L2 Near Real Time (NRT) products.

  • MIR_WMO_1C: NRT L1C Light (i.e. reduced) BUFR product over Land, geo-located on Gaussian grid (T511/N256), in BUFR format, disseminated within 3 hours from sensing.
  • MIR_SMNRT2: NRT L2 Soil Moisture estimates from Neural Network, in netCDF format, disseminated within 4 hours from sensing.

1. Creating an EO-SSO Account and Applying for SMOS Data

You can search for and browse the available data products openly, but you must be registered to "SMOS L1 and L2 SM/OS Science products" (see Section 1.1 below) to download files from the system.

In order to provide a unique username and password for all EO services available on the web, ESA has created the Earth Observation Users Single Sign On service, or EO-SSO.

Those users not having an EO-SSO account should follow the instructions for creating one beforehand. See also the EO-SSO FAQ page.

1.1 Apply for SMOS Data (Fast Registration)

In order to be enabled to download files from any of the available collections (incuding NRT), you have to register to SMOS data using your own EO-SSO account as follows:

  • Login with your EO-SSO account into the "My Earthnet" section available at the top of each web page on the ESA Earth Online web portal
  • Click "Add ESA products", add "SMOS Science data products", accept both "Terms&Conditions" and finally click on the "Save" button.
NOTE: Download of files via FTP/FTPES/HTTP will be enabled within a few days of registration. You will be notified by EOHelp.

1.2 Login

You can login into the system following these steps:

2. Search for Data

The SMOS Online Dissemination Service provides two different mechanisms for searching data, one based on HTTP via the dedicated web portal and another one based on FTP. Below the details on how to use the two methods.

2.1 Search via the Web Portal

To start searching for data, navigate to http://smos-diss.eo.esa.int/, click on the "Search" button and choose a collection. Three product search mechanisms are provided and listed.

2.1.1 Search by Tree Navigation

This function allows navigating a dynamic structure via the Web Browser where data is organised in collections. Inside these the user can navigate the available data types by Processing Level, Product Type, Year, Month and Day (of the sensing or validity date).

2.1.2 Search by File Name

This function allows searching a product by its file name. You can use the whole name of the file of interest, or part of it (using “*” and “?” wildcards) in order to search for multiple products.

Before starting using this functionality we recommend getting familiar with the different SMOS file naming conventions, described in the relevant Product Format Specifications documents available here. A few examples on the usage of this specific function are reported hereafter:

  1. SM_REPR_MIR_SCLF1C_20140318T023030_20140318T032345_620_001_1.zip
    only the product with "file-type" MIR_SCLF1C, in EE format, with that specific file name will match
  2. *REPR_MIR_SCLF1C_20140329*
    all products with "file-type" MIR_SCSF1C and "sensing-start-time" 2014-03-29 will match
  3. *REPR_MIR_SCLF1C_*_20140331*
    all products with "file-type" MIR_SCSF1C and "sensing-stop-time" 2014-03-31 will match
  4. *REPR_MIR_OSUDP2*.zip
    all reprocessed products with "file-type" MIR_OSUDP2, in EE format, irrespective of sensing time, will match
  5. *REPR_MIR_SMUDP2*.nc
    all reprocessed products with "file-type" MIR_SMUDP2, in netCDF format, irrespective of sensing time, will match
  6. W_ES-ESA-ESAC,SMOS,N256_C_LEMM_20150821000435_20150820214935_20150820232714_bufr_v620.bin.bz2
    only the NRT Light BUFR product with that specific file name will match
  7. W_ES-ESA-ESAC,SMOS,N256_C_LEMM_??????????????_20150820*
    all NRT Light BUFR products with "sensing-start-time" 2015-08-20 will match
  8. W_XX-ESA,SMOS,NRTNN_C_LEMM_20160120032139_20160119231118_20160120004240_o_v100_l2sm.nc
    only the NRT L2 SM NN product with that specific file name will match
  9. W_XX-ESA,SMOS,NRTNN_C_LEMM_*_*_20160220*
    all NRT L2 SM NN products with "sensing-stop-time" 2016-01-20 will match

To display the results in a list of products, tick the "Show search result as product list". To display the results in a short URL list, tick "Show search result as URL list". Note that in case of more than 500 results the view defaults to a "URL list".

2.1.3 Catalogue Search

This function allows searching for data using different parameters and obtaining different results.

2.1.3.1 Search Form

  • Tick the "draw bbox" checkbox and draw a rectangular area to define the bounding box of interest on the world map
  • When a bounding box is drawn on the world map, the "footprint" information gets filled by the coordinates of the displayed rectangular area. This information can be manually adjusted by changing latitude and longitude values. The area will be adjusted as well on the map
  • In order to refine your search, if needed, you can specify additional metadata parameters:
    • "Format": "HTML search response" is intended for detailed browsing, "lists of Download URL" and "Plain text metadata (index)" is intended for bulk download
    • "Items per page": it only refers to the "HTML search response"
    • "nativeProductFormat": it defines the format of the requested products, i.e. "Earth Explorer Format" or "netCDF Format" (for SMOS Level 2 products only)
    • "footprint": it allows indicating the coordinates, as min and max latitude and min and max longitude values, of the area of interest
    • "beginAcquisition": it defines the "sensing-start-time" and the (optional) "sensing-stop-time" of interest. Search goes from 00:00:00 of the start date to 23:59:59 of the stop date, or from 00:00:00 to 23:59:59 of the single date. The date can be entered either manually (as YYYY-MM-DD UTC, time specification is not allowed) or by means of the provided Date Selection widget
    • "orbitDirection": it can be used to select the orbit direction by optionally chosing one of the possible values from the drop-down menu, i.e. "ASCENDING" or "DESCENDING"
    • "productType": it allows selecting the product of interest by chosing one of the possible values from the drop-down menu.
  • Click on the "Search" button.

The search results can be adjusted as described in Section 2.1.3.2.

2.1.3.2 Search Results Format

The Simple Online Catalogue allows selecting among five different response formats, by clicking on the "Format" blank field and selecting one of the options from the drop-down menu:

  • "HTML search response": it returns an HTML page with the list of all requested products (ref. to Section 2.1.3.2.1 for further details)
  • "HTML list of download URLs": it returns an HTML page containing one single download URL per line
  • "Metalink list of download URLs": it returns an XML text file containing the download metalinks for the matched products
  • "Plain text list of download URLs": it returns a plain text page containing one single download URL per line
  • "Plain text metadata (index)": it returns a plain text page containing the full metadata records for the matched products, encoded in a tab separated value format.

It is also possible to set the pagination parameter ("Items per page") to define the maximum number of records per page of the "HTML search response".

2.1.3.2.1 HTML Search Response

The "HTML search response" (see Section 2.1.3.2) contains a list of data items (products or auxiliary files) matching the search parameters, organised into different pages each with as many items as indicated in "Items per page".

The displayed list of results contains the data item file name with a link to download it (it requires authentication), a link to view the associated metadata and, for L1C and L2 products, a link to the relevant browse image.

In order to identify a given item on the map, by clicking on its file name, the item will be highlighted in the map, zoomed to its extent. On the other hand, by clicking on a given product footprint displayed on the map, the corresponding search record in the list will be highlighted.

Please note that, in case of multiple overlapping footprints at the same location, a clickable list of product identifiers will appear in a tooltip, as shown below:

2.2 Searching via an FTP Client

This function allows navigating a dynamic structure via a standard FTP Client (FileZilla, CyberDuck, etc.). Data is organised into different collections the user can navigate by Processing Level, Product Type, Year, Month and Day (of the sensing or validity date).

The L2 Soil Moisture and Ocean Salinity user products available in netCDF format can be found under the [Product_Type]_nc subdirectories.

You can refer to Section 3.2 for additional details on the FTP access.

3. Download Products

There are different mechanisms to access the data and download the products of interest.

In order to download SMOS data, either via HTTP or FTP, you must be a registered SMOS user.

3.1 Download via HTTP

The SMOS data can be accessed and downloaded via HTTP as follows:

  • Login with your own EO-SSO credentials
  • For interactive or individual downloads, select the data of interest by using a search mechanism as from Section 2
  • Download the files by using the product download links.

3.2 Download via FTP

The SMOS data can be accessed and downloaded via FTP as follows:

  • Open an FTP Client (such as FileZilla, CyberDuck, Rumba FTP, etc.) or the command prompt
  • Choose "explicit" FTP over SSL/TLS settings (FTPS)
  • Set to Passive (PASV) connect mode
  • Enter "smos-diss.eo.esa.int" as Host
  • Enter your EO-SSO credentials (username and password)
  • The SMOS directory tree will be displayed.

In case you only see empty directories, please register to SMOS data following the procedure for fast registration available in Section 1.1.

In case of problems with FTPS (FTP encrypted with SSL/TLS), please try with plain FTP or contact smos.dissemination.support@esa.int for assistance.

It is also recommended to use the LFTP command-line client for UNIX and UNIX-like systems.

NOTE: Level 0 and Level 1A products and auxiliary data files are only available to SMOS CalVal users. Should you need to access restricted data, please send a request to the SMOS Mission Manager for authorisation.

3.3 Download in Bulk

It is possibile to download in bulk a set of data contained in a web page. In order to download multiple products all at once, it is suggested to use Firefox with the DownThemAll! extension able to automatically parse a list of URLs inputs:

  • Execute a search as described in either File name search or Catalogue search , respectively by selecting "Show search result as URL list" or one of the "Format" options to get a list of download URLs
  • In the result page, select "Tools" > "DownThemAll! Tools" > "DownThemAll!" from the Firefox Browser's Toolbar
  • In the "DownThemAll!" window use the "Filters" or "Fast Filtering" options in order to select ".zip" or ".nc" (in case of netCDF products)
  • Select "Start" for immediate download, or "Add paused" and activate the download at wish from the "DownThemAll! Manager" window.
Recommended "Preferences/Network" settings for "DownThemAll!":
  • Concurrent downloads: 8
  • Downloads per server: 4

3.4 Advanced Query and Download via Command Line

The catalogue can be queried via a command-line tool like cURL, saving the output as a URL list file. A tool supporting SSO authentication schemas (such e.g. https://sourceforge.net/projects/secp/) must then be used to perform URL downloads to a target directory.

This is an example of the sequence of commands to run on UNIX-like systems:

  1. To fetch a plain text url list from the catalogue and store it in a temp_file.
    In the following example the user is interested in 1 year of L2 SM data in EE Format, acquired on descending passes only, and intersecting a given area of interest (as a rectangular box). The query on the curl command can be adjusted as needed by setting any of the following parameters:
    • query.nativeProductFormat - ("Earth+Explorer+Format" or "netCDF+Format")
    • query.footprint.minlat - (latitude of bottom-left corner of a rectangular box)
    • query.footprint.minlon - (longitude of bottom-left corner of a rectangular box)
    • query.footprint.maxlat - (latitude of upper-right corner of a rectangular box)
    • query.footprint.maxlon - (longitude of upper-right corner of a rectangular box)
    • query.beginAcquisition.start - (start time as "YYYY-MM-DD")
    • query.beginAcquisition.stop - (stop time as "YYYY-MM-DD")
    • query.orbitDirection - ("ASCENDING" or "DESCENDING")
    • query.productType - (any of the Types listed here)

    curl --data "service=SimpleOnlineCatalogue&version=1.0&request=search&format=text%2Fplain&pageCount=10&query.nativeProductFormat=Earth+Explorer+Format&query.footprint.minlat=-12.305&query.footprint.minlon=-68.379&query.footprint.maxlat=-4.570&query.footprint.maxlon=-53.086&query.beginAcquisition.start=2015-04-01&query.beginAcquisition.stop=2016-03-31&query.orbitDirection=DESCENDING&query.productType=MIR_SMUDP2" https://smos-diss.eo.esa.int/socat/SMOS_Open/search > temp_file

  2. To clean-up the temp_file

    dos2unix temp_file

  3. To download all products, without compression, using the SSO credentials (requested the first time and then stored locally for further accesses) one at a time to the local directory.
    Another directory can be specified with -o /some/local/dir

    secp -U -F temp_file

4. Troubleshooting

In case of problems with the authentication process (inconsistent session), you can try to solve the issue by cleaning the cookie cache (Browser memory) or by closing and reopening again your Web Browser.