SDMX Implementation for Data Exchange English (en)

From SDMX and Metadata Standards

(Redirected from SDMX Implementation)
Jump to: navigation, search

Introduction

This page aims at explaining how SDMX is implemented in statistical domains, as well as giving an overview of the on-going implementation projects running within Eurostat and the European Statistical System (ESS).

For more information on on-going implementation project in the ESS, please see the detail page.

 

Implementation phases

The implementation of SDMX in a statistical domain is made progressively and is generally divided into four main steps.
The duration of each of these phases varies according to the size of the project, the involvement of other international organisations and the expected date of use of SDMX into production.
 
 Sdmx implementation.jpg

Preparation

The initial phase is called "preparation" and starts with first meetings organised between the concerned Eurostat Production Unit(s) and the Unit responsible for implementing SDMX.
During these first meetings, the objective is to have an overview of the current data collection/treatment/dissemination system in place and to analyse if and how SDMX could be used to facilitate the processes.
Therefore during this step, sample files are analysed and depicted in order to understand the structure of the data collected as well as the code lists which are used for exchanging those data.
In case other organisations are involved, it is also important to see if the data are collected on the same manner or differently and whether the data collection is regulated by some legal acts.
In this phase, the way reference metadata are collected/treated is also analysed especially in case the Production Unit is collecting national information from the National Statistical Institutes and/or exchanging metadata with other national/international organisations.
This phase ends with a set of decisions on the future work/tasks to be undertaken: continuation or abandon of the project; which datasets to be concerned, etc.
 

Compliance

Once the decision of further continuing the implementation of SDMX has been taken, the second phase concerns the "Compliance" to the SDMX standards as defined in the SDMX Information Model, the SDMX technical standards and the content-oriented guidelines.
During this phase, the first new structures for data exchange are drafted and the code lists used are more deeply analysed and possibly re-adapted in order to comply with the SDMX standards.
This phase ends with a proposed Data Structure Definition (DSD).
Regarding the analysis of reference metadata, the compliance phase consists in an alignment to the existing standards, i.e to the ESMS (Euro SDMX Metadata Structure) for the collection of reference Metadata and to the ESQRS (ESS Standard for Quality Reports Structure) for quality related metadata.
 

Implementation

The implementation phase encompasses the practical creation of the developed SDMX elements to be delivered to the production unit.
The created package of SDMX objects (DSD, code lists, etc.) is then generally used in a pilot project where several NSIs are involved. This pilot project consists in installation processes in Member States as well as a series of tests. These actions might result in changes in the structure of the DSDs or to certain re-adaptations which are also part of this Implementation Phase.
A series of test of validation of the generated SDMX-ML sample files created is also performed.
The possible re-adaptation of the Eurostat Production systems is also made during this third phase and could sometimes imply significant changes.
Concerning the implementation in terms of reference metadata, this phase consists in pilot projects where Member States will make use of the European standard structures (ESMS, ESQRS) and the dedicated IT application (National Reference Metadata Editor) to generate their first test files.
 

Production  

This phase marks the concrete application of SDMX for data and metadata exchange.

All countries are using a final version of the DSD and the transmission chain is fully based on SDMX.From that point onwards, normal maintenance tasks are required to keep code lists and validation mechanisms up to date.

National metadata files are created and sent by Member States to Eurostat via the NRME and those files are checked and validated by the Eurostat domain managers. The produced files may also be disseminated on the Eurostat website.

 

 

Contact

In Eurostat, the unique contact point for the implementation projects of SDMX is the following email address: mailto:estat-support-sdmx@ec.europa.eu
In National Statistical Institutes, the main contacts are the local transmission coordinators. These persons are generally familiar with SDMX and the use of the Eurostat Single Entry Point (eDamis) for data transmission.
Personal tools