Difference between revisions of "Mapping Assistant" English (en)

From SDMX and Metadata Standards

Jump to: navigation, search
m
 
(2 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The Mapping Assistant is meant to facilitate the mapping between the structural metadata provided by an SDMX-ML Data Structure Definition (DSD) and those that reside in a database of a dissemination environment.
+
The Mapping Assistant is meant to facilitate the mapping between the structural metadata provided by an SDMX-ML Data Structure Definition (DSD) and those that reside in a database of a dissemination environment.  
  
Mapping Assistant maintains a Mapping Store for keeping the mappings between the SDMX and the local data storage scheme. In the SDMX Reference Infrastructure, the Mapping Assistant provides mapping information to the Data Retriever. The Data Retriever module connects to the Mapping Store database and accesses the appropriate mappings to translate the SDMX-ML queries to SQL for the dissemination database.
+
Mapping Assistant maintains a Mapping Store for keeping the mappings between the SDMX and the local data storage scheme. In the SDMX Reference Infrastructure, the Mapping Assistant provides mapping information to the Data Retriever. The Data Retriever module connects to the Mapping Store database and accesses the appropriate mappings to translate the SDMX-ML queries to SQL for the dissemination database.  
  
 +
<br>The mapping process with the Mapping Assistant tool can be described in four steps:
  
The mapping process with the Mapping Assistant tool can be described in four steps:
+
Step 1 – loading of the SDMX structures - CategoryScheme, DataFlow, Data Structure Definition – from SDMX-ML structure files
  
Step 1 – loading of the  SDMX structures  - CategoryScheme, DataFlow, Data Structure Definition – from SDMX-ML structure files
+
Step 2 – loading of the local non-SDMX database schema and the creation of the Dataset  
 
+
Step 2 – loading of the local non-SDMX database schema and the creation of the Dataset
+
  
 
Step 3 – mapping of local concepts to SDMX Concepts of the Data Structure Definition  
 
Step 3 – mapping of local concepts to SDMX Concepts of the Data Structure Definition  
  
Step 4 – trancoding of local codes to SDMX Codes of the Codelists referenced in the Data Structure Definition
+
Step 4 – transcoding of local codes to SDMX Codes of the Codelists referenced in the Data Structure Definition  
 
+
 
+
The objective of the mapping process is to define the mappings for each of the mandatory SDMX Components to local data columns, residing in a local storage (Dissemination Database).
+
 
+
The mapping should be complete and well defined, in a machine readable way. This allows automated data retrieval from the Dissemination Database (DDB) by an SDMX-ML Query and the transformation of the dataset to an SDMX-ML format.
+
 
+
 
+
  
The Mapping Assistant (MA) employs an intermediate layer in order to simplify the mapping and hide complexity of the Local Storage.
+
<br>The objective of the mapping process is to define the mappings for each of the mandatory SDMX Components to local data columns, residing in a local storage (Dissemination Database).  
Therefore the MA defines a Dataset artefact that encapsulates the local storage. The Dataset presents the required data as they were in a single table.
+
  
These table columns have to be mapped with the SDMX Components.
+
The mapping should be complete and well defined, in a machine readable way. This allows automated data retrieval from the Dissemination Database (DDB) by an SDMX-ML Query and the transformation of the dataset to an SDMX-ML format.  
  
Therefore, defining a Dataset includes defining the columns of this virtual table and the SQL query that returns them.
+
<br>
Dataset concept resembles the concept of the SQL VIEW where data coming from a complex are presented in single table.
+
A complete definition of a Dataset requires also the Connection to the DDB that the query is executed upon.
+
  
Having defined a Dataset the mapping to the SDMX constructs is simplified.
+
The Mapping Assistant (MA) employs an intermediate layer in order to simplify the mapping and hide complexity of the Local Storage. Therefore the MA defines a Dataset artefact that encapsulates the local storage. The Dataset presents the required data as they were in a single table.  
  
Mapping Assistant utilizes the Mapping Set artefact that groups the mappings for a specific data retrieval case.
+
These table columns have to be mapped with the SDMX Components.  
Since data exchange in SDMX is based on Dataflow, the Mapping Set is defined using a specific Dataflow and a Dataset.
+
The mappings are performed between the data fields (Concepts) of the Data Structure Definition related with the particular Dataflow & the Dataset’s columns.
+
  
Also matching between the codes defined (in Codelists) for the DSD Concepts and the codes of the Dataset’s columns (Local codes) has to be done. This is called transcoding.
+
Therefore, defining a Dataset includes defining the columns of this virtual table and the SQL query that returns them. Dataset concept resembles the concept of the SQL VIEW where data coming from a complex are presented in single table. A complete definition of a Dataset requires also the Connection to the DDB that the query is executed upon.  
  
The tool is licenced under the terms of the [[Media:EUPL_v1.1.txt|European Union Public Licence V.1.1]]
+
Having defined a Dataset the mapping to the SDMX constructs is simplified.  
  
 +
Mapping Assistant utilizes the Mapping Set artefact that groups the mappings for a specific data retrieval case. Since data exchange in SDMX is based on Dataflow, the Mapping Set is defined using a specific Dataflow and a Dataset. The mappings are performed between the data fields (Concepts) of the Data Structure Definition related with the particular Dataflow &amp; the Dataset’s columns.
  
Complete Mapping Assistant package: [http://circa.europa.eu/Public/irc/dsis/stne/library?l=/x-dis/tools/reference_architecture/sdmx_mapping_assistant/2011/mappingassistant_4/mappingassistant_2011082/_EN_1.0_&a=d MappingAssistant_.NET_v2.7.1_2011.08.26 (ZIP)], containing an application, source code and documentation such as a User Manual, Installation Guide, Analysis and Design as well as Testing Documentation, stored in the CIRCA repository.
+
Also matching between the codes defined (in Codelists) for the DSD Concepts and the codes of the Dataset’s columns (Local codes) has to be done. This is called transcoding.  
  
 +
The tool is licenced under the terms of the [[Media:EUPL_v1.1.txt|European Union Public Licence V.1.1]]
  
All versions of the tool can be downloaded from [http://circa.europa.eu/Public/irc/dsis/stne/library?l=/x-dis/tools/reference_architecture/sdmx_mapping_assistant&vm=detailed&sb=Title CIRCA repository].
+
All versions of the tool can be downloaded from [https://circabc.europa.eu/w/browse/69d9fb16-939f-4f85-94d9-60962b5d25d8 CircaBC repository].

Latest revision as of 17:36, 1 March 2013

The Mapping Assistant is meant to facilitate the mapping between the structural metadata provided by an SDMX-ML Data Structure Definition (DSD) and those that reside in a database of a dissemination environment.

Mapping Assistant maintains a Mapping Store for keeping the mappings between the SDMX and the local data storage scheme. In the SDMX Reference Infrastructure, the Mapping Assistant provides mapping information to the Data Retriever. The Data Retriever module connects to the Mapping Store database and accesses the appropriate mappings to translate the SDMX-ML queries to SQL for the dissemination database.


The mapping process with the Mapping Assistant tool can be described in four steps:

Step 1 – loading of the SDMX structures - CategoryScheme, DataFlow, Data Structure Definition – from SDMX-ML structure files

Step 2 – loading of the local non-SDMX database schema and the creation of the Dataset

Step 3 – mapping of local concepts to SDMX Concepts of the Data Structure Definition

Step 4 – transcoding of local codes to SDMX Codes of the Codelists referenced in the Data Structure Definition


The objective of the mapping process is to define the mappings for each of the mandatory SDMX Components to local data columns, residing in a local storage (Dissemination Database).

The mapping should be complete and well defined, in a machine readable way. This allows automated data retrieval from the Dissemination Database (DDB) by an SDMX-ML Query and the transformation of the dataset to an SDMX-ML format.


The Mapping Assistant (MA) employs an intermediate layer in order to simplify the mapping and hide complexity of the Local Storage. Therefore the MA defines a Dataset artefact that encapsulates the local storage. The Dataset presents the required data as they were in a single table.

These table columns have to be mapped with the SDMX Components.

Therefore, defining a Dataset includes defining the columns of this virtual table and the SQL query that returns them. Dataset concept resembles the concept of the SQL VIEW where data coming from a complex are presented in single table. A complete definition of a Dataset requires also the Connection to the DDB that the query is executed upon.

Having defined a Dataset the mapping to the SDMX constructs is simplified.

Mapping Assistant utilizes the Mapping Set artefact that groups the mappings for a specific data retrieval case. Since data exchange in SDMX is based on Dataflow, the Mapping Set is defined using a specific Dataflow and a Dataset. The mappings are performed between the data fields (Concepts) of the Data Structure Definition related with the particular Dataflow & the Dataset’s columns.

Also matching between the codes defined (in Codelists) for the DSD Concepts and the codes of the Dataset’s columns (Local codes) has to be done. This is called transcoding.

The tool is licenced under the terms of the European Union Public Licence V.1.1

All versions of the tool can be downloaded from CircaBC repository.

Personal tools