SDMX-CSV - Revision history

Plazarou: /* Version 1.0 and Version 2.0 */

2023-10-10T10:11:19Z

‎Version 1.0 and Version 2.0

Plazarou: /* Version 1.0 and Version 2.0 */

2023-10-10T10:07:47Z

‎Version 1.0 and Version 2.0

Plazarou: /* Overview */

2023-10-10T10:06:40Z

‎Overview

Plazarou: /* Example */

2023-10-10T10:03:52Z

‎Example

Plazarou: /* Overview */

2023-10-10T10:01:27Z

‎Overview

Plazarou: /* Overview */

2023-10-10T10:00:34Z

‎Overview

Mnelson at 08:48, 29 September 2022

2022-09-29T08:48:31Z

Mnelson: /* Formatting Using Query Parameters */

2020-12-02T15:02:59Z

‎Formatting Using Query Parameters

Plazarou: /* Example */

2020-10-20T08:23:33Z

‎Example

Mnelson at 09:06, 10 August 2020

2020-08-10T09:06:53Z

← Older revision		Revision as of 10:11, 10 October 2023
Line 16:		Line 16:

	Version 2.0 has a different header where the first 2 "columns" are "STRUCTURE" and "STRUCTURE_ID". The Structure column can refer to a DSD, Dataflow or Provision Agreement and the STRUCTURE_ID is the [[URN]] postfix of the metadata structure.		Version 2.0 has a different header where the first 2 "columns" are "STRUCTURE" and "STRUCTURE_ID". The Structure column can refer to a DSD, Dataflow or Provision Agreement and the STRUCTURE_ID is the [[URN]] postfix of the metadata structure.
		+
		+	This can be demonstrated by example. The following are queries for the same data but with different format versions specified:
		+
		+	* SDMX-CSV V2: https://demo11.metadatatechnology.com/FusionRegistry/sdmx/v2/data/dataflow/WB/GCI/1.0/?c%5BREF_AREA%5D=GHA&c%5BINDICATOR%5D=GCI&format=sdmx-csv-2.0.0
		+	* SDMX-CSV V1: https://demo11.metadatatechnology.com/FusionRegistry/sdmx/v2/data/dataflow/WB/GCI/1.0/?c%5BREF_AREA%5D=GHA&c%5BINDICATOR%5D=GCI&format=sdmx-csv-1.0.0

	=Formatting Using Query Parameters=		=Formatting Using Query Parameters=

@@ Line 11: / Line 11: @@
 == Version 1.0 and Version 2.0 ==
-Version 1.0 only supported Datasets against a Dataflow. The first column must be called '''Dataflow''' and each row of the dataset must contain the [[URN]] postfix of the Dataflow that dataset is for.
+There are a number of differences between version 1.0 and version 2.0 of the standard but the most apparent is the identifier against the target Data Structure.
-Version 2.0 has a different header where the first 2 "columns" are "STRUCTURE" and "STRUCTURE_ID".  The Structure column can refer to a DSD, Dataflow or Provision Agreement and the STRUCTURE_ID is the short URN of the metadata structure.
+Version 1.0 only supports Datasets against a Dataflow. The first column must be called '''Dataflow''' and each row of the dataset must contain the [[URN]] postfix of the Dataflow that dataset is for.
 =Formatting Using Query Parameters=

@@ Line 8: / Line 8: @@
 The format is comma-separated format where each row describes a single Observation value.  The columns describe each component of the Observation value, reflecting the Components in the [[Data_Structure_Definition#Data_Structure_Components|Data Structure Definition]].
-The dataset can contain both Code labels, in addition to the Code IDs.  The order of the columns is not important, however each column header will contain the ID of the corresponding Component.  The first column must be called '''Dataflow''' where each value contains the [[URN]] postfix of the Dataflow that dataset is for.
+The dataset can contain both Code labels, in addition to the Code IDs.  The order of the columns is not important, however each column header will contain the ID of the corresponding Component.
 =Formatting Using Query Parameters=

@@ Line 48: / Line 48: @@
 <i>https://demo.metadatatechnology.com/FusionRegistry/ws/public/sdmxapi/rest/data/WB,GCI,1.0/GHA.GCI..?format=sdmx-csv&labels=both</i>
-An example dataset with IDs only, spaces have been added to this example to assist readability.
+An example dataset with IDs only, spaces have been added to this example to assist readability. Note this is version 2.0 of the SDMX CSV format:
 <pre>
-DATAFLOW,    REF_AREA, INDICATOR, SUB_INDICATOR, FREQ, TIME_PERIOD, OBS_VALUE
+STRUCTURE, STRUCTURE_ID, REF_AREA, INDICATOR, SUB_INDICATOR, FREQ, TIME_PERIOD, OBS_VALUE
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2008,        102
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2008,        102
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2009,        114
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2009,        114
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2010,        114
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2010,        114
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2011,        114
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2011,        114
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2012,        103
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2012,        103
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2013,        114
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2013,        114
-WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2014,        111
+dataflow,  WB:GCI(1.0),  GHA,      GCI,       RANK,          A,    2014,        111
 </pre>
 The same dataset in SDMX-CSV with labels included.
 <pre>
-DATAFLOW,                                  REF_AREA:Reference Area, INDICATOR:Indicator,               SUB_INDICATOR:Sub Indicator, FREQ:Frequency, TIME_PERIOD:Time period, OBS_VALUE:Observation
+STRUCTURE, STRUCTURE_ID,                              REF_AREA:Reference Area, INDICATOR:Indicator,               SUB_INDICATOR:Sub Indicator, FREQ:Frequency, TIME_PERIOD:Time period, OBS_VALUE:Observation
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2008,                    102
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2008,                    102
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2009,                    114
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2009,                    114
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2010,                    114
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2010,                    114
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2011,                    114
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2011,                    114
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2012,                    103
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2012,                    103
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2013,                    114
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2013,                    114
-WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2014,                    111
+dataflow,  WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2014,                    111
 </pre>

@@ Line 2: / Line 2: @@
 [[Category:SdmxDataFormat]]
 =Overview=
-The SDMX-CSV Data format is an official SDMX format described on the [https://github.com/sdmx-twg/sdmx-csv/tree/master/data-message SDMX GitHub Repository].
+The SDMX-CSV Data format is an official SDMX format described on the [https://github.com/sdmx-twg/sdmx-csv/tree/master/data-message SDMX GitHub Repository]. There are currently two major versions and if in doubt it is recommended to use the latest version, version 2.
 It can be used as both an import and export format for the Fusion Registry, and an export format for the Fusion Edge Server and Fusion Data Browser.

← Older revision		Revision as of 08:48, 29 September 2022
Line 1:		Line 1:
		+	see [https://wiki.sdmxcloud.org/Category:SdmxDataFormat Data Formats]
	[[Category:SdmxDataFormat]]		[[Category:SdmxDataFormat]]
	=Overview=		=Overview=

@@ Line 15: / Line 15: @@
 * timeFormat = normalized
 * bom = include | exclude  (Include or Exclude the [https://en.wikipedia.org/wiki/Byte_order_mark '''B'''yte '''O'''rder '''M'''ark] (BOM).<br/> The BOM helps Excel interpret non Latin characters when opening a CSV file)
 The output type "all-lang" will output both id and name and will add an element to the start of each row which will be the language. For each series and each language a row will be output where the name in the row is for the specified language. If there is no name value for that language, the name is simply not output. See below for an example output/