Data Publication Web Service - Revision history

Mnelson: /* Overview */

2023-02-14T13:50:45Z

‎Overview

Mnelson: /* HTTP Headers */

2022-02-17T15:14:58Z

‎HTTP Headers

Plazarou: /* Overview */

2021-02-04T09:26:20Z

‎Overview

Mnelson: /* Overview */

2020-06-19T10:15:58Z

‎Overview

Mnelson: /* Check Import Progress */

2020-06-04T14:27:42Z

‎Check Import Progress

Mnelson: /* Progress Check Response */

2020-05-22T14:46:05Z

‎Progress Check Response

Mnelson: /* HTTP Query Parameters */

2020-05-22T14:42:55Z

‎HTTP Query Parameters

Mnelson: /* Check Import Progress */

2020-05-22T14:42:26Z

‎Check Import Progress

Mnelson: Created page with "Category:WebService = Overview = Importing data is an asynchronous request to the Fusion Registry, which recieves a token in the response which can be used to track the pr..."

2020-05-18T20:19:38Z

Created page with "Category:WebService = Overview = Importing data is an asynchronous request to the Fusion Registry, which recieves a token in the response which can be used to track the pr..."

New page

[[Category:WebService]]
= Overview =
Importing data is an asynchronous request to the Fusion Registry, which recieves a token in the response which can be used to track the progress of import. This page describes both the submission web service, and the progress report web service.

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/public/data/publish
|-
|style="background-color:#eaecf0"|Access|| Secure Admin or Data Provider
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Accepts|| CSV, XLSX, SDMX-ML, SDMX-EDI (any format for which there is a Data Reader)
|-
|style="background-color:#eaecf0"|Compression || Zip files supported, if loading from URL gzip responses supported
|-
|style="background-color:#eaecf0"|Content-Type|| 1. multipart/form-data (if attaching file) – the attached file must be in field name of uploadFile
2. application/text or application/xml (if submitting data in the body of the POST)
|-
|style="background-color:#eaecf0"|Response Format|| Determined by [[HTTP_Accept_Headers|Accept Header]] - default SDMX 2.1 Structure Specific
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Submisson Accepted
400 - Submission could not be performed (either an unreadable dataset, or unresolvable reference to a required structure)
401 - Unauthorized (if access has been restricted)
500 - Server Error
|}

= HTTP Headers =
The Accept Header is used to define the output format, to transform the data to. The supported accept header values are

In addition, the following optional header parameters can be used to provide further details on the incoming dataset. If these details are not provided, the Fusion Registry will interrogate the dataset header to get the information. If the dataset is a non-SDMX format, or does not contain the required information in the header, then an error response will be returned.

{| class="wikitable"
|-
!|HTTP Header || Purpose || Allowed Values
|-
|style="background-color:#eaecf0"|ProvisionURN ||This can be used to tell the Fusion Registry which Provision Agreement to use for Data Import. This will override the Dataset Header information if it exists. ||
A valid Provision Agreement URN
|-
|style="background-color:#eaecf0"|DatasetAction || Specifies the action to perform when submitting the dataset. If this value is specified it will override any action specified in the dataset.
|| Append, Replace or Delete
|}

== Server Response ==
The server will respond with a JSON object containing a token to track the progress of the import.

{
"Id" : "unique-token-as-a-UUID"
}

== Error Response ==
If the server rejects the submission request, it will reply with an error reponse in JSON format:

{
"Error": [
"Error while attempting to import data",
"Data Publishing is not supported for Provision Agreement 'urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=WB:WDI_ECON_POL_DEBT(1.0)'. Provision Agreement must be configured to using a Registry Managed or Fusion Data Store. Provision Agreement has been configured to use data from the following source: 'Registration (FILE)'"
]}

= Check Import Progress =
The progress of the import is checked by posting the Id response from the data submission to poll the following service

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/public/data/status/find
|-
|style="background-color:#eaecf0"|Access|| Secure Admin or Data Provider
|-
|style="background-color:#eaecf0"|Http Method|| GET
|-
|style="background-color:#eaecf0"|Reponst Format|| application/json
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Import Check Success
401 - Unauthorized (if access has been restricted)
500 - Server Error
|}

== Progress Check Response ==

The response status, any errors, import action, and general details about the Dataset.

{
"Status": "SUCCESS",
"Username": "user_id",
"Start": 1464781420000,
"Completion": 1464781422000,
"Errors": [],
"Keys": 960,
"Obs": 11220,
"Action": "Information",
"DataflowUrn": "urn…",
"ProvisionUrn": "urn…",
"DatasetId": "2401cfbb-1783-4ee2-ad4f-d900332e0034"
}

Status returned can be one of the following:
* '''PENDING''' – the data is undergoing initial validation
* '''QUEUED''' – the data has been validated and is awaiting database imoport
* '''PROCESSING''' – the data is being loaded into the database
* '''PROCESSED''' – the data has been imported but not yet indexed
* '''INDEXING''' – the fusion registry is reindexing the data
* '''SUCCESS''' – the data import has completed with no errors
* '''ERROR''' – the import resulted in error

If the status is PROCESSING then the response Object will also contain PercentageComplete as a number, for example:
{
"Status": "PROCESSING",
"PercentageComplete": 72, ...
}

The DatasetId is the unique identifier for the dataset, and will match the 'id' query parameter.

'''NOTE''': Start and End Dates is the measured in number of milliseconds since January 1st 1970.

== Publish Error ==
If the data publication resulted in error the Errors feild is populated with the error message(s). The data publish is set to fail on first error, so the array of errors are only relevent if the error message contains a number of levels of detail. For a more comprehensive error report, use the [[Data_Validation_Web_Service|Data Validation web service]], or the [[Asynchronous_Data_Validation_and_Transformation|Asynchronous equivalent]].

{
Status: "ERROR",
Username: "root",
Start: 1589832922000,
Completion: 1589832922000,
Errors: [ "C=Constraint,P=SERIES,DS=0,K=A:BN_KLT_DINV_CD:AUS,Comp=REF_AREA:AUS,M=Disallowed Dimension Value: REF_AREA=AUS"],
Action: "Information",
DataflowUrn: "urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=WB:WDI_ECON_POL_DEBT(1.0)",
ProvisionUrn: "urn:sdmx:org.sdmx.infomodel.registry.ProvisionAgreement=WB:WB_ECON_DP1(1.0)",
DatasetId: "1af3cb4f-979c-4956-b595-a107f6926ff7"
}

@@ Line 5: / Line 5: @@
 {| class="wikitable"
 |-
-|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/public/data/publish</b>
+|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/secure/data/publish</b>
 |-
 |style="background-color:#eaecf0"|<b>Access</b>|| <span style='color:red'><b>Secure</b></span> Admin or Data Provider

@@ Line 39: / Line 39: @@
 |-
 |style="background-color:#eaecf0"|<b>DatasetAction</b> || <p>Specifies the action to perform when submitting the dataset. If this value is specified it will override any action specified in the dataset.
-|| Append, Replace or Delete
+|| Append, Replace, Delete, or FullReplace
 |}
@@ Line 58: / Line 58: @@
     "Data Publishing is not supported for Provision Agreement 'urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=WB:WDI_ECON_POL_DEBT(1.0)'.  Provision Agreement must be configured to using a Registry Managed or Fusion Data Store.  Provision Agreement has been configured to use data from the following source: 'Registration (FILE)'"
   ]}
 = Check Import Progress =

@@ Line 18: / Line 18: @@
 <p>2.	application/text or application/xml (if submitting data in the body of the POST)</p>
 |-
-|style="background-color:#eaecf0"|<b>Response Format</b>|| Determined by [[HTTP_Accept_Headers|Accept Header]] - default SDMX 2.1 Structure Specific
+|style="background-color:#eaecf0"|<b>Response Format</b>|| Determined by [[Data Formats|Accept Header]] - default SDMX 2.1 Structure Specific
 |-
 |style="background-color:#eaecf0"|<b>Response Statuses</b>|| <p><b>200</b> - Submisson Accepted</p>

@@ Line 65: / Line 65: @@
 {| class="wikitable"
 |-
-|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/public/data/status/find</b>
+|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/secure/data/status/find</b>
 |-
 |style="background-color:#eaecf0"|<b>Access</b>|| <span style='color:red'><b>Secure</b></span> Admin or Data Provider

@@ Line 79: / Line 79: @@
 = HTTP Query Parameters =
 {| class="wikitable"
 |-
-!| Request Parameter || Purpose
+!| Request Parameter || Required || Purpose
 |-
-|style="background-color:#eaecf0"|<b>id</b>|| This is the token that was passed back from the Fusion registry server on the publication request
+|style="background-color:#eaecf0"|<b>id</b>|| Yes || This is the token that was passed back from the Fusion registry server on the publication request
 |-
 |}

@@ Line 1: / Line 1: @@
 [[Category:WebService]]
 = Overview =
-Importing data is an asynchronous request to the Fusion Registry, which recieves a token in the response which can be used to track the progress of import.  This page describes both the submission web service, and the progress report web service.
+Importing data is an asynchronous request to the Fusion Registry, which receives a token in the response which can be used to track the progress of import.  This page describes both the submission web service, and the progress report web service.
 {| class="wikitable"

@@ Line 116: / Line 116: @@
 * '''ERROR''' – the import resulted in error
-If the status is PROCESSING then the response Object will also contain PercentageComplete as a number, for example:
+If the status is PROCESSING then the response Object will also contain PercentComplete as a number, for example:
+  {
     "Status": "PROCESSING",
-    "PercentageComplete": 72,    ...
+    "PercentComplete": 72,    ...
+  }

@@ Line 77: / Line 77: @@
 <p><b>500</b> - Server Error</p>
 |}
 == Progress Check Response ==