Asynchronous Data Validation and Transformation

From Fusion Registry Wiki
Revision as of 06:37, 27 April 2020 by Mnelson (talk | contribs)
Jump to navigation Jump to search

Overview

Entry Point /ws/public/data/load
Access Public (default). Configurable to Private
Http Method POST
Accepts CSV, XLSX, SDMX-ML, SDMX-EDI (any format for which there is a Data Reader)
Compression Zip files supported, if loading from URL gzip responses supported
Content-Type

1. multipart/form-data (if attaching file) – the attached file must be in field name of uploadFile

2. application/text or application/xml (if submitting data in the body of the POST)

Response Format JSON
Response Statuses

200 - Data file recieved

400 - Trasformation could not be performed (either an unreadable dataset, or unresolvable reference to a required structure)

401 - Unauthorized (if access has been restricted)

500 - Server Error


Response

The response to a data load is a token, which can be used in subsequent calls to track the data load and validation process and, once validation is complete, the token can be used to perform actions such as a pulish, obtain validation report, export in a different format, or export with mapping. 

{
  "Success" : true,
   "uid"    : "unique token"
}


Load Status

Entry Point /ws/public/data/loadStatus
Access Public (default). Configurable to Private
Http Method GET
Response Format JSON

Load Status Response

The response is a validation report, to either indicate success or error. 777

The report Status indicates how far into the validation process the server has reached. The following table shows the various stages:

Status Description
Initialising Initial status
Analysing The dataset is being analysed for series and obs count, and which dsd's it references
Validating The dataset is being validated
Complete The dataset validation process has finished, there may/man not be errors
Consolidating The dataset is being consolidated, duplicate series and observations are being merged into one final dataset
IncorrectDSD The dataset references a DSD that can not be used to validate the data
InvalidRef The dataset references a DSD/Dataflow/Provision that does not exist in the Registry
MissingDSD The dataset does not not reference any structure so the system can not read the dataset
Error The dataset can not be read at all
Retrieved from "https://wiki.sdmxcloud.org/index.php?title=Asynchronous_Data_Validation_and_Transformation&oldid=1536"