Vmurrell at 13:44, 16 August 2023

2023-08-16T13:44:19Z

Mnelson: /* Overview */

2021-02-11T09:28:24Z

‎Overview

Mnelson: /* Environment Security */

2021-02-11T09:27:00Z

‎Environment Security

Mnelson: Created page with "Category:WebService = Overview = Fusion Edge Server is a high speed secure server for data and metadata dissemination. Publication of data from the Fusion Registry to th..."

2021-02-11T09:26:43Z

Created page with "Category:WebService = Overview = Fusion Edge Server is a high speed secure server for data and metadata dissemination. Publication of data from the Fusion Registry to th..."

New page

[[Category:WebService]]

= Overview =
Fusion Edge Server is a high speed secure server for data and metadata dissemination. Publication of data from the Fusion Registry to the Fusion Edge Server is achieved by a number of steps, which include:

# Defining a Edge Environment - this defines information such as what Dataflows to include data for in the published content.
# Generating the Edge content - this is an asynchronous service which runs in the background after it receives instruction to launch
#

= Manage Edge Environments =
An Edge Environment is a container for a description of what gets put into the edge content. For example if there are 2 Edge Server clusters, one for public data and one for internal use within the organisation, the there would be 2 Edge Environment definitions in the Fusion Registry - where the Dataflows in each environment differ based on what data is to be made public and which data is to be disseminated internally. The Identity and description of the environment is just for local use, there is no connection between this information and what is published in the edge content.

== Add Environment ==
To add a new Environment, the Id and description need to be provided - after an environment is created, it is possible to use other web services to define what is published to the environment.

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/addEnvironment
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId",
"Description" : "Environment Description"
}

== Edit Environment ==
This web service allows for the Id and/or description of the environment to be updated

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/editEnvironment
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId_New",
"OldId" : "EnvironmentId_Old"
"Description" : "Environment Description Updated"
}

== Environment Dataflows ==
This web service is used to supply the list of Dataflows to be included in the publication for the environment, the list overwrites any list that was previously provided

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/setIncludedDataflows
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId",
"IncludedDataflows" : ["urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ECB:EXR(1.0)", "urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ECB:TRD(1.0)"]
}

== Environment Metadata ==
This web service is used to define if reference metadata is to be included in the content or not

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/setIncludedMetadata
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId",
"IncludeMetadata" : true
}

== Environment Provision Agreements ==
This web service is used to define if Provision Agreements are to be included in the content, if they are included it will be possible to query a dataset by both Dataflow and Data Provider, if they are not included then all data for a Dataflow will be stored at the level of the dataflow - the data provider information will be lost.

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/setIncludedProvisions
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId",
"IncludeProvisions" : false
}

== Environment Security ==
In generating the Edge Server content, the Fusion Registry will treat the Edge Server Environment as a user, by default the user will be anonymous and will be grated the same permissions on structures and data as a user who has not logged in. Any secure content will not be included in the published content. It is possible to link the Fusion Edge Server to a specific Organisation (SDMX Agency, Data Provider, Data Consumer) in order to grant the edge server access to secure content. The Fusion Registry security rules should then be used to grant permissions to the Organisation that the Fusion Edge Environment is linked to.

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/setOrganisation
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId",
"OrganisationUrn" : " urn:sdmx:org.sdmx.infomodel.base.DataProvider=ORG1:DATA_CONSUMERS(1.0).EDGE_PUBLIC"
}

= Build Publication Content =
These web services are used to request the edge content is built from the environment description, and exported. There are three steps:
# Build Request
# Poll Build Status
# Export node.zip file, used to publish to the Fusion Edge Server

== Build Request ==
The build request is an asynchronous request, which starts off the build process.

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/buildEdgeServerPublication
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| POST
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

POST Syntax
{
"Id" : "EnvironmentId",
"PublicationType" : "FullReplace" | "Replace"
"EmbargoDate" : 2345679987 //Unix time millliseconds
}

A publication type of FullReplace will generate a node.zip file which will replace all Fusion Edge server content, a Replace action will produce a delta, which only includes data that have changed since the last publication time. A delta publication can create new series and observations, or replace existing ones that have been modified.

The Embargo Date is a Unix timestamp (number of milliseconds since 1970). The content of the node.zip file does not change in any way (there is no encryption) the timestamp is simply an indicator for the Fusion Edge Server not to make the data live until the server time has passed the data indicated by the given timestamp. The Fusion Edge server does this by importing the node.zip file and building the new environment in memory, but only links the environment up when the time has passed. This allows for the data to be released exactly on the time specified.

== Poll Build Status ==
The build request is an asynchronous request, which starts off the build process.

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/environment/status/{ENVIRONMENT_ID}
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| GET
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| JSON
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

Response Syntax
{
"id" : 1,
"processorStartDate" : 123456,
"environmentId" : "PUBLIC_EDGE",
"status" : "PROCESSING" | "WRITING_STRUCTURES" | "WRITING_DATA" | "SUCCESSFUL" | "ERRORED" | "CANCELLED",
"auditTxId" : "uuid"
}

== Export Edge Content ==
The request pulls down the latest node.zip file that was successfully generated for the given environment

{| class="wikitable"
|-
|style="background-color:#eaecf0"|Entry Point|| /ws/secure/edge/export/{ENVIRONMENT_ID}
|-
|style="background-color:#eaecf0"|Access|| Secure (Admin)
|-
|style="background-color:#eaecf0"|Http Method|| GET
|-
|style="background-color:#eaecf0"|Compression || gzip supported
|-
|style="background-color:#eaecf0"|Response Format|| octet-stream
|-
|style="background-color:#eaecf0"|Response Statuses|| 200 - Ok
400 - Bad URL Syntax
401 - Unauthorised (authenticated no permission to perform action)
403 - Forbidden (not authenticated)
500 - Server Error
|}

← Older revision		Revision as of 13:44, 16 August 2023
Line 1:		Line 1:
	[[Category:WebService]]		[[Category:WebService]]
		+	[[Category:Fusion Edge Server]]

	= Overview =		= Overview =

@@ Line 6: / Line 6: @@
 # Defining a Edge Environment - this defines information such as what Dataflows to include data for in the published content.
 # Generating the Edge content - this is an asynchronous service which runs in the background after it receives instruction to launch
+# Exporting Edge Content - this exports the edge content generated in step 2
 = Manage Edge Environments =

@@ Line 179: / Line 179: @@
+  {
     "Id" : "EnvironmentId",
-    "OrganisationUrn" : " 	urn:sdmx:org.sdmx.infomodel.base.DataProvider=ORG1:DATA_CONSUMERS(1.0).EDGE_PUBLIC"
+    "OrganisationUrn" : "urn:sdmx:org.sdmx.infomodel.base.DataProvider=ORG1:DATA_CONSUMERS(1.0).EDGE_PUBLIC"
+  }
 = Build Publication Content =

Edge Server Data Preperation Web Service - Revision history

Vmurrell at 13:44, 16 August 2023

Mnelson: /* Overview */

Mnelson: /* Environment Security */

Mnelson: Created page with "Category:WebService = Overview = Fusion Edge Server is a high speed secure server for data and metadata dissemination. Publication of data from the Fusion Registry to th..."