Difference between revisions of "Edge Server - Publish Content"

From Fusion Registry Wiki
Jump to navigation Jump to search
(→‎Part 1 - Source Data and Metadata Files)
Line 18: Line 18:
  
 
Where agency id, dataflow id, and dataflow version are specific to the Dataflows that the data are for.  The content can be in any SDMX format, each folder can contain multiple files, the compiler will merge the information where required.
 
Where agency id, dataflow id, and dataflow version are specific to the Dataflows that the data are for.  The content can be in any SDMX format, each folder can contain multiple files, the compiler will merge the information where required.
−
 
−
'''Note''': The Fusion Edge Compiler can build this local file system automatically from content pulled from compliant SDMX web services such as those provided by Fusion Registry. Information is provided later about how this is achieved.
 
  
 
An example folder/file content is given below:
 
An example folder/file content is given below:
Line 41: Line 39:
 
  |--metadataset2.zip
 
  |--metadataset2.zip
  
−
The files in the file system must be in SDMX format, and may be individually zipped.  Each folder may contain multiple files.  The compilation process will combine all the files in each folder to create a consolidated output.  For example a dataflow folder may contain multiple dataset instances with different series or time periods, the output will be a single compiled dataset instance built from all the dataset files.
 
  
 +
It is possible to manually build this file system, however the Fusion Edge Compiler can be used to obtain information directly from an SDMX REST API to automate the process of building the File system.
  
 
== Part 2 - Compiled Environment ==
 
== Part 2 - Compiled Environment ==
 +
The Fusion Edge Compiler compiles the source file system into an ([[Edge_Server_Environment|Environment]]).
 +
 +
 +
== Part 3 - Publish ==
 +
There are 2 main modes of publication, static mode and dynamic mode. 
 +
 +
=== ​Static Mode ===
 +
Static mode is termed as such because the Fusion Edge Server loads the Environment on application startup, and it never changes the Environment after this.  The contents of the Fusion Edge Server are static, and can only be changed by restarting the web application server.
 +
 +
To use static mode, the contents of the ([[Edge_Server_Environment|Environment]]) folder is zipped to producing a single zip file.  The zip file is placed on the file system in the home directory of the Fusion Edge Server.  The edgeserver.properties file is configured to define the name of the Environment zip file that the Fusion Edge Server should load on application startup.
 +
 +
The Fusion Edge Server will read the zip file on application startup and load the Environment into memory.  The only way a new Environment can be deployed is by restarting the web application server. 
 +
 +
=== Dynamic Mode ===
 +
Static mode is termed as such because the Fusion Edge Server loads the Environment on application startup, and then it polls the Environment's

Revision as of 01:32, 26 August 2022

Overview

There are three parts to publishing content

  1. Source Data and Metadata Files
  2. Compiled Data and Metadata Files (the Environment)
  3. Published (Environment)

Part 1 - Source Data and Metadata Files

Content is published to the Fusion Edge Server by compiling datasets, structure files, and reference metadata files that are present in a local file system. The compilation process is run using the Fusion Edge Compiler. The Fusion Edge Compiler is given the root folder as an argument and it expects to find the following folder structure under the root folder: 

|- data
|-- [agency id]
|---- [dataflow id]
|------ [dataflow version]  (data files are placed in this folder)
|- structure (structure files are placed in this folder)
|- metadata (metadata files are placed in this folder)

Where agency id, dataflow id, and dataflow version are specific to the Dataflows that the data are for. The content can be in any SDMX format, each folder can contain multiple files, the compiler will merge the information where required.

An example folder/file content is given below: 

|- data
|-- WB
|---- POVERTY
|------ 1.0 
|-------- PovertyData.zip
|-------- PovertyUpdate.xml
|---- EDUCATION
|------ 1.0 
|-------- EduData_1990_2010.json
|-------- EduData2010_2020.xml
|- structure 
|-- corestructures.zip
|-- categories.xml
|-- msds.xml
|- metadata 
|--metadataset1.zip
|--metadataset2.zip


It is possible to manually build this file system, however the Fusion Edge Compiler can be used to obtain information directly from an SDMX REST API to automate the process of building the File system.

Part 2 - Compiled Environment

The Fusion Edge Compiler compiles the source file system into an (Environment).


Part 3 - Publish

There are 2 main modes of publication, static mode and dynamic mode.

​Static Mode

Static mode is termed as such because the Fusion Edge Server loads the Environment on application startup, and it never changes the Environment after this. The contents of the Fusion Edge Server are static, and can only be changed by restarting the web application server.

To use static mode, the contents of the (Environment) folder is zipped to producing a single zip file. The zip file is placed on the file system in the home directory of the Fusion Edge Server. The edgeserver.properties file is configured to define the name of the Environment zip file that the Fusion Edge Server should load on application startup.

The Fusion Edge Server will read the zip file on application startup and load the Environment into memory. The only way a new Environment can be deployed is by restarting the web application server.

Dynamic Mode

Static mode is termed as such because the Fusion Edge Server loads the Environment on application startup, and then it polls the Environment's

Retrieved from "https://wiki.sdmxcloud.org/index.php?title=Edge_Server_-_Publish_Content&oldid=5009"