Difference between revisions of "Main Page"

From Fusion Registry Wiki
Jump to navigation Jump to search
(Fusion Data Browser Topics)
(99 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
{{DISPLAYTITLE:<span style="opacity:0;position:absolute;">{{FULLPAGENAME}}</span>}}
 
{{DISPLAYTITLE:<span style="opacity:0;position:absolute;">{{FULLPAGENAME}}</span>}}
 
__NOTOC__
 
__NOTOC__
=The Fusion Registry Knowledge Base=
+
<span style="font-size:2.0rem;">Fusion Registry 10 Knowledge Base</span>
<big><strong>Statistical data management software for official statistics using [https://metadatatechnology.com/about-sdmx.php SDMX]</strong></big>
 
  
=Fusion Registry 10=
+
==Fusion Registry 10==
Fusion Registry 10 is for central banks, international organisations, national statistics offices, development banks and regional statistics authorities who need to collect, integrate, store, process and publish aggregated official statistics.
+
Fusion Registry 10 is a <strong>statistical data warehouse</strong> for official statistics organisations who need to collect, integrate, store, process and disseminate aggregated data using the SDMX standard.
  
It implements the SDMX international standard for official statistics making it robust, non-proprietary and easy to exchange both data and metadata.
+
Version 10.0 was released in January 2020
 +
* On-premises - the Fusion Registry 10 suite of components and models can be used to build custom SDMX data and metadata solutions
 +
* Cloud - SDMX data warehouse delivered as a managed software as a service (SaaS)
  
Version 10.0 was released in January 2020 and replaces Fusion Registry 9.
+
See the [[Change Log]] for details of recent changes, bug fixes and improvements.
::[[File:FR10.png|frameless|750px]]
 
  
=Fusion Registry Topics=
+
 
 +
::[[File:FR10.png|450px]]
 +
 
 +
==Fusion Registry Components and Modules==
 +
It consists of a number of modules centered on the Fusion Registry Core data engine, including a secure high-performance data dissemination server and a configurable data explorer and analytics tool for data consumers and analysts. It also incorporates the same SDMX structural metadata registry as Fusion Metadata Registry.
 +
 
 +
====Fusion Registry Core====
 +
SDMX statistical data engine and structural metadata registry.<br>
 +
The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API.<br>
 +
The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.<br>
 +
[[Fusion Registry Core|More]]
 +
 
 +
====Fusion Edge Server====
 +
High-performance data dissemination server.<br>
 +
A light-weight SDMX data and metadata REST API server designed to be deployed in clusters in the DMZ or at the network edge for scalable, fault-tolerant public data dissemination use cases.<br>
 +
[[Fusion Edge Server|More]]
 +
 
 +
====Fusion Data Browser====
 +
Web tool for data consumers to explore the Fusion Registry data catalogue, create charts and tables, and download data.<br>
 +
Fusion Data Browser is a web application suitable for both internal and external data consumers to search and explore the catalogue of datasets, select series and cubes of interest, build visualisations using tables and charts, and retrieve selected data in a variety of formats including SDMX, Excel and CSV.<br>
 +
It can connect to either a Fusion Registry Core engine, or a Fusion Edge Server cluster.<br>
 +
[[Fusion Data Browser|More]]
 +
 
 +
====FusionXL====
 +
Microsoft Excel addin for Fusion Registry.<br>
 +
FusionXL is an Excel addin for browsing and modifying structural metadata, exploring the data catalogue, and downloading data directly into a spreadsheet.
 +
 
 +
====Fusion Data Mapper====
 +
Web tool for creating and maintaining Fusion Registry FAME to SDMX series mappings.<br>
 +
Web tool for Fusion Registry Core for mapping series with a single dimension to richer multi-dimensional structures using SDMX structure mapping rules.<br>
 +
The main use case is mapping FAME series which typically have a single 'series code' dimension to SDMX multi-dimensional structures including dimensions and attributes like: FREQ, INDICATOR, REF_AREA and UNIT_MULT. However, it will work with source data with a single dimension like 'Indicator'.
 +
 
 +
====Fusion ETL Server (beta)====
 +
Module for Extracting, Transforming and Loading data into Fusion Registry.<br>
 +
Fusion ETL extracts observation data from one or more local SQL databases and loads them into a Fusion Registry Core data engine, transforming the data into the structure specified for the target SDMX Dataflows.
 +
Any number of jobs can be created that can be executed interactively through the web user interface, executed automatically on a defined schedule, or triggered programmatically using the Fusion ETL REST API.
 +
 
 +
====Fusion Registry Command Line====
 +
Windows and Linux command line interface.<br>
 +
A command line interface to Fusion Registry's REST API providing utility scripts for loading structures, data and metadata, validating data, exporting content and deleting content.
 +
 
 +
==Fusion Registry Core Topics==
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 26: Line 67:
 
|-
 
|-
 
| [[:Category:SdmxDataFormat|'''Data Formats''']] || A description of supported Data Formats
 
| [[:Category:SdmxDataFormat|'''Data Formats''']] || A description of supported Data Formats
 +
|-
 +
| [[:Category:DynamicData|'''Dynamic Data''']] || Calculations, Transformations and Dynamic Attributes on queried data
 
|-
 
|-
 
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
 
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
 
|-
 
|-
 
| [[:Category:WebService|'''Web Services''']] || Registry Web Services
 
| [[:Category:WebService|'''Web Services''']] || Registry Web Services
 +
|-
 +
| [[Fusion_Registry_Command_Line|'''Command Line''']] || Structure, data, and metadata publishing from the command line
 
|-
 
|-
 
| [[:Category:Glossary|'''Glossary of Terms''']] || Glossary of SDMX and Fusion Registry terminology
 
| [[:Category:Glossary|'''Glossary of Terms''']] || Glossary of SDMX and Fusion Registry terminology
 
|}
 
|}
  
= Fusion Edge Server =
+
==FusionXL==
 +
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
 +
{| class="wikitable"
 +
|-
 +
| [[FusionXL_Install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
 +
|-
 +
| [[FusionXL_FXLData|'''Inline Data Retrieval''']] || Data Retrieval using the FXLData function
 +
|-
 +
| [[FusionXL_DataBrowser|'''Data Browser''']] || Graphical Data Browsing via the Excel Ribbon
 +
|-
 +
| [[FusionXL_StructureBrowser|'''Structure Browser''']] || Graphical Structure Browsing via the Excel Ribbon
 +
|-
 +
| Videos || [https://www.youtube.com/watch?v=zpslTahktak&feature=youtu.be FXLData]
 +
|}
 +
 
 +
==Fusion Data Mapper==
 +
Fusion Data Mapper provides a web user Interface to administer Structure Maps in the Fusion Registry, where the maps support a specific use case – mapping from Single Dimension Data Structures to multidimensional Data Structures.  This use case is borne out of statistical applications which model time series datasets as a single unique identifier for the Series alongside a Time Dimension.  By modelling these dataset as a DSD with one Dimension (and a Time Dimension), it is very easy to overlay SDMX metadata to describe the Time Series as a multidimensional Cube with additional series Attributes to carry richer metadata.
 +
 
 +
{| class="wikitable"
 +
|-
 +
| [[DataMapperServerInstall|'''Installation''']] || How to Install Fusion Data Mapper
 +
|-
 +
| [[Fusion_Data_Mapper_-_Add_Dataset|'''Add Dataset''']] || Creating new Mapped Dataflows
 +
|-
 +
| [[Fusion_Data_Mapper_-_Maintaining_Mapping_Rules|'''Maintaining Rules''']] || How to create and maintain mapping rules
 +
|-
 +
| [[Fusion_Data_Mapper_-_Query_Forwarding|'''Query Forwarding''']] || How data queries are forwarded from the Target to Source Dataflow using mappings
 +
|-
 +
| '''[https://www.youtube.com/channel/UC0zlrpIaiIivnWWIIRkqXYA Videos]''' || [https://www.youtube.com/watch?v=5dZJi9OysC8 Data Mapper in Action]
 +
 
 +
|}
 +
 
 +
==Fusion ETL Server==
 +
Fusion ETL is a web application which connects to one or more local databases in order to extract data for publication to a remote Fusion Registry instance via the standard Fusion Registry web services.
 +
 
 +
Fusion ETL provides the ability to map Dataflows to Database tables and Dimensions to specific database columns.  Schedules can be configured for automated extraction, views can be defined allowing for extraction of full updates, sub-cubes and deltas. 
 +
 
 +
Fusion ETL provides a sophisticated auditing, logging and error notification framework to ensure full visibility of data publication history.
 +
 
 +
{| class="wikitable"
 +
|-
 +
| [[ETLServerInstall|'''Installation''']] || How to Install Fusion ETL Server
 +
|-
 +
| [[:Category:ETLServerConfigure|'''Configuration Reference''']] || Configuration of the Fusion ETL Server
 +
|-
 +
| '''[https://www.youtube.com/channel/UC0zlrpIaiIivnWWIIRkqXYA Videos]''' || [https://www.youtube.com/watch?v=h3V6kKU1Reg Set up and Configuration]
 +
 
 +
|}
 +
 
 +
==Fusion Edge Server==
 
Fusion Edge Server is a solution for supporting high performance data and metadata dissemination, with high throughput. The Fusion Edge Server serves all data, related metadata, including reference metadata, from its highly optimised in memory store and as such has no reliance on a database. The Fusion Edge Server has been designed to support the use case of high availability of data through load balancing.
 
Fusion Edge Server is a solution for supporting high performance data and metadata dissemination, with high throughput. The Fusion Edge Server serves all data, related metadata, including reference metadata, from its highly optimised in memory store and as such has no reliance on a database. The Fusion Edge Server has been designed to support the use case of high availability of data through load balancing.
 
Fusion Edge Server provides access to its stored data and metadata through SDMX compliant web services, and includes support for non-SDMX data formats such as CSV and Excel. The Fusion Edge Server extends the SDMX web services by providing additional information such as data metrics, and data search.
 
Fusion Edge Server provides access to its stored data and metadata through SDMX compliant web services, and includes support for non-SDMX data formats such as CSV and Excel. The Fusion Edge Server extends the SDMX web services by providing additional information such as data metrics, and data search.
  
 
Fusion Edge Server is compatible with Fusion Data Browser and Fusion XL, providing an off-the-shelf solution for both web based data exploration, and from within Microsoft Excel. Connectivity to statistical software such as R and Matlab can be achieved using third party SDMX Connector libraries such as the [https://github.com/amattioc/SDMX SDMX Connectors project].
 
Fusion Edge Server is compatible with Fusion Data Browser and Fusion XL, providing an off-the-shelf solution for both web based data exploration, and from within Microsoft Excel. Connectivity to statistical software such as R and Matlab can be achieved using third party SDMX Connector libraries such as the [https://github.com/amattioc/SDMX SDMX Connectors project].
 +
 +
  
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
| [[EdgeServerInstall|Installation]] || How to Install Fusion Edge Server
+
| [[EdgeServerInstall|'''Installation''']] || How to Install Fusion Edge Server
 
|-
 
|-
| [[:Category:EdgeServerFunctions|'''Functions Reference''']] || Features and functions reference
+
| [[:Category:EdgeServerConfigure|'''Configuration Reference''']] || Configuration of the Fusion Edge Server
 
|}
 
|}
  
=Fusion Data Browser Topics=
+
==Fusion Data Browser Topics==
 
Fusion Data Browser is a HTML and JavaScript client side web application which provides a user interface for querying, graphing, and exporting data from a Fusion Registry or Fusion Edge Server.   
 
Fusion Data Browser is a HTML and JavaScript client side web application which provides a user interface for querying, graphing, and exporting data from a Fusion Registry or Fusion Edge Server.   
  
Line 54: Line 150:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
| [[Data_Browser_Install|Installation]] || How to Install/Deploy the Data Browser
+
| [[Data_Browser_Install|'''Installation''']] || How to Install/Deploy the Data Browser
 
|-
 
|-
 
| [[:Category:DataBrowserConfigure|'''Configuration Reference''']] || Configuration of the Browser, Charts, and Series
 
| [[:Category:DataBrowserConfigure|'''Configuration Reference''']] || Configuration of the Browser, Charts, and Series

Revision as of 08:12, 14 December 2020


Fusion Registry 10 Knowledge Base

Fusion Registry 10

Fusion Registry 10 is a statistical data warehouse for official statistics organisations who need to collect, integrate, store, process and disseminate aggregated data using the SDMX standard.

Version 10.0 was released in January 2020

  • On-premises - the Fusion Registry 10 suite of components and models can be used to build custom SDMX data and metadata solutions
  • Cloud - SDMX data warehouse delivered as a managed software as a service (SaaS)

See the Change Log for details of recent changes, bug fixes and improvements.


FR10.png

Fusion Registry Components and Modules

It consists of a number of modules centered on the Fusion Registry Core data engine, including a secure high-performance data dissemination server and a configurable data explorer and analytics tool for data consumers and analysts. It also incorporates the same SDMX structural metadata registry as Fusion Metadata Registry.

Fusion Registry Core

SDMX statistical data engine and structural metadata registry.
The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API.
The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.
More

Fusion Edge Server

High-performance data dissemination server.
A light-weight SDMX data and metadata REST API server designed to be deployed in clusters in the DMZ or at the network edge for scalable, fault-tolerant public data dissemination use cases.
More

Fusion Data Browser

Web tool for data consumers to explore the Fusion Registry data catalogue, create charts and tables, and download data.
Fusion Data Browser is a web application suitable for both internal and external data consumers to search and explore the catalogue of datasets, select series and cubes of interest, build visualisations using tables and charts, and retrieve selected data in a variety of formats including SDMX, Excel and CSV.
It can connect to either a Fusion Registry Core engine, or a Fusion Edge Server cluster.
More

FusionXL

Microsoft Excel addin for Fusion Registry.
FusionXL is an Excel addin for browsing and modifying structural metadata, exploring the data catalogue, and downloading data directly into a spreadsheet.

Fusion Data Mapper

Web tool for creating and maintaining Fusion Registry FAME to SDMX series mappings.
Web tool for Fusion Registry Core for mapping series with a single dimension to richer multi-dimensional structures using SDMX structure mapping rules.
The main use case is mapping FAME series which typically have a single 'series code' dimension to SDMX multi-dimensional structures including dimensions and attributes like: FREQ, INDICATOR, REF_AREA and UNIT_MULT. However, it will work with source data with a single dimension like 'Indicator'.

Fusion ETL Server (beta)

Module for Extracting, Transforming and Loading data into Fusion Registry.
Fusion ETL extracts observation data from one or more local SQL databases and loads them into a Fusion Registry Core data engine, transforming the data into the structure specified for the target SDMX Dataflows. Any number of jobs can be created that can be executed interactively through the web user interface, executed automatically on a defined schedule, or triggered programmatically using the Fusion ETL REST API.

Fusion Registry Command Line

Windows and Linux command line interface.
A command line interface to Fusion Registry's REST API providing utility scripts for loading structures, data and metadata, validating data, exporting content and deleting content.

Fusion Registry Core Topics

Structural Metadata Detailed explanations of the main SDMX structural metadata artefacts
How To Guidance on specific tasks using Fusion Registry
Behaviours Answers to 'What happens when...?' questions
Components and Modules Components and modules reference
Functions Reference Features and functions reference
Data Formats A description of supported Data Formats
Dynamic Data Calculations, Transformations and Dynamic Attributes on queried data
Security Authentication and Authorisation Features
Web Services Registry Web Services
Command Line Structure, data, and metadata publishing from the command line
Glossary of Terms Glossary of SDMX and Fusion Registry terminology

FusionXL

FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.

Installation How to Install Fusion XL and connect to a Fusion Registry server
Inline Data Retrieval Data Retrieval using the FXLData function
Data Browser Graphical Data Browsing via the Excel Ribbon
Structure Browser Graphical Structure Browsing via the Excel Ribbon
Videos FXLData

Fusion Data Mapper

Fusion Data Mapper provides a web user Interface to administer Structure Maps in the Fusion Registry, where the maps support a specific use case – mapping from Single Dimension Data Structures to multidimensional Data Structures. This use case is borne out of statistical applications which model time series datasets as a single unique identifier for the Series alongside a Time Dimension. By modelling these dataset as a DSD with one Dimension (and a Time Dimension), it is very easy to overlay SDMX metadata to describe the Time Series as a multidimensional Cube with additional series Attributes to carry richer metadata.

Installation How to Install Fusion Data Mapper
Add Dataset Creating new Mapped Dataflows
Maintaining Rules How to create and maintain mapping rules
Query Forwarding How data queries are forwarded from the Target to Source Dataflow using mappings
Videos Data Mapper in Action

Fusion ETL Server

Fusion ETL is a web application which connects to one or more local databases in order to extract data for publication to a remote Fusion Registry instance via the standard Fusion Registry web services.

Fusion ETL provides the ability to map Dataflows to Database tables and Dimensions to specific database columns. Schedules can be configured for automated extraction, views can be defined allowing for extraction of full updates, sub-cubes and deltas.

Fusion ETL provides a sophisticated auditing, logging and error notification framework to ensure full visibility of data publication history.

Installation How to Install Fusion ETL Server
Configuration Reference Configuration of the Fusion ETL Server
Videos Set up and Configuration

Fusion Edge Server

Fusion Edge Server is a solution for supporting high performance data and metadata dissemination, with high throughput. The Fusion Edge Server serves all data, related metadata, including reference metadata, from its highly optimised in memory store and as such has no reliance on a database. The Fusion Edge Server has been designed to support the use case of high availability of data through load balancing. Fusion Edge Server provides access to its stored data and metadata through SDMX compliant web services, and includes support for non-SDMX data formats such as CSV and Excel. The Fusion Edge Server extends the SDMX web services by providing additional information such as data metrics, and data search.

Fusion Edge Server is compatible with Fusion Data Browser and Fusion XL, providing an off-the-shelf solution for both web based data exploration, and from within Microsoft Excel. Connectivity to statistical software such as R and Matlab can be achieved using third party SDMX Connector libraries such as the SDMX Connectors project.


Installation How to Install Fusion Edge Server
Configuration Reference Configuration of the Fusion Edge Server

Fusion Data Browser Topics

Fusion Data Browser is a HTML and JavaScript client side web application which provides a user interface for querying, graphing, and exporting data from a Fusion Registry or Fusion Edge Server.

Live Demo running against data from the Registry Demo Server.

Installation How to Install/Deploy the Data Browser
Configuration Reference Configuration of the Browser, Charts, and Series
Functions Reference Features and functions reference