Data Sources Manager

Data Sources Manager

Overview

The Data Sources Manager is accessed by an Admin user in the Fusion Registry Web Interface. The Data Sources Manager provides information on the data located in File, Web Service, and the Fusion Data store. In addition, it also provides the means to create a connection to both Registry Managed and External databases.

The Data Sources Manager provides an overview of all the available data sources in the Fusion Registry along with a count of how many Provision Agreements are linked to the data source and a total count of available Series and Observations over all the Provision Agreements. The Data Sources Manager provides the means to bulk add or remove the connection of Provision Agreements to Data Stores. Individual Series or Observations can be deleted from Registry Managed data stores, external URLs can be deleted in bulk.

The remainder of this section describes the functionality of each of the actions in the image above.

Adding a New Database Connection

A database connection establishes a link to a Relational Database (MySQL, Oracle, SQL Server) for the purpose of exposing the data in a read only externally managed database, or creating a Registry Managed connection.

To create a new Database Connection, first click the Maintenance Button in the Data Sources Manager page as shown below.

Clicking on the Add button will open a form which must be completed to establish the database connection, this is shown in the image below.

The Connection Id is used entirely for identification purposes within the Registry itself, it must however contain no spaces and only consist of alphanumeric characters and underscores. The Connection Type refers to if the database connection is for the purpose of a Registry Managed database or an Externally Managed database. If it is Registry Managed then database tables are built on demand as data are loaded. If it is Externally Managed then database tables are expected to exist for any Provision Agreement which is linked to the Data Store.

The remainder of the form is used to establish the connection to the database. Single Sign On (SSO) is supported for SQL Server databases. SSO uses the authentication credentials of the user who launched the Tomcat server to authenticate with the SQL Server database. The Setup Guide provides more information about how to configure the Fusion Registry for SSO support.

Linking Provision Agreements

A Provision Agreement is linked to a Data Store to inform the Fusion Registry where the data for that Provision Agreement should be obtained from. If the Provision Agreement is linked to a Registry Managed data store then the action of linking allows data to be published to the data store. If there is already in the data store on link, then the data will be indexed and made available for query.

Note: If the Provision Agreement is linked to a read only relational database, then it will be indexed on link, and the data will be made available for query. Re-indexing a read only store is required when data is updated in the data store. Re-Indexing can be performed from the Data Dashboard page, by selecting the provision agreement and clicking the ‘Re-Register’ button. An automated process can be set up, by submitting a Data Registration to the web service. To generate a Data Registration SDMX file, click the ‘Automate Registration’ button in the Data Dashboard.

Linking to a Web Service or File data store allows Data Providers to register the URL of the data web service or file. All data publication and registration details are discussed in the Data Providers Guide.

Provision Agreements can be linked to a data store via the Dataflow Wizard, or via the Provision Agreement page, this is discussed in the Structural Metadata Manager Guide. The Data Source Manager page allows multiple Provision Agreements to be linked to a data store in one action. A Provision Agreement can only link to one data store, so the action of linking a Provision Agreement to one data store will unlink it from another. The action of unlinking from a relational database (both registry managed and read only) will not delete the data from the database. The action of unlinking from web service or file stores will delete any Data Registrations made against the Provision Agreements and therefore relinking will require a re-registration. Unlinking from the Fusion Data Store will delete the data from the Fusion Data Store.

Data Rollback

If using the Fusion Data Store, data submissions can be rolled back to a previous state. The last 5 submissions are stored for each Provision Agreement.

To rollback data to a previous version, the Data Sources Manager page allows data submitted against a Fusion Data Store to be rolled back.

Within the Data Sources Manager, selected the appropriate Provision Agreement and a table of updates will be displayed in the lower right of the screen. This will show the submission time, action and number of series in each data submission.

Rolling back the data submission will create a new submission of action “Replace” which will restore the data to the selected point in time. This makes it possible to roll back a roll-back. Note: data rollbacks are only possible against data stored in a Fusion Data Store.

Drop Connection

Dropping a connection to a database will not delete any data in the database, however the stored connection details will be deleted and the active connection will be dropped. Data from the data store will no longer be accessible from the Fusion Registry user interface or web service.

Any Provision Agreement linked to the data store will be unlinked and linked to the data store type “Web Service or File”. The series count for each unlinked Provision Agreement will revert to zero.

Any Data Registrations for Provision Agreements linked to the data store, will be deleted.

Delete all Data

The Delete all data action is only possible for a Registry Managed data store. The effect of deleting all data in a data store will drop all database tables related to any linked Provision Agreements. The Provision Agreements will still be linked to the data store, however the series count will revert to zero.

Any Data Registrations for Provision Agreements linked to the data store, will be deleted.

Delete Registrations

The Delete Registrations action is only possible for a Web Service or File data store. On selecting this action a table will be displayed showing all Data Registrations against the store, with the option to select any number of Data Registrations to delete. The action of deleting a Data Registration will mean the series count indexes will be updated and the data which was accessible from the deleted Registrations, will no longer be available via the web service or user interface.

Edit Data

The Edit Data button opens the Data Browser as shown in the image below.

On selecting Run Query and then clicking on a row, the Observation Details will be displayed and you can either Delete the observation or the Series.