ETL Server Directory

From Metadata Technology Wiki
Jump to navigation Jump to search


Overview

Fusion ETL Server does not use a tranditional database, instead it makes use of the file system and as such requires a 'home directory' from which all sub-directories are derived. This is called the 'ETL Server Directory and sub-directories are used for:

  • Configuration Files
  • Embedded Database (Nitrite)
  • Temporary Files
  • Data Files

The Fusion ETL Server creates the ETL Server Directory on startup.

Content

Sub-Directories

  • dbconnections – This folder is used to store the database connection details. - Connections are created through User Interface
  • dbmappings – Contains user defined mapping rules from linking Dataflows to the Database Table(s) that hold the data
  • registries – Contains the license file(s) which contain details about the Fusion Registry server which the ETL Server publishes to. License files are generated from the Fusion Registry Server.
  • schedules – Contains user defined CRON schedules which automate the data publication process
  • subscriptions/error - Contains any user defined subscriptions to error events in the ETL server
  • views - Contains any user defined subcube definitions for the Dataflows
  • test - Contains data files that have been produced as part of a Manual Run of a Test Publication

Files

  • auth.json - Connection to Active Directory or Open LDAP server. OPtional. Created manually.
  • etl.db - Embedded database. Created by ETL server on application startup.
  • emailconfig.json - Email server configuration. Optional. Created through User Interface.

Embedded Database Content

  • Audit Logs - A record of all audited events including schedule runs, data publication events, and manual schedule invocations.
  • Error Logs - A record of all error events, which link back to the audited event
  • Application Logs - A record of all the application logging information which links back to audited events

Information Security

Sensitive information which includes database and email server passwords are encrypted using AES-256

Default Edge Server Directory Location

The location of the Fusion Edge Server Directory defaults to:

<user home>/MetadataTechnology/FusionETL

On a Windows 10 Operating System this will typically be:

C:\users\<your user name>\MetadataTechnology\FusionETL

On a Unix Operating System, the likely location is:

/home/<your user name>/MetadataTechnology/FusionETL

Custom Edge Server Directory Location

The location of the Edge ETL Directory can be changed by supplying a Java System variable to the JVM on tomcat launch.

The easiest way to achieve this is to create a new file named setenv.bat (or setenv.sh on Unix environments) and place it in the Tomcat bin directory. The contents of this file should state the full location of the properties directory which should be in the appropriate format for your operating system.

An example of setenv.bat

SET JAVA_OPTS=-DFusionETLHome=c:/dir/MyDirectory

(For Windows systems)

An example of setenv.sh

export JAVA_OPTS=-DFusionETLHome=/dir/MyDirectory

(For Unix systems)