Data Jobs (DP) - Revision history

Vmurrell at 14:19, 19 February 2024

2024-02-19T14:19:37Z

Vmurrell: /* Run Data Job */

2023-08-17T15:52:34Z

‎Run Data Job

Vmurrell: /* Create Data Job */

2023-08-17T13:01:39Z

‎Create Data Job

Vmurrell: /* Run Data Job */

2023-08-17T12:35:34Z

‎Run Data Job

Vmurrell at 12:32, 17 August 2023

2023-08-17T12:32:15Z

Vmurrell: Created page with "Category:Data Portal Functions == Overview == Having set up your connections you can now create and run data jobs to pull the information into your Registry. File:Data..."

2023-08-17T12:12:39Z

Created page with "Category:Data Portal Functions == Overview == Having set up your connections you can now create and run data jobs to pull the information into your Registry. File:Data..."

New page

[[Category:Data Portal Functions]]
== Overview ==
Having set up your connections you can now create and run data jobs to pull the information into your Registry.

[[File:Data Jobs.png|800px]]<br>

== Data Jobs ==

{| class="wikitable"
|-
| ID || Each Job must have an unique ID, a maximum of 20 Characters are allowed.
|-
| Sync Schedule || Optional Cron expression to automate the execution of this job against a schedule.
Click [https://www.freeformatter.com/cron-expression-generator-quartz.html Here] for help building an expression.
|-
| Connection || Select which connection you want to use for the Job.
|-
| Data Store || The default is the "in memory" Fusion Data Store. If you have set up any additional Data Sources, they will appear on the list.
|-
| Validation Level || Some datasets may fail validation due to inconsistencies between the metadata and the data (for example reported code Ids which are not in the supporting codelist). Disable validation to ensure datasets are imported even if they are not technically valid against the SDMX metadata
|}

[[File:Create-Data-Job.png|500px]]

@@ Line 28: / Line 28: @@
 == Run Data Job ==
-Select the data job and click the Run Job button.
+Select the data job and click the Run Job button, note that you can select and deselect all, and click the Run Job button.
-[[File:Data Job Run.png|500px]]
 You will be prompted to confirm and then be advised that a sync has been requested. Fusion Portal will now obtain all the relevant structures and data for your Dataflow(s). Once the job has successfully completed, this data will be available in both Fusion Registry (structures and registrations) and the data will be available to view in Fusion Data Browser.

@@ Line 32: / Line 32: @@
 [[File:Data Job Run.png|500px]]
-You will be prompted to confirm and then be advised that a sync has been requested. Fusion Portal will now obtain all the relevant structures and data for your Dataflow(s).
+You will be prompted to confirm and then be advised that a sync has been requested. Fusion Portal will now obtain all the relevant structures and data for your Dataflow(s). Once the job has successfully completed, this data will be available in both Fusion Registry (structures and registrations) and the data will be available to view in Fusion Data Browser.
 == Data Transaction History ==

@@ Line 1: / Line 1: @@
 [[Category:Data Portal Functions]]
 == Overview ==
 Having set up your connections you can now create and run data jobs to pull the information into your Registry.

@@ Line 9: / Line 9: @@
 {| class="wikitable"
 |-
-| ID || Each Job must have an unique ID, a maximum of 20 Characters are allowed.
+| ID || Each Job must have an unique ID, a maximum of 20 Characters are allowed. If you leave this blank and select a specific Dataflow, the system will use this.
 |-
 | Sync Schedule || Optional Cron expression to automate the execution of this job against a schedule.

@@ Line 5: / Line 5: @@
 [[File:Data Jobs.png|800px]]<br>
-== Data Jobs ==
+== Create Data Job ==
 {| class="wikitable"
@@ Line 15: / Line 15: @@
 |-
 | Connection || Select which connection you want to use for the Job.
 |-
 | Data Store || The default is the "in memory" Fusion Data Store. If you have set up any additional Data Sources, they will appear on the list.
@@ Line 22: / Line 24: @@
 [[File:Create-Data-Job.png|500px]]