Change Log FR - V11 and subsequent releases

From Fusion Registry Wiki
Revision as of 01:02, 14 September 2023 by Plazarou (talk | contribs) (→‎Version 11.10.0 (Released: 2023/09/11))
Jump to navigation Jump to search


Changelog for Fusion Registry Version 11

Version 11.10.0 (Released: 2023/09/11)

Based on FMR 11.10.1

New Features:
FR11-195 Publication Tables: Allow dependant variables to be overridden from the API
FR11-197 Default response formats in v2 web service are now configurable
FR11-203 Kafa: new topic to publish structure transaction events to
FR11-204 Kafa: new topic to publish reference metadata events to

Change:
FR11-202 Structure Format response is no longer auto detected if not explicitly set (takes value from default setting)

Improvement:
FR11-198 Querying for data by observation attributes

Bug fixes:
FR11-199 saveas=zip is not working on v2 API
FR11-200 Publication table: formatting cells ignores UNIT_MULT when it is reported as a dataset level attribute
FR11-201 Publication table: Conditional Formatting being applied when condition references a Component which is not present in DSD

Version 11.9.6 (Released: 2023/08/25)

Changes:
FR11-163 - Validator: "Missing Attributes" renamed to "Missing Components" and reports on missing Measures
FR11-164 - Add "Output Required" UI control to Structure Mapping
FR110188 - Adding a observation formatting rule treats comma separeted component ids as an AND operator (was OR)

Improvements:
FR11-146 - Update Apache POI to version 5
FR11-147 - Update Commons-fileupload third-party dependency
FR11-148 - Update Spring dependencies
FR11-149 - Update Jettison third-party dependency
FR11-150 - Web Service to refresh caches
FR11-158 - Merging a Codelist should merge Locale information
FR11-160 - Update Spring and Jackson Dependencies
FR11-161 - Implment caching on publication tables
FR11-167 - Improve Mapping functionality so that it can perform Validation
FR11-171 - Portal File system sweep - allow files to be placed in root of sweep directory
FR11-173 - Update Kryo to version 5.5.0
FR11-174 - Publication Tables - not formatting numbers with scientific notation
FR11-175 - Components with Time based Mappings cannot be edited in the Structure Map Wizard
FR11-176 - Support Custom Plugins
FR11-190 - Add SDMX-JSON v2 to the UI
FR11-191 - Restore Fusion Security as an authentication mechanism
FR11-192 - FMR#481: Duplicated group IDs for schema queries in SDMX-JSON
FR11-193 - Mapped Datasets Configuration - new option "Output Required"

Bug Fixes:
FR11-151 - Item Validity not returning most recent entry prior to today's date
FR11-152 - Subscript out of range error with FusionXL when carriage return is in the name of a structure
FR11-153 - No transactions returned on SDMX Transactions page when using date slider
FR11-155 - Publicaiton Table Editor: Dependent Variables, remove variable button does not work
FR11-156 - Publication Table availabililty algorithm modified to better calculate default values when using multiple dataflows
FR11-157 - FusionXL resolve carriage returns in concept names
FR11-159 - Cross references not updated correctly when structure already exists
FR11-162 - Update Vulnerable Dependencies for FMR 11.7.0
FR11-165 - Validating Generic Data can result in IllegalElement error
FR11-166 - FMR#418 - Conversion and mapping services fail to read SDMX CSV 2.0 format files with BOM
FR11-168 - CSV-TS data format outputting names as labels does not output labels in last Dimension/Attribute column
FR11-169 - RegEx Capture Groups for Representation Map only support 0-9 (\10 and above does not work)

Version 11.9.5 (Released: 2023/03/08)

Changes:
FR11-140 - Modification of interpration of SDMX-CSV V2 row https://github.com/bis-med-it/fmr/issues/349

Improvements:
FR11-145 - Fusion Excel Export Data API now takes an additional 'attachment' parameter

Bug Fixes:

FR11-137 - Removal of dependencies: gson, protobuf, closure, args4j
FR11-138 - Unable to load Structure File when Mapping refers to DSD with ValueList coded dimension https://github.com/bis-med-it/fmr/issues/359
FR11-141 - Caching in reference metadata requests https://github.com/bis-med-it/fmr/issues/356
FR11-142 - Parsing EDI fails to detect malformed period https://github.com/bis-med-it/fmr/issues/362
FR11-143 - EDI REL segment must be followed by ARR segment https://github.com/bis-med-it/fmr/issues/322
FR11-144 - Publication Table - Calculations which contain missing values should not output a value

Version 11.9.4 (Released: 2023/02/17)

Bug Fixes
FR11-135 Publication Table - calculated values could come back as enginnering string when standard form is requested
FR11-136 Regression mapped data store was not mapping rows with null attribute values

Version 11.9.3 (Released: 2023/02/16)

Bug Fixes
FR11-129 Temporary "MV" files not being cleared up after data transformation https://github.com/bis-med-it/fmr/issues/346

Improvements
FR11-131 Improve Performance of large Codelists which have Item Validity https://github.com/bis-med-it/fmr/issues/353
FR11-133 Improve linking to external datasource by auto detecting foreign key

Version 11.9.2 (Released: 2023/02/13)

Change
FR11-132 Supported 'latest' in v2 query to prevent errors in connected tools

Version 11.9.1 (Released: 2023/02/10)

Bug Fixes:
FR11-128 Security rule on OBS Attribute blocks observation when value is null

Version 11.9.0 (Released: 2023/02/03)

Improvements:
FR11-117 - Provide option to skip Validation phase in Transform Web Service https://github.com/bis-med-it/fmr/issues/237
FR11-120 - Publication Table - apply number formatting to calculated cells
FR11-121 - Publication Table - Apply number scaling before calculations

Change:
FR11-118 - Use the public fastUtil library https://github.com/bis-med-it/fmr/issues/338

Bug Fixes:
FR11-116 - Timezone issue with REST API resulting in no data
FR11-119 - Codelists with Annotations not being correctly parsed in SDMX-JSON https://github.com/bis-med-it/fmr/issues/345
FR11-123 - EDI should prevent Quadruplet attributes in Footnote Section https://github.com/bis-med-it/fmr/issues/347
FR11-124 - EDI file missing mandatory Period Information validates successfully https://github.com/bis-med-it/fmr/issues/348
FR11-125 - Improve validation of EDI DTM segment https://github.com/bis-med-it/fmr/issues/351
FR11-126 - Publication Table - decimal places (fixed) truncated trailing zero places when locale was also set
FR11-127 - Duplicate elements in SDMX CSV result in unexpected output https://github.com/bis-med-it/fmr/issues/342

Version 11.8.2 (Released: 2023/01/17)

Improvements:
FR11-108 - Prevent contradictory excel mappings (generted from Fame Metadata Manager) worksheets being uploaded to Registry
FR11-109 - Flag contradictory excel mappings (generted from Fame Metadata Manager) in generated worksheet
FR11-134 - Improve Metrics Report with IncUnmappedReport option

Changes:
FR11-111 - Change ID Generation of Representation Map from FusionFameMapper xlsx files.
FR11-115 - Ensure Implicit Mappings in Structure Maps validate codelists for valid intersection

Bug Fixes:
FR11-107 - Parent codes should be included in reference partial
FR11-110 - Complete Cube web service feature outputs empty measure values (not 0 values)
FR11-112 - Data export formats excel-ts and excel-series can miss observations in output series if series contains observations with no measure values and no attribute values
FR11-113 - Fix 'too much recursion' error with reverse engineer data structure with large number of Dimensions
FR11-114 - Validate submission of Structure Map's Implicit mappings to ensure Coded mappings have a valid intersection from source to target

Version 11.8.1 (Released: 2023/01/11)

Bug Fixes:
FR11-101 - Unable to import Codelists with multiple levels of inheritence in same submission as structures that use inherited codes
FR11-102 - Bulk actions - detail=raw is ignored
FR11-103 - REST query: detail=raw is not applied to referenced codelists
FR11-104 - IncValid output may contain empty series when it should not be generated
FR11-105 - Annotations are ignored when importing 2.1 XML Hierarchical Codelists
FR11-106 - format=csv-series null pointer

Version 11.8.0 (Released: 2022/12/21)

Important Note for MySQL users: due to changes in the distribution of the MySQL connector (change FR11-94 below), please read this article.

New Feature:
FR11-98 - Publication Table - support a scaling function applied to numerical observations

Improvement:
FR11-93 - Improve Performance of Viewing a Hierarchy https://wiki.sdmxcloud.org/Publication_Table#Formatting_Numerical_Observation_Values

Changes:
FR11-91 - Validation and Transformation should preserve original format unless output format specified
https://wiki.sdmxcloud.org/Data_Validation_Web_Service#HTTP_Headers see Accept Header
https://wiki.sdmxcloud.org/Data_Transformation_Web_Service#HTTP_Headers see Accept Header
FR11-94 - Remove MySQL Connector https://wiki.sdmxcloud.org/Install_MySQL#Installing_MySQL_5.7

Bug Fixes:
FR11-89 - Support dataset attribute consolidation (detection of duplicates)
FR11-90 - Observation deletions are no longer written in EDI format
FR11-92 - NPE when publishing EDI data from Command Line
FR11-95 - Publication Table - UI changes table definition (looses column) when clicking preview table
FR11-96 - Publication Table - Preview table error when table heading Rowspan > 1
FR11-97 - Publication Table - bug fix on formatting obs which removed values from output table
FR11-99 - Publication Table - Scientific Notation shows NaN in UI and is not applied to observation values
FR11-100 - Refreshable workbook - issue security challenge when connecting to private server
FMR11-207 - Transforming Data with "Delete" action can leave erroneous output in EDI format
FMR11-217 - Publication Tables: Numerical Observation, "Add Rule" button does not work
FMR11-224 - Failure to report unmapped data when using TIME_PERIOD mapping

Version 11.7.2 (Released: 2022/11/22)

Change:
FR11-84 - In SDMX CSV, SERIES_KEY and OBS_KEY will output with the quote marks around the whole key

Bug Fixes:
FR11-80 - Validation on Group ID should ignore case
FR11-82 - Size of column in Portal Table not large enough to contain Data Source ID
FR11-85 - When loading via Registry UI, Data file with Duplicate Series results in MV Store Error
FR11-86 - Viewing a Representation Map has issue with "valid from" on Mappings

Version 11.7.1 (Released: 2022/11/15)

New Feature:
FR11-83 - Support Categorise Publication Table via the UI

Version 11.7.0 (Released: 2022/11/11)

New Feature:
FR11-78 - Publication Table Support Numerical Observation formatting

Bug Fixes:
FR11-79 - Fix an issue when "edi.datagroup.identifier" specified with value of asterisk
FR11-80 - Validation on Group ID should ignore case
FR11-81 - Data Portal - ESTAT Connector fix

Version 11.6.0 (Released: 2022/11/31)

New Features:
FR11-76 - Audit Structure Queries
FR11-77 - Audit Data Queries

Improvements:
FR11-75 - Support for extended UTF-8 characters https://github.com/MEDIT-Org/fmr/issues/297

Change:
FR11-65 - Update FontAwesome to use Version 6

Bug Fixes:
FR11-71 - Corrected issue with Dynamic Attributes
FR11-72 - Version Drop-Down not Displayed when Selected Item on Page Load
FR11-74 - Security Group Web Service does not output UTF-8 characters
FR11-65 - Update Web Services "structure" page so the plus symbol is used for "latest"

Version 11.5.2 (Released: 2022/10/14)

New Feature:
FMR11-194 - Publication Table - support casting dates more then once in same expression

Improvement:
FMR11-202 - Update dependencies: Hibernate, Spring-Kafka, Jettison and Jersey https://github.com/MEDIT-Org/fmr/issues/294

Bug Fixes:
FR11-66 - "Metadata Provider Scheme" not listed on Web Service -> Structures page
FR11-67 - "Metadata Provider Wizard" does not allow Providers to start with a numeric yet the SDMX Standard allows this
FR11-68 - "Metadata Provider Scheme" not listed on Cross References page
FR11-69 - Attaching Reference Metadata to Items in an item scheme fails to add link (cache issue)
FMR11-189 - Behaviour of web service resolving multiple Codelist inheritance with "details=raw" https://github.com/MEDIT-Org/fmr/issues/257
FMR11-196 - Unable to read Excel Spreadsheet where all Frequencies are Formulas
FMR11-197 - Request for Codelist in SDMX-JSON, "details=raw", references="children" doesn't return children as raw https://github.com/MEDIT-Org/fmr/issues/257
FMR11-198 - Request for Codelist in SDMX-JSON, "details=raw", references="descendants" doesn't return descendants https://github.com/MEDIT-Org/fmr/issues/257
FMR11-199 - "Metadata Provider Scheme" not listed on Web Service -> Structures page
FMR11-200 - "Metadata Provider Wizard" does not allow Providers to start with a numeric yet the SDMX Standard allows this
FMR11-201 - "Metadata Provider Scheme" not listed on Cross References page
FMR11-203 - Attaching Reference Metadata to Items in an item scheme fails to add link (cache issue)
FMR11-205 - Version Number not Displayed when Selected Item on Page Load

Version 11.5.1.1 (Released: 2022/10/10)

Change:
FR11-73 - Modify SQL to ensure GMT Timezone is used

Version 11.5.1 (Released: 2022/10/05)

New Features:
FR11-60 - Publication Table - support casting dates more then once in same expression
FR11-61 - orderMeasure web service parameter to sort observations by date asc/desc

Version 11.5.0 (Released: 2022/09/28)

New Features:
FMR11-187/FR11-53 - Support recovery of structure transactions from backup on installation https://github.com/MEDIT-Org/fmr/issues/277
FR11-52 - Support variable number of max arguments to a SQL IN statement
FR11-58 - Support import of datafile containing mulitple datasets

Improvements:
FMR11-183 - Update dependencies https://github.com/MEDIT-Org/fmr/issues/277
FR11-48 - Update third-party dependencies: apache-jstl, commons-dcp2, joda-time, spring, tika-core
FR11-45 - Add Version Number to Provision Agreement Selector in Data Source Manager Modal

Bug Fixes:
FMR11-181/FR11-46 - SDMX CSV v 2.0 Data Format does not report its format https://github.com/MEDIT-Org/fmr/issues/110
FMR11-184/FR11-49 - Support '*' for structure type in REST API https://github.com/MEDIT-Org/fmr/issues/273
FMR11-185/FR11-51 - Failure on Compare Revision shows Stack Trace https://github.com/MEDIT-Org/fmr/issues/278
FMR11-186/FR11-55 - SDMX-JSON Writer not outputting DSD as stub https://github.com/MEDIT-Org/fmr/issues/274
FR11-50 - FameMapper code is creating Excel files in temporary storage which are not deleted
FR11-54 - EDI data not fully consolidated when publishing data via web service

Version 11.4.3 (Released: 2022/09/13)

New Features:
FMR11-174 - Publication Tables: Formatting of Missing Values support ObsValue=NaN https://github.com/MEDIT-Org/fmr/issues/253
FMR11-175 - Support Reading of Reference Metadata (with restrictions) in version 2.1 SDMX https://github.com/MEDIT-Org/fmr/issues/250

Improvement:
FMR11-182 - Modify ID generation strategy for importing Reference Metadata at v2.1

Bug Fixes:
FR11-39 - OBS_PRE_BREAK validation corrected
FR11-43 - SDMX DSD output may include invalid attribute "isMultiLingual"
FR11-44 - SQL Server does not delete temporary tables
FMR11-169 - SDMX CSV v1 Data Reader does not allow header fields (row 1) to be double quoted
FMR11-170 - Bulk Export - spelling mistake on Raw help text
FMR11-171 - Order of revisions not as expected
FMR11-176 - OBS_PRE_BREAK validation corrected
FMR11-178 - Quadruplet in EDI vs the other formats in the context of data consolidation or integration https://github.com/MEDIT-Org/fmr/issues/118
FMR11-179 - SDMX DSD output may include invalid attribute "isMultiLingual" https://github.com/MEDIT-Org/fmr/issues/255


Version 11.4.2 (Released: 2022/08/19)

Bug Fixes:
FR11-30 - Fusion Security Manager will not accept AD group names longer than 30 characters
FR11-34 - Updated third-party dependencies

Version 11.4.1 (Released: 2022/08/10)

Bug Fixes:
FR11-21 - Issue loading data into SQL Server in Registry managed table FR11-26 - Bulk Export - spelling mistake on Raw help text
FR11-27 - Order of revisions not as expected
FR11-28 - JNDI Registry Managed Database - unable to load data
FR11-25 - SDMX CSV v1 Data Reader does not allow header fields (row 1) to be double quoted

Version 11.4.0 (Released: 2022/07/22)

New Feature:
FR11-18 - Support Reference Metadata https://wiki.sdmxcloud.org/Reference_Metadata

Improvement:
FMR11-146 - Dependency updates
FMR11-159 - Add "Raw" option to Download structures and remove "Include internal settings"
FMR11-160 - Allow a Concept Representation to be a ValueList

Bug Fixes:
FR11-14 - Issue loading Data Constraint with a Component element under Cube Region element
FR11-17 - Attribute variable does not work in table heading
FR11-19 - Increase max size of security group ID from 30 to 100 characters
FMR11-162 - SERIES_KEY column should be reserved for series keys
FMR11-161 - Issue loading Data Constraint with a Component element under Cube Region element
FMR11-163 - Attribute variable does not work in table heading
FMR11-166 - Duplicate attribute is not detected anymore in FMR
FMR11-167 - Publication Table - Array Variables fail to save
FMR11-168 - Error encountered whist attempting to deserialize structure

Version 11.3.0 (Released: 2022/07/06)

New Feature:
FR11-10/FMR11-153 - Environment Synchronisation - compare the structural metadata content with that from another Registry, push or pull changes

Improvement:
FMR11-158 - Publication Table: Footnotes support

Bug Fixes:
FMR11-152 - Comma not being processed correctly in Web Service query for ItemBeans
FMR11-154 - Unexpected structure validation error from FMR
FMR11-155 - Structure validation errors are not detected anymore in FMR
FMR11-156 - v2 structure REST API does not support queries for multiple items using the comma syntax
FMR11-157 - Typo in header of Representation Maps page

Version 11.2.0 (Released: 2022/06/27)

New Feature:
[FMR11-150] Specify the validation and transformation behaviour when duplicate observations are found - https://wiki.sdmxcloud.org/Data_Transformation_Web_Service#HTTP_Headers

Bug Fixes:
[FR11-3] Schema Query in structure format does not work
[FR11-4] Fusion Cache ignoring parameters like normalise freq on V1 data queries
[FR11-5] Data Point Security - allow rule against a series attribute value that does not exist in the dataset restricts all data
[FR11-8] Comma not being processed correctly in Web Service query for ItemBeans
[FMR11-141] Re-opening of Alternate Half year Issue
[FMR11-143] TIME_FORMAT not being generated correctly
[FMR11-144] Problem with Date data type when reverse engineering a DSD from a CSV dataset
[FMR11-145] Problem with Description data type when reverse engineering a DSD from a CSV dataset
[FMR11-147] DSD Wizard step 3 - not allowing Enumerated Representation
[FMR11-148] Schema Query in Structure Format does not work

Improvement:
 [FMR11-140] Revert xsi:type=SiblingType, change the schema to output the Group as Sibling not SiblingType -

Version 11.1.0 (Released: 2022/06/01)

New Feature:
Publication Tables
[FR-5130] Publication Table - Support Dynamic Rows
[FR-5131] Publication Table - Format rules on cells with no data
[FR-5132] Publication Table - Rules to derive a Dimension Value from another Dimenison's value
[FR-5133] Publication Table - Option to output rows with no data
[FR-5134] Publication Table - Ouput Materialised Table in Excel

Improvement:
[FR-5129] - Support for MySQL 8 and above

Version 11.0.0 (Released: 2022/05/22)

Changes and New Features:

  • Change to the SDMX 3.0 information model as Fusion Registry’s internal metadata object model
  • SDMX 3.0 REST API support
  • Backward compatibility for SDMX 2.1 and 2.0 structures and formats
  • Backward compatibility with SDMX 2.1 REST API (REST API v1 specification)
  • Improved data and metadata access control
  • Granular control over admin rights
  • Fusion Portal – scheduled and on demand data and structure loads
  • Refreshable Excel data format
  • JNDI database connection option
  • Publication Tables - publish curated but dynamic user interactive web data tables that show precisely the statistics required even if they come from multiple different data sets

Breaking Changes from Fusion Registry 10:

  • Change to the operating database schema - upgrading from Fusion Registry 10 to Fusion Registry 11 requires a fresh install and migration of the metadata, data and settings
  • Environment sync function temporarily removed
  • Fusion Security support deprecated - Active Directory and LDAP are the supported authentication services in Fusion Registry 11
Retrieved from "https://wiki.sdmxcloud.org/index.php?title=Change_Log_FR_-_V11_and_subsequent_releases&oldid=6387"