Difference between revisions of "Audit Web Services"

From Fusion Registry Wiki
Jump to navigation Jump to search
(→‎Response)
(→‎Overview)
 
(5 intermediate revisions by the same user not shown)
Line 14: Line 14:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/registry/tx/searchTransactions</b>
+
|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/registry/tx/searchAudits</b>
 
|-
 
|-
 
|style="background-color:#eaecf0"|<b>Access</b>|| <span style='color:red'><b>Restricted to '''Audit Manager, Structure Manager, Portal Manager'''</b></span>
 
|style="background-color:#eaecf0"|<b>Access</b>|| <span style='color:red'><b>Restricted to '''Audit Manager, Structure Manager, Portal Manager'''</b></span>
Line 104: Line 104:
  
 
== Overview ==  
 
== Overview ==  
−
Audited system events are categorised by the process id (which process initiated the event) and the event type for that process.  For example a process id of STRUCTURE may have a sub-event of SAVE to declare it is a save request, or AFTER_COMMIT to declare it is a process which is initiated after a save event.
+
Audited system events are categorised by the process id (which process initiated the event) and the event type for that process.  For example a process id of STRUCTURE may have a sub-event of SAVE to declare it is a save request, or AFTER_COMMIT to declare it is a process which is initiated after a save event.  Each audit event is uniquely identified by its UID, and can be retrieved by this UID.  
  
 
A single event (for example a Portal Sync) is likely to consist of a chain of audited events, where one process and event pair PORTAL.MANUAL_SYNC and likely to call other services which capture their own audits, such as SEND_API_REQUEST.GET which captures the information about the HTTP Get Request from a web service.  This chaining of events enables users to search for specific events (find all audits which call a web service) or search for a chain of events (find a PORTAL.MANUAL_SYNC and include all children).  If an event is found in a chain, for example a specific web service call event, it is possible to request the parent event that led to that call, or all the ancestors - or for the full chain, the ancestors and children events.
 
A single event (for example a Portal Sync) is likely to consist of a chain of audited events, where one process and event pair PORTAL.MANUAL_SYNC and likely to call other services which capture their own audits, such as SEND_API_REQUEST.GET which captures the information about the HTTP Get Request from a web service.  This chaining of events enables users to search for specific events (find all audits which call a web service) or search for a chain of events (find a PORTAL.MANUAL_SYNC and include all children).  If an event is found in a chain, for example a specific web service call event, it is possible to request the parent event that led to that call, or all the ancestors - or for the full chain, the ancestors and children events.
  
−
Each audit event captures the same high level information (unique id, process id, event type, username, thread, start time, duration, status, machine identification, and software version), when a specific event is retrieved by its unique identification any event specific properties will also be retrieved.  An example of event specific property is a for a REST GET event, which may include the HTTP Request and response Headers as well as the URL called.  Properties are also returned if child or parent/ancestors are included on a query for a specific audit.
+
Each audit event captures the same high level information (unique id, process id, event type, username, thread, start time, duration, status, machine identification, and software version), when a specific event is retrieved by its unique identification any event specific properties will also be retrieved.  An example of event specific property is a for a REST GET event, which may include the HTTP Request and response Headers as well as the URL called.   
 +
 
 +
'''Note:''' Properties specific to the event are only returned if a specific audit event is queried, (may include child or parent/ancestors for the audit).
  
 
== Search Audits ==
 
== Search Audits ==
Line 153: Line 155:
 
|-
 
|-
 
| status|| no ||  the response status for the event
 
| status|| no ||  the response status for the event
 +
|-
 +
| username|| no ||  the username associated (that caused) the event
 
|}
 
|}
  
Line 177: Line 181:
 
   ]
 
   ]
  
 +
 +
=== Search Structure Queries ===
 +
Set process_id to SDMX_GET and event_type to structure
 +
ws/secure/audit/?process_id=SDMX_GET&event_type=structure
 +
 +
=== Search Data Queries ===
 +
Set process_id to SDMX_GET and event_type to data
 +
ws/secure/audit/?process_id=SDMX_GET&event_type=data
  
 
== Audit Details ==
 
== Audit Details ==
Line 225: Line 237:
 
         '''"sun.jnu.encoding": "Cp1252", ....'''
 
         '''"sun.jnu.encoding": "Cp1252", ....'''
 
       }
 
       }
 +
    },...
 +
  ]
 +
 +
 +
 +
== Logs for Audit ==
 +
Retrieve the logs for a specific audit event (and optionally child & ancestor events).  The granularity of the log events are dependent on the system log level, which can be modified under the admin pages.
 +
 +
{| class="wikitable"
 +
|-
 +
|style="background-color:#eaecf0"|<b>Entry Point</b>|| <b> /ws/secure/audit/log/{uid}</b>
 +
|-
 +
|style="background-color:#eaecf0"|<b>Access</b>|| <span style='color:red'><b>Restricted to '''Audit Manager, Structure Manager, Portal Manager'''</b></span>
 +
|-
 +
|style="background-color:#eaecf0"|<b>Http Method</b>|| GET
 +
|}
 +
 +
 +
{| class="wikitable"
 +
|+ Query Parameters
 +
|-
 +
! Parameter !! Required !! Description
 +
|-
 +
| ancestors || no ||  true/false to include ancestor audit events (parent events, and their parents, and so on)
 +
|-
 +
| children || no ||  true/false to include child audit events (includes all decendants)
 +
|}
 +
 +
 +
=== Response ===
 +
A JSON Array of log events
 +
 +
[
 +
    {
 +
      "AuditId": "6fa6c67c-37ac-48e5-b687-da2ebb316857",
 +
      "Level": 2,
 +
      "Logger": "org.bis.fusion.utils.spring.ApplicationStarterImpl",
 +
      "Message": "Starting class org.bis.fusion.metadataregistry.manager.settings.RegistryAuditSettingsManagerImpl",
 +
      "Thread": "main",
 +
      "LogTime": 1644827100277
 
     },...
 
     },...
 
   ]
 
   ]

Latest revision as of 01:07, 29 September 2023

Overview

Fusion Registry 11 provides 2 main audit functions

  1. Audit of structure events, including data registrations
  2. System audits, example Fusion Portal events

Audited Structure Events

Search Transaction

Search structure transactions

Entry Point /ws/registry/tx/searchAudits
Access Restricted to Audit Manager, Structure Manager, Portal Manager
Http Method GET
Query Parameters
Parameter Required Description
dateFrom no transaction date from in SDMX date format
dateTo no transaction date to in SDMX date format
max no the maximum number of results
user no Username of user which initiated the transaction
orderDesc no Order results in date descending order

Response

A JSON array of audited transactions 

 [
   {
     "Date": 1645027254245,
     "Id": 53,
     "User": "root",
     "Action": "Replace",
     "Type": "Structure Submission"
   },...
 ]

Audit Details for Transaction

Return the audit details for a transaction. The transaction ID can be obtained from the RSS feed, or from the results of the searchTransactions API

Entry Point /ws/secure/audit/getDetailsByTransactionId
Access Restricted to Audit Manager, Structure Manager, Portal Manager
Http Method GET
Query Parameters
Parameter Required Description
id yes Transaction Id

Response

A JSON response in the following format: 

{
   "UID": "6d6ef80c-d19f-4506-94cb-05e53ebd06e1",
   "TxId": 1,
   "Type": "Structure Submission",
   "ServerName": "localhost",
   "VMID": "g6ee3edeb60c835d:55ba23af:19df811c3be:-8000",
   "MachineId": "DESKTOP-E6MP45C/192.168.4.70",
   "ServerContext": "/FusionRegistry",
   "ServletPath": "/ws/registry",
   "ServerPort": 8080,
   "PathInfo": "/json/save",
   "Parameters": "",
   "Protocol": "HTTP/1.1",
   "AcceptHeaders": "application/json, text/javascript, */*; q=0.01",
   "AcceptLanguage": "en-US,en;q=0.9",
   "SubmissionMethod": "POST",
   "HttpResponseStatus": 200,
   "RequestTime": 1644827120293,
   "Duration": 385,
   "Username": "root",
   "RequestIP": "0:0:0:0:0:0:0:1",
   "ProductVersion": "11.0.0.13",
   "UserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.82 Safari/537.36",
   "BrowserName": "Chrome",
   "BrowserVersion": "98",
   "OS": "Windows 10"
 }


Audited System Events

Overview

Audited system events are categorised by the process id (which process initiated the event) and the event type for that process. For example a process id of STRUCTURE may have a sub-event of SAVE to declare it is a save request, or AFTER_COMMIT to declare it is a process which is initiated after a save event. Each audit event is uniquely identified by its UID, and can be retrieved by this UID.

A single event (for example a Portal Sync) is likely to consist of a chain of audited events, where one process and event pair PORTAL.MANUAL_SYNC and likely to call other services which capture their own audits, such as SEND_API_REQUEST.GET which captures the information about the HTTP Get Request from a web service. This chaining of events enables users to search for specific events (find all audits which call a web service) or search for a chain of events (find a PORTAL.MANUAL_SYNC and include all children). If an event is found in a chain, for example a specific web service call event, it is possible to request the parent event that led to that call, or all the ancestors - or for the full chain, the ancestors and children events.

Each audit event captures the same high level information (unique id, process id, event type, username, thread, start time, duration, status, machine identification, and software version), when a specific event is retrieved by its unique identification any event specific properties will also be retrieved. An example of event specific property is a for a REST GET event, which may include the HTTP Request and response Headers as well as the URL called.

Note: Properties specific to the event are only returned if a specific audit event is queried, (may include child or parent/ancestors for the audit).

Search Audits

Search all system audits

Entry Point /ws/secure/audit
Access Restricted to Audit Manager, Structure Manager, Portal Manager
Http Method GET


Query Parameters
Parameter Required Description
min_duration no minimum processing duration in milliseconds
max_duration no maximum processing duration in milliseconds
event_type no desc
ancestors no true/false to include ancestor audit events (parent events, and their parents, and so on)
children no true/false to include child audit events (includes all decendants)
limit no maximum number of results
offset no used to page the results, the offset skips an [offset] number of results before writing
parent no retrieve any parent audit events of this event
process_id no filter by the audited process
process_start no filter by the timestamp of the event (no sooner then process start)
process_end no filter by the timestamp of the event (no later then process end)
software_version no the version of the software that was running when the even was captured
status no the response status for the event
username no the username associated (that caused) the event


Response

A JSON Array of matched events 

 [
   {
     "uid": "6fa6c67c-37ac-48e5-b687-da2ebb316857",
     "process_id": "APPLICATION_START",
     "thread": "main",
     "event_type": "START",
     "username": "SYSTEM",
     "process_start": 1644827100088,
     "process_end": 1644827102046,
     "duration": 1958,
     "status": 200,
     "vmid": "f5da5cfeb60c837d:44ff22ec:17ef755c3be:-8000",
     "machine_id": "DESKTOP-E6MP45C/192.168.4.70",
     "software_version": "11.0.0.13",
     "properties": {}
   },...
 ]


Search Structure Queries

Set process_id to SDMX_GET and event_type to structure 

ws/secure/audit/?process_id=SDMX_GET&event_type=structure

Search Data Queries

Set process_id to SDMX_GET and event_type to data 

ws/secure/audit/?process_id=SDMX_GET&event_type=data

Audit Details

Retrieve the audit details for a specific audit event (and optionally child/ancestor events). This web service call includes the UID which can be found in the audit response of the search audits, this UID is also reported in error events (such as an emailed error notification).

Entry Point /ws/secure/audit/{uid}
Access Restricted to Audit Manager, Structure Manager, Portal Manager
Http Method GET


Query Parameters
Parameter Required Description
ancestors no true/false to include ancestor audit events (parent events, and their parents, and so on)
children no true/false to include child audit events (includes all decendants)


Response

A JSON Array of matched events, this is the same as the response to search audits, but with the properties object complete with event specific properties. 

 [
   {
     "uid": "6fa6c67c-37ac-48e5-b687-da2ebb316857",
     "process_id": "APPLICATION_START",
     "thread": "main",
     "event_type": "START",
     "username": "SYSTEM",
     "process_start": 1644827100088,
     "process_end": 1644827102046,
     "duration": 1958,
     "status": 200,
     "vmid": "f5da5cfeb60c837d:44ff22ec:17ef755c3be:-8000",
     "machine_id": "DESKTOP-E6MP45C/192.168.4.70",
     "software_version": "11.0.0.13",
     "properties": {
        "MEM_FREE_START": "8.468993E9",
        "java.specification.version": "16",
        "sun.cpu.isalist": "amd64",
        "sun.jnu.encoding": "Cp1252", ....
     }
   },...
 ]


Logs for Audit

Retrieve the logs for a specific audit event (and optionally child & ancestor events). The granularity of the log events are dependent on the system log level, which can be modified under the admin pages.

Entry Point /ws/secure/audit/log/{uid}
Access Restricted to Audit Manager, Structure Manager, Portal Manager
Http Method GET


Query Parameters
Parameter Required Description
ancestors no true/false to include ancestor audit events (parent events, and their parents, and so on)
children no true/false to include child audit events (includes all decendants)


Response

A JSON Array of log events 

[
   {
     "AuditId": "6fa6c67c-37ac-48e5-b687-da2ebb316857",
     "Level": 2,
     "Logger": "org.bis.fusion.utils.spring.ApplicationStarterImpl",
     "Message": "Starting class org.bis.fusion.metadataregistry.manager.settings.RegistryAuditSettingsManagerImpl",
     "Thread": "main",
     "LogTime": 1644827100277
   },...
 ]
Retrieved from "https://wiki.sdmxcloud.org/index.php?title=Audit_Web_Services&oldid=6413"