Using Predix Audit Service

Publishing Predix Audit Events

Predix Audit Service enables publishing events by sending requests to the following endpoint:

<audit-receiver-url>/v2/audit

The POST request must include the following information:
Table 1.
HeaderDescription
AuthorizationA valid token that includes the scopes under "audit-pub-client-scope" , which was issued by a trusted UAA that was provided at instance provisioning.
Predix-Zone-IdAudit Service zone id parameters that represents the AuditService tenant id. This value must match the scopes in the provided token.
Contentapplication/JSON
EventsEvents that need to be published in the event format.
For example:
curl -X POST \
  https://audit-receiver-sysint-blue-dev.run.aws-usw02-dev.ice.predix.io[audit-receiver-url]/v2/audit \
  -H 'Authorization: Bearer <token>
  -H 'Content-Type: application/json' \
  -H 'Predix-Zone-Id: 5dd76136-f897-4371-89e0-38d5e9a0dbbb' \
  -d '[{
"messageId": "525b3e34-5648-46d4-8901-acb6aec53490",
"timestamp":"1546337722",
"classifier":"SUCCESS",
"publisherType":"APP_SERVICE",
"categoryType": "OPERATIONS",
"eventType":"LOGIN_SUCCESS",
"payload":"payload",
"correlationId":"525b3e34-5648-46d4-8901-acb6aec53491",
"tenantUuid":"525b3e34-5648-46d4-8901-acb6aec53492",
"ownerTenant":"525b3e34-5648-46d4-8901-acb6aec53493"}
]'
The event structure should be written in the following format:
[{
	"appName": "string",
	"categoryType": "AUDIT_ACCOUNTABILITY",
	"classifier": "SUCCESS",
	"correlationId": "string",
	"eventType": "CUSTOM",
	"messageId": "string",
	"operatorTenant": "string",
	"ownerTenant": "string",
	"payload": "string",
	"publisherType": "NETWORK_DEVICE",
	"tenantUuid": "string",
	"timestamp": 0
}]
Table 2.
TypeField NameMandatory for user?Details
StringmessageIdYesMust be a random, unique UUID.
longtimestampYestimestamp in milliseconds
AuditEnums.ClassifierclassifierYes
AuditEnums.PublisherTypepublisherTypeYes
AuditEnums.CategoryTypecategoryTypeYes
AuditEnums.EventTypeeventTypeYes
StringpayloadNo
StringcorrelationIdNoMaximum 64 characters
StringtenantUuidNoMaximum 36 characters
StringownerTenantNoMaximum 36 characters
StringoperatorTenantNoMaximum 36 characters
StringappNameNoMaximum 100 characters
The following list shows the available values for the request:
  • AuditEnums.Classifier

    Classifier is a field that is created to classify the event, whether it was a success or a failure. Each message is classified automatically as a successful message. Supported types: SUCCESS, FAILURE, UNRECOGNIZED

  • AuditEnums.PublisherType

    This field enables the developer to provide the auditor type. Supported types: NETWORK_DEVICE, DB_SYSTEM, APP_SERVICE, OS, UNRECOGNIZED

  • AuditEnums.CategoryType

    This field mentions which type of category is related to the event. Supported fields: AUDIT_ACCOUNTABILITY, OPERATIONS, ADMINISTRATIONS, AUTHENTICATIONS, AUTHORIZATION, MALICIOUS, DATA_INTEGRITY, API_CALLS, UNRECOGNIZED

  • AuditEnums.EventType

    This field mentions the event in general terms (It is also a provided CUSTOM value in case there is no matching for the client’s needs). Supported fields: CUSTOM, LOG_START, LOG_STOP, LOG_DELETION, LOG_DEACTIVATION, LOG_MODIFICATION, UNAVAILABILITY, EXCEPTION, SERIOUS_ERROR, STARTUP_EVENT, SHUTDOWN_EVENT, START_SERVICE, STOP_SERVICE, ACCOUNT_PRIVILEGE_SUCCESS_MODIFICATION, ACCOUNT_PRIVILEGE_FAILURE_MODIFICATION, ADD_ADMIN_ACCOUNT, CHANGE_PASSWD_SUCCESS, CHANGE_PASSWD_FAILURE, CHANGE_CONFIGURATIONS_SUCCESS, CHANGE_CONFIGURATIONS_FAILURE, ADD_ADMIN_GROUP_ACCOUNT, CHANGE_CONFIGURATIONS, ADD_ROLE, REMOVE_ROLE, SECURITY_POLICY_CHANGE_SUCCESS, SECURITY_POLICY_CHANGE_FAILURE, LOGIN_SUCCESS, LOGIN_FAILURE, ACCOUNT_LOCKOUT, AUTHENTICATION_ERROR, VPN_CONNECTION_ESTABLISHED_SUCCESS, VPN_CONNECTION_ESTABLISHED_FAILURE, CHANGE_CRITICAL_FILE, PRIVILEGE_ACCOUNT_ACTION, CHANGE_CRITICAL_RESOURCE, INBOUND_CONNECTION_DENIED, OUTBOUND_CONNECTION_DENIED, INVALID_INPUTS, INVALID_APP_ABUSE, COMPONENT_INSTALLATION, COMPONENT_MODIFICATION, COMPONENT_DELETION, DOS_ATTACK, EAVSDROPPING_ATTACK, USER_UNAPPROVED_OUTBOUND_TRAFFIC, VIRUS_ALERT, MALWARE_ALERT, ACTION, CREATE, TRIGGER, DROP, INSERT, UPDATE, DELETE, SUCCESS_API_REQUEST, FAILURE_API_REQUEST, UNRECOGNIZED

    The HTTP response shall include the following parameters:
    Table 3.
    HTTP StatusDescription
    200 okRequest was successfully processed . The user should examine the results.
    400 Bad Request

    Request has a bad format:

    • Max size was reached.
    • Audit messageId is not unique.
    • Audit events are malformed.
    • Missing mandatory headers.
    401 Unauthorized

    Request is not authorized:

    • Token is invalid, missing scopes or was not issued by a known trusted issuer.
    • Zone-Id header does not match the token scopes.
    500 Internal Server ErrorAn unknown error occurred.

    When the request is successfully processed ,the user receives a 200 OK HTTP Status with a list of message statuses. This information provides insight regarding the status of each audit event. The user should check the status to ensure that each event was successfully published, and if not, what is the current status and reason.

    Table 4.
    StatusDetailsNotes
    SUCCESSThe audit event was successfully published.
    FAILURE_INVALID

    The audit event is invalid:

    • Mandatory fields are missing.
    • Exceeds limitations per field.
    FAILURE

    The event was not published:

    • Communication failure to the Predix Audit Service.
    • The process was timed out.
    It is recommended to resend the event. The messageStatus description may include additional relevant information.
    The following examples show possible responses. The first two audit events were accepted, whereas the third was rejected due to a missing mandatory "Classifier" field.
    {
        "messageStatus": [
             {
                "messageId": "525b3e34-5648-46d4-8901-acb6aec53490",
                "status": "SUCCESS",
                "description": "message was accepted"
            }
            
            {
                "messageId": "525b3e34-5648-46d4-8901-acb6aec53492",
                "status": "SUCCESS",
                "description": "message was accepted"
            },
            
            {
                "messageId": "525b3e34-5648-46d4-8901-acb6aec53493",
                "status": "FAILURE_INVALID",
                "description": "classifier - must not be null, "
            },
           
        ]
    }

Sending a Predix Audit Message using AuditEvent

On creation, the AuditClient is authenticated by sending an OAuth token to the Predix Audit Service, after which audit messages can be sent to the Predix Audit Service. Compose the audit message to be sent and send it with the audit method of the AuditClient. This section details all the information and methods of the AuditEvent. To create a new AuditEvent, a builder is provided so all other required fields are automatically inserted to the object. The following code example shows the builder:
AuditEvent eventV2 = AuditEventV2.builder()
        .payload("GET: /v2/resources was unsuccessful")
        .classifier(AuditEnums.Classifier.FAILURE)
        .publisherType(AuditEnums.PublisherType.APP_SERVICE)
        .categoryType(AuditEnums.CategoryType.API_CALLS)
        .eventType(AuditEnums.EventType.FAILURE_API_REQUEST)
        .tenantUuid(TENANT)
        .correlationId(CORRELATION_ID)
        .appName(“Application name”)
        .build();
auditClient.audit(eventV2);
The following table describes the various values of Predix canonical model implementation. Note that not all fields are mandatory:
Table 5.
TypeField NameMandatory for user?Editable?Default ValueDetails
StringmessageIdNoYesAuto generated in UUID format by AuditEventV2 builderMaximum 36 characters
intversionNoNoAuto generated by AuditEventV2 builder with the value "2".The field is protected. Changing this field may cause an exception.
StringauditServiceIdNoYesAuto generated in Audit-SubscriberThe field is an Audit Subscriber Component and should not be used by Audit SDK users
longtimestampNoYesAuto generated by AuditEventV2 builder with system current time in milliseconds
AuditEnums.ClassifierclassifierYesYesAuto generated by AuditEventV2 builder to "SUCCESS"Audit SDK throws a Null Point Exception if AuditEventV2 is initialized without this value
AuditEnums.PublisherTypepublisherTypeYesYesThe developer must set this in the builderAudit SDK throws a Null Point Exception if AuditEventV2 is initialized without this value
AuditEnums.CategoryTypecategoryTypeYesYesThe developer must set this in the builderAudit SDK throws a Null Point Exception if AuditEventV2 is initialized without this value
AuditEnums.EventTypeeventTypeYesYesThe developer must set this in the builderAudit SDK throws a Null Point Exception if AuditEventV2 is initialized without this value
StringpayloadNoYesCustom field. not mandatoryMaximum 2048 characters
StringcorrelationIdNoYesNot mandatoryMaximum 64 characters
StringtenantUuidNoYesShould be in UUID formatMaximum 254 characters
StringAppNameNoYesDefines the application friendly nameMaximum 100 characters
Predix canonical model is a method for making a log standardization in all platform services or application, and thus, the user needs to set all mandatory fields without the default values. Otherwise, a NullPointerException is thrown since those fields are mandatory to canonical model. The development team aimed to simplify the work of the developers and combined the canonical model requirements via the AuditEnums type:
  • AuditEnums.Classifier

    Classifier is a field that is created to classify the event, whether it was a success or a failure. Each message is classified automatically as a successful message. Supported types: SUCCESS, FAILURE

  • AuditEnums.PublisherType

    This field enables the developer to provide the auditor type. Supported types: NETWORK_DEVICE, DB_SYSTEM, APP_SERVICE, OS

  • AuditEnums.CategoryType

    This field mentions which type of category is related to the event. Supported fields: AUDIT_ACCOUNTABILITY, OPERATIONS, ADMINISTRATIONS, AUTHENTICATIONS, AUTHORIZATION, MALICIOUS, DATA_INTEGRITY, API_CALLS

  • AuditEnums.EventType

    This field mentions the event in general terms (It is also a provided CUSTOM value in case there is no matching for the client’s needs). Supported fields: CUSTOM, LOG_START, LOG_STOP, LOG_DELETION, LOG_DEACTIVATION, LOG_MODIFICATION, UNAVAILABILITY, EXCEPTION, SERIOUS_ERROR, STARTUP_EVENT, SHUTDOWN_EVENT, START_SERVICE, STOP_SERVICE, ACCOUNT_PRIVILEGE_SUCCESS_MODIFICATION, ACCOUNT_PRIVILEGE_FAILURE_MODIFICATION, ADD_ADMIN_ACCOUNT, CHANGE_PASSWD_SUCCESS, CHANGE_PASSWD_FAILURE, CHANGE_CONFIGURATIONS_SUCCESS, CHANGE_CONFIGURATIONS_FAILURE, ADD_ADMIN_GROUP_ACCOUNT, CHANGE_CONFIGURATIONS, ADD_ROLE, REMOVE_ROLE, SECURITY_POLICY_CHANGE_SUCCESS, SECURITY_POLICY_CHANGE_FAILURE, LOGIN_SUCCESS, LOGIN_FAILURE, ACCOUNT_LOCKOUT, AUTHENTICATION_ERROR, VPN_CONNECTION_ESTABLISHED_SUCCESS, VPN_CONNECTION_ESTABLISHED_FAILURE, CHANGE_CRITICAL_FILE, PRIVILEGE_ACCOUNT_ACTION, CHANGE_CRITICAL_RESOURCE, INBOUND_CONNECTION_DENIED, OUTBOUND_CONNECTION_DENIED, INVALID_INPUTS, INVALID_APP_ABUSE, COMPONENT_INSTALLATION, COMPONENT_MODIFICATION, COMPONENT_DELETION, DOS_ATTACK, EAVSDROPPING_ATTACK, USER_UNAPPROVED_OUTBOUND_TRAFFIC, VIRUS_ALERT, MALWARE_ALERT, ACTION, CREATE, TRIGGER, DROP, INSERT, UPDATE, DELETE, SUCCESS_API_REQUEST, FAILURE_API_REQUEST

  • Payload

    The following fields are currently parsed by the Audit Log Viewer UI and can be a part of the Payload string per the consuming application needs: OLDVALUE, NEWVALUE, RESOURCEUUID, RESOURCE, ACTOR, ACTORUUID, ACTORDISPLAYNAME, BATCHUUID, ORIGINATOR, ACTIONTYPE, DESCRIPTION, RESOURCETYPE.

    For example, the following code snippet can be used by a consuming application to ingest data in all the above payload fields:
    try{
      payload = gson.fromJson(this.getPayload(),
                      AuditEventV2Payload.class);
    }catch (Exception e){
      String msg = "error parsing audit payload as JSON: ";
      log.error(msg, e);
      throw new AuditException(msg);
    }
    Audit audit = Audit.builder()
      .resourceUuid(payload.getResourceUuid())
      .resource(payload.getResource())
      .actor(payload.getActor())
      .actorUuid(payload.getActorUuid())
      .actorDisplayName(payload.getActorDisplayName())
      .tenantUuid(this.getTenantUuid())
      .ownerTenant(this.getOwnerTenant())
      .operatorTenant(this.getOperatorTenant())
      .batchUuid(payload.getBatchUuid())
      .originator(payload.getOriginator())
      .actionType(payload.getActionType())
      .description(payload.getDescription())
      .eventAt(""+this.timestamp)
      .resourceType(payload.getResourceType())
      .oldValue(payload.getOldValue())
      .newValue(payload.getNewValue())
      .build();
    return audit;
    

Querying the Predix Audit Service

Procedure

Predix Audit Service provides REST API with query interface for receiving audit logs according to transmitted parameters. The developer can use the REST API to control auditing and run secure queries against the audit data.
Table 6.
TypeParameterDescription
OptionaltenantUuidReturns audit events that are generated by the specified tenant. This is a second-level tenancy.
MandatorystartDate/endDateReturns audit events within the date/time range (in milliseconds). The current maximal range is 3 months.
MandatorypageDefines response specific page of audit events. Must be larger than 0.
MandatorypageSizeDefines response page size of audit events. The range is between 1 and 1000.
OptionalcorrelationId
Optionalpayload
Optionalclassifier
OptionalpublisherType
OptionalcategoryType
OptionalAppNameDefines the auditing application friendly name. Maximum 100 characters.
OptionaleventType
The most important thing is to secure the audit data access. The client shall set the HTTP headers with the following headers:
Table 7.
HeaderDescription
AuthorizationUAA Token
Predix-Zone-IdAudit Service zone id parameters that represents the AuditService tenant id
Contentapplication/JSON
The following code sample shows an example of a valid query:
curl -X POST   https://audit-service-release.run.aws-usw02-pr.ice.predix.io/v2/query   -H 'Authorization: Bearer ${access_token}'   
-H 'Cache-Control: no-cache'   -H 'Content-Type: application/json'   -H 'Predix-Zone-Id: ${audit_service_instance_guid}'   -d '{
"pageSize": 1000,
"page": 1,
"startDate": 1520470313460,
"endDate":  1520471078764
}'
The HTTP response shall include the following parameters:
Table 8.
HTTP StatusDescription
200 okSuccessful
401 UnauthorizedThe user is not allowed to use the service.
404 Not FoundThe requested resource does not exist
406 Not AcceptableThe request was malformed. The response body includes an error providing additional information

The following procedure describes the methods of performing a query:

  1. Specify your UAA instance as the intended target.
    uaac target <uaa_instance_url>
  2. Request a token for your querying OAuth2 client.
    curl -X GET "https://<client_id>:<client_secret>@<uaa_instance_url>/oauth/token?grant_type=client_credentials" | sed 's/.*access_token":"\([^"]*\).*/\1/'
  3. Navigate to the Swagger documentation of Audit Service query API URL, which you can find in the VCAP credentials of an application bound to the Audit Service.
    https://audit-query.<cf_host>/swagger-ui.html
  4. Drag the Model Schema into the query box, and alter the values in accordance with the query you wish to run.
  5. In the Predix-Zone-Id box, insert the <Predix-Zone-Id> value that can be found in the VCAP credentials of an application bound to the Audit Service. For example:
    "audit-query-api-scope": "audit.zones.<Predix-Zone-Id>.user",
  6. In the Authorization box, insert the token obtained from the querying OAuth2 client.
    Bearer <token>
  7. You should receive non-empty results if the query criteria matched the events that your have audited. For example:
    {  
       "content":[  
          {  
             "messageId":"a0793d4a-0e25-4dcb-92a4-01fd6725aa27",
             "version":2,
             "auditServiceId":"4e807ce7-e9fc-49e3-ab7b-36fcac115cc8",
             "timestamp":1488144400212,
             "classifier":"FAILURE",
             "publisherType":"APP_SERVICE",
             "categoryType":"ADMINISTRATIONS",
             "eventType":"ACCOUNT_LOCKOUT",
             "payload":null,
             "correlationId":null,
             "tenantUuid":"3f878967-282f-3c84-a305-835311655f3d",
             "appName": "myApp"
          },
          {  
             "messageId":"8e7d6a0b-cbcc-4a50-9213-4bbdea9885c0",
             "version":2,
             "auditServiceId":"4e807ce7-e9fc-49e3-ab7b-36fcac115cc8",
             "timestamp":1488144400212,
             "classifier":"FAILURE",
             "publisherType":"APP_SERVICE",
             "categoryType":"ADMINISTRATIONS",
             "eventType":"ACCOUNT_LOCKOUT",
             "payload":null,
             "correlationId":null,
             "tenantUuid":"3f878967-282f-3c84-a305-835311655f3d",
             "appName": "myApp"
          },
          ....
       ],
       "totalPages":10,
       "totalElements":50,
       "last":false,
       "sort":null,
       "numberOfElements":5,
       "first":true,
       "size":10,
       "number":0
    }

Querying Archived Data

Predix Audit Service provides REST API with query interface for retrieving archived audit events, which are stored in BlobStore, according to transmitted parameters. The developer can use the REST API to control auditing and run secure queries against the audit data.
Note: Currently, the archived data is stored in the Blobstore for a period of up to a year.
First, the developer needs to send a request that lists all available files in the Blobstore, which meets the required timeframe in ISO-8601 format (i.e., between startDate and endDate) using the List Request, and afterwards, send the File Request to get that file. The following tables show the parameters that are needed for each request:

POST /v2/archive/list

TypeParameterDescription
MandatorystartDateThe start date of the list in ISO-8601 format
MandatoryendDateThe end date of the list in ISO-8601 format
MandatoryversionThe version of the audit event message (should be V2)
OptionalpageSize
Optionalpage
OptionaltenantUUID Filters data based on the tenant identifier.
Secure the archived audit data access by setting the HTTP headers with the following headers:
HeaderDescription
Predix-Zone-IdAudit Service zone id parameters that represents the AuditService tenant id
AuthorizationUAA token

The HTTP response shall include the following parameters:
HTTPS StatusDescription
200 okSuccessful
201Created
401Unauthorized
403Forbidden
404Not Found
For example:
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/xml' --header 'Predix-Zone-Id: {audit-service-id}' --header 
'Authorization: Bearer {token}' -d '{ 
   "startDate": "2018-01-01", 
   "endDate": "2018-01-31", 
   "version": "V2", 
   "pageSize": 10,  
   "page": 1
   “tenantUUID” : “83fed947-8619-4989-a6bd-ed44378e8cf1” 
 }' 'https://audit-service-sysint-blue-dev.run.aws-usw02-dev.ice.predix.io/v2/archive/list'

POST /v2/archive/file

TypeParameterDescription
MandatorypathThe full path of the archived file.
OptionaltenantIdFilters data based on the tenant identifier.
Secure the archived audit data access by setting the HTTP headers with the following headers:
HeaderDescription
Predix-Zone-IdAudit Service zone id parameters that represents the AuditService tenant id
AuthorizationUAA toekn

The HTTP response shall include the following parameters:
HTTPS StatusDescription
200 okSuccessful
201Created
401Unauthorized
403Forbidden
404Not Found
For example:
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/xml' --header 'Predix-Zone-Id: {audit-service-id}' --header 
'Authorization: Bearer {token}' -d '{ 
   "path": "{path/to/your/file}" 
 }' 'https://audit-service-sysint-blue-dev.run.aws-usw02-dev.ice.predix.io/v2/archive/file'
Note: The name of the path is received from the list request.

Updating Plans

The user may want to update its subscription plan while working with an existing one to move to a dedicated database or enlarge its database storage. To move from one subscription plan to another, the user should use the “update-service” API via the service broker.

The following update scenarios in Predix Audit service are supported using the update-service API:
  • Beta to a Dedicated-A50: A one-time change for existing beta customers.It is recommended for a production environment where a dedicated database is required and the audit events storage volume is considered as medium.

  • Beta to a Shared-A10 : A one-time change for existing beta customers.It is recommended for testing and evaluating the Predix Audit service where the required audit events storage volume is considered small and the DB is shared with other tenants.

Note: After the update, the audit events, which were saved in the current Customer’s Plan do not migrate to the newly selected Plan. Therefore, it is highly recommended that before an update, the Customer backs up all existing Audit events data on Customer’s servers.
The following command is an example of how to use the update-service request using the CF CLI:
cf update-service<audit-instance-name-with-old-plan>-p<new-plan>
For example, assuming the user’s audit service instance name is “audit-test”, which is subscribed to a “Beta” plan, and the user wants to update to a Dedicated-A50 plan then he should use the following CF CLI syntax:
cf update-service audit-test -p Dedicated-A50
Note: All other update-service scenarios that are not written above (such as downgrading to different DB plan or moving from one dedicated plan to another) are currently not supported and return an error message.

Upgrading from a Shared Plan to a Dedicated Plan

The user can upgrade its subscription plan from a Shared plan to a Dedicated plan. A dedicated plan includes a dedicated database, enlarged storage, and an extended retention period for the audit events data. In order to achieve that, the user should perform the following steps:
  1. Delete the current Audit service instance (either via “Delete Service Instance” option in the Marketplace UI or via CF CLI command)
  2. Create a new Audit service instance with Dedicated plan (either via selecting the dedicated subscription plan in the Marketplace UI or via CF CLI command).
  3. Perform the instructions on creating a new Predix Audit instance, as seen in the Getting Started section (https://docs.predix.io/en-US/content/service/security/audit/getting-started-with-predix-audit).
Note: All information that was written in the shared database is deleted once the Predix Audit Service is removed. It is the user's responsibility to backup it before deletion.
Note:

To retain data integrity, the user is required to sign each audit event message with a digital signature (e.g., RSA PKI) and the Predix Audit Service Query API should verify that the signature is valid. The digital signature should be owned and maintained by the customer and used within the message payload field to uniquely identify the original message.

Note:

When a delete-service request is sent to the Predix Audit service broker (via CF CLI or marketplace UI), the customer’s Predix Audit Database is immediately and permanently deleted with all its current audit events data. In order to keep the data before its deletion (for example, due to compliancy regulations), it is the customer's responsibility to transfer all data from the Predix to its own server prior of completing this action.