Orchestration Issues

General Issues

You may experience one of the following issues when using the Analytics Runtime service.

Unanticipated System Errors

If you receive one of the following errors, it is an unanticipated system error. It could indicate that the request signature or body is malformed.

CONFIG_ORCH002, CONFIG_ORCH003, CONFIG_ORCH005, CONFIG_ORCH007, CONFIG_ORCH008, CONFIG_ORCH010, CONFIG_ORCH012, CONFIG_ORCH014, CONFIG_ORCH015, CONFIG_ORCH020, CONFIG_ORCH021

Cause

One of the following causes may exist.

  • API signature is wrong.
  • API method type (GET, POST, etc) is wrong.
  • Headers are missing or malformed.
  • Specified resource does not exist.
Solution

Try the following.

  • Verify the request parameters (URI, HTTP method, headers) against the API documentation.
  • Check that the request body is valid. For example, if the API sends JSON, use a JSON validator to verify the validity of the JSON structure, and verify it matches the expected structure from the API documentation.
  • If a resource ID was specified in the URI, verify that it is a valid ID.
  • Verify that another API works. If they are both unsuccessful, contact the Support team to verify that the service and dependent database are running. See Support Ticket Information.

500 Internal Error, 400 Bad Request, 404 Not Found, 405 Method Not Allowed

Cause

If you receive one of these errors, it is an unanticipated system error. It could indicate that the request signature or body are malformed.

One of the following causes may exist.

  • API signature is wrong.
  • API method type (GET, POST, etc) is wrong.
  • Headers are missing or malformed.
  • Specified resource does not exist.
Solution

Try the following.

  • Verify the request parameters (URI, HTTP method, headers) against the API documentation.
  • Check that the request body is valid. For example, if the API sends JSON, use a JSON validator to verify the validity of the JSON structure, and verify it matches the expected structure from the API documentation.
  • If a resource ID was specified in the URI, verify that it is a valid ID.
  • Verify the REST API headers are configured correctly. For information about how to find the URI values needed, see Binding an Application to an Analytics Framework Instance. For information about configuring headers, see Configuring REST Request Headers.
  • Verify that another API works. If they are both unsuccessful, contact the Support team to verify that the service and dependent database are running. See Support Ticket Information.

401 Unauthorized

Cause

Provided user token is invalid or has expired.

Solution

Obtain a new token from your identity provider.

403 Forbidden

Cause

The user does not have the proper scopes or access for the zone.

Solution

Verify that the token being passed has the correct scope. This can be verified when the access token is obtained in the "scopes" field of the token response. For more information, see Updating the OAuth2 Client for Services.

Orchestration Execution

You may experience one of the following issues when running an orchestration of analytics.

ORCH001, ORCH002, ORCH003

Cause

The platform was unable to execute the orchestration because the BPMN workflow was not formatted correctly.

Solution

Try the following.

  • Verify that the orchestration's BPMN workflow matches the documented specification.
  • If the problem persists, contact the Support team. See Support Ticket Information.

ORCH004

Cause
The platform received an unexpected response when trying to execute the specified analytic(s).
Solution

Try the following.

  • Verify the input that is being passed to the analytic(s) in the orchestration execution request.
  • Call the Analytics Catalog's Validation or Deployment API again to redeploy the analytic(s).
  • If the problem persists, call the Support team. See Support Ticket Information.

ORCH006

Cause

The platform was unable to execute the specified analytic(s) because the analytic details in the BPMN workflow do not match those of the analytic in the catalog.

Solution

Try the following.

  • Verify the analytic detains in the BPMN workflow against those in the catalog.
  • If the problem persists contact the Support team. See Support Ticket Information.

Scheduler Service

You may experience the following issues with scheduled jobs.

Scheduled Job Fails

The scheduled job is fails after execution.

Cause

In the case where 25 jobs have failed, the job will be automatically suspended by the system.

Remedy

Check the job's history in the scheduler service by following these steps.

  1. Retrieve job execution history to get the status of past job executions as follows.
    GET /api/v1/scheduler/jobs/{jobId}/history 

    The following is a sample response of job history retrieval.

    [
        {
            "id": "de6f80af-fa8b-4b03-8f8f-12bcc4c0440e",
            "jobId": "b3142c28-18f9-4cc5-87e2-883bf7d3f7dc",
            "fireTime": "2015-10-06 17:14:00.0",
            "scheduledFireTime": "2015-10-06 17:14:00.0",
            "cron": "0/30 * * * * ?",
            "timeZoneId": "UTC",
            "httpMethod": "POST",
            "url": "<JOB_EXECUTION_ENDPOINT>",
            "statusMessage": "suspended"
        },
        {
            "id": "de6f80af-fa8b-4b03-8f8f-12bcc4c0440e",
            "jobId": "b3142c28-18f9-4cc5-87e2-883bf7d3f7dc",
            "fireTime": "2015-10-06 17:14:00.0",
            "scheduledFireTime": "2015-10-06 17:14:00.0",
            "cron": "0/30 * * * * ?",
            "timeZoneId": "UTC",
            "httpMethod": "POST",
            "url": "<JOB_EXECUTION_ENDPOINT>",
            "statusMessage": "failed <failure reason>"
        },
        {
            "id": "fe04fd6a-e7ad-4f1f-85ec-012a1af40963",
            "jobId": "b3142c28-18f9-4cc5-87e2-883bf7d3f7dc",
            "fireTime": "2015-10-06 17:13:30.0",
            "scheduledFireTime": "2015-10-06 17:13:30.0",
            "cron": "0/30 * * * * ?",
            "timeZoneId": "UTC",
            "httpMethod": "POST",
            "url": "<JOB_EXECUTION_ENDPOINT>",
            "statusMessage": "failed <failure reason>"
        },
        {
            "id": "3adbb8e1-0ddd-4c5d-af39-8e84cb4c9e38",
            "jobId": "b3142c28-18f9-4cc5-87e2-883bf7d3f7dc",
            "fireTime": "2015-10-06 17:13:00.0",
            "scheduledFireTime": "2015-10-06 17:13:00.0",
            "cron": "0/30 * * * * ?",
            "timeZoneId": "UTC",
            "httpMethod": "POST",
            "url": "<JOB_EXECUTION_ENDPOINT>",
            "statusMessage": "failed <failure reason>"
        }
    ]
  2. If your job has been suspended by the scheduler service, determine the cause of the job failures and correct them.
  3. Re-activated the job by setting the "state" to "Active" using the following API call.
    PUT /api/v1/scheduler/jobs/<job_id>

    The following is a sample request to update a job.

    {
      "name": "Sample Job Name",
      "id": "b3142c28-18f9-4cc5-87e2-883bf7d3f7dc",
      "description": "Sample Job Description",
      "state": "Active",
      "cron": {
        "seconds": "0/30",
        "minutes": "*",
        "hours": "*",
        "dayOfMonth": "*",
        "months": "*",
        "dayOfWeek": "?",
        "years": "*",
        "timeZoneId": "UTC"
      },
      "executionRequest": {
        "url": "<JOB_EXECUTION_ENDPOINT>",
        "httpMethod": "POST",
        "httpHeaders": [
          {
            "name": "Content-Type",
            "value": "application/json"
          }
        ],
        "inputData": "<JOB_EXECUTION_DATA>"
      }
    }
  4. After the job has been re-activated, continue checking the job history to make sure that the failures have been corrected.

Scheduled Job Fails with 404 Not Found Error

A job is scheduled to run an orchestration every two minutes. The job execution history (see Managing Suspended Jobs) shows the job execution failed with a 404 Not Found Error. For example:
Cause
In this example, the orchestration execution request URL is incorrect as it contains only the route name. It does not include the API context path.
Remedy
The correct format for the orchestration execution request URL is as follows: https://predix-analytics-execution-service.run.aws-usw02-pr.ice.predix.io/api/v2/execution.

Support Ticket Information

When contacting the Support team regarding an issue, have the following information available.

  • The operations you were trying to perform when you encountered the error.
  • Details such as the request URI, request headers, request body, and HTTP method.
  • The value of the X-App-Correlation-Id header from the response, which can be used to isolate log messages generated by your operation.