API Proxy

Invoke internal or third-party REST APIs to fulfill data subject access, deletion, opt-out, and identifier retrieval requests.

Connection Configuration

The API Proxy connection uses the standard Connection object schema to define the API contract using the APIProxy ConnectorType. This ConnectorType requires an array of APIProxyQuery objects for its queries. Once the object has been created and the credentials have been stored, place the object in the RMAgentConfig connections array.

Example Configuration

{
    "name": "User Service",
    "uuid": "44d9e703-b8cf-40a8-a138-3cc110319b0d",
    "capabilities": ["privacy/access", "privacy/delete", "privacy/optout", "privacy/identifiers"],
    "mode": "live",
    "connector_type": "APIProxy",
    "queries": {
        "test": [
            {
                "url": "https://api.acme.com/v0/health-check",
                "headers": {
                    "Authorization": "Basic {credentials}"
                },
                "verb": "GET",
                "verify_ssl": "true"
            }
        ],
        "identifiers": {
            "phone_number": [
                {
                    "url": "https://api.acme.com/v0/identifiers/get-phone",
                    "headers": {
                        "Authorization": "Basic {credentials}",
                        "Content-Type": "application/json"
                    },
                    "body": "{{\"email\": \"{email}\"}}",
                    "verb": "POST",
                    "verify_ssl": "true",
                    "valid_response_codes": [200]
                }
            ]
        },
        "access": [
            {
                "url": "https://api.acme.com/v0/users?{email}",
                "headers": {
                    "Authorization": "Basic {credentials}",
                    "Content-Type": "application/json"
                },
                "verb": "GET",
                "verify_ssl": "true",
                "valid_response_codes": [200]
            }
        ],
        "delete": [
            {
                "url": "https://api.acme.com/v0/users",
                "headers": {
                    "Authorization": "Basic {credentials}",
                    "Content-Type": "application/json"
                },
                "body": "{{\"email\": \"{email}\"}}",
                "verb": "DELETE",
                "verify_ssl": "true",
                "valid_response_codes": [200, 404]
            }
        ],
        "optout": [
            {
                "url": "https://api.acme.com/v0/users/optout",
                "headers": {
                    "Authorization": "Basic {credentials}",
                    "Content-Type": "application/json"
                },
                "body": "{{\"email\": \"{email}\"}}",
                "verb": "PUT",
                "verify_ssl": "true",
                "valid_response_codes": [200, 201, 404]
            }
        ]
    },
    "credentials_location": "arn:aws:secretsmanager:Region:AccountId:secret:datagrail.user-service-hfbdhy"
}

APIProxyQuery

Field
url	string (required) The full URL of the API endpoint.
headers	object(Headers) (optional) The headers to include in the API request.
body	string (optional) The body to include in the API request.
verb	string (required) The HTTP method for the API request. All HTTP methods are supported
verify_ssl	string (required) Determines whether to verify the SSL certificate of the URL. Accepted values are "true" or "false".
valid_response_codes[]	integer (optional) Status codes that the Agent should consider successful. The default value is [200]. Note: An instance of a data subject not existing should be handled the same as a successful request. If the API returns 404, for example, 404 should be added to the array.

JSON Representation
{ "url": string, "headers": { object(Headers) }, "body": string, "verb": enum(Verb), "verify_ssl": string, "valid_response_codes": [integer] }

Headers

Field
<header_name>	string The value of the header.

Substitutions

The url, headers, and body fields support named substitutions to format the strings with identifiers coming from DataGrail, and values stored in your credentials manager. To make a substitution, enclose the variable name in curly braces in the string.

URL

"url": "https://api.acme.com/v2/data-subject-request?{email}"

Headers

"headers": {
    "Authorization": "Basic {credentials}"
}

Credentials to authenticate API requests will be stored in JSON format in your credentials manager. The key/value pairs in the secret will be dictated by substitutions you need to make in your headers. If the API uses Basic Authentication, like the above example, the credentials should be stored as the following key/value pair:

{"credentials": "<base64 encoded username:password>"}

Body

The body of the request should be in JSON format where the value must be enclose in double curly braces so that they are escaped.

"body": "{{\"email\": \"{email}\"}}"

System Requirements

The API Proxy connector requires the following criteria of the target system to be met.

Authentication

The API must support static token-based authentication and the token must be stored in you credentials manager.

Synchronous Flow

The API must support a synchronous response flow whereby all operations occur over a single open connection.

Each request must be stateless and perform deletion, identifier, or data retrieval without any context of previously executed queries.

Response Body Format

Access Requests

The response body of an access request must be an array of objects. The value of each key can be of any datatype, including arrays and nested objects. Each object in the array will be converted into a separate file for your data subject.

The environment variable LOGLEVEL can be set to DEBUG to get more detailed feedback if responses are malformed. Be aware that this level of logging has the potential to expose sensitive data.

[
    {
        "first_name": "Peter",
        "last_name": "Gibbons",
        "company": "Initech",
        "friends": [
            "Samir",
            "Michael"
        ]
    },
    {
        "address_type": "Home",
        "address": {
          "street": "191 N. Lamar",
          "city": "Flander",
          "state": "Illinois",
          "zip_code": "77070"
        }
    }
]

Deletion Requests

The status code is the only signal used to determine if a deletion request was successful. A body can be included for logging purposes but will not propagate to the DataGrail platform.

Identifier Requests

The response body of an identifier retrieval request must be an array of objects with the below key/value pair.

[
    {
      "user_id": "3ef6159b-a523-4ae4-a2b8-6b3ddedf1ab4"
    }
]

Note: The identifier category key is a snake_cased version of the identifier category that the identifier is assigned in the DataGrail application.