Setup

The Request Manager Agent establishes connections to internal systems through a set of componentized integrations. DataGrail provides and is regularly adding to a standard set of integrations for use with common systems (databases, APIs, etc), however, the DataGrail Agent is designed to be extended through the use of custom components to encompass any needs specific to your organization.

Overview

Each connection is defined as an object in the connections array in the DATAGRAIL_AGENT_CONFIG environment variable. All connection configurations share the same schema and required parameters, but the parameter binding syntax of the queries, and credentials/connection formats will differ. Refer to the connection's respective document for more information.

Connection

Defines a connection to an internal system, including all metadata regarding the system type, capabilities, and business logic for different request types.

Fields
name	string (required) Name of the target system used for the integration in the platform. Characters should be ASCII-only.
uuid	string (required) An arbitrary and unique version 4 UUID to identify the connection.
capabilities[]	enum(Capability) (required) The capabilities of the connection.
mode	enum(Mode) (required) The status of the connection.
connector_type	enum(ConnectorType) (required) The supported connection to use.
queries	object(Queries) (required) The associated queries of each capability.
credentials_location	string (required) The location of the connection credentials (e.g. Amazon ARN).

JSON Representation
{ "name": string, "uuid": string, "capabilities": [ enum(Capability) ], "mode": enum(Mode), "connector_type": enum(ConnectionType), "queries": { object(Queries) }, "credentials_location": string }

Capability

The supported privacy capabilities for a given Connection.

Enums
privacy/access	string Supports data subject access requests.
privacy/delete	string Supports data subject deletion requests.
privacy/optout	string Supports opt-out requests.
privacy/identifiers	string Supports identifier retrieval requests.

Mode

The status of a Connection to determine whether it should be included in data subject requests.

Enums
live	string The connection should be included in data subject requests.
test	string The connection should not be included in data subject requests.

ConnectorType

The type of system to connect.

Enums
APIProxy	string Call a REST API.
BigQuery	string Connect to a BigQuery data warehouse.
DynamoDB	string Connect to a DynamoDB database.
MySQL	string Connect to a MySQL database.
Oracle	string Connect to an Oracle database.
Postgres	string Connect to a Postgres database.
Redshift	string Connect to a Redshift data warehouse.
Snowflake	string Connect to a Snowflake data warehouse.
SQLServer	string Connect to a SQLServer database.
SSH	string Create an SSH tunnel and execute a command line argument.

Queries

The business logic to be executed for the different request types.

Fields
identifiers	object(Identifier) (optional) Queries to retrieve additional data subject identifiers.
access[]	string or object(APIProxyQuery) (optional) Queries to execute an access request.
delete[]	string or object(APIProxyQuery) (optional) Queries to execute a deletion request.
optout[]	string or object(APIProxyQuery) (optional) Queries to execute an opt-out request.
test[]	object(APIProxyQuery) (optional) Required health check query for the APIProxy connector.

JSON Representation
{ "identifiers": { object(Identifier) }, "access": [ string ], "delete": [ string ] }

Identifier

The business logic to retrieve a configured identifier.

Fields
<identifier_name>[]	string Queries to retrieve additional data subject identifiers.

JSON Representation
{ "<identifier_name>": [string] }

Identifier Name

The name of the field in the configuration is the name of the identifier in DataGrail in "snake_case", e.g. User ID in DataGrail would be user_id in the configuration. Learn more about identifier setup.