Configuration

The Request Manager Agent’s behavior is dictated by environment variables set in the container. These variables define the target systems to connect to, the operations to perform, and other metadata to instruct actions.

To configure the Request Manager Agent properly, you will need to:

• Determine the systems to connect, their associated connectors, and the applicable business logic.
• Create and store credentials for:
  • Each connector.
  • The OAuth Client Credentials to authorize operations by the DataGrail application.
  • The callback token to authenticate requests to the DataGrail platform.
• Set the environment variable(s).

Setup

Connections

Take a look at the Connections directory for all available connections. Each connection will have its configuration set in the environment variable in the connections array. Example configurations for each connection is available in its respective document.

Credentials

A supported credentials manager will be used to store all access tokens, client credentials, connection strings, PEM keys, etc.

Connection Credentials

Each connection requires credentials to connect to the system. These credentials will be stored in a JSON format with the required key/value pairs of the connection. You can refer to the respective connection's documentation in the Connections directory to find the required credential parameters.

OAuth Client Credentials

DataGrail is authorized access to the Agent's resources using the OAuth Client Credentials grant type. DataGrail will use the resulting access token to authenticate subsequent requests.

The client_id and client_secret are arbitrary, customer-controlled values that can be rotated at any time. The raw contents of the credentials should be stored in JSON format with the following key/value pairs.

{
    "client_id": "<identifer for the credential e.g. 'datagrail'>",
    "client_secret": "<generated password>"
}

Callback Token

The Agent uses an API access token to authorize its requests with the DataGrail platform.

The token value will be provided by your DataGrail representative. The raw contents of the secret should be in JSON format with the following key/value pairs.

{
    "token": "<your DataGrail token>"
}

Environment Variables

The Agent's core functionality will be derived from the DATAGRAIL_AGENT_CONFIG environment variable. This variable will determine the systems the Agent connects to, where the system's credentials are stored, and the business logic to invoke on those systems for different request types.

Name
DATAGRAIL_AGENT_CONFIG	object(RMAgentConfig) JSON object that contains metadata about connectors, credentials, cloud storage, etc.

RMAgentConfig

Field
connections[]	object(Connection) (required) Defines the target systems, their capabilities and business logic.
customer_domain	string (required) Your DataGrail domain.
datagrail_agent_credentials_location	string (required) Location of the OAuth Client Credentials.
datagrail_credentials_location	string (required) Location of the Callback Token.
platform	object(Platform) (required) Credentials and cloud storage platforms used.
redis_url	string (optional) Used for multi-node deployments that need to share an external Redis instance.
job_timeout_seconds	integer (optional) Duration in seconds before a job times out. The default is 180 (3 minutes).

JSON Representation
{ "connections": [ object(Connection) ], "customer_domain": string, "datagrail_agent_credentials_location": string, "datagrail_credentials_location": string, "platform": { object(Platform) }, "redis_url": string, "job_timeout_seconds": integer }

Cloud Provider Credentials

When not using role-based access controls, the following cloud provider-specific environment variables need to be set.

Amazon Web Services

Name	Value
AWS_ACCESS_KEY_ID	AWS access key associated with an IAM account.
AWS_SECRET_ACCESS_KEY	Secret key associated with the access key. This is essentially the "password" for the access key.
AWS_REGION	The AWS Region to send the request to.

Google Cloud Platform

Name	Value
GOOGLE_APPLICATION_CREDENTIALS_JSON	Extracted Google credentials file JSON.

Microsoft Azure

Name	Value
AZURE_TENANT_ID	The Azure Active Directory tenant (directory) ID.
AZURE_CLIENT_ID	The client (application) ID of an App Registration in the tenant.
AZURE_CLIENT_SECRET	A client secret that was generated for the App Registration.