Environment Variable

After creating the Agent and API key in DataGrail, you're ready to configure the environment variable used by the agent container.

Each container requires an DATAGRAIL_AGENT_CONFIG environment variable, which is set to a JSON object:

{
  "customer_domain": "yoursubdomain.datagrail.io",
  "datagrail_credentials_location": "<DG API Key secret location>",
  "platform": {
    "credentials_manager": {
      "provider": "<AWSSSMParameterStore|AWSSecretsManager|JSONFile|GCP|AzureKeyVault>",
      "options": {
        "optional": "some modules may have additional required fields",
      }
    }
  }
}

Example Environment Variable

In this example, the environment variable is configured to use AWS Secrets Manager:

{
  "customer_domain": "example.datagrail.io",
  "datagrail_credentials_location": "arn:aws:secretsmanager:us-west-2:1234:secret:RDD_API_KEY",
  "platform": {
    "credentials_manager": {
      "provider": "AWSSecretsManager"
    }
  }
}

Schema

Name
DATAGRAIL_AGENT_CONFIG	object(RDDAgentConfig) JSON object that contains metadata about connectors, credentials, cloud storage, etc.

RDDAgentConfig

Field
customer_domain	string (required) Your DataGrail domain.
datagrail_credentials_location	string (required) Location of the DataGrail API Key.
platform	object(Platform) (required) Credentials manager used.

JSON Representation
{ "customer_domain": string, "datagrail_credentials_location": string, "platform": { object(Platform) } }

---

Platform

Fields
credentials_manager	object(CredentialsManager) (required) Credentials manager settings.

JSON Representation
{ "credentials_manager": { object(CredentialsManager) } }

---

CredentialsManager

Fields
provider	enum(CredentialsManagerProvider) (required) The credentials manager provider.
options	object(CredentialsManagerOptions) (optional) Metadata about the credentials manager.

JSON Representation
{ "provider": enum(CredentialsManagerProvider), "options": { object(CredentialsManagerOptions) } }

---

CredentialsManagerProvider

Enums
AWSSSMParameterStore	Amazon Web Services Parameter Store.
AWSSecretsManager	Amazon Web Services Secrets Manager.
AzureKeyVault	Microsoft Azure Key Vault.
GCP	Google Secret Manager.
JSONFile	JSON file for secrets injection.

---

CredentialsManagerOptions

Fields
secret_vault	string (optional) Specifies the vault name. Required when using AzureKeyVault.
project_id	string (optional) Specifies the project ID. Required when using GCP.
filename	string (optional) Specifies the file location/name. Required when using JSONFile.

JSON Representation
{ "secret_vault": string, "project_id": string, "filename": string }

---

Integration Variables

BigQuery

Optional environment variables for BigQuery integrations:

Name
MAX_BIGQUERY_BLOCKS	integer (default: 5) Maximum number of blocks used in TABLESAMPLE. We recommend no more than 10.