Skip to main content

Environment Variables

The Request Manager Agent uses environment variables to configure its runtime behavior and connections. You'll need to set variables for your DataGrail domain, Agent API Key, credentials manager provider, and optionally cloud storage for handling sensitive data. This page documents all required and optional environment variables, including cloud provider credentials for local development.

Variables

NameDescriptionDefault
RM_CUSTOMER_DOMAINstring (required)
Your DataGrail domain (e.g. yourcompany.datagrail.io).
None
RM_PLATFORM_CREDENTIALS_LOCATIONstring (required)
Secret location for platform API key (e.g. AWS Secrets Manager ARN).
None
RM_CREDENTIALS_MANAGERobject(CredentialsManager) (required)
JSON string describing the credentials provider.
None
RM_STORAGE_MANAGERobject(StorageManager) (optional)
JSON string describing the storage provider.
None
RM_REDIS_URLstring (optional)
Connection string for a remote Redis instance.
redis://localhost:6379
RM_JOB_TIMEOUT_SECONDSinteger (optional)
Max time (seconds) for a single job before timeout.
3600
LOGLEVELstring (optional)
Logging level.
WARNING

CredentialsManager Schema

The CredentialsManager object defines metadata about the credentials manager used by the Agent. A credentials manager is used to store client credentials, connection strings, PEM keys, API tokens, etc. that authenticate the Agent's various requests.

FieldsDescription
providerenum(CredentialsManagerProvider) (required)
The credentials manager provider.
optionsobject(CredentialsManagerOptions) (optional)
Metadata about the credentials manager.

CredentialsManagerProvider

The CredentialsManager field specifies which credentials management provider the Agent connects to for secure retrieval of secrets.

ProvidersDescriptionRequired Options
AWSSSMParameterStoreAmazon Web Services Parameter Store.-
AWSSecretsManagerAmazon Web Services Secrets Manager.-
AzureKeyVaultMicrosoft Azure Key Vault.secret_vault
GCPGoogle Secret Manager.project_id
JSONFileJSON file for secrets injection.filename

CredentialsManagerOptions

The CredentialsManagerOptions field contains configuration settings for your chosen credentials manager. Required settings vary by provider.

FieldsDescription
secret_vaultstring (optional)
Specifies the Azure Key Vault name.
project_idstring (optional)
Specifies the Google Cloud project ID.
filenamestring (optional)
Specifies the credentials file location/name.
Example CredentialsManager Environment Variable Values
# AWS Secrets Manager
RM_CREDENTIALS_MANAGER='{"provider": "AWSSecretsManager"}'

# AWS SSM Parameter Store
RM_CREDENTIALS_MANAGER='{"provider": "AWSSSMParameterStore"}'

# GCP Secret Manager
RM_CREDENTIALS_MANAGER='{"provider": "GCP", "options": {"project_id": "my-project-123"}}'

# Azure Key Vault
RM_CREDENTIALS_MANAGER='{"provider": "AzureKeyVault", "options": {"secret_vault": "my-vault-name"}}'

# Local JSON file
RM_CREDENTIALS_MANAGER='{"provider": "JSONFile", "options": {"filename": "/path/to/secrets.json"}}'

StorageManager Schema

The StorageManager configuration is optional. When provided, the Agent will upload access request and identifier retrieval results to the specified cloud storage bucket, keeping Personally Identifiable Information (PII) within your private network. DataGrail will then retrieve and process the files from your bucket. When StorageManager is not provided, the Agent will send the data back inline for immediate processing by DataGrail.

FieldsDescription
providerenum(StorageManagerProvider) (required)
The cloud storage manager provider.
optionsobject(StorageManagerOptions) (optional)
Metadata about the storage manager. Required for some providers.

StorageManagerProvider

The StorageManagerProvider field specifies which cloud storage manager provider the Agent connects to for secure upload of access request and identifier retrieval results.

ProviderDescriptionRequired Options
AWSS3string
Amazon Web Services S3
bucket
AzureBlobstring
Microsoft Azure Blob
bucket, project_id
BackblazeB2string
Backblaze B2
bucket, endpoint, region
GCPCloudStorestring
Google Cloud Storage
bucket, project_id

StorageManagerOptions

The StorageManagerOptions field contains configuration settings for your chosen storage manager. Required settings vary by provider.

Fields
bucketstring (required)
The name of the cloud storage bucket
project_idstring (optional)
Required when using AzureBlob or GCPCloudStore to denote the project ID.
regionstring (optional)
Required when using BackblazeB2 to denote the region.
endpointstring (optional)
Required when using BackblazeB2 to denote the endpoint.
# AWS S3
RM_STORAGE_MANAGER='{"provider": "AWSS3", "options": {"bucket": "my-results-bucket"}}'

# GCP Cloud Storage
RM_STORAGE_MANAGER='{"provider": "GCPCloudStore", "options": {"project_id": "my-project-123", "bucket": "my-results-bucket"}}'

# Azure Blob Storage
RM_STORAGE_MANAGER='{"provider": "AzureBlob", "options": {"project_id": "mystorageaccount", "bucket": "my-container"}}'

# Backblaze B2
RM_STORAGE_MANAGER='{"provider": "BackblazeB2", "options": {"bucket": "my-bucket", "region": "us-west-002", "endpoint": "https://s3.us-west-002.backblazeb2.com"}}'

Cloud Provider Credentials

Set these environment variables when role-based access controls are unavailable, such as in local development environments.

Recommended Approach

Use role-based access controls instead of long-lived credentials to avoid storing secrets in plain text.

Amazon Web Services

NameDescription
AWS_ACCESS_KEY_IDAWS access key associated with an IAM account.
AWS_SECRET_ACCESS_KEYSecret key associated with the access key. This is essentially the "password" for the access key.
AWS_REGIONThe AWS Region to send the request to.

Google Cloud Platform

NameDescription
GOOGLE_APPLICATION_CREDENTIALS_JSONExtracted Google credentials file JSON.

Microsoft Azure

NameDescription
AZURE_TENANT_IDThe Azure Active Directory tenant (directory) ID.
AZURE_CLIENT_IDThe client (application) ID of an App Registration in the tenant.
AZURE_CLIENT_SECRETThe client secret for the App Registration.

 

Need help?
If you have any questions, please reach out to your dedicated Account Manager or contact us at support@datagrail.io.

Disclaimer: The information contained in this message does not constitute as legal advice. We would advise seeking professional counsel before acting on or interpreting any material.