Request Manager Agent Quick Start
This guide provides a streamlined approach to deploying the Request Manager Agent, configuring it to connect to your internal systems, and processing DSRs with minimal setup.
Deploy the Agent
The first half of the guide focuses on configuring and deploying the Request Manager Agent in your environment before connecting it to any internal systems.
Before performing the steps below, ensure you have the following:
- Access to the DataGrail platform.
- DataGrail image registry credentials.
- Privileges to create and manage containerized workloads in your cloud environment.
Create a Request Manager Agent
- In the DataGrail platform, under Integration Network, select the Agents tab.
- If you don't have any existing Agents, select the Add New Agent button in the middle of the screen, otherwise select the button in the top right corner.
- In the modal that appears, in the Agent Name field, enter a name for your Agent (e.g., "AWS Production").
- For the Agent Type, select Request Manager.
- Select Add New Agent.
Generate an API Key
The Request Manager Agent requires an API key to authenticate with the DataGrail platform. This key is generated in DataGrail and securely stored in your credentials manager. Each API key is associated with a specific Agent. If you intend to use multiple Agents, you must generate a unique API key for each one.
- On the page of the Agent you just created, select Generate New API Key.
- In the modal that appears, optionally overwrite the default API Key Name.
- Select Copy to copy the generated API key to your clipboard. Temporarily store this key in a secure location, as you will need it in the next steps.
- Select Save API Key Name to close the modal.
Store your API Key
The API key you generated must be securely stored in your credentials manager. This ensures that the Agent can access it during runtime without exposing sensitive information. The list of supported credentials managers can be found in the CredentialsManagerProvider documentation.
In your credentials manager of choice, create a new secret with the following JSON structure, replacing the placeholder value with the API key you generated in the previous step.
{
"token": "privacy_api_key..."
}
For detailed instructions on securely managing credentials using your preferred credentials manager, refer to the Storing Credentials Guide.
Define the Environment Variables
The Request Manager Agent requires several environment variables to be configured before deployment. These variables define how the Agent connects to your credentials manager and cloud storage. Take note of the following variables you'll need, as you'll apply them when you run the Agent in the next step.
| Name | Description | Default |
|---|---|---|
| RM_CUSTOMER_DOMAIN | string (required) Your DataGrail domain (e.g. yourcompany.datagrail.io). | None |
| RM_PLATFORM_CREDENTIALS_LOCATION | string (required) Secret location for platform API key (e.g. AWS Secrets Manager ARN). | None |
| RM_CREDENTIALS_MANAGER | object(CredentialsManager) (required) JSON string describing the credentials provider. | None |
| RM_STORAGE_MANAGER | object(StorageManager) (optional) JSON string describing the storage provider. | None |
| RM_REDIS_URL | string (optional) Connection string for a remote Redis instance. | redis://localhost:6379 |
| RM_JOB_TIMEOUT_SECONDS | integer (optional) Max time (seconds) for a single job before timeout. | 3600 |
| LOGLEVEL | string (optional) Logging level. | WARNING |
Source the Image
Request Manager Agent images are distributed via DataGrail's self-hosted Docker registry. Accessing the images requires basic authentication credentials provided by your DataGrail representative.
# Authenticate with the DataGrail registry
docker login contairium.datagrail.io -u $DATAGRAIL_SUBDOMAIN
# Pull the Request Manager Agent image
docker pull contairium.datagrail.io/rm-agent:$VERSION
If you prefer to use a pull-through cache instead of hosting the image yourself, you can set up your container orchestration platform's deployment manifest to fetch the image directly during the deployment process.
Run the Agent
The Request Manager Agent can be deployed using any container orchestration platform, such as Google Cloud Run, Amazon ECS, Azure Container Apps, or Kubernetes.
Refer to the specific deployment guides for detailed instructions on running the Agent on your chosen platform:
Connect to DataGrail
Once the Agent is deployed and running, it will automatically connect to DataGrail and begin polling for work. You can verify that the connection is successful by either checking the Agent module in the DataGrail platform or the Agent's logs in your container orchestration platform. Look for log entries indicating that the Agent has started successfully and is polling for tasks.
Connect to Data Sources
With the Agent successfully running and connected to DataGrail, the next step is to connect it to your internal systems. This process involves creating an integration in the DataGrail platform. Refer to the integration's documentation for detailed instructions on connecting the Agent to your internal systems:
Disclaimer: The information contained in this message does not constitute as legal advice. We would advise seeking professional counsel before acting on or interpreting any material.