Google Cloud Run

Overview

If you are hosting in Google Cloud Platform, DataGrail recommends using Google Cloud Run to deploy the Request Manager Agent. This guide covers deploying the egress-only Request Manager Agent, which initiates outbound connections to DataGrail and does not require incoming traffic or load balancing. Cloud Run simplifies the management of service uptime and networking, offering a reliable and streamlined deployment process. For more information about Cloud Run, refer to Google's documentation.

Sourcing the Agent Image

The Request Manager Agent Docker image is hosted in DataGrail's private image registry. Once you have obtained the credentials from your DataGrail representative, you can pull the image using the following command:

Pulling the image

# Authenticate with the DataGrail registry
docker login contairium.datagrail.io -u $DATAGRAIL_SUBDOMAIN

# Pull the Request Manager Agent image
docker pull contairium.datagrail.io/rm-agent:$VERSION

If you prefer to use a pull-through cache instead of hosting the image yourself, you create an Artifact Registry remote repository.

Quick Setup Guide

The following sections contain the core steps to creating an egress-only Cloud Run Agent service. Since the Agent only makes outbound connections and does not receive incoming traffic, no load balancer or TLS certificate configuration is required. Please note that depending on your GCP environment's pre-existing configuration, you may need to take additional steps to configure your VPC, subnets, etc. Those are not covered in this document but we are happy to provide you with any assistance we can offer.

Create a Service Identity

Cloud Run will use the Compute Engine default service account if you do not specify a user-managed service account to run the service under. This principal is granted the Editor role, which grants read and write access on all resources in your Google Cloud project. To follow the principle of least privilege, it is recommended that you create a new service account with the most minimal set of permissions.

1. In the Google Cloud console, navigate to IAM and click Service Account in the left-hand menu, and then Create Service Account in the top bar.
2. Under Service account details, enter the Service account name (e.g. rm-agent), and optionally a Service account description. The Service account ID will automatically be generated based on the Service account name.
3. In the Grant this service account access to project section, add at least the following three roles to the service account:
   1. Artifact Registry Service Agent - Pulling the Docker image from Artifact Registry.
   2. Secret Manager Secret Accessor - Accessing the various secrets used by the Agent.
   3. Storage Object Creator - Writing the results of an access request to Cloud Storage.

Create Cloud Run Service

1. In the Google Cloud console, navigate to Cloud Run and click Create Service at the top of the page to start the Service wizard.
2. Under Container Image URL, enter the container image URL of the uploaded Agent image.
3. Under Configure, enter a Service name and Region.
4. Under CPU Allocation and Pricing, select CPU is always allocated to ensure processing of background tasks.
   Important Setting

   If this step is missed, the Agent will not be able to process background tasks!

5. Under Service Autoscaling, set minimum number of instances to 1.
6. Under Ingress Control, select Internal (the Agent only requires outbound connectivity).
7. Expand the Container(s), Volumes, Networking, Security section, and select the Container tab.
8. Under Resources, set Memory to at least 4 Gib, and CPU to 1.
9. In the Variables & Secrets tab, select Add Variable and set the appropriate environment variables. The complete list of environment variables can be found in the Environment Variables documentation.
10. Under Requests, set the timeout to 300, and maximum concurrent requests to 80.
11. Under Revision Autoscaling, set both the minimum and maximum number of instances to 1 to avoid any automatic autoscaling actions.
12. With everything configured, click Create at the bottom of the wizard and your service will launch!

Your egress-only Agent service is now deployed and will initiate outbound connections to DataGrail and your configured systems!