Skip to main content

Backend on GKE

The current backend deployment path runs each collaborator as a Kubernetes workload in the shared GKE cluster.

Terraform owns the cloud infrastructure. GitHub Actions renders orphaned-wells-ui-server/deployment/kubernetes/backend.yaml with collaborator-specific values and applies it to GKE.

Architecture​

Terraform creates:

  • Shared GKE Autopilot cluster.
  • One Cloud Storage upload bucket per unique backend bucket name.
  • One global static IP per collaborator backend.
  • Optional test DNS records at <collaborator>-k8s-server.uow-carbon.org.
  • Primary backend DNS records at <collaborator>-server.uow-carbon.org.
  • kubernetes_deploy_targets, the JSON map consumed by GitHub Actions.

Kubernetes creates one namespace per collaborator, using the uow-<collaborator> naming pattern. Each namespace contains:

  • Deployment/backend
  • Service/backend
  • BackendConfig/backend-config
  • ManagedCertificate/backend-cert
  • FrontendConfig/backend-frontend-config
  • Ingress/backend
  • Secret/backend-runtime-env
  • Secret/backend-runtime-files
  • Secret/dockerhub-pull

Prerequisites​

Required local tools for manual operations:

  • gcloud
  • kubectl
  • terraform
  • jq
  • gh

Required Google APIs:

  • Kubernetes Engine API: container.googleapis.com
  • Compute Engine API: compute.googleapis.com
  • Cloud DNS API: dns.googleapis.com
  • Cloud Storage API: storage.googleapis.com

Other required access:

  • Ability to edit GitHub Secrets and Actions for the OGRRE GitHub project CATALOG-HISTORIC-RECORDS.
  • Permissions to manage GKE, Compute addresses, Cloud DNS, Cloud Storage buckets, OAuth, and project services in Google Cloud.

Prepare deployment targets​

Apply Terraform first:

cd orphaned-wells-ui-server/deployment/terraform
terraform init
terraform workspace select ogrre
terraform plan
terraform apply

Then export and store the target map. The gh command requires a GitHub account with permission to edit repository Actions secrets:

terraform workspace select ogrre
gh auth login
gh secret set K8S_DEPLOY_TARGETS \
  --repo CATALOG-Historic-Records/orphaned-wells-ui-server \
  --body "$(terraform output -json kubernetes_deploy_targets | jq -c .)"

The host field controls the Kubernetes Ingress host and the Google-managed certificate domain. The storage_bucket_name field controls the runtime STORAGE_BUCKET_NAME written to Secret/backend-runtime-env. DNS or bucket changes in Terraform are not enough by themselves; update K8S_DEPLOY_TARGETS and redeploy the backend.

Required backend secrets​

The GKE deployment workflows require:

  • PROJECT_ID
  • DOCKERHUB_USERNAME
  • DOCKERHUB_ACCESS_TOKEN
  • CREDS_JSON
  • SERVICE_KEY_JSON
  • K8S_DEPLOY_TARGETS
  • One runtime env secret per dispatch-supported collaborator, currently STAGING_ENV, CA_ENV, ISGS_ENV, NEWTS_ENV, OSAGE_ENV, and RRC_ENV

The environment-file secret should contain the same key/value pairs used by the VM .env file. The workflow overrides these Kubernetes-owned values:

  • ENVIRONMENT
  • BACKEND_URL
  • LOG_DIR
  • LOCAL_STORAGE_ROOT
  • LOCAL_STORAGE_URL_BASE
  • STORAGE_BUCKET_NAME

Keep COLLABORATOR in the runtime secret when the backend needs collaborator-specific processor metadata.

When adding a collaborator, update deploy-k8s-dispatch.yml with the new DEPLOY_ENV, <COLLABORATOR>_ENV secret, and runtime-env case branch before relying on a dedicated workflow. A dedicated deploy-k8s-<collaborator>.yml workflow calls the dispatch workflow and will fail until dispatch supports that collaborator.

Deploy with GitHub Actions​

The staging workflow builds and pushes the backend image, then deploys staging:

gh workflow run deploy-k8s-staging.yml \
  --repo CATALOG-Historic-Records/orphaned-wells-ui-server \
  --ref main

Collaborator workflows deploy an existing image tag, defaulting to latest:

gh workflow run deploy-k8s-isgs.yml --repo CATALOG-Historic-Records/orphaned-wells-ui-server --ref isgs
gh workflow run deploy-k8s-newts.yml --repo CATALOG-Historic-Records/orphaned-wells-ui-server --ref newts
gh workflow run deploy-k8s-osage.yml --repo CATALOG-Historic-Records/orphaned-wells-ui-server --ref osage
gh workflow run deploy-k8s-ca.yml --repo CATALOG-Historic-Records/orphaned-wells-ui-server --ref ca

The exact set of deployable collaborators is controlled by .github/workflows/deploy-k8s-dispatch.yml. Confirm the collaborator is present in workflow_dispatch.inputs.DEPLOY_ENV.options and in the runtime-env secret mapping before dispatching it.

Automatic deployments are controlled by repository variables:

ENABLE_GKE_DEPLOYMENTS=true

Or one collaborator at a time:

ENABLE_GKE_STAGING_DEPLOY=true
ENABLE_GKE_CA_DEPLOY=true
ENABLE_GKE_ISGS_DEPLOY=true
ENABLE_GKE_NEWTS_DEPLOY=true
ENABLE_GKE_OSAGE_DEPLOY=true

The current RRC workflow checks ENABLE_GKE_DEPLOYMENTS; add an ENABLE_GKE_RRC_DEPLOY workflow check before relying on an RRC-specific deploy variable.

Check rollout status​

Authenticate to the cluster:

gcloud container clusters get-credentials uow-backend-gke \
  --region us-central1 \
  --project <PROJECT_ID>

Check one backend:

kubectl -n uow-staging get deployment backend
kubectl -n uow-staging get pods -l app.kubernetes.io/name=orphaned-wells-ui-server -o wide
kubectl -n uow-staging get ingress backend
kubectl -n uow-staging get managedcertificate backend-cert

Read logs:

kubectl -n uow-staging logs deployment/backend --tail=200
kubectl -n uow-staging logs deployment/backend --tail=200 -f

Restart a backend deployment:

kubectl -n uow-staging rollout restart deployment/backend
kubectl -n uow-staging rollout status deployment/backend --timeout=10m

Check resource usage:

kubectl top pods -n uow-staging
kubectl top nodes

Notes​

  • GKE replaces VM nginx/certbot with GKE Ingress, ManagedCertificate, FrontendConfig, and BackendConfig.
  • Google-managed certificates require DNS to point at the GKE load balancer before they become active.
  • The rendered manifest in deployment/kubernetes/rendered/backend.yaml is an ephemeral artifact. Do not commit it.
  • The Kubernetes Deployment uses pod-local emptyDir volumes for /logs and /data. Production document storage should continue using the Terraform-managed Google Cloud Storage upload bucket.