Skip to main content

AI-driven database scaling with Crossplane

In this tutorial, you deploy an AI controller that manages an AWS RDS database. A CronOperation runs every minute. It reads live CloudWatch metrics from the database object, calls Claude, and decides whether to scale. If it scales, it writes its reasoning back to the object as an annotation.

No Go. No custom operator. No build pipeline. The controller is a single YAML file.

By the end of this tutorial, you can:

  • See live CloudWatch metrics surfaced directly on a Crossplane SQLInstance object
  • Deploy an AI scaling controller with a single kubectl apply
  • Read the model's reasoning from the Kubernetes object it acted on
  • Trigger a load test and watch the AI decide to scale up in real time

Prerequisites​

Install the following tools before starting:

Install the up CLI​

curl -sL "https://cli.upbound.io" | sh
sudo mv up /usr/local/bin/

If you don't have sudo access:

mkdir -p ~/.local/bin && mv up ~/.local/bin/
export PATH="$HOME/.local/bin:$PATH"

Add the export line to your shell profile (~/.bashrc, ~/.zshrc, etc) to make it permanent.

Install mysqlslap​

The load test in this tutorial uses mysqlslap, which ships with the MySQL client tools.

macOS:

brew install mysql-client
export PATH="$(brew --prefix mysql-client)/bin:$PATH"

Linux (Debian/Ubuntu):

apt-get install -y mysql-client

Clone the project​

git clone https://github.com/upbound/configuration-aws-database-ai demo
cd demo

All commands from this point run from inside the demo directory.

Configure credentials​

Export your AWS credentials and Anthropic API key. The setup steps below use these values to create Kubernetes secrets.

export AWS_ACCESS_KEY_ID=<your-access-key-id>
export AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
export ANTHROPIC_API_KEY=<your-anthropic-api-key>

Start the project​

Open a dedicated terminal and run from inside the demo directory:

up project run --local --ingress

This command:

  • Creates a kind cluster
  • Installs UXP
  • Builds and deploys the composition functions (function-rds-metrics and function-claude)
  • Installs the AWS providers declared in upbound.yaml
  • Applies the XRDs from apis/
  • Installs an ingress controller for the UXP console

Startup takes several minutes. The command exits when the cluster is ready.

warning

If up project run --local prints traces export: context deadline exceeded in stderr, check the provider status:

kubectl get providers

If providers appear, provisioning succeeded despite the telemetry error. If the list is empty, delete the cluster and retry:

kind delete cluster --name up-$(basename "$PWD")

Verify your network allows outbound connections to xpkg.upbound.io on port 443.

Configure kubectl​

In your second terminal, point kubectl at the new cluster. up project run --local names the cluster after the project directory:

CLUSTER_NAME=$(kind get clusters | grep "^up-" | head -1)
kind get kubeconfig --name "${CLUSTER_NAME}" > ~/.kube/config
warning

This overwrites your existing ~/.kube/config. To preserve existing contexts, merge instead:

kind get kubeconfig --name "${CLUSTER_NAME}" > ~/.kube/config-upbound
KUBECONFIG=~/.kube/config:~/.kube/config-upbound \
  kubectl config view --flatten > ~/.kube/config.merged
mv ~/.kube/config.merged ~/.kube/config

Verify the connection:

kubectl get nodes

Create the namespace and apply credentials​

  1. Create the database-team namespace:

    kubectl apply -f examples/ns-database-team.yaml

  2. Create the AWS credentials secret. The ProviderConfig and the function-rds-metrics function both read from this secret:

    kubectl create secret generic aws-creds \
      --namespace database-team \
      --from-literal=credentials="$(printf '[default]\naws_access_key_id = %s\naws_secret_access_key = %s\n' \
        "$AWS_ACCESS_KEY_ID" "$AWS_SECRET_ACCESS_KEY")" \
      --dry-run=client -o yaml | kubectl apply -f -

    kubectl create secret generic aws-creds \
      --namespace crossplane-system \
      --from-literal=credentials="$(printf '[default]\naws_access_key_id = %s\naws_secret_access_key = %s\n' \
        "$AWS_ACCESS_KEY_ID" "$AWS_SECRET_ACCESS_KEY")" \
      --dry-run=client -o yaml | kubectl apply -f -

  3. Create the Anthropic API key secret used by function-claude:

    kubectl create secret generic claude \
      --namespace crossplane-system \
      --from-literal=ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY}" \
      --dry-run=client -o yaml | kubectl apply -f -

Verify providers and functions​

Wait for both AWS providers and both functions to become healthy:

kubectl get providers
kubectl get functions

All four should show HEALTHY: True before continuing.

warning

If kubectl get providers or kubectl get functions returns No resources found, up project run --local didn't complete successfully. Delete the cluster and restart from Start the project.

Apply the ProviderConfig​

kubectl apply -f examples/providerconfig-aws-static.yaml

Provision the network​

kubectl apply -f examples/network-rds-metrics.yaml

Wait for the network composite resource to become ready (~5 minutes):

kubectl get network rds-metrics-database-ai-scale -n database-team -w

Wait until READY: True. Press Ctrl+C when it does.

Provision the database​

kubectl apply -f examples/mariadb-xr-rds-metrics.yaml

RDS provisioning takes 10 to 15 minutes. Watch the status:

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team -w

Wait until READY: True before continuing. Press Ctrl+C when it does.

info

While waiting, the function-rds-metrics composition step is already collecting CloudWatch data and writing it onto the object. By the time the database is ready, status.performanceMetrics will have live data.

Access the UXP console​

  1. Enable the web UI:

    up uxp web-ui enable

  2. In a new terminal, port-forward to the service:

    kubectl port-forward -n crossplane-system svc/webui 8080:80

  3. Open http://localhost:8080 in your browser.

Meet the database​

An RDS MariaDB instance is running on AWS, managed by Crossplane. Before wiring the AI into the loop, explore what the system already knows.

See the database object​

kubectl get sqlinstance -n database-team

You should see rds-metrics-database-ai-mysql with READY: True. That's a real AWS RDS instance, managed as a Kubernetes object.

In the UXP console, click View all Composite Resources. You'll see rds-metrics-database-ai-mysql listed. Click Relationship View to see the resources Crossplane provisioned.

Verify the AWS resource​

In the AWS Console, RDS in us-west-2 find rds-metrics-database-ai-mysql.

The Kubernetes object and the AWS resource are the same thing. Crossplane is the bridge.

Find the performance metrics​

kubectl describe sqlinstance rds-metrics-database-ai-mysql -n database-team

Find the status.performanceMetrics block. This block contains live CloudWatch data like CPU utilization, active connections, free storage. function-rds-metrics collects this data and writes it into the object.

The AI can only access this context. It never touches CloudWatch directly. The control plane is the authoritative source of state for both humans and the AI.

Or fetch just the metrics:

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.status.performanceMetrics}' | jq .

Open the controller​

Open operations/rds-intelligent-scaling-cron/operation.yaml in your editor.

That file is the entire scaling controller. The systemPrompt defines the scaling logic like thresholds, instance class progression, cooldown.

Apply the controller​

kubectl apply -f operations/rds-intelligent-scaling-cron/operation.yaml

Watch the first decision​

kubectl get cronoperation

It takes 30–45 seconds to start. Once running, watch for the first operation:

kubectl get operations -w

Wait until an operation shows SUCCEEDED: True, then press Ctrl+C and describe it:

kubectl describe operation <name>

Look at the Events section. That's the AI's output — its reasoning about whether to scale, and what it decided.

Then check the annotation written back to the database object:

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.metadata.annotations}' | jq .

The AI's reasoning is on the object. Not a black box — the control plane is the system of record for every decision the AI made.

In the UXP console, navigate to rds-metrics-database-ai-mysql and open the YAML tab. You'll see the intelligent-scaling/last-scaled-decision annotation with the model's last decision.

Read the room​

The CronOperation runs every minute. CPU is low right now. Watch what the AI decides when there's nothing to do — and understand exactly what it sees.

Watch operations fire​

kubectl get operations -w

A new operation appears roughly every minute. Press Ctrl+C after a few have run.

In the UXP console, select Operations in the left navigation to see the same list visually.

Read a decision​

Pick one of the operation names and describe it:

kubectl describe operation <name>

Look at the Events section. At low CPU, the AI should decide to hold. The cooldown logic is also in the prompt — it won't flip the instance class every minute even if thresholds are crossed.

See the current metrics​

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.status.performanceMetrics}' | jq .

This is exactly what the AI sees before making a decision. Live data, on the object.

See the current instance class​

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.spec.parameters.instanceClass}'

It's db.t3.micro. That's about to change.

You can also confirm the current instance type in the AWS Console, RDS in us-west-2.

Trigger a scale​

Time to put the controller under pressure. A load test drives CPU above the scaling threshold and the AI decides to act.

Confirm the starting instance class​

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.spec.parameters.instanceClass}'

It should be db.t3.micro.

Run the load test​

In a second terminal, run the load test from inside the demo directory:

bash perf-scale-demo.sh

This hammers the database with CPU-intensive queries. The script takes 5–10 minutes. If it finishes without triggering a scale, run it again.

Watch CPU climb​

In your first terminal, watch the metrics update every 10 seconds:

watch -n 10 "kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.status.performanceMetrics.metrics}' | jq ."

Watch the controller fire​

Press Ctrl+C to exit the watch command, then:

kubectl get operations -w

When CPU crosses the threshold (~60%), the next CronOperation will decide to scale up. Press Ctrl+C once you see a new operation start.

See the scale event​

Check the instance class:

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.spec.parameters.instanceClass}'

It should now be db.t3.small. Check the reasoning:

kubectl get sqlinstance rds-metrics-database-ai-mysql -n database-team \
  -o jsonpath='{.metadata.annotations.intelligent-scaling/last-scaled-decision}'

In the AWS Console, RDS in us-west-2, refresh the database list. The instance class change is in progress — RDS is modifying the live database. No Terraform. No manual AWS operation. The platform handled it.

The AI read the metrics, crossed the threshold, picked the next instance class, and wrote its reasoning to the object. The control plane made the call.

Clean up​

Delete the composite resources. Crossplane deletes all composed AWS resources (VPC, subnets, RDS instance) before removing the composite resources.

kubectl delete sqlinstance rds-metrics-database-ai-mysql -n database-team
kubectl delete network rds-metrics-database-ai-scale -n database-team

RDS deletion takes 5–10 minutes. Wait until the sqlinstance is fully removed:

kubectl get sqlinstance -n database-team -w

Once it's gone, delete the CronOperation and its history:

kubectl delete cronoperation rds-intelligent-scaling-cron
kubectl delete operations --all

Stop up project run with Ctrl+C in that terminal, then delete the cluster:

CLUSTER_NAME=$(kind get clusters | grep "^up-" | head -1)
kind delete cluster --name "${CLUSTER_NAME}"

Next steps​

In this tutorial, you:

  • Provisioned a real AWS RDS instance managed as a Crossplane SQLInstance
  • Observed live CloudWatch metrics surfaced directly on the Kubernetes object
  • Deployed an AI scaling controller with a single kubectl apply
  • Read the model's reasoning from the annotation it wrote back to the object
  • Ran a load test and watched the AI scale the database automatically

Continue with: