I set up a GKE Autopilot cluster in Singapore (asia-southeast1), ran a small app on it, and put it through a deploy, a rolling update, autoscaling and self-healing. Then I broke it on purpose in two ways, and a third problem turned up on its own. I tracked down each one with kubectl. Once it was all recorded I deleted the cluster, so this page costs nothing to keep online.
Picture a restaurant kitchen. The manager doesn't stand over every cook. They say "always have three plates of this ready", and if a cook goes home sick, someone else steps in. When the lunch rush hits, more cooks come in.
Kubernetes does that job for software. You write down what should be running (the desired state), and it keeps checking and fixing things until reality matches. It restarts apps that crash, moves them off broken machines, and adds or removes copies as needed.
02
The building blocks
Words you'll see on the rest of this page.
Container
An app packed together with everything it needs to run, so it works the same on a laptop as it does in the cloud.
Pod
The smallest thing Kubernetes runs. Usually one container, sometimes a few that need to sit together. Pods are throwaway: when one breaks, Kubernetes replaces it instead of fixing it.
Node
A machine that runs pods. With Autopilot, Google looks after these for you.
Cluster
All the nodes together, plus the control plane that manages them.
Control plane
The part that makes the decisions: which pod goes on which node, and what to do when something drifts from the plan. In the kitchen picture, this is the manager.
Deployment
Where you say which app to run and how many copies to keep. It also handles rolling updates, swapping old pods for new ones a few at a time.
Service
One fixed address for a group of pods. Pods come and go, but traffic always knows where to find them.
Ingress
Rules for traffic coming in from the internet: which web address or path goes to which Service.
NetworkPolicy
Decides which pods are allowed to talk to which, both coming in (ingress) and going out (egress). There's no "Egress" object to match Ingress; this is where outgoing traffic gets controlled. Without one, any pod can reach anything.
HorizontalPodAutoscaler (HPA)
Watches the load, such as CPU use, and adds or removes pods to match. The extra cooks at lunchtime.
Namespace
A way to split one cluster into separate areas, so different apps or teams don't trip over each other.
Requests & limits
Requests are how much CPU and memory a pod asks for, which decides where it can fit. Limits are the most it's allowed to use. Use too much memory and the pod gets killed (OOMKilled). Ask for more than there's room for and it waits (Pending).
kubectl
The command-line tool for talking to a cluster. I used it to apply changes, check status and read logs.
03
Kubernetes vs GKE
Kubernetes is the open-source software. GKE (Google Kubernetes Engine) is Google running it for you. Google keeps the control plane up and patched, and the cluster comes connected to the rest of Google Cloud: load balancers, IAM permissions, logging and monitoring, and Artifact Registry for container images.
AWS has EKS and Azure has AKS, which do the same job. You can also run Kubernetes yourself with tools like kubeadm, but then every upgrade and outage is on you.
04
GKE Standard vs Autopilot
GKE comes in two modes. The main difference is who looks after the machines underneath.
Aspect
Standard
Autopilot
Who manages the nodes
You, with help from Google
Google
Billing model
You pay for the machines, even when they sit idle
You pay mostly for the CPU, memory and storage your pods ask for
Control
You pick the machine types and node pools, and can SSH in
Google picks the machines and scales them
Security defaults
Up to you
Locked down from the start: no SSH into nodes, privileged pods restricted
Resource requests
Optional
Google fills in defaults if you leave them out
Standard makes sense if you need a say in the hardware or direct access to the machines.
Autopilot makes sense if you'd rather not manage machines at all and just want your apps to run.
This website runs on Cloud Run, not GKE, so it's worth saying why.
Cloud Run
You give it a container and it runs it. There's no cluster to look after, and it scales down to zero when nobody is visiting. For a site like this one, that's all I need.
GKE
Worth it when you need Kubernetes itself: lots of services working together, custom networking, background jobs or apps that keep data, and tools from the Kubernetes world like Helm.
So I ran GKE long enough to record the demo, then deleted the cluster. This site stays on Cloud Run.
06
What I built
One Autopilot cluster in Singapore (asia-southeast1). The app is Google's public sample image hello-app (version 1.0, later 2.0), run as a Deployment called hello with 3 replicas. Each pod asks for 250m CPU and 512Mi memory.
A Service of type LoadBalancer gets a public IP from a Google Cloud load balancer and passes the traffic on to the pods. I didn't use an Ingress. A HorizontalPodAutoscaler keeps the Deployment between 3 and 10 replicas, aiming for 50% CPU.
flowchart LR
User["Browser"] --> LB["Google Cloud load balancer"]
LB --> Svc["Service (LoadBalancer)"]
subgraph Cluster["GKE Autopilot cluster (asia-southeast1)"]
Svc --> Dep["Deployment"]
Dep --> P1["Pods"]
HPA["HorizontalPodAutoscaler"] -.-> Dep
end
Why Autopilot
Two things I saw in the console. On the create-cluster page, the default Standard cluster showed an estimate of $327.93 a month if left running. I only looked and didn't create it. Then the dialog for switching to Autopilot said Google recommends Autopilot for most workloads. I went with Autopilot.
The default Standard cluster form: $327.93 a month estimated.The switch dialog: Google recommends Autopilot for most workloads.
Autopilot bills for what the pods request. The Clusters page showed 0.75 vCPU and 1.5 GB while my 3 pods were running, which is exactly 3 × 250m CPU and 3 × 512Mi.
The Clusters page: 0.75 vCPU and 1.5 GB for 3 pods.
07
The demo, step by step
For each step: what I ran, a trimmed copy of the real output, and a screenshot from the console where I took one.
1. Deploy and load balancing
One command applied the Deployment and the Service. The pods waited in Pending for a short while before they started (the story is in section 08). Then the Service got a public IP. I sent six requests to it and all three pods answered, so the load balancer was spreading the traffic.
$ kubectl apply -f k8s/01-deployment.yaml -f k8s/02-service.yaml
deployment.apps/hello created
service/hello created
$ kubectl get svc hello
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello LoadBalancer 34.118.238.145 35.247.x.x 80:32361/TCP 109s
$ for i in 1 2 3 4 5 6; do curl -s http://35.247.x.x; echo; done
(output trimmed to requests 3 to 5 of 6)
Hello, world!
Version: 1.0.0
Hostname: hello-56677d9698-5jgrc
Hello, world!
Version: 1.0.0
Hostname: hello-56677d9698-v774l
Hello, world!
Version: 1.0.0
Hostname: hello-56677d9698-48prd
Workloads: hello, 3/3 pods, node type Autopilot-managed.Services: hello is an external load balancer in front of 3/3 pods.
2. Rolling update, 1.0 to 2.0
I changed the image from hello-app:1.0 to 2.0 in the Deployment and applied the file again. Kubernetes replaced the pods one at a time. Afterwards every response said Version: 2.0.0.
$ kubectl apply -f k8s/01-deployment.yaml
deployment.apps/hello configured
$ kubectl rollout status deployment/hello --timeout=600s
...
Waiting for deployment "hello" rollout to finish: 1 old replicas are pending termination...
deployment "hello" successfully rolled out
$ for i in 1 2 3; do curl -s http://35.247.x.x; echo; done (first response only)
Hello, world!
Version: 2.0.0
Hostname: hello-6996d777cf-sbq6c
3. Scaling with the HPA
The HPA aims for 50% CPU, between 3 and 10 replicas. To make traffic I used a busybox load generator Deployment and added more copies of it. With 4 generators, CPU levelled off around 34 to 37%. With 10 it hovered around 48 to 52%, which is within the HPA's tolerance, so nothing changed. With 25 it reached 55%, then 69%, and the HPA went from 3 to 6 replicas. CPU then settled at 46%. The nodes grew to 4. After I stopped the traffic and the default 5-minute stabilisation window passed, it scaled back from 6 to 3.
Before: 3 ready replicas.After: 6 ready replicas, CPU climbing.
4. Self-healing
I deleted one pod on purpose. The Deployment noticed and started a replacement. In the next listing the new pod was already Running at 3 seconds old, and the count was back to 3.
$ kubectl delete pod hello-6996d777cf-4vtlv --wait=false
pod "hello-6996d777cf-4vtlv" deleted from default namespace
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-6996d777cf-ngvvg 1/1 Running 0 3s
hello-6996d777cf-rmsfs 1/1 Running 0 22s
hello-6996d777cf-sbq6c 1/1 Running 0 32s
08
Breaking it on purpose
I caused two failures on purpose, CrashLoopBackOff and OOMKilled. The third, Pending, I didn't cause. It showed up right after the first deploy and Autopilot fixed it by itself. For each one I worked out what went wrong with kubectl describe pod and kubectl logs --previous. I left the two broken pods as they were until clean-up.
CrashLoopBackOff (on purpose)
payments-worker starts, finds no DATABASE_URL setting and exits with code 1. Kubernetes keeps restarting it, with longer waits each time. The status says it is looping, and the logs from the previous run say why.
$ kubectl get pods payments-worker report-builder
NAME READY STATUS RESTARTS AGE
payments-worker 0/1 CrashLoopBackOff 1 (9s ago) 12s
$ kubectl logs payments-worker --previous
Starting payments-worker v1.4.2
Connecting to database...
ERROR: DATABASE_URL is not set
$ kubectl describe pod payments-worker | sed -n '/^Events:/,$p' (events trimmed to the last line)
Warning BackOff 9s (x2 over 10s) kubelet spec.containers{worker}: Back-off restarting failed container worker in pod payments-worker_default(81a14308-31da-49c7-a8a5-e9fb21573a6c)
The console shows the same thing: Error with exit code 1.
OOMKilled (on purpose)
report-builder has a memory limit of 256Mi and tries to hold 600 MB. It gets killed. describe shows the reason and exit code 137.
$ kubectl get pods payments-worker report-builder
NAME READY STATUS RESTARTS AGE
report-builder 0/1 OOMKilled 0 11s
$ kubectl describe pod report-builder | sed -n '/State:/,/Restart Count/p'
State: Terminated
Reason: OOMKilled
Exit Code: 137
Started: Wed, 14 Oct 2026 02:34:56 +0800
Finished: Wed, 14 Oct 2026 02:35:01 +0800
Ready: False
Restart Count: 0
The console: OOMKilled with exit code 137, plus an out-of-memory warning.
Pending (not on purpose)
Right after kubectl apply, all 3 hello pods were Pending. The event says the scheduler had 1 node and it carried a taint the pods don't tolerate. Nothing was wrong with my YAML. The cluster had no room for the pods yet, so Autopilot added 2 new nodes and the pods became Running in about 1.5 minutes. I only had to wait.
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
hello-56677d9698-24km5 0/1 Pending 0 5s <none> <none> <none> <none>
hello-56677d9698-5jgrc 0/1 Pending 0 5s <none> <none> <none> <none>
hello-56677d9698-v774l 0/1 Pending 0 5s <none> <none> <none> <none>
$ kubectl describe pod/hello-56677d9698-24km5 | sed -n '/^Events:/,$p'
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 16s gke.io/optimize-utilization-scheduler 0/1 nodes are available: 1 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
gk3-gke-demo-default-pool-be5e45c5-hwqn Ready <none> 2m56s v1.36.4-gke.1391002
gk3-gke-demo-pool-1-8fa8dc24-skfl Ready <none> 31s v1.36.4-gke.1391002
gk3-gke-demo-pool-1-8ff46d88-5cjk Ready <none> 4s v1.36.4-gke.1391002
Workloads page with both broken pods flagged. hello shows 6/6 pods because autoscaling had already added three.
Cheat sheet
General rules of thumb, not results from my demo.
Status
Usually means
First command
CrashLoopBackOff
The app crashes as soon as it starts, over and over
kubectl logs <pod> --previous
OOMKilled
The app used more memory than its limit allows
kubectl describe pod <pod> (check Last State)
Pending
Kubernetes can't find room for the pod, e.g. it asks for too much or a quota is hit
kubectl describe pod <pod> (check Events)
09
Cost and clean-up
Once everything was recorded, I deleted the cluster. Nothing is left running, so keeping this page online costs $0 a month.
The whole demo cost under US$1 (estimated) and ran on Free Trial credit; every resource was deleted afterwards.
Everything existed for about 37 minutes. The cluster was created at 18:11 UTC and was gone by 18:48 UTC on 2026-10-13. What the cost was made of:
Autopilot pods. Billed per second for the CPU and memory they request. That was the 3 hello pods (up to 6 while autoscaling), plus the short-lived load generator, CrashLoopBackOff and OOMKilled pods.
External load balancer. The forwarding rule behind the LoadBalancer Service, for about 30 minutes.
To clean up I deleted the Service first (that releases the load balancer), then the cluster. Then I listed what was left.
$ kubectl delete service hello
service "hello" deleted from default namespace
$ gcloud container clusters delete gke-demo --region asia-southeast1
Deleting cluster gke-demo... (progress dots trimmed) done.
Deleted [https://container.googleapis.com/v1/projects/my-project/zones/asia-southeast1/clusters/gke-demo].
$ gcloud container clusters list
$ gcloud compute forwarding-rules list
Listed 0 items.
$ gcloud compute target-pools list
Listed 0 items.
$ gcloud compute disks list
Listed 0 items.
$ gcloud compute addresses list
Listed 0 items.
$ gcloud compute instances list
Listed 0 items.
$ gcloud compute instance-groups list
Listed 0 items.
10
The manifests
The actual Kubernetes YAML files from the demo. 01-deployment.yaml holds the 2.0 image, as it was after the update. The 1.0 version is kept as its own file.
01-deployment-v1.yamlThe hello Deployment as first applied, image version 1.0.
# The demo app (as first applied, version 1.0): Google's public sample "hello-app". 3 replicas, each asking for 250m CPU and 512Mi memory.
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
spec:
replicas: 3
selector:
matchLabels:
app: hello
template:
metadata:
labels:
app: hello
spec:
containers:
- name: hello
image: us-docker.pkg.dev/google-samples/containers/gke/hello-app:1.0
ports:
- containerPort: 8080
resources:
requests:
cpu: 250m
memory: 512Mi
readinessProbe:
httpGet:
path: /
port: 8080
01-deployment.yamlThe hello Deployment after the rolling update, image version 2.0.
# The demo app: Google's public sample "hello-app". 3 replicas, each asking for 250m CPU and 512Mi memory.
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
spec:
replicas: 3
selector:
matchLabels:
app: hello
template:
metadata:
labels:
app: hello
spec:
containers:
- name: hello
image: us-docker.pkg.dev/google-samples/containers/gke/hello-app:2.0
ports:
- containerPort: 8080
resources:
requests:
cpu: 250m
memory: 512Mi
readinessProbe:
httpGet:
path: /
port: 8080
02-service.yamlThe LoadBalancer Service that exposes the pods.
# One stable address in front of the pods, exposed to the internet through a Google Cloud load balancer.
apiVersion: v1
kind: Service
metadata:
name: hello
spec:
type: LoadBalancer
selector:
app: hello
ports:
- port: 80
targetPort: 8080
03-hpa.yamlThe autoscaler: 50% CPU target, 3 to 10 replicas.
# Horizontal Pod Autoscaler: keep average CPU near 50% of the request, between 3 and 10 pods.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: hello
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: hello
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
04-load-generator.yamlA temporary busybox traffic generator for the scaling demo.