I'm running an b2b application on a GKE cluster (1 pool, 2 nodes).
They are normal nodes, because the uptime is critical
However, I've created a secondary pool with Spot VMs with more powerful machines, but I want to allow kubernetes to use it only for scaling purposes.
Let's take this example:
- current state - default-pool: app-1 (3 replicas)
- wanted state
- default-pool: app-1 (3 replicas)
- spot-pool: app-1 (5 more replicas)
I want to give my clients a little boost in performance without worrying about uptime.
PS. currently on a test cluster, after adding the Spot-pool, GKE decided to move there the entire app (I didn't have a nodeselector) and I ended up with default-pool (just system services) and spot-pool with my app
LE. I may have found a way to do it but seems a bit complicated
- deploy initial app on default-pool using nodeSelector
- create a clone deployment for the same app, use nodeSelector to make it used the spot-pool and scale it to the required replicas
- use kubernetes ingress canary annotation to route traffic
- It ain't stupid if it works :D
Result on consecutive curl
Hostname: nginx-app-6f5db95856-xm8nk
Hostname: replica-app-5658bddc5d-jpqwp
Hostname: replica-app-5658bddc5d-jpqwp
Hostname: nginx-app-6f5db95856-xm8nk
Hostname: nginx-app-6f5db95856-xm8nk
Hostname: nginx-app-6f5db95856-xm8nk
Hostname: replica-app-5658bddc5d-jpqwp
Hostname: replica-app-5658bddc5d-jpqwp
Hostname: replica-app-5658bddc5d-jpqwp
Hostname: nginx-app-6f5db95856-xm8nk
Hostname: nginx-app-6f5db95856-xm8nk
Hostname: replica-app-5658bddc5d-jpqwp
Hostname: nginx-app-6f5db95856-xm8nk