🖊️
Notes
  • Notes
  • aws
    • CloudWatch
    • EKS
    • IAM
    • Key Management Service (KMS)
    • security
      • Attacks against AWS infrastructure
    • vpc
      • AWS Transit Gateway
  • azure
    • Azure AD
    • Azure CDN
    • DNS in Azure
    • Hub-spoke network topology
    • Identity and access management
    • Azure Landing zones
    • Storage
  • certifications
    • aws-sa-pro
    • Certified Kubernetes Administrator
  • containers
    • Examples
    • Linux Container Primitives
  • databases
    • Relational databases
  • gcp
    • IAM
  • git
    • Git
  • golang
    • Building Go projects
    • Concurrency
    • Project structure
  • infosec
    • SSH
    • SSL
  • Kubernetes
    • Admission Controllers
    • Autoscaling
    • Debugging
    • Multi-tenancy
    • Network Policies
    • Pod Priority
    • Pod Security Policies
    • Secrets
    • StatefulSet
    • additional-services
      • Debugging ArgoCD RBAC
      • open-policy-agent
  • misc
    • FFmpeg
    • PDFs
  • programming
    • Learning resources
    • concepts
      • Serialization
  • rabbitmq
    • Clustering and HA
    • Shovel plugin
  • shells
    • Bash
  • terraform
    • Moving resources between remote states
  • vim
    • Fzf (plugin)
    • Registers
    • Spell Check
  • linux
    • arch
      • Arch Linux installation
Powered by GitBook
On this page
  1. Kubernetes

StatefulSet

PreviousSecretsNextadditional-services

Last updated 4 years ago

Resizing EBS disks of StatefulSet without major downtime

Kubernetes has supported resizing of volumes without recreating PVCs and PVs from version 1.11 with certain disk types. EBS is one of them. However, if VolumeClaimTemplate is used certain tricks needs to be done.

  • Find PVCs belonging to StatefulSet, for example kubectl get pvc --all-namespaces

  • Edit PVC API object on-the-fly, for example: kubectl edit pvc/$PVC_NAME -n $NAMESPACE

  • Change value of field spec.resources.requests.storage and save changes

  • Ensure that resize is finished by inspecting PVC with kubectl describe pvc/$PVC_NAME -n $NAMESPACE. Events field should state that resize is finished and changes take place after pod restart

  • Restart pod in question

  • Repeat the steps for all PVCs of the StatefulSet

  • Delete StatefulSet without deleting its pods: kubectl delete sts --cascade=false $STS_NAME -n $NAMESPACE

  • Change disk size in VolumeClaimTemplate and apply changes

  • If needed, trigger StatefulSet rollout with kubectl rollout restart sts $STS_NAME -n rabbitmq. This will restart the pods one-by-one

In addition, Kubernetes has beta support for resizing an in-use PersistentVolumeClaim from version 1.15:

References:

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim
https://github.com/kubernetes/kubernetes/issues/68737#issuecomment-498470138
https://serverfault.com/a/989665