Error upgrading Helm chart #74
Description
I'm currently iterating on our Sourcebot deployment, using Helm and Terraform and deploying to EKS...
However when trying to upgrade the Helm release, I get the following error:
│ Error: Error upgrading chart
│
│ with helm_release.sourcebot,
│ on helm.tf line 29, in resource "helm_release" "sourcebot":
│ 29: resource "helm_release" "sourcebot" {
│
│ Upgrade failed: failed to replace object: PersistentVolumeClaim "sourcebot-data" is invalid: spec: Forbidden: spec is immutable after creation except
│ resources.requests and volumeAttributesClassName for bound claims
│ core.PersistentVolumeClaimSpec{
│ AccessModes: {"ReadWriteOnce"},
│ Selector: nil,
│ Resources: {Requests: {s"storage": {i: {...}, s: "10Gi", Format: "BinarySI"}}},
│ - VolumeName: "pvc-f7670064-f64a-4fc9-9440-dc30fab7bdcc",
│ + VolumeName: "",
│ StorageClassName: &"gp3",
│ VolumeMode: &"Filesystem",
│ ... // 3 identical fields
│ }
│
╵
The only change I've made to the persistence values is to set storageClassName to gp3 to match the config in our cluster.
I ran the issue through Claude and it came back with:
⏺ This is a Kubernetes limitation — PVC specs are immutable after creation. The storageClass on the existing sourcebot-data PVC cannot be changed in-place.
The fix is to add helm.sh/resource-policy: keep annotation to the PVC so Helm doesn't try to replace it during upgrades, and document that users must manually delete and recreate the PVC if they need to change the storage class.
However, the more immediate practical fix for the user is to delete the existing PVC and let Helm recreate it (with data migration if needed). But from a chart perspective, the right fix is to add the keep annotation to prevent Helm
from attempting immutable field updates:⏺ Update(templates/pvc.yaml)
⎿ Added 3 lines, removed 2 lines7 namespace: {{ $.Release.Namespace }} 8 labels: 9 {{- include "sourcebot.labels" . | nindent 4 }} 10 - {{- with .Values.sourcebot.persistence.annotations }} 10 annotations: 11 + helm.sh/resource-policy: keep 12 + {{- with .Values.sourcebot.persistence.annotations }} 13 {{- toYaml . | nindent 4 }} 13 - {{- end }} 14 + {{- end }} 15 spec: 16 accessModes: 17 {{- range .Values.sourcebot.persistence.accessModes }}⏺ The helm.sh/resource-policy: keep annotation tells Helm to never delete or replace the PVC during upgrades — it will only create it if it doesn't exist.
To resolve the current error, you'll need to manually handle the migration since the PVC already exists with a different storage class:
- Backup your data if needed
- Delete the existing PVC: kubectl delete pvc sourcebot-data -n
- Re-run the Helm upgrade — it will create a new PVC with storageClass: gp3
Going forward with this chart fix, Helm will no longer attempt to replace the PVC on upgrades, avoiding this immutable field error entirely.
Happy to submit a patch if useful :)