Problem
DevWorkspace has a known unresolved webhook certificate / caBundle drift issue.
The failure is documented in devfile/devworkspace-operator#1157. In that issue, GitLab Runner calls the DevWorkspace admission webhook:
validate-exec.devworkspace-controller.svc
devworkspace-webhookserver.devworkspace-controller.svc:443/validate
and TLS verification fails. Restarting devworkspace-webhook-server temporarily fixes the issue, which suggests the webhook serving certificate and the Kubernetes webhook caBundle can drift out of sync.
Related upstream discussion
A related Che issue, eclipse-che/che#23184, discusses the cleaner fix: add cert-manager CA injection metadata to CRDs and mutating/validating webhooks so cert-manager cainjector keeps webhook caBundle fields updated automatically.
That discussion explicitly mentions applying the same approach to DevWorkspace. A DWO maintainer commented that adding this annotation to the webhook should hopefully resolve the issue, but the issue later went stale/closed without the DWO fix being completed.
Current gap
The dynamically generated DevWorkspace ValidatingWebhookConfiguration does not include the cert-manager CA injection annotation:
cert-manager.io/inject-ca-from: <namespace>/devworkspace-controller-serving-cert
Without this annotation, cert-manager may renew the serving certificate Secret, but Kubernetes can continue using a stale or incorrect webhook caBundle.
Proposed fix
Apply the same cainjector pattern used by Che: annotate the DevWorkspace validating webhook so cert-manager keeps the webhook caBundle synchronized with the cert-manager-managed serving certificate.
The generated DevWorkspace validating webhook should include:
metadata:
annotations:
cert-manager.io/inject-ca-from: <namespace>/devworkspace-controller-serving-certThis allows cert-manager cainjector to keep:
webhooks:
- clientConfig:
caBundle: ...updated automatically when the referenced certificate changes.
Expected result
The Kubernetes API server should continue to trust the DevWorkspace webhook after cert-manager certificate renewal, without requiring manual deletion or restart of devworkspace-webhook-server.
Related issues
Problem
DevWorkspace has a known unresolved webhook certificate /
caBundledrift issue.The failure is documented in devfile/devworkspace-operator#1157. In that issue, GitLab Runner calls the DevWorkspace admission webhook:
and TLS verification fails. Restarting
devworkspace-webhook-servertemporarily fixes the issue, which suggests the webhook serving certificate and the Kubernetes webhookcaBundlecan drift out of sync.Related upstream discussion
A related Che issue, eclipse-che/che#23184, discusses the cleaner fix: add cert-manager CA injection metadata to CRDs and mutating/validating webhooks so cert-manager cainjector keeps webhook
caBundlefields updated automatically.That discussion explicitly mentions applying the same approach to DevWorkspace. A DWO maintainer commented that adding this annotation to the webhook should hopefully resolve the issue, but the issue later went stale/closed without the DWO fix being completed.
Current gap
The dynamically generated DevWorkspace
ValidatingWebhookConfigurationdoes not include the cert-manager CA injection annotation:Without this annotation, cert-manager may renew the serving certificate Secret, but Kubernetes can continue using a stale or incorrect webhook
caBundle.Proposed fix
Apply the same cainjector pattern used by Che: annotate the DevWorkspace validating webhook so cert-manager keeps the webhook
caBundlesynchronized with the cert-manager-managed serving certificate.The generated DevWorkspace validating webhook should include:
This allows cert-manager cainjector to keep:
updated automatically when the referenced certificate changes.
Expected result
The Kubernetes API server should continue to trust the DevWorkspace webhook after cert-manager certificate renewal, without requiring manual deletion or restart of
devworkspace-webhook-server.Related issues