-
Notifications
You must be signed in to change notification settings - Fork 2
Add initialization.kcp.io/wait-for-ready annotation #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| # Waiting for Readiness | ||
|
|
||
| In some scenarios, simply creating objects during initialization is not enough. You may need to wait for | ||
| certain resources to become ready before the init-agent marks the workspace as initialized. For example, | ||
| a CRD must be `Established` before custom resources using it can be created by other initializers | ||
| further down the chain. | ||
|
|
||
| The `initialization.kcp.io/wait-for-ready` annotation allows you to express this requirement on | ||
| individual manifests. | ||
|
|
||
| ## How It Works | ||
|
|
||
| When the init-agent encounters a manifest (from any of its [init sources](./init-sources/)) with | ||
| the `initialization.kcp.io/wait-for-ready` annotation, it will: | ||
|
|
||
| 1. Create (or confirm the existence of) the object as usual. | ||
| 2. Re-fetch the object's current state from the API server. | ||
| 3. Check whether the condition type specified in the annotation's value has `status: "True"` in the | ||
| object's `status.conditions` list. | ||
| 4. If the condition is not yet `True`, the agent requeues reconciliation and tries again after a few | ||
| seconds. | ||
| 5. Only once **all** annotated objects across **all** sources have their required conditions met will | ||
| the agent remove the initializer from the workspace, completing initialization. | ||
|
|
||
| The annotation value must be the **name of a condition type** (e.g. `Established`, `Ready`, | ||
| `Available`). This condition must appear in the standard Kubernetes `status.conditions` array of | ||
| the resource. | ||
|
|
||
| !!! warning "Important" | ||
| Waiting for a condition to become `True` inherently means that **some process must set that | ||
| condition**. In some cases this happens automatically (e.g. the Kubernetes API server sets | ||
| `Established` on CRDs), but in other cases you may need a dedicated controller or operator to | ||
| act on the resource and update its status. | ||
|
|
||
| Due to the nature of kcp's workspace initialization, the workspace is not accessible through | ||
| the regular API while it still has initializers. Only processes that work through the same | ||
| `initializingworkspaces` virtual workspace – i.e. processes that are registered for the **same | ||
| initializer** as the init-agent – can see and modify objects in the workspace during | ||
| initialization. | ||
|
|
||
| This means that if you need an external controller to make a resource "ready", that controller | ||
| must also operate on the same initializer's `initializingworkspaces` view. Without this, the | ||
| controller will not be able to access the workspace and therefore cannot set the condition the | ||
| init-agent is waiting for. The initialization would be stuck indefinitely. | ||
|
|
||
| In practice, this is most relevant for custom operators that need to reconcile resources created | ||
| by the init-agent. Make sure these operators have the appropriate kcp permissions and are | ||
| configured to watch the same initializing workspaces. | ||
|
|
||
| ## Usage | ||
|
|
||
| Add the annotation to any manifest inside an `InitTemplate`'s `spec.template`. The following example | ||
| creates a CRD and waits for it to become `Established` before initialization is considered complete: | ||
|
|
||
| ```yaml | ||
| apiVersion: initialization.kcp.io/v1alpha1 | ||
| kind: InitTemplate | ||
| metadata: | ||
| name: widgets-crd | ||
| spec: | ||
| template: | | ||
| apiVersion: apiextensions.k8s.io/v1 | ||
| kind: CustomResourceDefinition | ||
| metadata: | ||
| name: widgets.example.com | ||
| annotations: | ||
| initialization.kcp.io/wait-for-ready: "Established" | ||
| spec: | ||
| group: example.com | ||
| names: | ||
| kind: Widget | ||
| listKind: WidgetList | ||
| plural: widgets | ||
| singular: widget | ||
| scope: Cluster | ||
| versions: | ||
| - name: v1alpha1 | ||
| served: true | ||
| storage: true | ||
| schema: | ||
| openAPIV3Schema: | ||
| type: object | ||
| ``` | ||
|
|
||
| In this example the init-agent will create the CRD and then wait until its `Established` condition | ||
| is `True` before considering this source complete. CRDs are just a nice example of a Kube-native | ||
| resource that on its own becomes ready. | ||
|
|
||
| The agent will keep retrying indefinitely. If a condition is never set, the workspace will remain | ||
| in the initializing state. Use kcp's workspace lifecycle management to handle stuck workspaces if | ||
| necessary. | ||
|
|
||
| The annotation works with **any Kubernetes resource** that follows the standard conditions pattern | ||
| in its status. Common examples include: | ||
|
|
||
| | Resource | Typical Condition | | ||
| | -------- | ----------------- | | ||
| | `CustomResourceDefinition` | `Established` | | ||
| | `Deployment` | `Available` | | ||
| | `APIBinding` (kcp) | `Ready` | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| /* | ||
| Copyright 2026 The kcp Authors. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package manifest | ||
|
|
||
| import ( | ||
| "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" | ||
| ) | ||
|
|
||
| // HasCondition checks if an unstructured object has the specified condition | ||
| // type with status "True". | ||
| func HasCondition(obj *unstructured.Unstructured, conditionType string) bool { | ||
| conditions, found, err := unstructured.NestedSlice(obj.Object, "status", "conditions") | ||
| if err != nil || !found { | ||
| return false | ||
| } | ||
|
|
||
| for _, c := range conditions { | ||
| condition, ok := c.(map[string]any) | ||
| if !ok { | ||
| continue | ||
| } | ||
|
|
||
| cType, _, _ := unstructured.NestedString(condition, "type") | ||
| cStatus, _, _ := unstructured.NestedString(condition, "status") | ||
| if cType == conditionType && cStatus == "True" { | ||
| return true | ||
| } | ||
| } | ||
|
|
||
| return false | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| /* | ||
| Copyright 2026 The kcp Authors. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package manifest | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" | ||
| ) | ||
|
|
||
| func TestHasCondition(t *testing.T) { | ||
| testcases := []struct { | ||
| name string | ||
| obj *unstructured.Unstructured | ||
| conditionType string | ||
| expected bool | ||
| }{ | ||
| { | ||
| name: "no status", | ||
| obj: newUnstructured("v1", "ConfigMap", "test"), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "no conditions", | ||
| obj: newUnstructuredWithStatus("v1", "ConfigMap", "test", map[string]any{}), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "empty conditions", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{}), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "condition type not found", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| map[string]any{"type": "Available", "status": "True"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "condition found but status is False", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| map[string]any{"type": "Ready", "status": "False"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "condition found but status is Unknown", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| map[string]any{"type": "Ready", "status": "Unknown"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "condition found with status True", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| map[string]any{"type": "Ready", "status": "True"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: true, | ||
| }, | ||
| { | ||
| name: "multiple conditions - target is True", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| map[string]any{"type": "Available", "status": "True"}, | ||
| map[string]any{"type": "Ready", "status": "True"}, | ||
| map[string]any{"type": "Progressing", "status": "False"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: true, | ||
| }, | ||
| { | ||
| name: "multiple conditions - target is False", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| map[string]any{"type": "Available", "status": "True"}, | ||
| map[string]any{"type": "Ready", "status": "False"}, | ||
| map[string]any{"type": "Progressing", "status": "True"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "CRD Established condition True", | ||
| obj: newUnstructuredWithConditions("apiextensions.k8s.io/v1", "CustomResourceDefinition", "test", []any{ | ||
| map[string]any{"type": "NamesAccepted", "status": "True"}, | ||
| map[string]any{"type": "Established", "status": "True"}, | ||
| }), | ||
| conditionType: "Established", | ||
| expected: true, | ||
| }, | ||
| { | ||
| name: "CRD Established condition False", | ||
| obj: newUnstructuredWithConditions("apiextensions.k8s.io/v1", "CustomResourceDefinition", "test", []any{ | ||
| map[string]any{"type": "NamesAccepted", "status": "True"}, | ||
| map[string]any{"type": "Established", "status": "False"}, | ||
| }), | ||
| conditionType: "Established", | ||
| expected: false, | ||
| }, | ||
| { | ||
| name: "malformed condition entry (not a map)", | ||
| obj: newUnstructuredWithConditions("v1", "ConfigMap", "test", []any{ | ||
| "not a map", | ||
| map[string]any{"type": "Ready", "status": "True"}, | ||
| }), | ||
| conditionType: "Ready", | ||
| expected: true, | ||
| }, | ||
| } | ||
|
|
||
| for _, tt := range testcases { | ||
| t.Run(tt.name, func(t *testing.T) { | ||
| result := HasCondition(tt.obj, tt.conditionType) | ||
| if result != tt.expected { | ||
| t.Fatalf("Expected %v.", tt.expected) | ||
| } | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func newUnstructuredWithStatus(apiVersion, kind, name string, status map[string]any) *unstructured.Unstructured { | ||
| obj := newUnstructured(apiVersion, kind, name) | ||
| obj.Object["status"] = status | ||
| return obj | ||
| } | ||
|
|
||
| func newUnstructuredWithConditions(apiVersion, kind, name string, conditions []any) *unstructured.Unstructured { | ||
| return newUnstructuredWithStatus(apiVersion, kind, name, map[string]any{ | ||
| "conditions": conditions, | ||
| }) | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hasConditioncan we make private?