Skip to content

docs: Document automated Prometheus metrics management #3096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tolusha wants to merge 1 commit into from
Closed

docs: Document automated Prometheus metrics management #3096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
565 changes: 565 additions & 0 deletions .../administration-guide/examples/checluster-properties/checluster-properties.adoc
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion modules/administration-guide/pages/monitoring-che.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@
[id="monitoring-{prod-id-short}"]
= Monitoring {prod-short} Server

You can configure {prod-short} to expose JVM metrics such as JVM memory and class loading for {prod-short} Server.
The {prod-short} Operator automatically configures monitoring to expose JVM metrics such as JVM memory and class loading for {prod-short} Server when metrics are enabled.

include::partial$proc_enabling-and-exposing-che-metrics.adoc[leveloffset=+1]

include::partial$proc_collecting-che-metrics-with-prometheus.adoc[leveloffset=+1]

include::partial$proc_viewing-che-metrics-on-grafana-dashboards.adoc[leveloffset=+1]

include::partial$proc_manually-configuring-prometheus-resources.adoc[leveloffset=+1]
7 changes: 6 additions & 1 deletion modules/administration-guide/pages/monitoring-the-dev-workspace-operator.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@
[id="monitoring-the-dev-workspace-operator"]
= Monitoring the {devworkspace} Operator

You can configure the OpenShift in-cluster monitoring stack to scrape metrics exposed by the {devworkspace} Operator.
The {prod-short} Operator automatically configures the OpenShift in-cluster monitoring stack to scrape metrics exposed by the {devworkspace} Operator when metrics are enabled.

include::partial$proc_enabling-dev-workspace-operator-metrics.adoc[leveloffset=+1]

include::partial$proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc[leveloffset=+1]

Expand All @@ -21,3 +23,6 @@ include::partial$proc_viewing-dev-workspace-operator-metrics-on-grafana-dashboar
include::partial$ref_grafana-dashboards-for-the-dev-workspace-operator.adoc[leveloffset=+1]


include::partial$proc_manually-configuring-prometheus-resources.adoc[leveloffset=+1]


93 changes: 8 additions & 85 deletions .../administration-guide/partials/proc_collecting-che-metrics-with-prometheus.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,100 +3,23 @@
[id="collecting-{prod-id-short}-metrics-with-prometheus"]
= Collecting {prod-short} Server metrics with Prometheus

To use the in-cluster Prometheus instance to collect, store, and query JVM metrics for {prod-short} Server:
The {prod-short} Operator automatically configures the in-cluster Prometheus instance to collect, store, and query JVM metrics for {prod-short} Server when metrics are enabled.

.Prerequisites

* Your organization's instance of {prod-short} is installed and running in Red Hat OpenShift.

* An active `oc` session with administrative permissions to the destination OpenShift cluster. See link:https://docs.openshift.com/container-platform/{ocp4-ver}/cli_reference/openshift_cli/getting-started-cli.html[Getting started with the CLI].
* {prod-short} metrics are enabled. See xref:enabling-and-exposing-{prod-id-short}-metrics[Enabling and exposing {prod-short} server JVM metrics].

* {prod-short} is exposing metrics on port `8087`. See xref:enabling-and-exposing-{prod-id-short}-metrics[Enabling and exposing {prod-short} server JVM metrics].
.Automatic configuration

.Procedure
When {prod-short} Server metrics are enabled via `spec.components.metrics.enable: true`, the {prod-short} Operator automatically creates and manages the following resources:

. Create the ServiceMonitor for detecting the {prod-short} JVM metrics Service.
+
.ServiceMonitor
====
[source,yaml,subs="+quotes,+attributes,+macros"]
----
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: che-host
namespace: {prod-namespace} <1>
spec:
endpoints:
- interval: 10s <2>
port: metrics
scheme: http
namespaceSelector:
matchNames:
- {prod-namespace} <1>
selector:
matchLabels:
app.kubernetes.io/name: {prod-deployment}
----
<1> The {prod-short} namespace. The default is `{prod-namespace}`.
<2> The rate at which a target is scraped.
====

. Create a Role and RoleBinding to allow Prometheus to view the metrics.
* A `ServiceMonitor` custom resource named `che-host` that monitors the {prod-short} Server metrics endpoint on port `8087`
* A `Role` and `RoleBinding` that grant the `prometheus-k8s` service account permission to scrape metrics from the {prod-short} namespace
* The `openshift.io/cluster-monitoring: "true"` label on the {prod-short} namespace to enable OpenShift cluster monitoring

+
.Role
====
[source,yaml,subs="+quotes,+attributes,+macros"]
----
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: prometheus-k8s
namespace: {prod-namespace} <1>
rules:
- verbs:
- get
- list
- watch
apiGroups:
- ''
resources:
- services
- endpoints
- pods
----
<1> The {prod-short} namespace. The default is `{prod-namespace}`.
====

+
.RoleBinding
====
[source,yaml,subs="+quotes,+attributes,+macros"]
----
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: view-{prod-id-short}-openshift-monitoring-prometheus-k8s
namespace: {prod-namespace} <1>
subjects:
- kind: ServiceAccount
name: prometheus-k8s
namespace: openshift-monitoring
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: prometheus-k8s
----
<1> The {prod-short} namespace. The default is `{prod-namespace}`.
====

. Allow the in-cluster Prometheus instance to detect the ServiceMonitor in the {prod-short} namespace. The default {prod-short} namespace is `{prod-namespace}`.
+
[source,terminal,subs="+attributes,quotes"]
----
$ oc label namespace {prod-namespace} openshift.io/cluster-monitoring=true
----
These resources are automatically updated or recreated if modified or deleted, ensuring continuous metrics collection.

.Verification

Expand Down
49 changes: 9 additions & 40 deletions ...de/partials/proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,56 +2,25 @@
= Collecting {devworkspace} Operator metrics

[role="_abstract"]
To use the in-cluster Prometheus instance to collect, store, and query metrics about the {devworkspace} Operator:
The {prod-short} Operator automatically configures the in-cluster Prometheus instance to collect, store, and query metrics about the {devworkspace} Operator.

.Prerequisites

* Your organization's instance of {prod-short} is installed and running in Red Hat OpenShift.

* An active `oc` session with administrative permissions to the destination OpenShift cluster. See link:https://docs.openshift.com/container-platform/{ocp4-ver}/cli_reference/openshift_cli/getting-started-cli.html[Getting started with the CLI].
* {devworkspace} metrics are enabled in the `CheCluster` Custom Resource. See xref:enabling-dev-workspace-operator-metrics[].

* The `devworkspace-controller-metrics` Service is exposing metrics on port `8443`. This is preconfigured by default.
.Automatic configuration

.Procedure
When {devworkspace} Operator metrics are enabled via `spec.devEnvironments.metrics: true` (the default), the {prod-short} Operator automatically creates and manages the following resources:

. Create the ServiceMonitor for detecting the Dev Workspace Operator metrics Service.
+
.ServiceMonitor
====
[source,yaml,subs="+quotes,+attributes,+macros"]
----
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: devworkspace-controller
namespace: {prod-namespace} <1>
spec:
endpoints:
- bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
interval: 10s <2>
port: metrics
scheme: https
tlsConfig:
insecureSkipVerify: true
namespaceSelector:
matchNames:
- openshift-operators
selector:
matchLabels:
app.kubernetes.io/name: devworkspace-controller
----
<1> The {prod-short} namespace. The default is `{prod-namespace}`.
<2> The rate at which a target is scraped.
====
* A `ServiceMonitor` custom resource named `devworkspace-controller` that monitors the {devworkspace} Operator metrics endpoint on port `8443`
* A `Role` and `RoleBinding` that grant the `prometheus-k8s` service account permission to scrape metrics from the {prod-short} namespace
* The `openshift.io/cluster-monitoring: "true"` label on the {prod-short} namespace to enable OpenShift cluster monitoring

include::example$snip_{project-context}-create-a-role-and-rolebinding-for-prometheus-to-view-metrics.adoc[]
These resources are automatically updated or recreated if modified or deleted, ensuring continuous metrics collection.

. Allow the in-cluster Prometheus instance to detect the ServiceMonitor in the {prod-short} namespace. The default {prod-short} namespace is `{prod-namespace}`.
+
[source,subs="+attributes"]
----
$ oc label namespace {prod-namespace} openshift.io/cluster-monitoring=true
----
NOTE: The `devworkspace-controller-metrics` Service exposes metrics on port `8443` by default. This is preconfigured by the {devworkspace} Operator.

.Verification

Expand Down
4 changes: 2 additions & 2 deletions modules/administration-guide/partials/proc_enabling-and-exposing-che-metrics.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
= Enabling and exposing {prod-short} Server metrics

{prod-short} exposes the JVM metrics on port `8087` of the `che-host` Service.
You can configure this behaviour.
When metrics are enabled, the {prod-short} Operator automatically configures Prometheus monitoring on OpenShift by creating the necessary ServiceMonitor, RBAC resources, and namespace labels.

.Procedure

Expand All @@ -17,4 +17,4 @@ spec:
metrics:
enable: __<boolean>__ <1>
----
<1> `true` to enable, `false` to disable.
<1> `true` to enable metrics and automatic Prometheus configuration, `false` to disable. Defaults to `true`.
19 changes: 19 additions & 0 deletions ...administration-guide/partials/proc_enabling-dev-workspace-operator-metrics.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// monitoring-the-dev-workspace-operator

[id="enabling-dev-workspace-operator-metrics"]
= Enabling {devworkspace} Operator metrics

The {devworkspace} Operator exposes metrics on port `8443` of the `devworkspace-controller-metrics` Service.
When metrics are enabled, the {prod-short} Operator automatically configures Prometheus monitoring on OpenShift by creating the necessary ServiceMonitor, RBAC resources, and namespace labels.

.Procedure

* Configure the `CheCluster` Custom Resource. See xref:using-the-cli-to-configure-the-checluster-custom-resource.adoc[].
+
[source,yaml,subs="+attributes,+quotes"]
----
spec:
devEnvironments:
metrics: __<boolean>__ <1>
----
<1> `true` to enable {devworkspace} Operator metrics collection and automatic Prometheus configuration, `false` to disable. Defaults to `true`.
Loading
Loading