Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(operators): updates to the operator overviews #11163

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,25 @@
= Strimzi operators

[role="_abstract"]
Strimzi operators are purpose-built with specialist operational knowledge to effectively manage Kafka on Kubernetes.
Each operator performs a distinct function.

Cluster Operator:: The Cluster Operator handles the deployment and management of Apache Kafka clusters on Kubernetes.
It automates the setup of Kafka brokers, and other Kafka components and resources.
Topic Operator:: The Topic Operator manages the creation, configuration, and deletion of topics within Kafka clusters.
User Operator:: The User Operator manages Kafka users that require access to Kafka brokers.

When you deploy Strimzi, you first deploy the Cluster Operator.
The Cluster Operator is then ready to handle the deployment of Kafka.
You can also deploy the Topic Operator and User Operator using the Cluster Operator (recommended) or as standalone operators.
You would use a standalone operator with a Kafka cluster that is not managed by the Cluster Operator.

The Topic Operator and User Operator are part of the Entity Operator.
The Cluster Operator can deploy one or both operators based on the Entity Operator configuration.

[IMPORTANT]
====
To deploy the standalone operators, you need to set environment variables to connect to a Kafka cluster.
These environment variables do not need to be set if you are deploying the operators using the Cluster Operator as they will be set by the Cluster Operator.
====
Strimzi uses operators to deploy and manage Kafka components.
The operators monitor Strimzi custom resources (like `Kafka`, `KafkaTopic`, and `KafkaUser`) and ensure that the Kafka components are configured and running as specified in the resources.

Strimzi provides the following operators, each handling different aspects of a Kafka deployment:

Cluster Operator (required):: Handles the deployment and management of Apache Kafka clusters on Kubernetes. It automates the setup of Kafka nodes, and other Kafka components and resources.
Entity Operator (recommended):: Incorporates the following to manage topics and users:
* *Topic Operator* for managing Kafka topics.
* *User Operator* for managing Kafka users.
ifdef::Section[]
Access Operator (optional):: Helps manage and share Kafka cluster connection details.
endif::Section[]

Additionally, Strimzi provides Drain Cleaner, which supports the Cluster Operator in managing pod evictions for Kafka clusters.

The Cluster Operator is required and must be deployed first.
It handles the deployment of Kafka and, if configured, can also deploy the Entity Operator, which may include the Topic Operator, User Operator, or both.

The Topic Operator and User Operator can also be deployed standalone to manage topics and users for a Kafka cluster that is not managed by Strimzi.
Comment on lines +25 to +28
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to list the Access Operator and Drain Cleaner in the introduction in sich a prominent way, we should probably also cover here how they are installed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1


//operator namespace
include::../../modules/operators/con-operators-namespaces.adoc[leveloffset=+1]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@
You can perform a standalone deployment of the Topic Operator and User Operator.
Consider a standalone deployment of these operators if you are using a Kafka cluster that is not managed by the Cluster Operator.

You deploy the operators to Kubernetes.
Kafka can be running outside of Kubernetes.
For example, you might be using a Kafka as a managed service.
You adjust the deployment configuration for the standalone operator to match the address of your Kafka cluster.
You deploy the operators to Kubernetes, Kafka can be running outside of Kubernetes.
For example, you might be using a managed Kafka service.

To connect the standalone operators to your Kafka cluster, you must set environment variables that specify the cluster address and authentication details.
These variables are automatically configured when deploying the operators with the Cluster Operator.

//Procedure for standalone deployment of Topic Operator
include::../../modules/deploying/proc-deploy-topic-operator-standalone.adoc[leveloffset=+1]
Expand Down
37 changes: 22 additions & 15 deletions documentation/assemblies/overview/assembly-overview-components.adoc
Original file line number Diff line number Diff line change
@@ -1,29 +1,28 @@
// This assembly is included in:
//
// overview/overview.adoc
// deploying/assembly_deploy-intro.adoc

[id="overview-components_{context}"]
= Strimzi Operators
= Strimzi operators

[role="_abstract"]
Operators are a method of packaging, deploying, and managing Kubernetes applications.
They provide a way to extend the Kubernetes API and simplify the administration tasks associated with specific applications.
Operators are Kubernetes components that package, deploy, and manage applications by extending the Kubernetes API.
They simplify administrative tasks and reduce manual intervention.

Strimzi operators support tasks related to a Kafka deployment.
Strimzi custom resources provide the deployment configuration.
This includes configuration for Kafka clusters, topics, users, and other components.
Leveraging custom resource configuration, Strimzi operators create, configure, and manage Kafka components within a Kubernetes environment.
Using operators reduces the need for manual intervention and streamlines the process of managing Kafka in a Kubernetes cluster.
Strimzi operators automate the deployment and management of Apache Kafka components on Kubernetes.
Strimzi custom resources define the deployment configuration.

Strimzi provides the following operators for managing a Kafka cluster running within a Kubernetes cluster.
The following operators manage Kafka in a Kubernetes cluster:

Cluster Operator:: Deploys and manages Apache Kafka clusters, Kafka Connect, Kafka MirrorMaker, Kafka Bridge, Kafka Exporter, Cruise Control, and the Entity Operator
Entity Operator:: Comprises the Topic Operator and User Operator
Topic Operator:: Manages Kafka topics
User Operator:: Manages Kafka users
Cluster Operator:: Manages Kafka clusters and related components.
Entity Operator:: Comprises the Topic Operator and User Operator.
Topic Operator:: Creates, configures, and deletes Kafka topics.
User Operator:: Manages Kafka users and their authentication credentials.
ifdef::Section[]
Access Operator:: Simplifies sharing Kafka connection information and credentials across namespaces.
endif::Section[]

The Cluster Operator can deploy the Topic Operator and User Operator as part of an *Entity Operator* configuration at the same time as a Kafka cluster.
Additionally, Strimzi provides Drain Cleaner, a separate tool that can be used alongside the Cluster Operator to assist with safe pod eviction during maintenance or upgrades.

.Operators within the Strimzi architecture

Expand All @@ -38,5 +37,13 @@ include::../../modules/overview/con-overview-components-topic-operator.adoc[leve
//description of User Operator
include::../../modules/overview/con-overview-components-user-operator.adoc[leveloffset=+1]

ifdef::Section[]
//description of Access Operator
include::../../modules/overview/con-overview-components-access-operator.adoc[leveloffset=+1]
endif::Section[]

//description of Drain cleaner
include::../../modules/overview/con-overview-components-drain-cleaner.adoc[leveloffset=+1]

//description of feature gates
include::../../modules/overview/con-feature-gates-overview.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,13 @@

[role="_abstract"]
The Strimzi Access Operator is an optional feature that simplifies the sharing of Kafka connection information and credentials between namespaces

The Access Operator is deployed separately and manages the distribution of connection details, which are stored as a `Secret` resource.
The secret contains cluster connection details and optional user credentials, and can be referenced by applications across different namespaces.

The `KafkaAccess` resource configures Kafka cluster connection information and credentials.
When you create, modify, or delete connection information using a `KafkaAccess` resource, the Access Operator updates a service binding secret.
This secret stores the details in multiple formats, allowing applications across different namespaces to connect to the clusters.

//Procedure for deployment of Access Operator
include::../../modules/deploying/proc-deploy-access-operator.adoc[leveloffset=+1]
//Procedure for running the Access Operator
Expand Down
5 changes: 0 additions & 5 deletions documentation/assemblies/security/assembly-security.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,3 @@ include::../../modules/security/proc-configuring-external-clients-to-trust-clust
//user-supplied certs
include::assembly-using-your-own-certificates.adoc[leveloffset=+1]

//access-operator
ifdef::Section[]
include::assembly-access-operator.adoc[leveloffset=+1]
endif::Section[]

4 changes: 4 additions & 0 deletions documentation/deploying/deploying.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ include::assemblies/security/assembly-securing-access.adoc[leveloffset=+1]
include::assemblies/oauth/assembly-oauth-security.adoc[leveloffset=+1]
//managing tls certificates
include::assemblies/security/assembly-security.adoc[leveloffset=+1]
//using the access-operator
ifdef::Section[]
include::assemblies/security/assembly-access-operator.adoc[leveloffset=+1]
endif::Section[]
//security context for all pods
include::assemblies/configuring/assembly-security-providers.adoc[leveloffset=+1]
//Scaling clusters
Expand Down
2 changes: 1 addition & 1 deletion documentation/modules/configuring/con-pod-management.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ You must not create, update, or delete `StrimziPodSet` resources.
The `StrimziPodSet` custom resource is used internally and resources are managed solely by the Cluster Operator.
As a consequence, the Cluster Operator must be running properly to avoid the possibility of pods not starting and Kafka clusters not being available.

NOTE: Kubernetes `Deployment` resources are used for creating and managing the pods of other components: Kafka Bridge, Kafka Exporter, Cruise Control, User Operator and Topic Operator.
NOTE: Kubernetes `Deployment` resources are used for creating and managing the pods of other components.
6 changes: 6 additions & 0 deletions documentation/modules/operators/con-operators-namespaces.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,16 @@
Operators watch and manage Strimzi resources in Kubernetes namespaces.
The Cluster Operator can watch a single namespace, multiple namespaces, or all namespaces in a Kubernetes cluster.
The Topic Operator and User Operator can watch a single namespace.
ifdef::Section[]
The Access Operator can watch a single namespace or all namespaces in a Kubernetes cluster.
endif::Section[]

* The Cluster Operator watches for `Kafka` resources
* The Topic Operator watches for `KafkaTopic` resources
* The User Operator watches for `KafkaUser` resources
ifdef::Section[]
* The Access Operator watches for `KafkaAccess` resources.
endif::Section[]

The Topic Operator and the User Operator can only watch a single Kafka cluster in a namespace.
And they can only be connected to a single Kafka cluster.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// Module included in the following assemblies:
//
// overview/assembly-overview-components.adoc

[id='overview-concepts-access-operator-{context}']
= Access Operator

[role="_abstract"]
If you are using the Cluster Operator to manage your Kafka cluster, you can deploy and use the Access Operator to manage the distribution of connection details
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you are using the Cluster Operator to manage your Kafka cluster, you can deploy and use the Access Operator to manage the distribution of connection details
If you are using the Cluster Operator to manage your Kafka cluster, you can deploy and use the Access Operator to manage the distribution of connection details.

Kafka instances and Kafka users are specified in a `KafkaAccess` resource.
Optionally, the names of the listeners used to connect to Kafka clusters can also be specified.
When the the resource is created, a service binding `Secret` is created to store the connection details and optional user credentials.
The secret can then be referenced by applications across different namespaces.
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
// Module included in the following assemblies:
//
// overview/assembly-overview-components.adoc
// assembly-using-the-cluster-operator.adoc

[id='overview-components-cluster-operator-{context}']
= Cluster Operator
Expand Down Expand Up @@ -32,11 +31,6 @@ The Cluster Operator can also deploy the following Strimzi operators through con

The Topic Operator and User Operator function within the Entity Operator on deployment.

You can use the Cluster Operator with a deployment of Strimzi Drain Cleaner to help with pod evictions.
By deploying the Strimzi Drain Cleaner, you can use the Cluster Operator to move Kafka pods instead of Kubernetes.
Strimzi Drain Cleaner annotates pods being evicted with a rolling update annotation.
The annotation informs the Cluster Operator to perform the rolling update.

.Example architecture for the Cluster Operator

image:cluster-operator.png[The Cluster Operator creates and deploys Kafka clusters]
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// Module included in the following assemblies:
//
// overview/assembly-overview-components.adoc

[id='overview-concepts-drain-cleaner-{context}']
= Drain Cleaner

[role="_abstract"]
If you are using the Cluster Operator to manage your Kafka cluster, you can deploy and use the Drain Cleaner to streamline the process of moving Kafka pods from Kubernetes nodes scheduled for maintenance.

By deploying the Strimzi Drain Cleaner, you can use the Cluster Operator to move Kafka pods instead of Kubernetes.
Strimzi Drain Cleaner annotates pods being evicted with a rolling update annotation.
The annotation informs the Cluster Operator to perform the rolling update.

Drain Cleaner ensures that no partition replicas become under-replicated during node draining, maintaining data availability and fault tolerance.
This controlled approach minimizes potential disruptions to your Kafka cluster when draining pods.
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
// Module included in the following assemblies:
//
// overview/assembly-overview-components.adoc
// assembly-using-the-topic-operator.adoc

[id='overview-concepts-topic-operator-{context}']
= Topic Operator

[role="_abstract"]
The Topic Operator provides a way of managing topics in a Kafka cluster through `KafkaTopic` resources.
The Cluster Operator can deploy the Topic Operator as part of the *Entity Operator* configuration at the same time as a Kafka cluster.
The operator can also be deployed standalone to help manage topics for Kafka clusters not operated by Strimzi.

.Example architecture for the Topic Operator

Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
// Module included in the following assemblies:
//
// overview/assembly-overview-components.adoc
// assembly-using-the-user-operator.adoc

[id='overview-concepts-user-operator-{context}']
= User Operator

[role="_abstract"]
The User Operator provides a way of managing users in a Kafka cluster through `KafkaUser` resources.
The Cluster Operator can deploy the User Operator as part of the *Entity Operator* configuration at the same time as a Kafka cluster.
The operator can also be deployed standalone to help manage users for Kafka clusters not operated by Strimzi.

The User Operator manages Kafka users for a Kafka cluster by watching for `KafkaUser` resources that describe Kafka users,
and ensuring that they are configured properly in the Kafka cluster.
Expand Down
4 changes: 2 additions & 2 deletions documentation/overview/overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ include::shared/attributes.adoc[]

//key features
include::modules/overview/ref-key-features.adoc[leveloffset=+1]
//Product components (operators)
include::assemblies/overview/assembly-overview-components.adoc[leveloffset=+1]
//Kafka components
include::assemblies/overview/assembly-kafka-components.adoc[leveloffset=+1]
//Kafka concepts
Expand All @@ -19,8 +21,6 @@ include::assemblies/overview/assembly-kafka-connect-components.adoc[leveloffset=
//description of kafka bridge
include::modules/overview/con-overview-components-kafka-bridge.adoc[leveloffset=+1]
include::modules/overview/con-overview-components-kafka-bridge-clients.adoc[leveloffset=+2]
//Product components
include::assemblies/overview/assembly-overview-components.adoc[leveloffset=+1]
//Main configuration points
include::modules/overview/con-configuration-points.adoc[leveloffset=+1]
//security options
Expand Down
Loading