Kueue Operator

Kueue Operator provides the ability to deploy kueue using different configurations

Dependencies

The Kueue Operator needs CertManager installed to operate correctly

Releases

ko version	ocp version	kueue version	k8s version	golang
1.0.0	4.19 - 4.20	0.11.z	1.32	1.23

Kueue releases around 6 times a year. For the latest Openshift version, we will take the latest version that was build with that underlying Kubernetes version.

See Kueue Release for more details on the Kueue release policy.

Deploy the Operator

Quick Development Operator

1. Login into podman and have a repository created.

2. Set OPERATOR_IMAGE to point to your repostory ie export OPERATOR_IMAGE=quay.io/testrepo/kueue-operator:test

3. Build operator image: make operator-build

4. Push operator image to repository: make operator-push

5. Set $KUEUE_IMAGE to point to kueue operand image

6. Run make deploy-cert-manager to deploy OperatorGroup and Subscription in cert-manager-operator namespace.

7. Run make deploy-ocp to deploy the operator using the $OPERATOR_IMAGE and $KUEUE_IMAGE for operator and operand, respectively.

8. Run make undeploy-ocp to remove operator from ocp cluster

Operator Bundle Development

1. Login into podman and have a repository created for the operator bundle.

2. Set BUNDLE_IMAGE to point to your repostory and a tag of choice

3. Run make bundle-generate to generate the bundle manifests

4. Run make bundle-build to build the bundle.Dockerfile.

5. Run make bundle-push to push the bundle image to your repository.

6. Run make deploy-cert-manager to deploy OperatorGroup and Subscription in cert-manager-operator namespace.

7. Set OPERATOR_NAMESPACE, i.e, "kueue-operator"

8. Run oc new-project $OPERATOR_NAMESPACE to create a namespace for the operaotr

9. Run operator-sdk run bundle --namespace $OPERATOR_NAMESPACE ${BUNDLE_IMAGE} to deploy operator to $OPERATOR_NAMESPACE

Local Development

1. make

2. oc apply -f deploy/

3. oc apply -f deploy/crd

4. hack/run-locally.sh

5. Optionally run oc apply -f deploy/examples/job.yaml

Sample CR

apiVersion: operator.openshift.io/v1alpha1
kind: Kueue
metadata:
  labels:
    app.kubernetes.io/name: kueue-operator
    app.kubernetes.io/managed-by: kustomize
  name: cluster
  namespace: openshift-kueue-operator
spec:
  config:
    integrations:
      frameworks:
      - "batch/job" 

E2E Test

1. Set kubeconfig to point to a OCP cluster
2. Set OPERATOR_IMAGE to point to your operator image
3. Set KUEUE_IMAGE to point to your kueue image you want to test
4. make deploy-cert-manager test-e2e