Avoid CA certificate at producer end

[rgshenoy]

I have a working setup currently, but the problem is that I have clients which need the Strimzi cluster ca cert if they have to connect. I want to solve this by using an AWS certificate, which already exists for our domain. I read in many discussions here that the brokerCertchainAndKey property can be used to set a custom certificate, but AWS ACM does not allow to download cert. We can only use them on AWS resources like NLB/ALB.

My setup is as follows:

[Kafka brokers (Strimzi CA)(TLS Termination)] <--- [NGINX service with NLB Loadbalancer IP and TLS Passthrough] <--- [Ingresses with NLB IP] <--- [Golang code running in a remote cluster with Kafka client (has strimzi ca cert in local trust store)]

Looking for any suggestions that will help avoid the need for client to have the strimiz ca certificate to do TLS verification of the broker.

here's my Kafka setup:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  annotations:
    strimzi.io/kraft: enabled
    strimzi.io/node-pools: enabled

  name: kafka-ingress-test-cluster
spec:
  entityOperator:
    topicOperator: {}
    userOperator: {}
  kafka:
    authorization:
      type: simple
    config:
      default.replication.factor: 3
      min.insync.replicas: 2
      offsets.topic.replication.factor: 3
      transaction.state.log.min.isr: 2
      transaction.state.log.replication.factor: 3
    listeners:
    - authentication:
        type: scram-sha-512
      name: plain
      port: 9092
      tls: false
      type: internal
    - authentication:
        type: scram-sha-512
      configuration:
        bootstrap:
          alternativeNames:
          - kafka-bs.mydomain.net
          - kafka-ingress-test-cluster-kafka-tls-bootstrap.myns.svc
          annotations:
            external-dns.alpha.kubernetes.io/hostname: kafka-bs.mydomain.net.
            external-dns.alpha.kubernetes.io/ttl: "600"
            ingress.kubernetes.io/ssl-passthrough: "true"
            kubernetes.io/ingress.class: nginx
            nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
            nginx.ingress.kubernetes.io/ssl-passthrough: "true"
          host: kafka-bs.mydomain.net
        brokers:
        - advertisedHost: kafka-b0.mydomain.net
          advertisedPort: 443
          annotations:
            external-dns.alpha.kubernetes.io/hostname: kafka-b0.mydomain.net.
            external-dns.alpha.kubernetes.io/ttl: "600"
            ingress.kubernetes.io/ssl-passthrough: "true"
            kubernetes.io/ingress.class: nginx
            nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
            nginx.ingress.kubernetes.io/ssl-passthrough: "true"
          broker: 0
          host: kafka-b0.mydomain.net
        - advertisedHost: kafka-b1.mydomain.net
          advertisedPort: 443
          annotations:
            external-dns.alpha.kubernetes.io/hostname: kafka-b1.mydomain.net.
            external-dns.alpha.kubernetes.io/ttl: "600"
            ingress.kubernetes.io/ssl-passthrough: "true"
            kubernetes.io/ingress.class: nginx
            nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
            nginx.ingress.kubernetes.io/ssl-passthrough: "true"
          broker: 1
          host: kafka-b1.mydomain.net
        - advertisedHost: kafka-b2.mydomain.net
          advertisedPort: 443
          annotations:
            external-dns.alpha.kubernetes.io/hostname: kafka-b2.mydomain.net.
            external-dns.alpha.kubernetes.io/ttl: "600"
            ingress.kubernetes.io/ssl-passthrough: "true"
            kubernetes.io/ingress.class: nginx
            nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
            nginx.ingress.kubernetes.io/ssl-passthrough: "true"
          broker: 2
          host: kafka-b2.mydomain.net
        class: nginx
      name: tls
      port: 9094
      tls: true
      type: ingress
    metadataVersion: 3.8-IV0
    resources:
      limits:
        cpu: "2"
        memory: 4Gi
      requests:
        cpu: "1"
        memory: 2Gi
    version: 3.8.0

[scholzj]

Maybe you can consider using something like Let's Encrypt that can give you the certificate as a file that can be mounted directly into the broker?

If you would want to use the AWS certificates, you might need to figure it out from scratch. But keep in mind:

• AFAIK, the Amazon ALB load balancers are designed for HTTP traffic only. They do not support non-HTTP traffic or TLS passthrough. So you cannot use it for Kafka.
• Kubernetes Nginx Ingress is also designed for HTTP traffic. The easiest way to get non-HTTP traffic through is by using TLS passthrough. But to do TLS passthrough, the TLS certificate needs to be in the Kafka broker and that means you need to use the certificate in there which you siggest AWS doesn't support.

So I think if you want to use the AWS certificates, you would need to kick the Nginx Ingress aside and use the load balancers directly - either classic or NLB and do either TLS termination in the load balancer or TLS re-encryption from the AWS certs to the Strimzi certs in the load balancer. That might work, but I have never done this myself.