Headless Service

Kafka Example

kubectl get svc | grep kafka

kafka                      NodePort    10.96.3.138   <none>  9092:11981/TCP             279d
kafka-broker-headless      ClusterIP   None          <none>  9094/TCP,9092/TCP          279d
kafka-controller-headless  ClusterIP   None          <none>  9094/TCP,9092/TCP,9093/TCP 279d
kafka-jmx-metrics          ClusterIP   10.96.2.241   <none>  5556/TCP                   279d
kafka-metrics              ClusterIP   10.96.1.108   <none>  9308/TCP                   279d
kafka-ui                   NodePort    10.96.2.31    <none>  80:42423/TCP               363d

• What's diff between kafka and kafka-broker-headless.
  • Kafka configuration advertised.listeners
  • Client broker params: kafka:9092 vs kafka-broker-0,kafka-broker-1,...

NodePort Service

apiVersion: v1
kind: Service
metadata:
  annotations:
    field.cattle.io/publicEndpoints: '[{"port":11981,"protocol":"TCP","serviceName":"sit:foobar-kafka","allNodes":true}]'
    meta.helm.sh/release-name: foobar-kafka
    meta.helm.sh/release-namespace: sit
  creationTimestamp: "2024-08-19T10:17:57Z"
  labels:
    app.kubernetes.io/component: kafka
    app.kubernetes.io/instance: foobar-kafka
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: foobar-kafka
    app.kubernetes.io/version: 3.5.1
    helm.sh/chart: foobar-kafka-3.5.1
  name: foobar-kafka
  namespace: sit
  resourceVersion: "75606877"
  uid: 1f1d382d-6b32-42b3-9da2-48be689d421d
spec:
  clusterIP: 10.96.3.138
  clusterIPs:
  - 10.96.3.138
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: tcp-client
    nodePort: 11981
    port: 9092
    protocol: TCP
    targetPort: client
  selector:
    app.kubernetes.io/instance: foobar-kafka
    app.kubernetes.io/name: foobar-kafka
    app.kubernetes.io/part-of: kafka
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

Broker Headless Service

apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: foobar-kafka
    meta.helm.sh/release-namespace: sit
  creationTimestamp: "2024-08-19T10:17:57Z"
  labels:
    app.kubernetes.io/component: broker
    app.kubernetes.io/instance: foobar-kafka
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: foobar-kafka
    app.kubernetes.io/part-of: kafka
    app.kubernetes.io/version: 3.5.1
    helm.sh/chart: foobar-kafka-3.5.1
  name: foobar-kafka-broker-headless
  namespace: sit
  resourceVersion: "74758533"
  uid: 89748b11-0445-4a3a-8145-7eeeb5492627
spec:
  clusterIP: None
  clusterIPs:
  - None
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: tcp-interbroker
    port: 9094
    protocol: TCP
    targetPort: interbroker
  - name: tcp-client
    port: 9092
    protocol: TCP
    targetPort: client
  publishNotReadyAddresses: true
  selector:
    app.kubernetes.io/component: broker
    app.kubernetes.io/instance: foobar-kafka
    app.kubernetes.io/name: foobar-kafka
    app.kubernetes.io/part-of: kafka
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Controller Headless Service

apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: foobar-kafka
    meta.helm.sh/release-namespace: sit
  creationTimestamp: "2024-08-19T10:17:57Z"
  labels:
    app.kubernetes.io/component: controller-eligible
    app.kubernetes.io/instance: foobar-kafka
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: foobar-kafka
    app.kubernetes.io/part-of: kafka
    app.kubernetes.io/version: 3.5.1
    helm.sh/chart: foobar-kafka-3.5.1
  name: foobar-kafka-controller-headless
  namespace: sit
  resourceVersion: "74758536"
  uid: 4ecd472e-3e6a-40e9-8009-d32996b7a528
spec:
  clusterIP: None
  clusterIPs:
  - None
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: tcp-interbroker
    port: 9094
    protocol: TCP
    targetPort: interbroker
  - name: tcp-client
    port: 9092
    protocol: TCP
    targetPort: client
  - name: tcp-controller
    port: 9093
    protocol: TCP
    targetPort: controller
  publishNotReadyAddresses: true
  selector:
    app.kubernetes.io/component: controller-eligible
    app.kubernetes.io/instance: foobar-kafka
    app.kubernetes.io/name: foobar-kafka
    app.kubernetes.io/part-of: kafka
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Broker Configuration

apiVersion: v1
data:
  server.properties: |-
    # Listeners configuration
    listeners=CLIENT://:9092,INTERNAL://:9094
    listener.security.protocol.map=CLIENT:PLAINTEXT,INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
    advertised.listeners=CLIENT://advertised-address-placeholder:9092,INTERNAL://advertised-address-placeholder:9094

    # KRaft node role
    process.roles=broker

    #node.id=
    controller.listener.names=CONTROLLER
    controller.quorum.voters=0@foobar-kafka-controller-0.foobar-kafka-controller-headless.sit.svc.cluster.local:9093,1@foobar-kafka-controller-1.foobar-kafka-controller-headless.sit.svc.cluster.local:9093,2@foobar-kafka-controller-2.foobar-kafka-controller-headless.sit.svc.cluster.local:9093
    log.dir=/bitnami/kafka/data

    # Interbroker configuration
    inter.broker.listener.name=INTERNAL
    offsets.topic.replication.factor=3
    transaction.state.log.replication.factor=3