Kubernetes Ingress 服务负载均衡

本文基于 Kubernetes 1.4.5 + Calico CNI 流水操作记录，关于 Kubernetes Ingress 资源的理论知识可参考官方文档 Ingress Resources。

1. 准备 Ingress 所需配置文件

# ls
dashboard-ingress.yaml  default-http-backend.yaml  default-tcp-configmap.yaml  nginx-ingress-lb.yaml

default-http-backend.yaml 用于 HTTP 服务暴露

# cat default-http-backend.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: default-http-backend
spec:
  replicas: 1
  selector:
    app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.0
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi

default-tcp-configmap.yaml 用于 TCP 服务暴露

# cat default-tcp-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: default-tcp-configmap

nginx-ingress-lb.yaml Ingress controller

因为使用的 Calico cni 网络，基于主机的端口绑定有问题，所以这里采用的 hostNetwork 方式。具体关于 CNI 的 issue 可参考 HostPort seemingly not working

# cat nginx-ingress-lb.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nginx-ingress-lb
spec:
  template:
    metadata:
      labels:
        name: nginx-ingress-lb
    spec:
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
        name: nginx-ingress-lb
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        # use downward API
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=kube-system/default-http-backend
        - --tcp-services-configmap=kube-system/default-tcp-configmap

2. 创建 `Ingress` 所需要的资源

# kubectl create -f default-tcp-configmap.yaml --namespace=kube-system
# kubectl create -f default-http-backend.yaml --namespace=kube-system
# kubectl create -f nginx-ingress-lb.yaml --namespace=kube-system
# kubectl get configmap --namespace=kube-system
NAME                    DATA      AGE
default-tcp-configmap   1         1m
# kubectl get rc default-http-backend --namespace=kube-system
NAME                   DESIRED   CURRENT   READY     AGE
default-http-backend   1         1         1         1m
# kubectl get daemonsets --namespace=kube-system
NAME               DESIRED   CURRENT   NODE-SELECTOR   AGE
nginx-ingress-lb   2         2         <none>          1m

3. 服务暴露

3.1. HTTP 服务暴露

HTTP 服务需要创建 kind 为 Ingress 的资源，以 kubernetes dashboard 为示例：

# cat dashboard-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kubernetes-dashboard-ingress
  namespace: kube-system
spec:
  rules:
  - host: k8sui.test.com
    http:
      paths:
      - path: /
        backend:
          serviceName: kubernetes-dashboard
          servicePort: 80
# kubectl create -f dashboard-ingress.yaml
# kubectl get ingress -o wide
NAME                           HOSTS                ADDRESS                     PORTS     AGE
kubernetes-dashboard-ingress   k8sui.test.com       192.168.0.10,192.168.0.11   80        8s

修改 hosts 文件绑定域名到任何一个 nginx-ingress-lb 节点，即可通过 k8sui.test.com 即可成功访问 Kubernetes dashboard

3.2. TCP 服务暴露

TCP 服务暴露需要更新 configmap ，以 redis 服务为例:

# cat default-tcp-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: default-tcp-configmap
data:
  9000: "default/redis:6379"
# kubectl replace -f default-tcp-configmap.yaml --namespace=kube-system

以上表示暴露 default namespace 下服务名为 redis ，端口为 6379 的服务到 nginx-ingress-lb 所在节点的 9000 端口。

更新 configmap 之后通过 <任何一个节点IP>:9000 即可访问集群内部的 redis 服务

# redis-cli -h 192.168.0.10 -p 9000
192.168.0.10:9000> info
# Server
redis_version:3.2.5

关于 Kubernetes ingress controllers nginx 的更详细内容可以参看 ingress/controllers/nginx，包括 HTTPS、UDP 等实现方式。

1. 准备 Ingress 所需配置文件

2. 创建 Ingress 所需要的资源

3. 服务暴露

3.1. HTTP 服务暴露

3.2. TCP 服务暴露

2. 创建 `Ingress` 所需要的资源