Skip to main content

Egress Gateway Guide

Overview​

This article explains how to use Cilium's Egress Gateway and CiliumEgressGatewayPolicy to flexibly control which egress IP is used for outbound traffic from the cluster.

Known Issues​

Using Cilium's Egress Gateway feature has the following known issues:

  1. Egress policy delay for new Pods: After a new Pod starts, if it matches an Egress policy, the expectation is that the Pod's outbound traffic should go through the specified egress gateway. However, during the initial period after Pod startup, this policy may not take effect immediately, though this delay is typically very short and doesn't affect most scenarios.
  2. Incompatibility with Cilium's Cluster Mesh and CiliumEndpointSlice features.

Enabling Egress Gateway​

To enable Egress Gateway, the following conditions must be met:

  1. Enable cilium to replace kube-proxy.
  2. Enable IP masquerade using BPF implementation instead of the default iptables implementation.
  3. VPC-CNI Native Routing only: you MUST also configure ipMasqAgent.config.nonMasqueradeCIDRs to cover ALL VPC CIDRs (main + every secondary CIDR). Otherwise BPF masquerade will incorrectly SNAT cross-node Pod-to-Pod traffic to the node IP or a link-local address; the receiving node then can't resolve the SNATed src IP back to the original Pod's cilium identity, and cross-node NetworkPolicy breaks.
Overlay mode is exempt

In Overlay mode (VPC-CNI / GR), cross-node Pod-to-Pod traffic is wrapped in vxlan: the outer header is the node IP and the inner Pod IP is preserved. BPF masquerade only acts on the outer, egressing-the-physical-NIC traffic — it never sees the inner Pod IP — so nonMasqueradeCIDRs is neither needed nor desired.

Everything below about nonMasqueradeCIDRs (script behavior, helm flags, rationale) applies only to Native Routing.

One-Click Enable​

Use the script to enable Egress Gateway in one shot (handles helm upgrade and component restart automatically):

bash -c "$(curl -sfL https://raw.githubusercontent.com/imroc/tke-guide/main/static/scripts/cilium.sh)" -- enable-egress-gateway

If your network can't reach GitHub, use the site mirror:

bash -c "$(curl -sfL https://imroc.cc/tke/scripts/cilium.sh)" -- enable-egress-gateway
On Native Routing the script handles nonMasqueradeCIDRs automatically

When enable-egress-gateway runs against a VPC-CNI Native Routing cluster, the script resolves nonMasqueradeCIDRs in this order:

  1. The NON_MASQ_CIDRS="10.0.0.0/8 172.16.0.0/12 ..." environment variable (space-separated) — for non-interactive use (CI / Terraform).
  2. Auto-reuse of the cluster's existing kube-system/ip-masq-agent-config ConfigMap (TKE's ip-masq-agent addon writes the VPC main + secondary CIDRs there).
  3. Interactive prompt (default: RFC 1918 three ranges 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 — covers any valid Tencent Cloud VPC config).

On Overlay clusters the script neither asks nor injects this setting.

Manual Enable​

Pick the tab matching your cilium routing mode:

Method to enable Egress Gateway during cilium installation (highlighted lines are added/changed relative to the default install):

helm upgrade --install cilium cilium/cilium --version 1.19.4 \
  --namespace kube-system \
  --set image.repository=quay.tencentcloudcr.com/cilium/cilium \
  --set envoy.image.repository=quay.tencentcloudcr.com/cilium/cilium-envoy \
  --set operator.image.repository=quay.tencentcloudcr.com/cilium/operator \
  --set certgen.image.repository=quay.tencentcloudcr.com/cilium/certgen \
  --set hubble.relay.image.repository=quay.tencentcloudcr.com/cilium/hubble-relay \
  --set hubble.ui.backend.image.repository=quay.tencentcloudcr.com/cilium/hubble-ui-backend \
  --set hubble.ui.frontend.image.repository=quay.tencentcloudcr.com/cilium/hubble-ui \
  --set nodeinit.image.repository=quay.tencentcloudcr.com/cilium/startup-script \
  --set preflight.image.repository=quay.tencentcloudcr.com/cilium/cilium \
  --set preflight.envoy.image.repository=quay.tencentcloudcr.com/cilium/cilium-envoy \
  --set clustermesh.apiserver.image.repository=quay.tencentcloudcr.com/cilium/clustermesh-apiserver \
  --set authentication.mutual.spire.install.agent.image.repository=docker.io/k8smirror/spire-agent \
  --set authentication.mutual.spire.install.server.image.repository=docker.io/k8smirror/spire-server \
  --set operator.tolerations[0].key="node-role.kubernetes.io/control-plane",operator.tolerations[0].operator="Exists" \
  --set operator.tolerations[1].key="node-role.kubernetes.io/master",operator.tolerations[1].operator="Exists" \
  --set operator.tolerations[2].key="node.kubernetes.io/not-ready",operator.tolerations[2].operator="Exists" \
  --set operator.tolerations[3].key="node.cloudprovider.kubernetes.io/uninitialized",operator.tolerations[3].operator="Exists" \
  --set operator.tolerations[4].key="tke.cloud.tencent.com/uninitialized",operator.tolerations[4].operator="Exists" \
  --set operator.tolerations[5].key="tke.cloud.tencent.com/eni-ip-unavailable",operator.tolerations[5].operator="Exists" \
  --set routingMode=native \
  --set endpointRoutes.enabled=true \
  --set ipam.mode=delegated-plugin \
  --set devices=eth+ \
  --set cni.chainingMode=generic-veth \
  --set cni.customConf=true \
  --set cni.configMap=cni-config \
  --set cni.externalRouting=true \
  --set extraConfig.local-router-ipv4=169.254.32.16 \
  --set localRedirectPolicies.enabled=true \
  --set sysctlfix.enabled=false \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=$(kubectl get ep kubernetes -n default -o jsonpath='{.subsets[0].addresses[0].ip}') \
  --set k8sServicePort=60002 \
  --set egressGateway.enabled=true \
  --set enableIPv4Masquerade=true \
  --set bpf.masquerade=true \
  --set ipMasqAgent.enabled=true \
  --set ipMasqAgent.config.masqLinkLocal=true \
  --set ipMasqAgent.config.nonMasqueradeCIDRs[0]=10.0.0.0/8 \
  --set ipMasqAgent.config.nonMasqueradeCIDRs[1]=172.16.0.0/12 \
  --set ipMasqAgent.config.nonMasqueradeCIDRs[2]=192.168.0.0/16

Then restart cilium components to take effect:

kubectl rollout restart ds cilium -n kube-system
kubectl rollout restart deploy cilium-operator -n kube-system
Picking values for nonMasqueradeCIDRs

ipMasqAgent.config.nonMasqueradeCIDRs must cover every VPC CIDR (main + all secondary CIDRs, including both node subnets and VPC-CNI Pod subnets). The example above just uses the RFC 1918 three ranges as a catch-all — works for any valid Tencent Cloud VPC config and is the lowest-effort option.

If you'd rather be precise, pull the values straight from the cluster's built-in kube-system/ip-masq-agent-config ConfigMap (TKE's ip-masq-agent addon writes the VPC main + secondary CIDRs there automatically):

kubectl -n kube-system get cm ip-masq-agent-config -o jsonpath='{.data.config}'
Already installed via this guide — short form

If you already installed cilium using the Install cilium using helm provided in Installing Cilium, the command to enable Egress Gateway can be simplified to:

helm upgrade cilium cilium/cilium --version 1.19.4 \
  --namespace kube-system \
  --reuse-values \
  --set egressGateway.enabled=true \
  --set enableIPv4Masquerade=true \
  --set bpf.masquerade=true \
  --set ipMasqAgent.enabled=true \
  --set ipMasqAgent.config.masqLinkLocal=true \
  --set ipMasqAgent.config.nonMasqueradeCIDRs[0]=10.0.0.0/8 \
  --set ipMasqAgent.config.nonMasqueradeCIDRs[1]=172.16.0.0/12 \
  --set ipMasqAgent.config.nonMasqueradeCIDRs[2]=192.168.0.0/16
Why ipMasqAgent.config.nonMasqueradeCIDRs and not ipv4NativeRoutingCIDR?

cilium offers two ways to tell BPF masquerade which traffic NOT to SNAT:

OptionTypeSufficient?
ipv4NativeRoutingCIDRSingle CIDR❌ No. A Tencent Cloud VPC supports a "main CIDR + multiple secondary CIDRs", and VPC-CNI Pods can be allocated from any of them — a single CIDR can't express that.
ipMasqAgent.config.nonMasqueradeCIDRsCIDR list✅ Lists every VPC CIDR. TKE's built-in ip-masq-agent addon uses the same field name, so its config is reusable as-is.

So Native Routing + Egress Gateway must use nonMasqueradeCIDRs. This guide standardizes on it everywhere Native + Egress is involved.

Creating Egress Nodes​

You can create a node pool as an Egress node pool, which can later be configured to route certain Pods' outbound traffic through these nodes. Refer to the Creating New Node Pool section in Installing Cilium for creation methods.

Things to note:

  1. Use the node pool to label the scaled-out nodes (e.g., egress-node=true) to identify them as Egress Gateway nodes.
  2. If internet access is needed, assign public IPs to the nodes.
  3. To prevent regular Pods from being scheduled there, add taints.
  4. Egress node pools typically don't enable auto-scaling and set a fixed number of nodes.
  5. If you need Egress Gateway to coexist with a NAT gateway (some Pods use NAT gateway, others use Egress nodes), the egress node pool should use a separate subnet whose route table does not have NAT gateway routes. See FAQ How to make Egress Gateway coexist with NAT Gateway? for details.

Below are specific operational considerations for creating node pools:

If creating through the console, make sure to check Create Elastic Public IP:

Add Labels and Taints (optional):

If creating via terraform, refer to the following code snippet:

resource "tencentcloud_kubernetes_native_node_pool" "cilium" {
  name       = "cilium"
  cluster_id = tencentcloud_kubernetes_cluster.tke_cluster.id
  type       = "Native"
  annotations {
    name  = "node.tke.cloud.tencent.com/beta-image"
    value = "ts4-public"
  }
  # Label the scaled-out nodes with this label
  labels {
    name = "egress-node"
    value = "true"
  }
  # (Optional) Add taints to nodes to prevent regular Pods from scheduling to Egress nodes
  taints {
    key    = "egress-node"
    effect = "NoSchedule"
    value  = "true"
  }
  native {
    # Set egress node replica count
    replicas = 1
    internet_accessible {
      # Pay by traffic
      charge_type       = "TRAFFIC_POSTPAID_BY_HOUR"
      # Maximum outbound bandwidth 100Mbps
      max_bandwidth_out = 100
    }
    # Omit other necessary but unrelated configurations
  }

After the node pool is created and nodes are initialized, check which nodes are egress nodes and what public IPs are assigned using:

$ kubectl get nodes -o wide -l egress-node=true
NAME            STATUS   ROLES    AGE     VERSION         INTERNAL-IP     EXTERNAL-IP      OS-IMAGE               KERNEL-VERSION           CONTAINER-RUNTIME
172.22.48.125   Ready    <none>   3h17m   v1.32.2-tke.6   172.22.48.125   43.134.181.245   TencentOS Server 4.4   6.6.98-40.2.tl4.x86_64   containerd://1.6.9-tke.8
172.22.48.48    Ready    <none>   3h17m   v1.32.2-tke.6   172.22.48.48    43.156.74.191    TencentOS Server 4.4   6.6.98-40.2.tl4.x86_64   containerd://1.6.9-tke.8
172.22.48.64    Ready    <none>   3h17m   v1.32.2-tke.6   172.22.48.64    43.134.178.226   TencentOS Server 4.4   6.6.98-40.2.tl4.x86_64   containerd://1.6.9-tke.8

Configuring CiliumEgressGatewayPolicy​

By configuring CiliumEgressGatewayPolicy, you can flexibly define which egress IPs are used for which Pods' traffic leaving the cluster. Refer to the official documentation Writing egress gateway policies for configuration methods.

Usage Examples​

Outbound Traffic Through Fixed Egress Nodes​

If you want outbound traffic to go through fixed Egress nodes (when accessing the internet, the source IP will be fixed to the public IP bound to the Egress node), refer to the following configuration method.

Deploy an nginx workload:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest

Configure CiliumEgressGatewayPolicy to specify that this workload uses a specific egress node for internet access:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: egress-test
spec:
  selectors:
  - podSelector: # Specify which Pods this egress policy applies to
      matchLabels:
        app: nginx # Specify Pods with app=nginx label
        io.kubernetes.pod.namespace: default # Specify default namespace
  destinationCIDRs:
  - "0.0.0.0/0"
  - "::/0"
  egressGateway:
    nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.119 # egress node name
    # Important: Testing shows that in TKE environment, the internal IP of the egress node must be specified here,
    # used to determine the source IP when the egress node forwards outbound traffic. Whether forwarding internal
    # or public network traffic, the source IP used when leaving the egress node is the node's internal IP.
    egressIP: 172.22.49.119

Check the egress node:

$ kubectl get nodes -o wide 172.22.49.119
NAME            STATUS   ROLES    AGE   VERSION         INTERNAL-IP     EXTERNAL-IP    OS-IMAGE               KERNEL-VERSION           CONTAINER-RUNTIME
172.22.49.119   Ready    <none>   69m   v1.32.2-tke.6   172.22.49.119   129.226.84.9   TencentOS Server 4.4   6.6.98-40.2.tl4.x86_64   containerd://1.6.9-tke.8

You can see the node's public IP is 129.226.84.9. Enter the Pod to test the current egress IP:

$ kubectl -n default exec -it deployment/nginx -- curl ifconfig.me
129.226.84.9

The final egress IP is 129.226.84.9, which meets expectations.

Outbound Traffic Through a Group of Egress Nodes​

If you want outbound traffic to go through a fixed group of Egress nodes (when accessing the internet, the source IP will be fixed to the public IP bound to the Egress node), refer to the following configuration method.

Deploy an nginx workload:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: default
spec:
  replicas: 10
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest

Configure CiliumEgressGatewayPolicy to specify that this workload uses a group of Egress nodes for outbound traffic:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: egress-test
spec:
  selectors:
  - podSelector: # Specify which Pods this egress policy applies to
      matchLabels:
        app: nginx # Specify Pods with app=nginx label
        io.kubernetes.pod.namespace: default # Specify namespace
  destinationCIDRs:
  - "0.0.0.0/0"
  - "::/0"
  egressGateway: # This field is required. If you want to specify multiple egress nodes, you must still specify one here, otherwise it will error: spec.egressGateway: Required value
    nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.20 # egress node name
    egressIP: 172.22.49.20 # egress node internal IP
  egressGateways: # Add remaining egress nodes to this list
  - nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.147
    egressIP: 172.22.49.147
  - nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.119
    egressIP: 172.22.49.119

Testing shows that different Pods in the workload may use different egress public IPs:

$ kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' | xargs -I {} sh -c 'kubectl exec -n default -it {} -- curl -s ifconfig.me 2>/dev/null || echo "Failed"; printf ":\t%s\n" "{}"'
129.226.84.9:   nginx-54c98b4f84-5wlpc
43.156.123.70:  nginx-54c98b4f84-6jx8n
43.156.123.70:  nginx-54c98b4f84-82wmq
129.226.84.9:   nginx-54c98b4f84-8ptvh
129.226.84.9:   nginx-54c98b4f84-jfr2x
129.226.84.9:   nginx-54c98b4f84-jlrr7
43.156.123.70:  nginx-54c98b4f84-mpvpz
129.226.84.9:   nginx-54c98b4f84-s7q4s
43.156.123.70:  nginx-54c98b4f84-vsnng
43.156.123.70:  nginx-54c98b4f84-xt8bs

But all use the public IPs bound to the currently defined group of egress nodes:

$ kubectl get nodes -o custom-columns="NAME:.metadata.name,EXTERNAL-IP:.status.addresses[?(@.type=='ExternalIP')].address" -l egress-node=true
NAME            EXTERNAL-IP
172.22.49.119   129.226.84.9
172.22.49.147   43.156.123.70
172.22.49.20    43.163.1.23

All Cluster Outbound Traffic Through Egress Nodes​

If you want all Pods' outbound traffic in the cluster to go through Egress nodes, you can use podSelector: {} to select all cluster Pods:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: egress-test
spec:
  selectors:
  - podSelector: {} # Select all cluster Pods
  destinationCIDRs:
  - "0.0.0.0/0"
  - "::/0"
  egressGateway: # This field is required. If you want to specify multiple egress nodes, you must still specify one here, otherwise it will error: spec.egressGateway: Required value
    nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.20 # egress node name
    egressIP: 172.22.49.20 # egress node internal IP
  egressGateways: # Add remaining egress nodes to this list
  - nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.147
    egressIP: 172.22.49.147
  - nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.119
    egressIP: 172.22.49.119

Different Environments or Business Pods Using Different Egress Nodes​

If different environments or business Pods are isolated by namespace, you can specify that Pods in a certain namespace use specific Egress nodes for outbound traffic:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: egress-test
spec:
  selectors:
  - podSelector:
      matchLabels:
        io.kubernetes.pod.namespace: prod # Specify all Pods in prod namespace
  destinationCIDRs:
  - "0.0.0.0/0"
  - "::/0"
  egressGateway:
    nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.119
    egressIP: 172.22.49.119

If different businesses are distinguished by labels, you can specify that Pods with specific labels across all namespaces use specific Egress nodes for outbound traffic:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: egress-test
spec:
  selectors:
  - podSelector:
      matchLabels:
        business: mall # Specify all Pods with business=mall label
  destinationCIDRs:
  - "0.0.0.0/0"
  - "::/0"
  egressGateway:
    nodeSelector:
      matchLabels:
        kubernetes.io/hostname: 172.22.49.119
    egressIP: 172.22.49.119

FAQ​

Network Connectivity Issues After Policy Configuration​

First confirm if the CiliumEgressGatewayPolicy configuration method is correct. In TKE environment, ensure that egressGateway's nodeSelector only selects one node, and egressIP must be configured as that node's internal IP, otherwise connectivity issues may occur.

You can also log into the cilium pod on the egress node and execute cilium-dbg bpf egress list to view current egress bpf rules on the node:

$ kubectl -n kube-system exec -it cilium-nz5hd -- bash
root@VM-49-119-tencentos:/home/cilium# cilium-dbg bpf egress list
Source IP      Destination CIDR   Egress IP       Gateway IP
172.22.48.4    0.0.0.0/0          172.22.49.119   172.22.49.119
172.22.48.10   0.0.0.0/0          0.0.0.0         172.22.49.147
172.22.48.14   0.0.0.0/0          0.0.0.0         172.22.49.147
172.22.48.37   0.0.0.0/0          0.0.0.0         172.22.49.147
172.22.48.38   0.0.0.0/0          172.22.49.119   172.22.49.119
172.22.48.39   0.0.0.0/0          172.22.49.119   172.22.49.119
172.22.48.41   0.0.0.0/0          0.0.0.0         172.22.49.147
172.22.48.42   0.0.0.0/0          0.0.0.0         172.22.49.147
172.22.48.43   0.0.0.0/0          172.22.49.119   172.22.49.119
172.22.48.44   0.0.0.0/0          172.22.49.119   172.22.49.119
172.22.48.45   0.0.0.0/0          172.22.49.119   172.22.49.119
172.22.48.46   0.0.0.0/0          0.0.0.0         172.22.49.147
172.22.48.47   0.0.0.0/0          0.0.0.0         172.22.49.147

Source IP is the Pod IP, Egress IP is the source IP used when traffic goes through the current node, 0.0.0.0 means the current node is not forwarding traffic for the corresponding Pod IP. If all are 0.0.0.0, it means no egress rules are selecting the current node.

Unexpected Egress IP​

Egress Gateway traffic ultimately exits from the egress node. If the VPC route table of the subnet where the egress node is located has a 0.0.0.0/0 route pointing to a NAT gateway, traffic leaving the egress node for the internet will be intercepted by the NAT gateway, resulting in the egress IP being the NAT gateway's IP instead of the EIP bound to the egress node.

Troubleshooting steps:

  1. Check if the route table bound to the egress node's subnet has a 0.0.0.0/0 route pointing to a NAT gateway.
  2. If so, remove that route rule or migrate the egress node to a subnet without NAT gateway routes.
Note

The key here is the route table of the egress node's subnet, not the work node (client Pod's node) subnet's route table. Egress Gateway forwards traffic from work nodes to egress nodes via tunnel, and that tunnel communication uses VPC internal routing, which is not affected by the work subnet's public network routes.

How to make Egress Gateway coexist with NAT Gateway?​

Scenario: Most Pods in the cluster use NAT gateway for internet access by default, while only specific workloads need to go through Egress Gateway with a fixed EIP.

Solution: Place work nodes and egress nodes in different subnets, with different route tables bound to each:

  • Work subnet route table: 0.0.0.0/0 next hop is NAT gateway (Pods not matching Egress policy access internet through NAT gateway)
  • Egress subnet route table: no NAT gateway route (traffic matching Egress policy exits through the egress node's EIP)

This works because Cilium Egress Gateway uses tunnel encapsulation (VXLAN) to forward traffic from work nodes to egress nodes. The outer destination IP of the tunnel is the egress node's internal IP, which uses VPC internal routing and will not be intercepted by the work subnet's NAT gateway route:

Configuration steps:

  1. Create two subnets in the VPC (e.g., work-subnet and egress-subnet), each associated with different route tables.
  2. Add 0.0.0.0/0 → NAT Gateway to the work-subnet route table.
  3. Do not add NAT gateway routes to the egress-subnet route table.
  4. Use work-subnet for the work node pool and egress-subnet for the egress node pool.
  5. Configure CiliumEgressGatewayPolicy as normal.

Pods not matching the Egress policy will follow the normal path (BPF masquerade SNATs Pod IP to work node IP → exits from work node → work subnet route table directs traffic to NAT gateway).

Note
  1. Ensure security groups between work subnet and egress subnet allow UDP 4789 (VXLAN tunnel port).
  2. egressIP must be the egress node's internal IP, not the EIP.
  3. If the egress node itself needs internet access (e.g., pulling images), you can configure specific routing rules for the egress subnet separately, or ensure the egress node has an EIP with security group outbound rules allowing traffic.

How to route outbound traffic through machines outside the VPC?​

In certain specific scenarios, you may want some Pod outbound traffic to go through designated machines outside the VPC (for example, when the business egress IP is in another VPC, another cloud, or an IDC data center, and a third party has whitelisted that IP, making it inconvenient to change, so you need the outbound traffic to exit through the machine where this IP is located). However, when Cilium uses CiliumEgressGatewayPolicy to configure policies, it requires the Egress machine to be a node in the current cluster. Normally, nodes added to a TKE cluster are machines within the VPC. So how can you route outbound traffic through machines outside the VPC?

You can add machines outside the VPC to the TKE cluster as registered nodes, and then configure the egress gateway in CiliumEgressGatewayPolicy to be that node.

The specific steps are:

  1. Before installing Cilium, enable registered nodes first in the TKE cluster's basic information page, and check to enable dedicated line connection support (after enabling, the cluster's apiserver address will change, and since Cilium replaces kube-proxy, it needs to be aware of the apiserver address - this is why it should be enabled before installing Cilium).
  2. Install Cilium and enable Egress Gateway.
  3. Prepare the Egress machine outside the VPC. The main requirements are that the network is connected to the VPC where the TKE cluster is located, and the Linux kernel version is >= 5.10.
  4. Create a new registered node pool. It's recommended to add both Labels and Taints (Taints example: egress-node=true:NoSchedule, which prevents regular Pods from being scheduled to this node, because registered nodes cannot use the VPC-CNI network plugin, cannot be assigned Pod IPs, and can only use HostNetwork).
  5. Enter the newly created registered node pool, click to create a new node, copy the registration script as prompted and execute it on the Egress machine outside the VPC to add that machine as a node to the TKE cluster.
  6. Configure CiliumEgressGatewayPolicy as needed to route specified outbound traffic through the machine outside the VPC. Example (Note that you need to replace the hostname and the egressIP): 
    apiVersion: cilium.io/v2
    kind: CiliumEgressGatewayPolicy
    metadata:
      name: egress-test
    spec:
      selectors:
      - podSelector: # Specify which Pods this egress policy applies to
          matchLabels:
            app: nginx # Specify Pods with the app=nginx label
            io.kubernetes.pod.namespace: test # Specify the test namespace
      destinationCIDRs:
      - "0.0.0.0/0"
      - "::/0"
      egressGateway:
        nodeSelector:
          matchLabels:
            kubernetes.io/hostname: node-10.111.128.148 # Egress registered node name
        # Important: Testing shows that in TKE environments, you must specify the egress node's
        # internal IP here. This determines what source IP the egress node uses when forwarding
        # outbound traffic. Whether forwarding internal or public network traffic, the source IP
        # when leaving the egress node is the node's internal IP.
        egressIP: 10.111.128.148

Reference Materials​