GKE Networking with Antrea

In this post I will explore deploying Antrea on a GKE Cluster. Antrea is an OpenSource CNI Plugin for Kubernetes. It has some good performance and features advantages compared to other CNI Plugins in the market.
Antrea leverage OVS as a DataPlane, which is widely adopted and high-performance programmable virtual switch.
From features prospective, Antrea extends native K8s NetworkPolicy to have more granular policy by adding hierarchy of security policies called “Tiers”, cluster wide policies, priorities, and more.
Other than the policy advantages, Antrea has great flexibility when it comes to the deployment options. It support inter-nodes encryption using IPSec, overlays, or no-encap modes.
Antrea is infrastructure agnostic, and it could run on top of any cloud, and it supports multiple Operating Systems including Windows. Thanks to the portability of Open vSwitch, Antrea can use the same data plane implementation on both Linux and Windows Kubernetes Nodes.
For more information about Antrea please visit https://antrea.io
If you don’t have any experience on Antrea, then please take a look to its architecture on antrea.io
Antrea just joined CNCF recently
https://www.cncf.io/sandbox-projects/
https://blogs.vmware.com/opensource/2021/05/05/antrea-joins-cncf-sandbox/

In this post I am exploring how easy to deploy Antrea on a GKE cluster so I can use Kubernetes and Antrea Network Policies and some of the advanced Antrea feature on GKE.

Deploying a GKE Cluster

This is my first ever GKE cluster deployment. To keep it simple, I will do it from GCP UI.

From GCP Navigation Menu, go to
Compute >> Kubernetes Engine >> Clusters >> Create

I will deploy a Standard Kubernetes Cluster

You can find my “Cluster Basics” configs below, the only other change I did is under Node Pools to use Ubuntu as OS and number of nodes to 2.

Now my cluster is created, let us try to access it. We can find cluster CLI access command under “CONNECT” on the top

Here is what you will find under “CONNECT”

Note: Before accessing our cluster, we need to deploy gcloud command line and login to our google account. We can find the procedure here

We can access our cluster by pasting the command in our local terminal

gcloud container clusters get-credentials gke-antrea-cluster-1 --zone europe-central2-a --project nsx-project-147708

Lets see the running pods in our cluster

We can see that there is no CNI solution deployed in our Cluster.

I am going to create a cluster-admin ClusterRoleBinding before deploying Antrea

kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user xxx@gmail.com

Deploying Antrea

In this section I will show how to easily deploy Antrea on our GKE cluster.

1. Prepare the Cluster Nodes

Deploy antrea-node-init DaemonSet to enable kubelet to operate in CNI mode.

kubectl apply -f https://raw.githubusercontent.com/antrea-io/antrea/main/build/yamls/antrea-gke-node-init.yml

2. Deploy latest release of Antrea

kubectl apply -f https://raw.githubusercontent.com/antrea-io/antrea/main/build/yamls/antrea-gke.yml

3. Check the status of Antrea Pods

kubectl get pods -n kube-system | grep antrea

We can see that All Antrea Pods are running.
Now we have a GKE cluster running with Antrea!

Testing Kubernetes NetworkPolicy with Antrea on GKE

Lets start by deploying an application.
1. Create a new namespace for the app

kubectl create ns yelb

2. Deploy test application

kubectl apply -f https://raw.githubusercontent.com/aidrees/yelb/main/rest-review.yaml

3. View the status of the application and services, then access application using External IP

4. Test K8s Network Policy
Let us apply a NetworkPolicy between yelb-ui and yelb-appserver

kubectl apply -f https://raw.githubusercontent.com/aidrees/yelb/main/yelb-network-policy.yaml

//yleb-network-policy.yaml

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: yelb-network-policy
  namespace: yelb
spec:
  podSelector:
    matchLabels:
      app:  yelb-appserver
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: yelb-ui
    ports:
    - protocol: TCP
      port: 4567

5. Test NetworkPolicy.
Application should work, but yelb-ui should not be able to ping yleb-appserver

That shows that Antrea is enforcing K8s NetworkPolicy.

TraceFlow

Antrea has a very useful feature that emulates traffic between K8s pods. Let us check if the traffic is actually blocked by our NetworkPolicy.

To do that I am going to use Octant UI. I wrote about Antrea Plugin for Octant in a previous post.

Below is for ICMP Traffic,

Below is for TCP 4567, Which is the port we allowed in our NetworkPolicy,

We can see that our NetworkPolicy is doing what is supposed to do.

In Conclusion, it was a straight forward and simple task to deploy Antrea on GKE. Now I can start using Antrea advanced functionality on GKE. I have not tested Antrea with GKE On-Prem (Anthos), but I would assume it would pretty similar. If you did such a test, then please let me know in the comments section.

Thank you for reading!