Antrea is an OpenSource CNI Plugin for Kubernetes. It has some good performance and features advantages compared to other CNI Plugins in the market.
Antrea leverage OVS as a DataPlane, which is widely adopted and high-performance programmable virtual switch.
From features prospective, Antrea extends native K8s NetworkPolicy to have more granular policy by adding hierarchy of security policies called “Tiers”, cluster wide policies, priorities, and more.
Other than the policy advantages, Antrea has great flexibility when it comes to the deployment options. It support inter-nodes encryption using IPSec, overlays, or no-encap modes.
Antrea is infrastructure agnostic, and it could run on top of any cloud, and it supports multiple Operating Systems including Windows. Thanks to the portability of Open vSwitch, Antrea can use the same data plane implementation on both Linux and Windows Kubernetes Nodes.
For more information about Antrea please visit https://antrea.io
If you don’t have any experience on Antrea, then please take a look to its architecture on antrea.io
Antrea just joined CNCF recently
https://www.cncf.io/sandbox-projects/
https://blogs.vmware.com/opensource/2021/05/05/antrea-joins-cncf-sandbox/
https://github.com/vmware-tanzu/antrea
In this post I will show how to easily deploy an OpenShift Cluster with Antrea using Antrea Operator.
I am using OpenShift IPI on vSphere, but same steps applies on any other Cloud Provider.
Please note that Antrea is already certified CNI for OpenShift with a certified Operator.
https://catalog.redhat.com/software/operators/detail/5fb647d793431a8f675c4e15
https://access.redhat.com/articles/5436171
Infrastructure Prerequisites
For this procedure, we need a DHCP Server and two DNS entries.
DNS Entries | DNS Entries Example | IP Address | IP Address Example |
api.{clusterID}.{domain_name} | api.ocp.vmwdxb.com | API VIP | 192.168.25.201 |
*.apps.{clusterID}.{domain_name} | *.apps.ocp.vmwdxb.com | Ingress VIP | 192.168.25.202 |
Creating OpenShift Manifests
First we will create an OpenShift Installation Directory and create install-config.yaml
mkdir antrea-test
cd antrea-test
nano install-config.yaml
In the install-config.yaml, we need to add networking section with antrea as networkType.
apiVersion: v1
baseDomain: vmwdxb.com
compute:
- hyperthreading: Enabled
name: worker
replicas: 3
platform:
vsphere:
cpus: 4
coresPerSocket: 2
memoryMB: 16384
osDisk:
diskSizeGB: 120
controlPlane:
hyperthreading: Enabled
name: master
replicas: 3
platform:
vsphere:
cpus: 4
coresPerSocket: 2
memoryMB: 16384
osDisk:
diskSizeGB: 120
metadata:
name: ocp
networking:
networkType: antrea
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineCIDR: 192.168.25.0/24
serviceNetwork:
- 172.30.0.0/16
platform:
vsphere:
vcenter: vcsa-tkg-01.vmwdxb.com
username: administrator@vsphere.local
password: VCENTER-PWD
datacenter: Datacenter
defaultDatastore: OCP-VSAN
fips: false
network: OCP-Nodes-Segment
cluster: Ali-OCP-Cluster
apiVIP: 192.168.25.201
ingressVIP: 192.168.25.202
pullSecret: '...'
sshKey: '...'
Please modify the inputs as per your environment and add the pullSecret and sshKey as per the standard OpenShift deployment.
Now we can create OpenShift Manifests, but before doing that, we should backup our install-config.yaml somewhere because it will be consumed and deleted in the manifests creation process.
cp install-config.yaml ../install-config.yaml.backup
openshift-install create manifests
Adding Antrea Operator Manifests
Now we will get the Antrea Operator Manifests from github
cd..
git clone https://github.com/vmware/antrea-operator-for-kubernetes.git
Before copying Antrea Manifests to OpenShift folder, we may need to do some changes. first we need to point to an Antrea Operator Image in the operator.yaml.
I published an image on aalidrees/antrea-operator. Feel free to use it.
nano antrea-operator-for-kubernetes/deploy/operator.yaml
- name: antrea-operator
# Replace this with the built image name
# image: REPLACE_IMAGE
image: aalidrees/antrea-operator:v0.0.9-test
Optionally Antrea config and version could be modified in below yaml.
nano antrea-operator-for-kubernetes/deploy/operator.antrea.vmware.com_v1_antreainstall_cr.yaml
Now we can copy Antrea Deploy folder to OpenShift Manifests Folder
cp antrea-operator-for-kubernetes/deploy/* antrea-test/manifests
Creating OpenShift Cluster
Now we can create our OpenShift Cluster with Antrea
cd antrea-test
openshift-install create cluster
Note: if the cluster is being deployed on vSphere, then we need to add vCenter Certificates before creating the cluster. You can find how to do that and other good steps related to creating OpenShift on vSphere.
https://veducate.co.uk/deploy-vsphere-openshift-machine-resources/
Ubuntu Example,
curl -O https://{vCenter_FQDN}/certs/download.zip unzip download.zip cp certs/win/* /usr/local/share/ca-certificates update-ca-certificates
OCP 4.6 Installation Issues
Note: Antrea needs podCIDR in the nodes specifications to work, there is an issue in OCP4.6 that the podCIDR is not configured in the nodes specs. You may need to configure that manually during the bootstrap. This Problem is solved in Antrea 1.0 by delegating the Pods IPAM to Antrea itself.
example,oc edit node ocp-xxxx-master-0
specs:
podCIDR: 10.128.0.0/23
oc edit node ocp-xxxx-master-1
specs:
podCIDR: 10.128.2.0/23
….
repeat for all nodes
OCP 4.7 Installation Issues
Can not access K8s API once it is migrated from Bootstrap VM to Master Nodes. This Problem is solved in Antrea 1.0 by delegating the Pods IPAM to Antrea itself.
Commands Summary
mkdir antrea-test
cd antrea-test
nano install-config.yaml
cp install-config.yaml ../install-config.yaml.backup
openshift-install create manifests
cd..
git clone https://github.com/vmware/antrea-operator-for-kubernetes.git
nano antrea-operator-for-kubernetes/deploy/operator.yaml
nano antrea-operator-for-kubernetes/deploy/operator.antrea.vmware.com_v1_antreainstall_cr.yaml
cp antrea-operator-for-kubernetes/deploy/* antrea-test/manifests
cd antrea-test
openshift-install create cluster
In this post I showed how to deploy an OpenShift 4.x cluster with Antrea as a CNI Plugin.
Thank you for Reading!