currently I had explore Multus CNI for my experimental project about migrate Virtual Machine or some of Application that we don't want (cannot change source code) about IP address so I had constraint to challenge that I need to use fixed IP
currently I had succeed in using Multus with Pod that have secondary IP via MacVLAN
accross Node in my Kubernetes already.
I had two subnet 192.168.15.0/24 (Public Zone)
and 192.168.16.0/24(Private Zone)
for each IP that reside in their zone (subnet) it can connect successfully between Kubernetes Node like Worker A contain pod 192.168.15.10
can ping to Worker B contain pod 192.168.15.11
because they are same subnet (same MACVLAN)
**But I require to have pod with different MACVLAN (different subnet)can connect together ** like this Worker A contain pod 192.168.15.10
can ping to Worker B contain pod 192.168.16.10
(Public subnet connect to Secure subnet zone)
In normal Network concept we will require Router
right ? Because currently my topology will look similar like this picture
credit: https://www.practicalnetworking.net/stand-alone/routing-between-vlans/
Example Picture of Logical Switch when using VLAN
As we see in the picture currently we have different subnet and it don't have some intermediate cable link between switch so it cannot routing/connect. So how can I create logical Router to succeed this attempt ? Is it possible ? Or I'm misleading in any Networking/Kubernetes design concept ?
In this picture you will see that my pod which attach to Public Subnet Interface
can ping/connect together across Kubernetes Node already due to macvlan
but if I using bridge it will require both of them reside in same host.
Picture of my Pod and Network Interface
I want to try something like this because I want to simulate network topology similar to on-premise Virtual Machine so I can have some automated Migrate Virtual Machine to Kubernetes without any changing code. (Currently I had explore KubeVirt too)
But in my context I don't have enterprise solution like ESXI or anything like that just plain Virtual Machine snapshot and upload to Kubernetes by using KubeVirt only so Network Topology is foundation of this migration experimental.
Thank you so much for reply any idea/helping in advance :)
Here is my example
Example of my NetworkAttachmentDefinition
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: macvlan-public
namespace: legacy-company
spec:
config: >-
{
"cniVersion": "0.3.1",
"type": "macvlan",
"bridge": "macvlan-public-zone",
"ipam": {
"type": "host-local",
"subnet": "192.168.15.0/24",
"rangeStart": "192.168.15.10",
"rangeEnd": "192.168.15.200"
}
}
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: macvlan-secure
namespace: legacy-company
spec:
config: >-
{
"cniVersion": "0.3.1",
"type": "macvlan",
"bridge": "macvlan-secure-zone",
"ipam": {
"type": "host-local",
"subnet": "192.168.16.0/24",
"rangeStart": "192.168.16.10",
"rangeEnd": "192.168.16.200"
}
}
---
Example Pod with Multus Secondary Network Interface
kind: Deployment
apiVersion: apps/v1
metadata:
name: bridge-public
labels:
app: bridge-public
spec:
replicas: 1
selector:
matchLabels:
app: bridge-public
template:
metadata:
labels:
app: bridge-public
annotations:
k8s.v1.cni.cncf.io/networks: '[
{ "name": "macvlan-public"}]'
spec:
nodeSelector:
kubernetes.io/hostname: 10.111.147.164
serviceAccountName: default
containers:
- name: bridge-public
image: "quay.io/linxianer12/nettool:0.0.4"
imagePullPolicy: Always
securityContext:
capabilities:
add:
- "NET_RAW"
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: bridge-secure
labels:
app: bridge-secure
spec:
replicas: 1
selector:
matchLabels:
app: bridge-secure
template:
metadata:
labels:
app: bridge-secure
annotations:
k8s.v1.cni.cncf.io/networks: '[
{ "name": "macvlan-secure"}]'
spec:
serviceAccountName: default
containers:
- name: bridge-secure
image: "quay.io/linxianer12/nettool:0.0.4"
imagePullPolicy: Always
securityContext:
capabilities:
add:
- "NET_RAW"