How Kubernetes Networking Evolved: From CNI to Cilium, and What eBPF Changed

Networking: The Part Kubernetes Deliberately Left Alone

Networking was the thing that impressed me most when I first got my hands on Kubernetes. Pods get IPs on their own, they talk to each other, and Services even handle load balancing. Then I started operating clusters, looked under the hood, and found something odd. Kubernetes itself barely implements any networking.

The assumption Kubernetes makes when it schedules a Pod is surprisingly short.

"This Pod must get an IP address, and it must be able to reach everything else."

That's the whole declaration. Actually making it true is somebody else's job. Creating Linux network namespaces, building routing tables, standing up overlays like VXLAN, programming iptables rules, managing IP allocation — Kubernetes never picked up any of that heavy lifting. It handed the whole thing off. The recipient is the Container Network Interface.

At first it looked like an unfinished design to me. Why push something this important outside the box? It took a while before I understood this was a deliberate choice.

Container Network Interface: The Contract for Outsourcing Networking

CNI is both a specification and a plugin ecosystem. It defines the contract between Kubernetes and whatever actually does the networking. The flow is short. Kubernetes calls out "set up networking for this Pod," the CNI plugin does the real configuration work, and when it's done it reports back "finished, here's the networking info." The runtime lifecycle has just three operations: ADD to create networking for a new Pod, DELETE to tear it down when the Pod dies, and CHECK to verify state.

When kubelet creates a Pod, it executes the CNI binary. The plugin allocates an IP address (IPAM), creates a virtual ethernet pair (veth), pushes one end of that interface into the Pod's namespace, and finally configures routing and policy rules.

Without a CNI, Kubernetes networking simply stops. Pods get no IPs, and Services have nowhere to send traffic. Build a fresh cluster and forget to install a CNI, and you'll watch Pods sit in ContainerCreating forever. That's the moment "Kubernetes doesn't do networking" stops being a sentence and becomes something you feel.

Why Kubernetes Chose the Plugin Model

Think back to the early Kubernetes era. Every networking vendor showed up with their own product. Cloud providers wanted native integration, virtualization vendors pushed their SDN stacks, and the open source community built overlays. Everyone wanted to be the standard.

Kubernetes made an interesting call here. It didn't pick a winner. It standardized the interface instead of the implementation.

That's why the same application moves between environments. A simple overlay on a local cluster, direct attachment to the provider network on managed cloud, high-performance routing on bare metal. The application stays the same and only the layer underneath gets swapped. It's also why new networking ideas can grow outside the Kubernetes core. A new technology doesn't require surgery on the main body. The CNI spec itself kept widening too — from basic IP allocation to bandwidth control, multiple network attachments, and native kernel acceleration.

I've had the chance to look inside a number of clusters, and the networking stacks were all different even though they were all running the same Kubernetes. The plugin model is what made that possible. Each of them picked what fit their environment without touching the core. Leaving the middle empty and fixing only the interface is what raised the pace of experimentation across the whole ecosystem.

Where Traditional CNIs Hit the iptables Wall

Open up an existing implementation like Flannel and the sequence looks familiar. The CNI plugin allocates an IP from the node subnet, a veth pair connects the Pod to the host network, routing and forwarding rules go in, and cross-node traffic gets encapsulated into a tunnel (usually VXLAN).

Play it out as Pod A sending a packet to Pod B. The packet leaves the Pod through the virtual interface, the node wraps it inside a VXLAN packet and ships it over UDP to the destination node, which decapsulates it and delivers it to the target Pod.

At small scale this works fine. The trouble starts when the cluster grows. Traditional CNIs lean on iptables for everything — routing decisions, load balancing, policy enforcement. But iptables evaluates rules sequentially, so as Pod count grows, rule count grows with it. Hundreds of Pods means hundreds of rules; thousands of Pods means thousands of rules. On large clusters this shows up as measurable packet latency and CPU overhead.

Every single packet has to walk a long rule chain to the end, inside the kernel.

This was an architectural bottleneck. Past a certain line, the cost of walking the chain climbs to a level you can't ignore. Every individual Pod looks fine, but the network as a whole gets heavy — a pattern you've probably seen somewhere before.

Cilium and eBPF: Changing How, Not What

Let me clear up one misconception. Cilium doesn't replace CNI. It's another plugin implementing the CNI spec. The difference isn't in what it does, it's in how.

What Cilium puts in place of iptables is the extended Berkeley Packet Filter — a programmable framework built directly into the Linux kernel. Where iptables evaluates a long sequential rule chain from the top down, eBPF attaches pre-compiled programs at kernel hook points and runs them right there. Hooks exist at several stages: before the packet reaches the network stack, during ingress and egress processing, and down at the per-application socket level. Cilium implements routing, service load balancing, network policy, and observability on top of these points.

Here's the biggest change. Instead of walking thousands of rules, the packet does an O(1) lookup against a kernel map. The model shifts from walking a rule chain to poking a map once. Add more Pods and the lookup cost doesn't grow linearly with them.

A Look Inside the Cilium Architecture

Every Kubernetes node runs a Cilium agent as a DaemonSet, handling five roles.

역할	설명
CNI 서버	kubelet의 ADD/DELETE 호출에 응답
IP 주소 관리	클러스터/클라우드 통합으로 Pod 주소 할당
eBPF 매니저	컴파일된 프로그램을 커널 훅에 로드
Kubernetes Watcher	클러스터 상태를 eBPF 맵으로 동기화
Hubble	네트워크 플로우 실시간 가시성 제공

The center of gravity sits between the Watcher and the eBPF maps. Rather than reprogramming thousands of rules every time, Cilium continuously syncs Kubernetes state into kernel data structures. Kubernetes says "this Pod was added," and that change lands straight in a kernel map.

If a traditional CNI behaves like a static router, Cilium is closer to a programmable network OS running inside the kernel. It feels less like applying policy and more like defining network behavior as code.

What Changes in Real Traffic Flow

Traffic leaving a Pod gets evaluated by an eBPF policy program, skips the iptables chains entirely, and routes straight to its destination. Latency drops to the microsecond range.

Hitting a Service IP works the same way. Cilium does the load balancing directly in kernel space. No kube-proxy, no NAT chains, no rule explosion. Dropping kube-proxy is a pretty big deal from an operations seat. A whole category of headaches — rule sync delays, conntrack issues — disappears with it.

Layer-7 Security Policy

Cilium doesn't only look at IP addresses. It enforces policy based on application behavior: allowing HTTPS traffic only to specific domains, denying unknown external destinations, controlling access at the HTTP path level. Policy moving from L3/L4 up to L7 means you draw the security boundary by "what is this traffic trying to do" rather than by IP.

Observability: Unexpectedly the Biggest Win

Honestly, the part of Cilium I felt most was not performance — it was this. Debugging Kubernetes networking used to be packet captures and guesswork on repeat. Fire up tcpdump, narrow down by instinct which node was blocking what. Some days I'd dig into three wrong nodes before hitting the actual cause.

Cilium opens up real-time flow telemetry through Hubble. Which Pod talked to which Service, and whether policy allowed or denied that traffic, is right there. You get DNS and HTTP level visibility too, and you can go back and trace flows that already happened.

You move from asking "why did the network fail?" to just looking at what the network is doing. During an incident that difference matters more than you'd think. The guessing stage drops out entirely.

Checklist Before Going to Production

1. 충분한 eBPF 지원을 갖춘 Linux 커널 버전
2. 최신 Cilium 릴리스
3. kube-proxy 교체 활성화
4. Hubble을 통한 관찰가능성
5. 적절한 라우팅 또는 BGP 통합

The first one trips people up most, especially on bare metal or with older OS images. If the kernel can't back the eBPF features properly, you only get half of what Cilium offers. If you're evaluating adoption, check the kernel version first.

From an operations angle, the interesting thing is that capabilities go up while complexity often goes down. Networking, load balancing, and security that used to live in separate places get consolidated into one platform. Fewer components to manage means a smaller failure surface.

So

Looking back, the starting point was that decision not to pick a networking winner. Kubernetes left the core empty and fixed only the CNI interface, and the next implementation grew in that empty space. Networking that leaned on overlay tunnels and iptables rules for years now runs on programmable kernel logic.

The reason Cilium gets attention is surprisingly plain. Kernel capabilities finally matured, so networking could keep up with the speed and scale Kubernetes promised from day one.

The number of teams moving to Cilium in 2026 sits on top of that context, I think. A better implementation grew in the space that was left open, and it's climbing into the standard slot now. Though how long that slot stays put, I honestly don't know.