Kubernetes Man-in-the-Middle with no patch in sight. What does CVE-2020-8554 mean to your organization?
Kubernetes gives users a lot of flexibility to customize the system in order to support various use cases.That’s why DevOps teams use it as their container orchestration platform of choice. This is a double-edged sword. Typically, organizations use RBAC to limit the access to resources, but any enterprise security pro knows that it's hard to manage when you have hundreds of users with unique access requirements to different parts of the cluster. What makes this vulnerability more dangerous is that the namespace boundary is crossed. Anyone who has access rights at the pod and service level, could circumvent established rules and processes.
Some security firms recommend solutions that introduce overhead to the DevOps process. Because NeuVector enforcers sit inside the cluster, it is able to protect application containers, as well as the system containers, actively. You can learn more about NeuVector's patented protection for Kubernetes here.
Before we dive into the protective measures, let's first take a look at how the attack is executed.
The following diagram demonstrates one scenario of the attack sequence.
The attacker first deploys an Man-in-the-middle (MitM) container in the cluster and exposes a service with the externalIPs option. Because the namespace users are typically allowed to deploy their own pods and services in their own namespaces, these activities look totally normal to the cluster administrators.
The following is a LoadBalancer service we created for the demo purpose. Notes that the external IP is defined as 22.214.171.124. This is the IP address that example.com is resolved into.
- nodePort: 32482
After the service is created, we can see that the external IP is accepted by Kubernetes.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
external LoadBalancer 172.21.220.18 126.96.36.199 80:32482/TCP 2d
Now, say the application in Joe's namespace wants to reach example.com. First, the application sends a DNS query asking for the IP address of example.com. The DNS response tells the application the IP address is 188.8.131.52.
Now the application sends a HTTP request to the resolved IP, instead of going to the real destination, because the IP address matches one of exposed services, Kubernetes network plugin directs the connections to the service and eventually reaches the attacker's pod.
It is not so interesting to just fake a destination, but the attacker's app can choose to be acting as a proxy and forward the connections to the real example.com. Now the attacker is capable of looking into requests and responses exchanged between the client and the server. If sensitive information are transmitted, they will be exposed to the attackers. The attacks can even manipulate the content. This is Man-in-the-Middle in action.
As an example, here is the connection we sent from the client.
# oc exec -it client -n client -- curl example.com
server_version=nginx: 1.10.0 - lua: 10001
-no body in requesdit svc external -n serverom
Edit cancelled, no changes made.
We simulate the attacker's application with a Nginx server. As you can see in the response of the client's request. The connection didn't reach example.com but the Nginx server.
An extra attack vector
Besides externalIPs, Kubernetes offers another "hidden" service type, ExternalName. This service allows users to map a service to a DNS name.
The attacker then can expose this service for other applications to access. Although the extra service doesn't change the actual network flow, it makes the behavior more complicated to be detected.
The Network View
Although we use the public server address as an example, the target can be legacy servers and cloud services as well. What makes this attack more stealthy are the facts that the attack can be initiated by the insiders, and use Kubernetes' native definitions without violating RBAC or namespace isolation. It looks totally legal from Kubernetes' point of view.
Most defensive actions discussed on the Internet focus on checking Kubernetes' configurations. We believe that is a necessary step. NeuVector supports Kubernetes admission control and can be integrated with OPA (Open Policy Agent) to audit the configurations. However, given the potential impacts of this Man-in-the-Middle attack, more proactive actions should be taken. The visibility into the network layer offers valuable insights in this case.
If we examine the network connections on the wire using NeuVector's network graph, we can see the following picture. (source-ip-app is the attacker's pod in the picture.) From the client's point of view, both DNS and HTTP requests go to the external network. The connection to the attacker's pod actually comes from a node in the cluster. The external IP service acts very much like a nodePort service.
This picture doesn't give the end-to-end view. In Part Two of this blog, we connect the dots by correlating Kubernetes' resources and security automation.
- Container Security
- Cloud Security
- Docker Security
- Kubernetes Security
- Press Releases
- Container Threats
- Network Security
- Container Networking
- Container Firewall
- RedHat OpenShift
- cloud-native security
- Ebooks + Guides
- data loss prevention
- security automation
- service mesh
- Cloud services
- Kubernetes system
- NeuVector Product
- AWS ECS
- CIS Benchmark
- Customer Stories
- Cyber Attacks
- Infrastructure as Code
- Open Source
- PCI Compliance
- Slider Ebooks & Guides
- security mesh
- shift left
- App Mesh
- Black Duck Software
- Docker Hub
- Docker Networking
- Docker container firewall
- Financial Services
- IBM Cloud
- Internet of Things
- VMware Photon
- Virtual Patching
- admission control
- application security
- container risk score
- cpu optimization
- data breach
- real-time containers
- sensitive data