WAF With EKS, Using Only k8s Controllers

Hadar Sonego, Senior DevOps Engineer in Engineering

5 min read

Devops Startup Heroes vol.1

As a young startup, we must leverage our creativity in order to build secure and resilient infrastructure while considering cost and budget, and here’s a classic example I recently encountered.

Our goal was to ensure that we protect our internet facing services with a proper WAF as it is a SOC2 mandatory security control.

The obvious (some would say - naive) approach is to use a paid service to do so. Unfortunately, this is an expensive option which is also difficult to manage and maintain as part of our entire GitOps operations.

Searching for a creative and cost-effective approach, I began searching for other solutions which would abide by five top considerations:

Cost effective
Secure
Easy to use
Easy to maintain
Easy to integrate with our existing infrastructure ecosystem

….and this is what I came up with.

A native AWS solution that can easily integrate with our entire EKS cluster ecosystem.

The problem

We found that the only load balancer that supports these capabilities on AWS is ALB (Amazon Application Load Balancer), and the only way to use ALB on top of EKS is to use the Amazon load balancer controller.

The problem is that the AWS K8S LB controller will create an ALB instance for each Ingress object we create in our cluster, resulting in one physical load balancer per application or service that we want to expose to the internet - and that can be very expensive and hard to maintain.

Determined to find an optimal solution, we searched for an ingress controller that provides a single point of connection to the cluster (one load balancer for all incoming traffic to the cluster), and came across one of the most common ingress controllers out there - Nginx ingress controller.

To learn more about NGINX Ingress Controller, click the image.

The Nginx ingress controller, however, can only create and manage NLB load balancers, which can’t act as WAF like ALBs can.

An appropriate question would be - why not manually create the ALB and forward the traffic to the single NLB that the Nginx controller creates? The answer is manual configurations are a hassle to deal with, and there is no way to add NLB to the Target group directly, only using a static IP address.

‍

The hybrid solution

Why not glue both ingress controllers together?

We decided to use the AWS load balancer controller to manage the ALB instance and its life cycle to gain the WAF control, and to use the Nginx ingress controller to control the routing of traffic to the internal services.

To do this, we had to install both controllers in the cluster.

IMPORTANT — When doing so, make sure to use the Nginx ingress controller only with NodePort type services.That’s the trick.

Now that both ingress controllers were installed, we created the connection between the two. We had to create a K8S ingress object (ALB ingress Class) that will act as the connector between them.


apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/certificate-arn: 
    alb.ingress.kubernetes.io/healthcheck-path: /nginx-healthcheck
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/subnets: 
    alb.ingress.kubernetes.io/wafv2-acl-arn: 
    kubernetes.io/ingress.class: alb
  name: alb-ingress-connect-nginx-{{.Values.mode}}
  namespace: 
spec:
  defaultBackend:
    service:
      name: 
      port:
        name: http

‍

Notes:

We had to make sure the Nginx health check endpoint is enabled, so that the ALB (target group) can validate the communication with the Nginx ingress.


alb.ingress.kubernetes.io/healthcheck-path: /nginx-healthcheck

It is important to create Web ACLs (WAF) rules.
There are managed ones that are easy to apply, and there are also some out-of-the-box free ones provided by AWS that are very useful (docs).
Once we created the web ACL, we then needed to attach it to our ALB using the following annotation(docs).


alb.ingress.kubernetes.io/wafv2-acl-arn:

DNS problem

We use a service called ExternalDNS. Using this service on the EKS cluster gives us the ability to annotate our services with the following annotation —


external-dns.alpha.kubernetes.io/hostname:

This annotation will create a CNAME DNS record that points to the load balancer that serves this service.

And then — we ran into a problem. We were using the Nginx ingress class on our ingress objects, and our nginx ingress controller did not create a load balancer, so the annotation did not work.

DNS solution

We noticed that ExternalDNS service allows us to annotate the service and force DNS value for the record —


external-dns.alpha.kubernetes.io/hostname: 
external-dns.alpha.kubernetes.io/target:

This way, all of our services will point to the same ALB and nginx will route the traffic to the right service (in our case, via hostname).

‍

Diagram

Creation workflow

Create the ingress object that will make a call to the AWS ingress controller, which makes the connection between the ALB and the Nginx NodePort service.
The ingress controller pod will span the ALB and forward all traffic to Nginx nodes.
The Nginx service will receive the requests and forward the traffic to the relevant service.

Traffic workflow

A request comes into the ALB with a specific hostsname\path (depends on the service ingress object).
The ALB will forward the traffic to the NGINX service.
The Nginx service will forward the traffic to the Nginx pods.
The Nginx pods will forward the traffic to the relevant service in the cluster.

The result

All the traffic that reaches the ALB will be forwarded to the Nginx service directly, then - depending on the request - the Nginx will forward the traffic to the right service.

In order to expose a service from your cluster to the internet, we needed to create an Ingress object and use Nginx ingress class on this ingress object (you can read more about it here).

Easy as that! :)

‍

See the Opus Platform in Action

Cybersecurity Consolidation: The Key to Stronger Security

February 13, 2025

SSVC Explained: How It Transforms Vulnerability Prioritization

February 5, 2025

How to Achieve and Maintain FedRAMP Compliance

February 2, 2025

Cloud-to-Code Integration: Enhancing Security from Cloud to Codebase

January 25, 2025

Cloud Security Posture Management (CSPM): A Comprehensive Guide

January 14, 2025

Remediation as Security Immunization: Protecting Against Exploits, Malware, and Ransomware

January 25, 2025

Security Remediation Insights - Looking Back at 2024, Ahead to 2025

January 15, 2025

No-Code Automation in 2025: Choosing and Implementing Tools

January 14, 2025

Build vs. Buy: A Guide for Security Remediation Platforms

January 14, 2025

How To Design an Effective Vulnerability Management Program

January 14, 2025

Exposure Assessment Platforms (EAPs): A Cybersecurity Essential

November 20, 2024

NYDFS Cybersecurity Regulation: Key Compliance Steps

January 14, 2025

Is Python the Right Tool for Your Cyber Security Strategy?

January 14, 2025

What Is DORA Compliance? A Practical Guide

January 14, 2025

Opus' Ownership Detection & No-Code Workflow Builder

November 5, 2024

Remediation vs. Mitigation: Key Differences Explained

November 7, 2024

What Is Vulnerability Remediation & How It Works

January 14, 2025

Vulnerability Assessment: Definitions, Methodology & Tools

November 7, 2024

Wrapping Up Season One of "30 Min on: Vulnerability Management"

October 21, 2024

CVE and CVSS: Vulnerability Assessment & Severity Ratings

October 29, 2024

CVSS Scores: How They Enhance Vulnerability Assessment

February 12, 2025

What CVE Scores Mean for Your Cybersecurity Strategy

October 29, 2024

The 2025 Guide to Risk-Based Vulnerability Management

January 12, 2025

Secure SDLC: Why and How to Integrate Security In Your Development Life Cycle

October 20, 2024

Application Security Posture Management (ASPM): How It Works & Top Tools in 2025

January 12, 2025

How to Prioritize Vulnerabilities Effectively in 2025

January 12, 2025

The Opus Multi-Layer Prioritization Engine: What is it All About?

September 10, 2024

EPSS Score: Improving Vulnerability Prioritization in 2025

January 12, 2025

Opus Platform Explained: Cloud-Native Vulnerability Remediation

October 20, 2024

It’s Time to Get Proactive About Vulnerability Remediation

October 29, 2024

The Vulnerability Strikes Back - TABLETOP PART TWO

August 13, 2024

Opus Security Joins CrowdStrike Marketplace

September 29, 2024

Cloud-to-Code Integration for Comprehensive Remediation

January 15, 2025

Understanding CTEM: How Opus Security Can Help

September 22, 2024

This Is What Your Security Champions Program Is Missing

September 29, 2024

Risk-Based Vulnerability Management (RBVM) in a Cloud-Native World

October 7, 2024

How Business Impact Analysis Facilitates Effective Remediation

October 20, 2024

[Case Study] How Opus Solved the Puzzle of Disparate Tools and Alerts

October 7, 2024

It’s Time for Security Operations to Ditch Excel

February 13, 2025

Security Remediation Budgeting - Everything You Need to Know

October 20, 2024

How to Achieve More Effective Vulnerability Management?

August 13, 2024

Checklist: Tracking Down Key Stakeholders for Effective Security Remediation

October 2, 2024

Streamlining Security Remediation: The Convergence of CTEM and ASPM into a Single Operating Platform

September 22, 2024

How to Locate Key Stakeholders for Security Remediation

October 7, 2024

The Evolution of Vulnerability Remediation: From RBVM to Security Remediation Platforms

September 29, 2024

Practical Vulnerability Remediation - When Should You Act?

October 20, 2024

The Ultimate Remediation Lifecycle for Vulnerability Management

September 22, 2024

Global Leader in Content Solutions Streamlines Remediation with Opus

August 13, 2024

Opus Security helps Global Technology Company Connect Security Tools for More Effective Security Remediaiton

August 13, 2024

Opus Security's VP of Marketing shares her thoughts on B2B marketing in a recent DemandBase podcast.

August 13, 2024

Opus Security Wins 2023 CyberSecurity Breakthrough Award

October 20, 2024

Vulnerability Management - A Story

September 22, 2024

6 Ways Orchestration Reduces Cloud Security Risk

October 2, 2024

Move Beyond Shift Left and Shift Smart

October 20, 2024

3 Tips for Bridging the Remediation Developer/Security Gap

October 2, 2024

WAF With EKS, Using Only k8s Controllers

August 13, 2024

Opus Recognized in the 2023 Gartner® Emerging Tech: Security — The Future of Cyber Is Automated Moving Target Defense report.

August 13, 2024

Wiz identifies, Opus fixes.

August 13, 2024

Writing your Cloud Opus: A Deep Dive into Orchestrating your Cloud Security

August 13, 2024

Snow in April, Alert Monsters, and Connect Four: 5 Takeaways from our SecOps Chat

August 13, 2024

Security in Sync: How Security Teams Can Make Cloud Remediation Less Painful for Engineers

October 20, 2024

Why We Need a New Approach to Remediation in the Cloud

August 13, 2024

Introducing Opus: Orchestrating Remediation and Reducing Risk in the Cloud

September 29, 2024

WAF With EKS, Using Only k8s Controllers

Read next

See the Opus Platform in Action