AWS Application Load Balancer Deployment: From Console Clicks to Control Planes

Introduction

The AWS Application Load Balancer (ALB) sits at a critical junction in modern cloud architectures: it's the entry point for user traffic, the routing layer for microservices, and the SSL termination point that protects application backends. Despite its ubiquity, ALB deployment remains surprisingly error-prone when managed manually, with common misconfigurations that undermine high availability, security, and operational stability.

This document traces the evolution of ALB deployment methodologies—from manual console provisioning to modern control-plane-based automation. It examines what makes ALBs architecturally complex (the interplay of listeners, target groups, health checks, and security groups), compares the major Infrastructure as Code (IaC) tools, and explains how Project Planton abstracts this complexity into a developer-friendly API designed for the 80% use case while enabling the 20% of advanced scenarios.

The paradigm shift is clear: production ALBs should never be created through the AWS Console. They should be declared as code, version-controlled, and continuously reconciled by a control plane.

The ALB Deployment Landscape

ALB management spans a spectrum from fully manual (AWS Console) to fully automated, continuously reconciled control planes.

Level 0: Manual Provisioning via AWS Console (The Anti-Pattern)

The AWS Management Console provides a wizard-driven workflow for creating load balancers. While approachable for learning, this method is highly susceptible to critical configuration errors:

Common Mistakes:

1. Single-AZ Deployment: The wizard allows selecting subnets but doesn't enforce multi-AZ distribution. A user can select multiple subnets in the same Availability Zone, creating an ALB with zero high availability—defeating the primary purpose of load balancing.

2. Misconfigured Security Groups: The most frequent production failure. Common errors include:

   • Creating overly permissive rules (allowing all traffic from all ports)
   • Failing to create the correct security group chaining pattern between the ALB and its targets
   • Allowing direct internet access to application targets, bypassing the load balancer

3. Insecure Listeners: Defaulting to HTTP-only listeners on port 80 without configuring HTTPS on port 443. This sends all application traffic unencrypted, unacceptable for any production workload.

4. Poor Health Checks: Accepting the default health check path (/). This path may return 200 OK from a web server (like Nginx) even if the underlying application is frozen, crashed, or returning errors—routing traffic to "zombie" instances.

Console Fragility: The console is merely a client for the AWS API and can experience its own transient failures. There are documented cases of the console displaying InternalFailure errors when trying to view ALB listeners, even while the underlying API and CLI commands function perfectly. This highlights the brittleness of relying on a UI layer.

Verdict: Acceptable only for learning and experimentation. Unacceptable for production environments requiring reproducibility, consistency, or security compliance.

Level 1: Scripted Provisioning with AWS CLI and SDKs

This approach moves from manual clicking to imperative scripting using the AWS CLI (aws elbv2 create-load-balancer) or SDKs like Boto3 (Python).

The Sequence Problem: A complete ALB deployment requires a precise sequence of imperative API calls:

1. create_load_balancer
2. create_target_group
3. create_listener
4. create_listener_rule (for advanced routing)

Managing these dependencies and ensuring idempotency becomes the script author's burden.

Key Advantage: The API documentation is explicit about production requirements. The Boto3 documentation for create_load_balancer states: "You must specify subnets from at least two Availability Zones". The console allows this mistake; the API documentation codifies the requirement. This shift forces developers to confront HA requirements directly.

Verdict: Suitable for simple, one-off automated tasks or embedding infrastructure provisioning into application code. Not ideal for declarative, state-managed infrastructure.

Level 2: Configuration Management (Ansible, Chef, Puppet)

Configuration management tools are designed primarily to manage software state on servers, not to provision cloud-native, serverless infrastructure. An ALB is a pure API-managed service with no server to configure.

The Anti-Pattern: Using Ansible's community.aws.elb_application_lb module to provision ALBs blurs the line between two distinct problems: infrastructure provisioning and server configuration. These tools lack the robust state management, drift detection, and dependency graphing of true IaC tools when applied to cloud resources.

The Correct Pattern: Industry best practice is a clear separation of concerns:

• IaC (Terraform/CloudFormation): Provisions immutable infrastructure (VPC, subnets, ALB, EC2 instances)
• Configuration Management (Ansible): Configures software on the provisioned instances (application deployment, config files)

Verdict: Not recommended for ALB provisioning. Use IaC tools for infrastructure; use configuration management for server state.

Level 3: Infrastructure as Code (Terraform, CloudFormation, CDK, Pulumi)

This is the modern, dominant paradigm for cloud infrastructure management. IaC tools are:

• Declarative: Define the desired end state (e.g., "an ALB with these subnets and this listener")
• Stateful: Track the current state of infrastructure and calculate the minimal diff to achieve the desired state
• Idempotent: Running the same configuration multiple times produces the same result

Terraform/OpenTofu

Resource Granularity: Highly granular, breaking an ALB into distinct resources:

• aws_lb: The load balancer itself
• aws_lb_listener: Listens on a port (80, 443) and defines default actions
• aws_lb_listener_rule: Routes requests based on conditions (path, host, headers)
• aws_lb_target_group: Defines a logical group of targets (EC2, containers, Lambdas)

Production Pattern: Teams rarely use these raw resources directly. They use modules, particularly the community-standard terraform-aws-modules/alb/aws module. This module abstracts the granular resources into a single, opinionated interface—effectively a "Level 2" abstraction.

State Management: Terraform's primary operational complexity. The state file must be stored in a remote backend (like S3) with locking (DynamoDB) to prevent concurrent, conflicting applies.

Multi-Environment: Uses Workspaces to manage multiple independent state files from a single set of configuration files.

AWS CloudFormation

AWS-Native: State is managed implicitly by AWS within the "stack" concept. No external state file to manage.

Resource Granularity: Similar to Terraform with granular resources like AWS::ElasticLoadBalancingV2::LoadBalancer and AWS::ElasticLoadBalancingV2::Listener.

Multi-Region: Uses StackSets for multi-account and multi-region deployments—a powerful AWS-native feature for centrally managing infrastructure across an organization.

Limitation: AWS-specific. Not suitable for multi-cloud strategies.

AWS CDK (Cloud Development Kit)

The Key Innovation: Abstraction levels.

• L1 (Level 1): Auto-generated, 1:1 mappings to CloudFormation resources
• L2 (Level 2): Curated constructs with intuitive APIs and sensible defaults (e.g., new alb.ApplicationLoadBalancer(...))
• L3 (Level 3): Patterns that create entire architectures (e.g., ApplicationLoadBalancedFargateService provisions an ALB, Fargate service, and all wiring)

Developer-Centric: Appeals to developers who prefer code over declarative YAML/HCL, allowing them to use familiar languages (TypeScript, Python, Java) with loops, functions, and classes.

The Leaky Abstraction: The CDK synthesizes to CloudFormation templates. Developers often encounter cryptic CloudFormation errors, revealing that the underlying declarative "quirks" aren't truly abstracted away—just hidden.

Pulumi

Multi-Language, Multi-Cloud: Similar to CDK but designed for multi-cloud from the start. Supports TypeScript, Python, Go, C#, and Java.

Key Differentiators:

• State: Can use the managed Pulumi Service (providing state, history, drift detection, CI/CD) or self-hosted backends
• Automation API: Embeds the Pulumi engine inside applications, enabling custom control planes
• Code Generation: Can generate programmatic code from existing cloud resources (Terraform only imports state)
• Policy as Code: CrossGuard is open-source and uses familiar languages (vs. Terraform's proprietary Sentinel)

Multi-Environment: First-class Stack concept. A single program can define multiple stacks (dev, staging, prod) as independent units.

Level 4: Control Planes (AWS Load Balancer Controller, Crossplane)

This is the most advanced deployment method, shifting from user-run CLI tools to long-running control planes that continuously monitor and reconcile desired state.

AWS Load Balancer Controller (for Kubernetes): Runs inside an EKS cluster and watches for Kubernetes Ingress or Service resources. When a developer creates a Kubernetes Ingress object, the controller automatically provisions a complete AWS ALB (load balancer, listeners, target groups) to satisfy the Ingress specification. This provides a Kubernetes-native abstraction.

Crossplane: Generalizes the controller concept. Crossplane extends the Kubernetes API to manage external infrastructure resources. A developer can kubectl apply a YAML file defining a Crossplane Custom Resource for an ALB, and Crossplane provisions and manages it in AWS.

The Fundamental Difference: A CLI-based tool like Terraform runs, applies changes, and exits. A control plane continuously observes infrastructure state and reconciles it against the desired state. This GitOps-style continuous reconciliation is the defining characteristic.

Project Planton Context: Project Planton, with its protobuf-defined APIs, is conceptually an API specification for this type of control plane, not just another CLI tool.

Production-Grade ALB Architecture

A production-ready ALB isn't a single resource—it's the correct configuration of interconnected components.

Network and Subnet Placement

Multi-AZ High Availability (Critical Requirement): A production ALB must be provisioned with subnets in at least two Availability Zones. This allows the ALB to remain available even if one AZ fails. The AWS API explicitly enforces this; any higher-level abstraction should actively validate it.

Public vs. Private Subnets (The internal Parameter):

• Internet-facing (internal: false): ALB nodes are placed in public subnets (subnets with routes to an Internet Gateway). The ALB receives traffic from the internet.
• Internal (internal: true): ALB nodes are placed in private subnets. Only routable from within the VPC for internal microservice-to-microservice communication.

Standard 3-Tier Architecture: An internet-facing ALB is placed in public subnets while backend targets (EC2 instances, Fargate containers) are placed in private subnets. The ALB routes traffic from its public nodes to the private IPs of targets. Return traffic flows from the private subnet back through the ALB to the client.

Configuration Linkage: The internal flag and subnets are inextricably linked. A common mistake is creating an internet-facing ALB but attaching it to private subnets—the ALB will provision but be unreachable from the internet. A robust API should validate this linkage.

Security Configuration

The Dual Security Group Pattern (Critical Best Practice):

A secure ALB setup requires two separate security groups:

1. ALB Security Group: Attached to the ALB itself

   • Ingress: Allow traffic on ports 80 and 443 from the desired source (e.g., 0.0.0.0/0 for public ALBs)
   • Egress: Allow traffic to the target port (e.g., 8080) to the Target Security Group

2. Target Security Group: Attached to backend instances/containers

   • Ingress: Allow traffic on the application port (e.g., 8080) only from the ALB Security Group ID (not from 0.0.0.0/0)
   • Egress: Standard application egress rules

Why This Matters: Referencing the ALB's security group ID in the target's ingress rules ensures that no traffic can bypass the load balancer and hit application targets directly. This is a core security best practice and a common point of confusion.

SSL/TLS Termination: Production listeners must use HTTPS. The ALB terminates the SSL/TLS connection using a certificate from AWS Certificate Manager (ACM), offloading decryption workload from backend targets.

Authentication: ALBs can natively integrate with Amazon Cognito or OIDC providers to authenticate users before forwarding requests to targets, offloading complex auth logic from applications.

Target Groups and Health Checks

Target Groups: A logical grouping of targets (e.g., "blue-fleet", "api-service") with routing settings like protocol and port.

Health Checks (Essential for Reliability): Configured per target group. The ALB actively polls each target on the HealthCheckPath. If a target fails UnhealthyThresholdCount consecutive checks, the ALB stops routing traffic to it.

Key Tunable Parameters:

• HealthCheckPath: Should be a specific endpoint (e.g., /healthz, /api/health) that validates application health, not just web server availability
• HealthCheckIntervalSeconds: Time between checks (default: 30s)
• HealthCheckTimeoutSeconds: How long to wait for a response (default: 5s)
• UnhealthyThresholdCount: Failures before marking unhealthy (default: 3)
• HealthyThresholdCount: Successes before marking healthy (default: 3)

Performance Tuning: Default settings can be too slow for critical applications (30s interval × 3 failures = 90s to detect a dead target). High-traffic services often use aggressive tuning (10s interval, 2s timeout, 2 unhealthy threshold = ~20s detection time).

Deregistration Delay (Connection Draining): When a target is deregistered (during deployments), the ALB waits for this delay (default: 300s) to allow in-flight requests to complete. This value must be longer than the application's graceful shutdown period to achieve zero-downtime deployments.

DNS and Route53 Integration

Route53 Alias Records (The Only Correct Method): ALBs have dynamic DNS names, not static IPs. A standard CNAME record cannot be used for a zone apex (e.g., example.com). Route53 Alias records are AWS-specific record types that can point apex domains to ALBs.

Evaluate Target Health: When configuring an Alias record, Route53 can be set to "Evaluate Target Health". This integrates Route53's own health checks with the ALB's health, enabling DNS-level failover. If an ALB becomes unhealthy, Route53 stops routing traffic to it and fails over to an ALB in another region.

Weighted Routing: Route53 supports weighted routing for canary or blue/green deployments at the DNS level, distributing traffic between different ALBs based on assigned weights.

Monitoring and Observability

CloudWatch Metrics (Essential for Alerting):

• HTTPCode_ELB_5XX_Count: Server-side errors (spike indicates targets are failing)
• HTTPCode_ELB_4XX_Count: Client-side errors (e.g., 404 Not Found)
• TargetResponseTime: Latency between ALB and targets
• UnHealthyHostCount: Number of targets failing health checks
• RequestCount: Total requests processed

Access Logs (Production Necessity): Disabled by default. When enabled, the ALB captures detailed logs for every request (client IP, path, latencies, status codes) and delivers them to an S3 bucket.

Analysis with Amazon Athena (The Standard Pattern): Athena can query ALB access logs directly in S3 using SQL. This requires a one-time CREATE EXTERNAL TABLE statement to map the log format.

Troubleshooting Power: During an outage, engineers can run queries like:

SELECT * FROM alb_access_logs
WHERE elb_status_code >= 500

This query is invaluable for identifying 5xx errors, slow requests, or problematic clients.

Request Tracing: ALBs integrate with AWS X-Ray to add trace IDs to requests, enabling end-to-end tracing as requests flow through the ALB to backend microservices.

Production Best Practices and Anti-Patterns

Advanced Features: The 20% Use Case

While the 80% use case is a simple internet-facing ALB with HTTPS and a single target group, production platforms must support advanced scenarios.

Listener Rules and Priority Management

ALB listeners evaluate rules in numerical order of priority (1 to 50,000). The first rule matching a request's conditions (path, host, headers) wins. A listener has a default rule (lowest priority) catching all unmatched traffic.

The Priority Trap: It's tempting to auto-calculate the next available priority. This is an anti-pattern. Dynamically calculating priority creates a fatal issue: on the next apply, the IaC tool sees a new rule, recalculates priority, and tries to change the priority of existing rules, leading to a constant, non-convergent state.

The Correct Approach: Force users to provide an explicit, static priority for every listener rule. This ensures deterministic, idempotent state that a control plane can safely reconcile.

Target Group Types: Instance vs. IP vs. Lambda

instance: Targets are EC2 instance IDs. Best for simple, EC2-based workloads managed by Auto Scaling Groups.

ip: Targets are IP addresses. The modern default for containerized workloads (ECS with Fargate). Also used to route traffic to services in other VPCs (via peering) or on-premise servers (via VPN).

lambda: The ALB invokes a Lambda function directly as its target. This requires an aws_lambda_permission resource granting the ALB service principal (elasticloadbalancing.amazonaws.com) permission to invoke the function. A high-level API should treat this as a special case.

Weighted Target Groups (Canary/Blue-Green Deployments)

A single listener rule's forward action can specify multiple target groups, each with a weight (e.g., 90% to "stable", 10% to "canary"). This is the core mechanism for progressive deployment strategies.

AWS WAF Integration

AWS Web Application Firewall (WAF) protects applications from SQL injection, XSS, and other exploits. Integration is straightforward: create a WAF Web ACL (collection of rules), then associate it with the ALB. This is a perfect "20%" feature—a single optional field providing immense security value.

Cost Optimization: Understanding LCUs

ALB pricing consists of a fixed hourly charge plus a variable charge based on Load Balancer Capacity Units (LCUs). An LCU measures workload, with billing based on the highest of four dimensions in any hour:

1. New Connections: 25 per second
2. Active Connections: 3,000 per minute
3. Processed Bytes: 1 GB per hour
4. Rule Evaluations: 1,000 per second (first 10 rules are free)

The Hidden Cost: "Rule Evaluations" is a non-obvious cost factor. A simple application with one listener and one default rule incurs minimal costs. A complex microservices gateway with 50 path-based listener rules processes all 50 rules for many requests, consuming significantly more LCUs and increasing monthly costs—even if connections and bytes are identical.

Trade-off: Platform architects must consider this when designing complex routing. Sometimes it's more cost-effective to use multiple simpler ALBs than one ALB with dozens of rules.

ALB vs. NLB vs. CLB: Choosing the Right Load Balancer

Application Load Balancer (ALB) - Layer 7

OSI Layer: 7 (Application)
Protocols: HTTP, HTTPS, WebSockets
Key Features: Advanced content-based routing (path, host, header, query string), SSL/TLS termination, WAF integration, Lambda/container targets
Use Case: The default choice for all modern web applications, APIs, microservices, and container-based applications

Network Load Balancer (NLB) - Layer 4

OSI Layer: 4 (Transport)
Protocols: TCP, UDP, TLS (pass-through)
Key Features: Ultra-low latency, millions of requests per second, static IP per AZ, AWS PrivateLink support
Use Cases:

1. Non-HTTP/S traffic (databases, game servers, IoT protocols)
2. When static IP is a hard requirement for client whitelisting
3. TCP pass-through where SSL should not terminate at the load balancer

Advanced Pattern: Using an ALB as a target for an NLB provides the best of both worlds—static IPs and low-latency entry point of NLB with advanced L7 routing of ALB.

Classic Load Balancer (CLB)

Status: Legacy only
Use Case: Not recommended for any new applications. Lacks advanced routing of ALB and performance/static IPs of NLB. Project Planton should not implement a resource for Classic Load Balancers.

What Project Planton Supports

Project Planton provides a Kubernetes-native API for deploying AWS Application Load Balancers that balances the 80% use case (simple, secure defaults) with extensibility for advanced scenarios.

Design Philosophy: 80/20 API Structure

The current API (see spec.proto) focuses squarely on the 80% use case with a simple, flat structure:

Core Fields (80% Case):

• subnets: List of subnet IDs (minimum 2, validated for HA)
• security_groups: Security group IDs to attach to the ALB
• internal: Boolean flag for internet-facing vs. internal
• delete_protection_enabled: Prevents accidental deletion
• idle_timeout_seconds: Connection idle timeout (default: 60s)

Integrated Abstractions:

• dns: Route53 DNS configuration with automatic Alias record creation
• ssl: Single toggle for SSL with ACM certificate ARN

Current Implementation Highlights

Multi-AZ Validation: The API enforces repeated.min_items = 2 on subnets, preventing single-AZ deployments at the API layer rather than just documenting the requirement.

DNS Integration: The AwsAlbDns message provides first-class Route53 integration:

• enabled: Toggle for DNS management
• route53_zone_id: Reference to hosted zone
• hostnames: List of domain names that will point to the ALB

SSL Simplification: The AwsAlbSsl message provides a simple abstraction:

• enabled: Toggle for HTTPS listener
• certificate_arn: ACM certificate reference

Foreign Key References

Project Planton uses a sophisticated StringValueOrRef pattern for references to other resources:

• subnets can reference AwsVpc resources
• security_groups can reference AwsSecurityGroup resources
• route53_zone_id can reference AwsRoute53Zone resources
• certificate_arn can reference AwsCertManagerCert resources

This enables declarative cross-resource dependencies managed by the control plane.

What's Not Yet Implemented (Future Enhancements)

Based on the 80/20 analysis from the research, future enhancements could include:

Target Group Management: Currently implicit. Future versions could add:

message AwsAlbTargetGroup {
  string name = 1;
  int32 port = 2;
  string protocol = 3;
  string target_type = 4; // "instance", "ip", "lambda"
  HealthCheck health_check = 5;
  int32 deregistration_delay = 6;
}

Advanced Listener Rules: For microservices with path/host-based routing:

message AwsAlbListenerRule {
  string name = 1;
  int32 priority = 2; // Explicit, not auto-calculated
  repeated Condition conditions = 3;
  Action action = 4;
}

Weighted Target Groups: For canary/blue-green deployments:

message WeightedTargetGroup {
  string target_group_name = 1;
  int32 weight = 2; // 1-1000
}

Access Logs Configuration:

message AwsAlbAccessLogs {
  bool enabled = 1;
  string s3_bucket_name = 2;
  string s3_prefix = 3;
}

WAF Integration:

string waf_web_acl_arn = N;

Multi-Environment Best Practice

Following AWS best practices, Project Planton encourages using separate VPCs (and thus separate ALBs) for each environment:

• company-dev-vpc → Dev ALBs
• company-staging-vpc → Staging ALBs
• company-prod-vpc → Production ALBs

Each environment is deployed as a separate resource instance with different configuration values, providing complete isolation for networking, security, and blast radius containment.

Security Best Practices

While the current API requires users to manually configure security groups, future versions could abstract the dual security group pattern:

message SimpleHttpsListener {
  string certificate_arn = 1;
  int32 target_port = 2;
  repeated string allow_client_cidrs = 3; // e.g., ["0.0.0.0/0"]
  // The control plane would automatically create:
  // - ALB SG allowing 443 from allow_client_cidrs
  // - Target SG allowing target_port from ALB SG
}

This abstraction would enforce the security best practice by default, eliminating a common misconfiguration.

Conclusion: The Control Plane Paradigm

AWS Application Load Balancer deployment represents a maturity journey: from manual, error-prone console clicking to scripted CLI operations to declarative Infrastructure as Code to continuously reconciled control planes.

The research makes clear that:

• Manual provisioning is an anti-pattern for production workloads
• IaC is the modern standard, with Terraform serving as the canonical implementation
• Control planes represent the future, continuously observing and reconciling infrastructure state

Project Planton builds on this foundation, providing a Kubernetes-native API that:

• Enforces best practices (multi-AZ validation, deletion protection defaults)
• Makes the simple case simple (80% use case with flat, intuitive fields)
• Makes the advanced case possible (extensible for future 20% features)
• Operates as a control plane, not a one-shot CLI tool

The paradigm shift is complete: production load balancers should be declared as code, version-controlled in Git, and continuously reconciled by a control plane. Project Planton embodies this philosophy, translating complex AWS primitives into developer-friendly abstractions that make doing the right thing the easy thing.

By codifying AWS best practices into validation rules, providing intelligent defaults, and abstracting security patterns, Project Planton helps teams deploy production-ready ALBs with confidence—eliminating the common pitfalls that plague manual and ad-hoc scripted approaches.