← Back to the blog

Published on 17 August 2026

Implementation guide

Palo Alto VM-Series behind a Gateway Load Balancer on AWS

Traffic inspection on AWS with two Palo Alto firewalls spread across two availability zones. Bump-in-the-wire mode: no NAT, no overlay routing, and no routing done by the firewall.

Zones
2 · one per AZ
Encapsulation
GENEVE · UDP 6081
Health check
TCP 80
Cross-zone
disabled
PAN-OS
11.1
Plugin
vm_series 5.x

ArchitectureWhat gets built

A Gateway Load Balancer spreads traffic across two VM-Series, one per zone. The firewall neither routes nor rewrites addresses: it receives the packet encapsulated in GENEVE, inspects it and returns it down the same tunnel.

Zonal symmetry is deliberate. With cross-zone load balancing disabled, traffic entering through one zone's endpoint is inspected on that same zone's firewall and returns the way it came. That keeps the session on a single firewall, which is what a stateful device needs, and avoids cross-zone data charges.

on-prem AS 65000 · advertises its prefixes over BGP IPsec · 2 tunnels virtual private gateway · AS 64512 inside 169.254.200.0/30 and 169.254.201.0/30 edge route table: 10.20.4.0/24 and 10.20.14.0/24 to their endpoint us-east-1a us-east-1b GWLB endpoint 10.20.3.129 gwlbe subnet 10.20.3.0/24 GWLB endpoint 10.20.13.64 gwlbe subnet 10.20.13.0/24 Gateway Load Balancer · target group GENEVE 6081 nodes 10.20.2.78 (AZ a) and 10.20.12.37 (AZ b) · in the fwdata subnets cross-zone off — each node delivers only to its own zone's firewall GENEVE GENEVE VM-Series eth1 10.20.2.89 · target eth0 mgmt 10.20.1.173 VM-Series eth1 10.20.12.224 · target eth0 mgmt 10.20.11.92 workload 10.20.4.10 app subnet 10.20.4.0/24 workload 10.20.14.10 app subnet 10.20.14.0/24 — — — inspected return to the same endpoint
Addressing from a real deployment. The GWLB node and the VM-Series share the fwdata: that is why the health check arrives from an address on that same network and is intrazone traffic.

AddressingFive subnets per zone

Each role needs its own subnet because each carries a different route table. Mixing them is the fastest way to create a loop between the endpoint and the workload.

RoleWhat lives thereAZ aAZ b
publicNAT gateway10.20.0.0/2410.20.10.0/24
mgmtVM-Series eth010.20.1.0/2410.20.11.0/24
fwdataVM-Series eth1 and the GWLB nodes10.20.2.0/2410.20.12.0/24
gwlbeGWLB endpoint10.20.3.0/2410.20.13.0/24
appworkloads10.20.4.0/2410.20.14.0/24

What route each table carries

  • app0.0.0.0/0 to its own zone's endpoint. Everything leaving the workload goes to inspection.
  • gwlbe0.0.0.0/0 to the NAT gateway, plus propagation from the virtual private gateway. This is the table that decides where already-inspected traffic goes.
  • fwdata — the local route only. GENEVE is born and dies inside the VPC.
  • VGW edge — the app subnets pointing at their endpoint. Without this table, traffic arriving from the tunnel goes straight to the workload without passing through the firewall.

ScopeWhat gets inspected and what does not

Worth deciding before writing the policy, because the three paths do not all come for free.

PathHow it reaches inspectionStatus
Hybrid
on-prem ↔ workload
Edge route table on the virtual private gateway works
Egress
workload → internet
0.0.0.0/0 on the app table towards the endpoint works
East-west
app-a ↔ app-b
Requires routes more specific than the VPC's local of the VPC see limitations

CostsWhat it costs to leave this running

Almost 3 dollars an hour, and four out of every five go to the firewall licence, not to AWS infrastructure.

Rates for us-east-1: infrastructure queried against the AWS pricing API, software against the PAYG listing's dimensions table. Data transfer not included.

ItemUSD/hour×USD/month
VM-Series licence — PAYG, m5.xlarge1.170021,708.20
VM-Series — compute m5.xlarge0.19202280.32
NAT gateway0.0450265.70
Site-to-site VPN connection0.0500136.50
Workloads t3.micro0.0104215.18
GWLB endpoint0.0100214.60
Public IPv4 in use0.0050414.60
EBS gp3 — 2×60 GB + 2×8 GB0.014910.88
Gateway Load Balancer0.012519.12
GWLB — capacity (minimum LCU)0.004012.92
Total2.95622,158.03
If left runningCostBreakdown
one hour2.96 USDlicence 79 % · compute 13 % · network 8 %
one day70.95 USD
one month2,158 USD

The licence rules

The software charge on the PAYG listing is six times the compute cost of the same instance: 1.17 against 0.192 an hour. It is billed through AWS Marketplace, which is why it shows up neither in the pricing API nor in estimates that only look at AWS resources.

The price goes by instance size , not by family: every xlarge pay 1.17, the large 0.99, the 2xlarge 1.80 and the 4xlarge 3.69. Dropping from m5.xlarge to m5.large would save 263 USD a month in licence — but the GWLB needs at least 10.0.2 and an instance with enough NICs, so check support before sizing down.

What to switch off

  • Stopping both firewalls cuts 92 % of the hourly spend — licence plus compute. It is the only lever that moves the needle.
  • With everything switched off, you still pay 0.20 USD/hour — around 143 a month: the NAT gateways, the GWLB, the endpoints, the VPN and the public IPv4 addresses all bill by the hour even if not one packet goes through.
  • For an on-and-off lab, terraform destroy and recreate beats switching off: it is 85 resources and the full cycle takes minutes.

Deployment reference: AMI PA-VM-AWS-11.1.15, product code e9yfvyj3uag5uo5j2hjikv74n, listing "VM-Series Next-Gen Virtual Firewall w/Advanced Threat Prevention (PAYG)".

Phase 1AWS infrastructure

terraform init
terraform plan     # leerlo entero, incluidas las lineas con "-"
terraform apply

# comprobar que el direccionamiento interno del tunel quedo bien
terraform output pa440_tunnel1
terraform output pa440_tunnel2

# antes de dar por buena cualquier regla de security group
terraform plan -detailed-exitcode    # 0 = sin deriva

Read the whole plan, not just the summary. The Plan: N to add, M to change does not distinguish between "adds a CIDR" and "revokes the ones that were there": in an aws_security_group with inline ingress blocks, an in-place change shows up as the old set with - and the new one with +.

The parameters that allow no variation:

ResourceValueWhy
Target groupGENEVE port 6081It is the only protocol a GWLB accepts
Target typeipWith instance it registers the primary interface, which is the management one
Health checkTCP port 80The GWLB cannot query an arbitrary PAN-OS path
Stickinesssource_ip_dest_ip_protoThe firewall does not rewrite the 5-tuple; the flow must always return to the same device
Data ENIsource_dest_check = falseWithout this, AWS drops traffic not addressed to the interface
Security groupUDP 6081 and TCP 80 from the VPC CIDRGENEVE and health check

Careful with the AMI

Use the public SSM parameter of the listing you are subscribed to. An AMI from another listing boots just the same but bills against a subscription you do not have. And do not solve it with most_recent: Palo Alto's catalogue has newer versions whose AMI carries an earlier date, so sorting by creation date picks the wrong one.

Verification

The endpoints in Available and the edge route table associated with the virtual private gateway, with the app subnets' routes pointing at their endpoints.

TerraformThe resources that matter

The full deployment is 85 resources. Most of it is plumbing repeated per zone; these are the ones that carry the design logic.

GroupResourcesDetail
Base network1 + 10VPC and five subnets per zone
Route tables8 + 11 routespublic, mgmt, fwdata, gwlbe, app and the VGW edge
Internet egress1 + 2 + 2IGW, NAT gateways and their EIPs
GWLB5balancer, listener, target group, endpoint service and 2 endpoints
Firewalls2 + 4 + 2instances, two ENIs each and the management EIPs
VPN4customer gateway, VGW, connection and route propagation
Security3one security group per role

The balancer and its target group

The GWLB lives in the fwdatasubnets, alongside the firewalls. The target is registered by IP and not by instance: with instance AWS would use the primary interface, which here is the management one.

resource "aws_lb" "gwlb" {
  load_balancer_type               = "gateway"
  subnets                          = [for k, v in aws_subnet.fwdata : v.id]
  enable_cross_zone_load_balancing = false
}

resource "aws_lb_target_group" "fw" {
  target_type = "ip"
  protocol    = "GENEVE"
  port        = 6081

  health_check {
    protocol = "TCP"
    port     = 80
  }

  # El firewall no reescribe la 5-tupla: el flujo debe volver
  # siempre a la misma instancia.
  stickiness {
    type    = "source_ip_dest_ip_proto"
    enabled = true
  }
}

resource "aws_lb_target_group_attachment" "fw" {
  for_each          = local.subnets
  target_id         = aws_network_interface.fw_data[each.key].private_ip
  availability_zone = local.az[each.key]
  port              = 6081
}

The edge association

This is the piece that makes inbound traffic get inspected. A route table associated with the virtual private gateway using gateway_id, not to a subnet, and with routes more specific than the VPC's local route.

resource "aws_route" "vgw_edge_to_app" {
  for_each               = local.subnets
  route_table_id         = aws_route_table.vgw_edge.id
  destination_cidr_block = each.value.app          # 10.20.4.0/24 y 10.20.14.0/24
  vpc_endpoint_id        = aws_vpc_endpoint.gwlbe[each.key].id
}

resource "aws_route_table_association" "vgw_edge" {
  gateway_id     = aws_vpn_gateway.vgw.id          # asociacion de BORDE
  route_table_id = aws_route_table.vgw_edge.id
}

The firewall interfaces

Two NICs: management on device_index 0 and data on 1, which is the one the VM-Series sees as ethernet1/1. The source_dest_check goes to false on the data one only.

resource "aws_network_interface" "fw_data" {
  for_each          = local.subnets
  subnet_id         = aws_subnet.fwdata[each.key].id
  security_groups   = [aws_security_group.fwdata.id]
  source_dest_check = false
}

resource "aws_instance" "fw" {
  for_each      = local.subnets
  instance_type = "m5.xlarge"

  network_interface { network_interface_id = ...fw_mgmt[each.key].id, device_index = 0 }
  network_interface { network_interface_id = ...fw_data[each.key].id, device_index = 1 }

  user_data = <<-EOT
    type=dhcp-client
    hostname=fw-gwlb-${each.key}
    dhcp-accept-server-hostname=no
    dhcp-accept-server-domain=no
  EOT
}

Choosing the AMI

With the public SSM parameter, never with most_recent: Palo Alto's catalogue has newer versions whose AMI carries an earlier date, so sorting by creation date picks the wrong one.

data "aws_ssm_parameter" "panos" {
  name = "/aws/service/marketplace/prod-<listing>/pan-os-<version>"
}

The three security groups

GroupIngressFrom
fwdataUDP 6081, TCP 80, ICMPVPC CIDR
mgmtTCP 22, TCP 443, ICMPadmin list + on-prem prefixes
appeverythingVPC CIDR + on-prem prefixes

The management one takes a list of CIDRs, not a string. With a single value the only way out is adding the real IP by hand in the console, and since the ingress blocks are inline, the next apply revokes it silently.

CodeThe full configuration

The seven files exactly as applied, with public addresses and account identifiers replaced by placeholders. This is the whole deployment: no external modules, no shared state.

FileContents
00-variables.tfProvider, variables and the per-zone subnet map.
01-vpc.tfVPC, the ten subnets, IGW, NAT gateways and the three security groups.
02-gwlb.tfBalancer, target group, listener, endpoint service and endpoints.
03-firewalls.tfAMI via SSM, the ENIs, the VM-Series and the workloads.
04-vpn.tfCustomer gateway, VGW and the connection with its two tunnels.
05-routing.tfThe six route tables, including the edge one associated with the VGW.
06-outputs.tfOutputs, among them the tunnels' inside addressing.
00-variables.tfProvider, variables and the per-zone subnet map.
terraform {
  required_version = ">= 1.5"
  required_providers {
    aws    = { source = "hashicorp/aws", version = "~> 5.0" }
    random = { source = "hashicorp/random", version = "~> 3.6" }
  }
}

provider "aws" {
  region = var.region
  default_tags {
    tags = {
      Project   = "kf-gwlb-lab"
      ManagedBy = "terraform"
      Owner     = "csegovia"
    }
  }
}

variable "region" {
  type    = string
  default = "us-east-1"
}

variable "vpc_cidr" {
  description = "Verificado libre contra la tabla de rutas del PA-410."
  type        = string
  default     = "10.20.0.0/16"
}

variable "azs" {
  type    = list(string)
  default = ["us-east-1a", "us-east-1b"]
}

# Cinco subnets por AZ. Cada una tiene un rol distinto en el path de GENEVE.
locals {
  az = { a = var.azs[0], b = var.azs[1] }

  subnets = {
    a = {
      public = "10.20.0.0/24" # NAT GW
      mgmt   = "10.20.1.0/24" # eth0 del VM-Series
      fwdata = "10.20.2.0/24" # eth1, GENEVE, target del GWLB
      gwlbe  = "10.20.3.0/24" # endpoints del GWLB
      app    = "10.20.4.0/24" # workloads
    }
    b = {
      public = "10.20.10.0/24"
      mgmt   = "10.20.11.0/24"
      fwdata = "10.20.12.0/24"
      gwlbe  = "10.20.13.0/24"
      app    = "10.20.14.0/24"
    }
  }

  # Rutas app -> GWLBE para cada prefijo del lab, por AZ.
  app_lab_routes = {
    for p in setproduct(keys(local.subnets), var.lab_cidrs) :
    "${p[0]}|${p[1]}" => { az = p[0], cidr = p[1] }
  }
}

# --- VPN / BGP ---------------------------------------------------------------

variable "cgw_public_ip" {
  description = "IP publica del PA-410 (ethernet1/1)."
  type        = string
  default     = "<IP-PUBLICA-CGW>"
}

variable "cgw_asn" {
  description = "AS local del PA-410, ya existente en el VR default."
  type        = number
  default     = 65000
}

variable "vgw_asn" {
  type    = number
  default = 64512
}

variable "tunnel1_inside_cidr" {
  description = "169.254.100.0/30 NO se puede: la usa el peer HEX-CASA."
  type        = string
  default     = "169.254.200.0/30"
}

variable "tunnel2_inside_cidr" {
  type    = string
  default = "169.254.201.0/30"
}

variable "lab_cidrs" {
  description = "Prefijos del home lab que se alcanzan por el tunel y se inspeccionan."
  type        = list(string)
  default = [
    "192.168.100.0/24",
    "192.168.106.0/24",
    "10.1.0.0/16",
    "10.142.0.0/16",
  ]
}

# --- Firewalls ---------------------------------------------------------------

# Parametro publico de SSM que publica AWS Marketplace. Resuelve al AMI
# correcto por region sin hardcodear IDs.
#
#   prod-<ID-DEL-LISTING> = listing "VM-Series Next-Gen Virtual Firewall
#                        w/Advanced Threat Prevention (PAYG)"
#
# OJO: usar el AMI de OTRO listing hace que la instancia facture contra una
# suscripcion que no tiene. El product code va atado al listing.
variable "panos_ssm_parameter" {
  type    = string
  default = "/aws/service/marketplace/prod-<ID-DEL-LISTING>/pan-os-11.1.15"
}

# Override manual. Si se setea, gana sobre el parametro SSM.
variable "panos_ami_id" {
  type    = string
  default = ""
}

variable "panos_instance_type" {
  description = "GWLB necesita minimo 10.0.2. m5.xlarge soporta las NICs necesarias."
  type        = string
  default     = "m5.xlarge"
}

variable "key_pair_name" {
  type = string
}

# Lista, no string: la IP publica de salida del equipo desde el que se
# administra NO siempre es la de ethernet1/1 del PA-410. Con un solo valor la
# unica salida era agregar la IP real a mano en la consola, y como los bloques
# ingress son inline, el siguiente terraform apply la revocaba en silencio.
variable "admin_cidrs" {
  description = "CIDRs con acceso de gestion a los VM-Series (SSH, HTTPS, ICMP)."
  type        = list(string)
  default     = ["<IP-PUBLICA-CGW>/32"]
}
01-vpc.tfVPC, the ten subnets, IGW, NAT gateways and the three security groups.
resource "aws_vpc" "lab" {
  cidr_block           = var.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags                 = { Name = "kf-gwlb-vpc" }
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.lab.id
  tags   = { Name = "kf-gwlb-igw" }
}

# --- Subnets -----------------------------------------------------------------

resource "aws_subnet" "public" {
  for_each          = local.subnets
  vpc_id            = aws_vpc.lab.id
  cidr_block        = each.value.public
  availability_zone = local.az[each.key]
  tags              = { Name = "kf-public-${each.key}" }
}

resource "aws_subnet" "mgmt" {
  for_each          = local.subnets
  vpc_id            = aws_vpc.lab.id
  cidr_block        = each.value.mgmt
  availability_zone = local.az[each.key]
  tags              = { Name = "kf-mgmt-${each.key}" }
}

resource "aws_subnet" "fwdata" {
  for_each          = local.subnets
  vpc_id            = aws_vpc.lab.id
  cidr_block        = each.value.fwdata
  availability_zone = local.az[each.key]
  tags              = { Name = "kf-fwdata-${each.key}" }
}

resource "aws_subnet" "gwlbe" {
  for_each          = local.subnets
  vpc_id            = aws_vpc.lab.id
  cidr_block        = each.value.gwlbe
  availability_zone = local.az[each.key]
  tags              = { Name = "kf-gwlbe-${each.key}" }
}

resource "aws_subnet" "app" {
  for_each          = local.subnets
  vpc_id            = aws_vpc.lab.id
  cidr_block        = each.value.app
  availability_zone = local.az[each.key]
  tags              = { Name = "kf-app-${each.key}" }
}

# --- NAT Gateway por AZ ------------------------------------------------------
# Uno por AZ. Con uno solo compartido el retorno cruza AZ y el path se vuelve
# dificil de explicar. Son US$0.045/hr cada uno: el segundo item mas caro
# despues de los firewalls.

resource "aws_eip" "nat" {
  for_each   = local.subnets
  domain     = "vpc"
  tags       = { Name = "kf-eip-nat-${each.key}" }
  depends_on = [aws_internet_gateway.igw]
}

resource "aws_nat_gateway" "nat" {
  for_each      = local.subnets
  allocation_id = aws_eip.nat[each.key].id
  subnet_id     = aws_subnet.public[each.key].id
  tags          = { Name = "kf-nat-${each.key}" }
  depends_on    = [aws_internet_gateway.igw]
}

# --- Security groups ---------------------------------------------------------

resource "aws_security_group" "mgmt" {
  name        = "kf-sg-mgmt"
  description = "Gestion de los VM-Series"
  vpc_id      = aws_vpc.lab.id

  ingress {
    description = "HTTPS GUI"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = concat(var.admin_cidrs, var.lab_cidrs)
  }
  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = concat(var.admin_cidrs, var.lab_cidrs)
  }
  ingress {
    description = "ICMP"
    from_port   = -1
    to_port     = -1
    protocol    = "icmp"
    cidr_blocks = concat(var.admin_cidrs, var.lab_cidrs)
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = { Name = "kf-sg-mgmt" }
}

# GOTCHA GWLB #1: si falta el 6081 el target queda unhealthy y AWS no dice
# por que. Es el error mas comun de toda la integracion.
resource "aws_security_group" "fwdata" {
  name        = "kf-sg-fwdata"
  description = "Interfaz de datos: GENEVE + health check del GWLB"
  vpc_id      = aws_vpc.lab.id

  ingress {
    description = "GENEVE desde el GWLB"
    from_port   = 6081
    to_port     = 6081
    protocol    = "udp"
    cidr_blocks = [var.vpc_cidr]
  }
  ingress {
    description = "Health check del target group"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = [var.vpc_cidr]
  }
  ingress {
    from_port   = -1
    to_port     = -1
    protocol    = "icmp"
    cidr_blocks = [var.vpc_cidr]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = { Name = "kf-sg-fwdata" }
}

resource "aws_security_group" "app" {
  name        = "kf-sg-app"
  description = "Workloads"
  vpc_id      = aws_vpc.lab.id

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = concat([var.vpc_cidr], var.lab_cidrs)
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = { Name = "kf-sg-app" }
}
02-gwlb.tfBalancer, target group, listener, endpoint service and endpoints.
# =============================================================================
# Gateway Load Balancer
# El GWLB encapsula en GENEVE (UDP 6081) y reparte hacia los VM-Series.
# Los firewalls trabajan como bump-in-the-wire: no rutean, no hacen NAT.
# =============================================================================

resource "aws_lb" "gwlb" {
  name                             = "kf-gwlb"
  load_balancer_type               = "gateway"
  subnets                          = [for k, v in aws_subnet.fwdata : v.id]
  # Probado en true (2026-08-17) para descartar que fuera la causa de que el
  # GWLB no entregue al target: NO cambio nada, el trafico sigue muriendo entre
  # el endpoint y el target. Se vuelve a false, que es el diseno buscado
  # (inspeccion local por AZ) y evita cargos de trafico cross-AZ.
  enable_cross_zone_load_balancing = false

  tags = { Name = "kf-gwlb" }
}

# GOTCHA GWLB #2: el health check no puede pegarle a un path cualquiera del
# PAN-OS. Se usa TCP:80 y hay que habilitar HTTP en el interface management
# profile de la interfaz de datos, o el target nunca pasa a healthy.
resource "aws_lb_target_group" "fw" {
  name        = "kf-gwlb-tg"
  target_type = "ip"
  protocol    = "GENEVE"
  port        = 6081
  vpc_id      = aws_vpc.lab.id

  health_check {
    protocol            = "TCP"
    port                = 80
    interval            = 10
    healthy_threshold   = 3
    unhealthy_threshold = 3
  }

  # El firewall no reescribe la 5-tupla, asi que el flujo debe volver siempre
  # a la misma instancia.
  stickiness {
    type    = "source_ip_dest_ip_proto"
    enabled = true
  }

  tags = { Name = "kf-gwlb-tg" }
}

# Se registra la IP de la ENI de datos, no el instance-id: con target_type
# "instance" el GWLB usaria la interfaz primaria, que aca es la de gestion.
resource "aws_lb_target_group_attachment" "fw" {
  for_each          = local.subnets
  target_group_arn  = aws_lb_target_group.fw.arn
  target_id         = aws_network_interface.fw_data[each.key].private_ip
  availability_zone = local.az[each.key]
  port              = 6081
}

resource "aws_lb_listener" "gwlb" {
  load_balancer_arn = aws_lb.gwlb.arn
  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.fw.arn
  }
}

# --- Endpoint service + endpoints por AZ -------------------------------------

resource "aws_vpc_endpoint_service" "gwlb" {
  acceptance_required        = false
  gateway_load_balancer_arns = [aws_lb.gwlb.arn]
  tags                       = { Name = "kf-gwlb-endpoint-service" }
}

resource "aws_vpc_endpoint" "gwlbe" {
  for_each          = local.subnets
  vpc_id            = aws_vpc.lab.id
  service_name      = aws_vpc_endpoint_service.gwlb.service_name
  vpc_endpoint_type = "GatewayLoadBalancer"
  subnet_ids        = [aws_subnet.gwlbe[each.key].id]
  tags              = { Name = "kf-gwlbe-${each.key}" }
}
03-firewalls.tfAMI via SSM, the ENIs, the VM-Series and the workloads.
# No se usa data "aws_ami" con most_recent: en la lista de AMIs de Palo Alto
# la 11.1.15 es una release MAS nueva que la 11.1.6-h35, pero su AMI tiene
# fecha ANTERIOR. Ordenar por CreationDate elige la version equivocada.
data "aws_ssm_parameter" "panos" {
  count = var.panos_ami_id == "" ? 1 : 0
  name  = var.panos_ssm_parameter
}

locals {
  panos_ami = var.panos_ami_id != "" ? var.panos_ami_id : data.aws_ssm_parameter.panos[0].value
}

# Dos NICs: gestion y datos. Sin overlay routing, sin NAT, sin interfaz
# untrust. Todo el trafico entra y sale encapsulado en GENEVE por eth1.

resource "aws_network_interface" "fw_mgmt" {
  for_each        = local.subnets
  subnet_id       = aws_subnet.mgmt[each.key].id
  security_groups = [aws_security_group.mgmt.id]
  tags            = { Name = "kf-fw-${each.key}-mgmt" }
}

resource "aws_network_interface" "fw_data" {
  for_each          = local.subnets
  subnet_id         = aws_subnet.fwdata[each.key].id
  security_groups   = [aws_security_group.fwdata.id]
  source_dest_check = false
  tags              = { Name = "kf-fw-${each.key}-data" }
}

resource "aws_eip" "fw_mgmt" {
  for_each          = local.subnets
  domain            = "vpc"
  network_interface = aws_network_interface.fw_mgmt[each.key].id
  tags              = { Name = "kf-eip-fw-${each.key}-mgmt" }
  depends_on        = [aws_internet_gateway.igw]
}

resource "aws_instance" "fw" {
  for_each      = local.subnets
  ami           = local.panos_ami
  instance_type = var.panos_instance_type
  key_name      = var.key_pair_name

  network_interface {
    network_interface_id = aws_network_interface.fw_mgmt[each.key].id
    device_index         = 0
  }
  network_interface {
    network_interface_id = aws_network_interface.fw_data[each.key].id
    device_index         = 1
  }

  user_data = <<-EOT
    type=dhcp-client
    hostname=kf-fw-gwlb-${each.key}
    dns-primary=169.254.169.253
    dns-secondary=8.8.8.8
    dhcp-accept-server-hostname=no
    dhcp-accept-server-domain=no
  EOT

  root_block_device {
    volume_size = 60
    volume_type = "gp3"
  }

  tags = { Name = "kf-fw-gwlb-${each.key}" }
}

# --- Workloads ---------------------------------------------------------------

data "aws_ami" "al2023" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["al2023-ami-2023.*-x86_64"]
  }
}

# Sin IP publica. Se prueban desde el home lab por el tunel, que es
# justamente el escenario que se quiere demostrar.
resource "aws_instance" "app" {
  for_each = local.subnets

  ami                    = data.aws_ami.al2023.id
  instance_type          = "t3.micro"
  subnet_id              = aws_subnet.app[each.key].id
  private_ip             = cidrhost(local.subnets[each.key].app, 10)
  vpc_security_group_ids = [aws_security_group.app.id]
  key_name               = var.key_pair_name

  user_data = <<-EOT
    #!/bin/bash
    dnf install -y nginx
    echo "kf-gwlb-lab · stack ${each.key} · $(hostname)" > /usr/share/nginx/html/index.html
    systemctl enable --now nginx
  EOT

  tags = { Name = "kf-app-${each.key}" }
}
04-vpn.tfCustomer gateway, VGW and the connection with its two tunnels.
# PSK: 8-64 chars, alfanumerico mas . y _ , no puede empezar con 0.
resource "random_password" "psk" {
  count            = 2
  length           = 32
  special          = true
  override_special = "._"
  numeric          = true
  upper            = true
  lower            = true
}

resource "aws_customer_gateway" "cgw" {
  bgp_asn    = var.cgw_asn
  ip_address = var.cgw_public_ip
  type       = "ipsec.1"
  tags       = { Name = "kf-cgw-pa410" }
}

resource "aws_vpn_gateway" "vgw" {
  vpc_id          = aws_vpc.lab.id
  amazon_side_asn = var.vgw_asn
  tags            = { Name = "kf-vgw" }
}

resource "aws_vpn_connection" "s2s" {
  customer_gateway_id = aws_customer_gateway.cgw.id
  vpn_gateway_id      = aws_vpn_gateway.vgw.id
  type                = "ipsec.1"
  static_routes_only  = false # BGP

  # --- Tunel 1 ---------------------------------------------------------------
  tunnel1_inside_cidr   = var.tunnel1_inside_cidr
  tunnel1_preshared_key = random_password.psk[0].result
  tunnel1_ike_versions  = ["ikev2"]

  tunnel1_phase1_encryption_algorithms = ["AES256"]
  tunnel1_phase1_integrity_algorithms  = ["SHA2-256"]
  tunnel1_phase1_dh_group_numbers      = [14]
  tunnel1_phase1_lifetime_seconds      = 28800

  tunnel1_phase2_encryption_algorithms = ["AES256"]
  tunnel1_phase2_integrity_algorithms  = ["SHA2-256"]
  tunnel1_phase2_dh_group_numbers      = [14]
  tunnel1_phase2_lifetime_seconds      = 3600

  tunnel1_startup_action      = "start" # AWS inicia; tenemos IP fija
  tunnel1_dpd_timeout_action  = "restart"
  tunnel1_dpd_timeout_seconds = 30

  # --- Tunel 2 ---------------------------------------------------------------
  tunnel2_inside_cidr   = var.tunnel2_inside_cidr
  tunnel2_preshared_key = random_password.psk[1].result
  tunnel2_ike_versions  = ["ikev2"]

  tunnel2_phase1_encryption_algorithms = ["AES256"]
  tunnel2_phase1_integrity_algorithms  = ["SHA2-256"]
  tunnel2_phase1_dh_group_numbers      = [14]
  tunnel2_phase1_lifetime_seconds      = 28800

  tunnel2_phase2_encryption_algorithms = ["AES256"]
  tunnel2_phase2_integrity_algorithms  = ["SHA2-256"]
  tunnel2_phase2_dh_group_numbers      = [14]
  tunnel2_phase2_lifetime_seconds      = 3600

  tunnel2_startup_action      = "start"
  tunnel2_dpd_timeout_action  = "restart"
  tunnel2_dpd_timeout_seconds = 30

  tags = { Name = "kf-vpn-pa410" }
}
05-routing.tfThe six route tables, including the edge one associated with the VGW.
# =============================================================================
# ROUTING
#
# El path completo de un flujo de egress:
#   app -> GWLBE (misma AZ) -> GWLB -> firewall -> GWLB -> GWLBE -> NAT GW -> IGW
#
# El path de un flujo hibrido entrante:
#   home lab -> VGW -> [edge route table] -> GWLBE -> firewall -> GWLBE -> app
# =============================================================================

# --- Public: NAT GW + IGW ----------------------------------------------------
# El retorno del NAT GW tiene que volver por el firewall, si no el flujo queda
# asimetrico y solo se inspecciona la ida.

resource "aws_route_table" "public" {
  for_each = local.subnets
  vpc_id   = aws_vpc.lab.id
  tags     = { Name = "kf-rt-public-${each.key}" }
}

resource "aws_route" "public_default" {
  for_each               = aws_route_table.public
  route_table_id         = each.value.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.igw.id
}

resource "aws_route" "public_return_to_fw" {
  for_each               = local.subnets
  route_table_id         = aws_route_table.public[each.key].id
  destination_cidr_block = each.value.app
  vpc_endpoint_id        = aws_vpc_endpoint.gwlbe[each.key].id
}

resource "aws_route_table_association" "public" {
  for_each       = aws_subnet.public
  subnet_id      = each.value.id
  route_table_id = aws_route_table.public[each.key].id
}

# --- Mgmt --------------------------------------------------------------------
# Trafico de gestion, no se inspecciona a proposito: si el firewall se cae,
# igual se puede llegar a administrarlo.

resource "aws_route_table" "mgmt" {
  vpc_id = aws_vpc.lab.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }
  tags = { Name = "kf-rt-mgmt" }
}

resource "aws_route_table_association" "mgmt" {
  for_each       = aws_subnet.mgmt
  subnet_id      = each.value.id
  route_table_id = aws_route_table.mgmt.id
}

# --- Fwdata ------------------------------------------------------------------
# Solo la ruta local. El GENEVE nace y muere dentro de la VPC.

resource "aws_route_table" "fwdata" {
  vpc_id = aws_vpc.lab.id
  tags   = { Name = "kf-rt-fwdata" }
}

resource "aws_route_table_association" "fwdata" {
  for_each       = aws_subnet.fwdata
  subnet_id      = each.value.id
  route_table_id = aws_route_table.fwdata.id
}

# --- GWLBE: salida despues de inspeccion -------------------------------------
# Aca cae el trafico ya inspeccionado que devuelve el firewall.

resource "aws_route_table" "gwlbe" {
  for_each = local.subnets
  vpc_id   = aws_vpc.lab.id
  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.nat[each.key].id
  }
  tags = { Name = "kf-rt-gwlbe-${each.key}" }
}

resource "aws_route_table_association" "gwlbe" {
  for_each       = aws_subnet.gwlbe
  subnet_id      = each.value.id
  route_table_id = aws_route_table.gwlbe[each.key].id
}

# Los prefijos del lab llegan por BGP. GOTCHA #1 del post anterior sigue
# vigente: la propagacion viene apagada por defecto.
resource "aws_vpn_gateway_route_propagation" "gwlbe" {
  for_each       = aws_route_table.gwlbe
  vpn_gateway_id = aws_vpn_gateway.vgw.id
  route_table_id = each.value.id
}

resource "aws_vpn_gateway_route_propagation" "mgmt" {
  vpn_gateway_id = aws_vpn_gateway.vgw.id
  route_table_id = aws_route_table.mgmt.id
}

# --- App: todo sale por el GWLBE ---------------------------------------------
# Sin propagacion del VGW aca: si se propagaran los prefijos del lab, el
# trafico iria directo al VGW sin pasar por el firewall.

resource "aws_route_table" "app" {
  for_each = local.subnets
  vpc_id   = aws_vpc.lab.id
  tags     = { Name = "kf-rt-app-${each.key}" }
}

resource "aws_route" "app_default" {
  for_each               = local.subnets
  route_table_id         = aws_route_table.app[each.key].id
  destination_cidr_block = "0.0.0.0/0"
  vpc_endpoint_id        = aws_vpc_endpoint.gwlbe[each.key].id
}

resource "aws_route" "app_to_lab" {
  for_each               = local.app_lab_routes
  route_table_id         = aws_route_table.app[each.value.az].id
  destination_cidr_block = each.value.cidr
  vpc_endpoint_id        = aws_vpc_endpoint.gwlbe[each.value.az].id
}

resource "aws_route_table_association" "app" {
  for_each       = aws_subnet.app
  subnet_id      = each.value.id
  route_table_id = aws_route_table.app[each.key].id
}

# --- Edge route table del VGW ------------------------------------------------
# Esto es lo que hace que el trafico hibrido se inspeccione. Sin esta
# asociacion el trafico del tunel entra directo a las subnets app.

resource "aws_route_table" "vgw_edge" {
  vpc_id = aws_vpc.lab.id
  tags   = { Name = "kf-rt-vgw-edge" }
}

resource "aws_route" "vgw_edge_to_app" {
  for_each               = local.subnets
  route_table_id         = aws_route_table.vgw_edge.id
  destination_cidr_block = each.value.app
  vpc_endpoint_id        = aws_vpc_endpoint.gwlbe[each.key].id
}

resource "aws_route_table_association" "vgw_edge" {
  gateway_id     = aws_vpn_gateway.vgw.id
  route_table_id = aws_route_table.vgw_edge.id
}
06-outputs.tfOutputs, among them the tunnels' inside addressing.
output "firewalls" {
  description = "EIP de gestion de cada VM-Series y la IP de datos registrada en el target group."
  value = {
    for k in keys(local.subnets) : k => {
      mgmt_eip = aws_eip.fw_mgmt[k].public_ip
      data_ip  = aws_network_interface.fw_data[k].private_ip
    }
  }
}

output "app_servers" {
  description = "Workloads. Sin IP publica: se prueban desde el home lab por el tunel."
  value       = { for k, v in aws_instance.app : k => v.private_ip }
}

output "gwlb_endpoints" {
  value = { for k, v in aws_vpc_endpoint.gwlbe : k => v.id }
}

output "target_group_arn" {
  description = "Para vigilar el estado con: aws elbv2 describe-target-health --target-group-arn ..."
  value       = aws_lb_target_group.fw.arn
}

# --- Datos para armar el tunel en el PA-410 ---------------------------------

# inside_local / inside_remote salen de los atributos que devuelve AWS, no de
# cidrhost(). AWS asigna el PRIMER host del /30 al VGW y el segundo al customer
# gateway; calcularlo a mano invita a dejarlo al reves, que es justo lo que
# paso antes: el PA quedaba con la .1 (que es del VGW) y peereaba contra la .2.
output "pa440_tunnel1" {
  value = {
    aws_peer_ip   = aws_vpn_connection.s2s.tunnel1_address
    inside_local  = aws_vpn_connection.s2s.tunnel1_cgw_inside_address
    inside_remote = aws_vpn_connection.s2s.tunnel1_vgw_inside_address
    bgp_peer_asn  = var.vgw_asn
  }
}

output "pa440_tunnel2" {
  value = {
    aws_peer_ip   = aws_vpn_connection.s2s.tunnel2_address
    inside_local  = aws_vpn_connection.s2s.tunnel2_cgw_inside_address
    inside_remote = aws_vpn_connection.s2s.tunnel2_vgw_inside_address
    bgp_peer_asn  = var.vgw_asn
  }
}

# CREDENCIAL VIVA. Nunca en el blog ni en screenshots.
output "psk_tunnel1" {
  value     = aws_vpn_connection.s2s.tunnel1_preshared_key
  sensitive = true
}

output "psk_tunnel2" {
  value     = aws_vpn_connection.s2s.tunnel2_preshared_key
  sensitive = true
}

Phase 2VM-Series configuration

Identical on both devices: the data interface takes its IP over DHCP, so there is nothing to customise per zone.

configure

set network profiles interface-management-profile GWLB-HC http yes
set network profiles interface-management-profile GWLB-HC ping yes

set network interface ethernet ethernet1/1 layer3 dhcp-client enable yes
set network interface ethernet ethernet1/1 layer3 dhcp-client create-default-route no
set network interface ethernet ethernet1/1 layer3 interface-management-profile GWLB-HC

set network virtual-router default interface ethernet1/1
set zone gwlb network layer3 ethernet1/1

The http yes on the management profile is what makes the firewall answer the health check. The zone path depends on multi-vsys: with off it is set zone; with on it is set vsys vsys1 zone. Verify it with show system info | match multi-vsys before writing the line: if the zone is not created, every rule referencing it fails later.

Policy: a single zone, everything intrazone

Traffic comes in and goes out through the same interface, so source and destination land in the same zone. The policy has to allow that explicitly.

The health check needs its own rule

Enabling HTTP on the management profile is necessary but not sufficient. The health check comes from a GWLB node in the same subnet towards the firewall's data IP: it is intrazone traffic and it goes through the policy. Any cleanup rule any/any kills it first, and the target stays unhealthy forever without AWS explaining a thing.

set service TCP-80 protocol tcp port 80
set address-group GWLB-NODES static [ FWDATA-A FWDATA-B ]

set rulebase security rules GWLB-HEALTHCHECK from gwlb
set rulebase security rules GWLB-HEALTHCHECK to gwlb
set rulebase security rules GWLB-HEALTHCHECK source GWLB-NODES
set rulebase security rules GWLB-HEALTHCHECK destination any
set rulebase security rules GWLB-HEALTHCHECK application any
set rulebase security rules GWLB-HEALTHCHECK service TCP-80
set rulebase security rules GWLB-HEALTHCHECK action allow

move rulebase security rules GWLB-HEALTHCHECK top

The move is not cosmetic: new rules get appended at the end, that is, below the cleanup, where they never match.

For the inspection rules use service any. The combination application any with service application-default matches nothing: with no specific applications in the rule there is no default port to derive the service from.

Verification

Both targets healthy. It takes about 30 seconds at the default interval.

aws elbv2 describe-target-health --region <region> \
  --target-group-arn <arn>

Phase 3Enabling GENEVE parsing

This is the step that makes the design work, and the only one that produces no error at all if you skip it.

PAN-OS does not decapsulate GENEVE until you ask it to. Without that it receives the UDP 6081 packets, never sees the inner addresses, inspects nothing and never returns them to the tunnel.

On both devices

request plugins vm_series aws gwlb inspect enable yes
request plugins vm_series aws gwlb overlay-routing enable no
show plugins vm_series aws gwlb

GWLB enabled has to say True. The message vpc-endpoint association not found is normal in bump-in-the-wire: that association belongs to overlay routing mode, which is deliberately ruled out here.

INSPECT ENABLE NO GWLB GENEVE 6081 VM-Series does not decapsulate nothing returns to the tunnel rules at 0 hits · no deny · no log INSPECT ENABLE YES GWLB GENEVE 6081 decap → policy → re-encap App-ID sees the inner packet GENEVE GWLB endpoint workload The TCP 80 health check crosses both paths unchanged: that is why the target looks healthy either way.
The health check does not depend on parsing, which is why the AWS dashboard looks healthy while not a single packet is being inspected. That is what makes this mistake so hard to find.

Final verification

A workload that answers proves traffic is flowing, not that it is being inspected. The real proof is in the log: the identified application has to be the real one and not not-applicable.

show rule-hit-count vsys vsys-name vsys1 rule-base security rules all
show log traffic rule equal <REGLA>

web-browsing   gwlb   49880   <origen>
HIBRIDO-IN     allow  gwlb    80    <workload>
                              tcp-fin

Do not use direction equal backward: it returns the oldest entries in the log, and with health checks running every few seconds you will never see your test there.

BGPThe control plane on the hybrid side

Only applies if you also terminate a VPN against the same architecture. Real output from the edge device, with the router identifier and the pre-existing peer redacted.

One screen with everything

The summary is the command with the best return: one view gives the state of each session and how many prefixes go in and out of each.

admin@fw-borde> show routing protocol bgp summary

  ==========
  router id:                     <router-id>
  virtual router:                default
  Local AS:                      65000
  Install BGP routes:            yes
  Graceful Restart:              supported
  Default local preference:      100
  mp-bgp-enable:                 yes
  afi-safi-ipv4-unicast:         yes
  rib-out entries:               current 140, peak 142
    peer PEER-PREEXISTENTE:      AS 65010, Established, IP <inside-peer>
      bgpAfiIpv4/unicast pfx:    Accepted pfx: 1, Advertised pfx: 54
    peer AWS-VGW-1:              AS 64512, Established, IP 169.254.200.1
      bgpAfiIpv4/unicast pfx:    Accepted pfx: 3, Advertised pfx: 43
    peer AWS-VGW-2:              AS 64512, Established, IP 169.254.201.1
      bgpAfiIpv4/unicast pfx:    Accepted pfx: 3, Advertised pfx: 43

What to read here: both AWS peers in Established, 3 prefixes accepted from each —the VPC and its two app subnets— and 43 advertised towards each. Both tunnels showing identical numbers is the sign that redundancy is live and not merely configured.

The peer that already existed keeps its own counter: Advertised pfx: 54. If that number changes when you add the new group, you touched a policy that belonged to another peer.

The detail of one session

admin@fw-borde> show routing protocol bgp peer

  ==========
  Peer:                          AWS-VGW-2 (id 5)
  virtual router:                default
  Peer router id:                169.254.201.1
  Remote AS:                     64512
  Peer group:                    AWS (id 1)
  Peer status:                   Established, for 20118 seconds
  Passive:                       no
  Multi-hop TTL:                 1
  Remote Address:                169.254.201.1:179
  Local Address:                 169.254.201.2:40411
  Prefix limit:                  100
  Holdtime:                      30 (config 30)
  Keep-Alive interval:           10 (config 10)
  Update messages:               in        4, out        5
  Total messages:                in     2018, out     2318
  Last error:
  Flap counts:                   1, established 1 times
  Nexthop set to self:           no
  ----------
  remove private AS number:      no
  ----------
  Capability:                    Multiprotocol Extensions(1)  value: IPv4 Unicast
  Capability:                    Route Refresh(yes)
  Capability:                    4-Byte AS Number(65)  value: 64512
  ----------
  Prefix counter for:            bgpAfiIpv4 / unicast
  Incoming Prefix:               Accepted 3, Rejected 0, Policy Rej 0, Total 3
  Outgoing Prefix:               43
  Advertised Prefix:             43

Four lines worth looking at that usually get overlooked:

  • Local Address: 169.254.201.2 against Remote Address: 169.254.201.1 — the local one is the .2. Swapped around, the tunnel comes up just the same and the session never establishes.
  • remove private AS number: no — with every ASN private, stripping them from the AS_PATH removes loop detection.
  • Flap counts: 1, established 1 times — it established once and never dropped again. A rising counter points at MTU or tunnel instability.
  • Accepted 3, Rejected 0, Policy Rej 0 — it separates "no routes arrive" from "they arrive and an import policy discards them".

What ends up in the forwarding table

admin@fw-borde> show routing route destination 10.20.0.0/16

flags: A:active, ?:loose, C:connect, H:host, S:static, ~:internal, R:rip, O:ospf, B:bgp,
       Oi:ospf intra-area, Oo:ospf inter-area, O1:ospf ext-type-1, O2:ospf ext-type-2, E:ecmp, M:multicast

VIRTUAL ROUTER: default (id 1)
  ==========
destination            nexthop              metric flags   age    next-AS
10.20.0.0/16           169.254.201.1        100    A?B     20117  64512
10.20.4.0/24           169.254.201.1        100    A?B     20117  64512
10.20.14.0/24          169.254.201.1        100    A?B     20117  64512
total routes shown: 3

All three prefixes come in over both tunnels, but only one survives in the FIB: the best path, with flags A?B —active, incomplete origin, learned via BGP. If that tunnel drops, the other takes over with no intervention. To see all six entries before selection, look at the BGP table with show routing protocol bgp loc-rib.

If the firewall advertises and AWS does not accept

When the summary shows advertised prefixes but the AWS console reports zero accepted routes, the suspect is export-nexthop: with resolve, routes relearned from a third peer can leave with a next-hop that AWS discards. use-self fixes it.

VerificationWhat it looks like when it is right

Screenshots from a working deployment. The two that matter most are the edge association and the log with the identified application: the first explains why traffic enters inspection, the second proves it actually is inspected.

In the AWS console

What the control plane has to report.

Target group · targets tab

Target group · pestaña de destinos
Both targets Healthy, one per zone, both on port 6081. Careful: this screen looks exactly the same with inspection working and without it, because the health check is TCP 80 and does not depend on GENEVE parsing. Green here proves nothing about inspection.

Edge route table · routes

Edge route table · rutas
The two routes that divert inbound traffic to inspection: each app subnet pointing at its own zone's endpoint. They are more specific than the local route, which is what makes them win.

Edge route table · edge association

Edge route table · asociación de borde
The piece that is usually missing: the table associated with the virtual private gateway. Without this association the routes in the previous screenshot are never evaluated, and tunnel traffic goes straight to the workload without passing through the firewall.

Gateway Load Balancer endpoints

Endpoints del Gateway Load Balancer
One per zone, both Available. Each lives in the gwlbe of its zone.

VPN connection · tunnel details

Conexión VPN · detalles del túnel
If you also connect on-prem: both tunnels up and with BGP routes accepted. This is the authoritative source for BGP state, above whatever the firewall reports.

Deployment instances

Instancias del despliegue
Four instances: two VM-Series as firewalls and two workloads, spread across both zones.

On the VM-Series

What confirms there is inspection and not just transit.

Policies · Security

Policies · Security
The correct rulebase order. The health check rule goes first; if it sits below the final deny, the target never reaches healthy. The counters confirm what is in use: hybrid traffic and egress accumulate hits, east-west stays at zero because in this design it never goes through the firewall.

Monitor · Traffic filtered by rule

Monitor · Traffic filtrado por regla
The verification that matters. The Application column says web-browsing, not not-applicable: App-ID is seeing the inner packet, which means GENEVE decap works. If what shows up there is not-applicable, traffic flows but is not inspected.

Network · Interfaces

Network · Interfaces
The data interface in Layer3, with its virtual router, the inspection zone and the management profile applied. DHCP assigns the IP and it has to match the target registered in the group.

On the edge device

Only if you also terminate a VPN against the same architecture.

IPSec Tunnels on the edge device

IPSec Tunnels en el equipo de borde
Both tunnels to AWS green, each on its own tunnel interface. The blank rows are production tunnels on the same device, hidden on purpose.

BGP · peers

BGP · peers
Both AWS peers in Established, with the local address in the .2 and the peer in the .1 from each /30. Above, a pre-existing peer with its own uptime: when adding the new peer group, it is worth confirming the existing ones did not flap.

MistakesTen that give no message

All of them turned up while building this architecture. None produces an error that points at the cause.

E1 · GWLB

GENEVE parsing is not enabled by default

Symptom
Everything looks healthy and not one packet is inspected. No errors.
Cause
PAN-OS does not decapsulate UDP 6081 until you ask it to. overlay-routing is a different setting.
Fix
request plugins vm_series aws gwlb inspect enable yes

E2 · policy

The final deny stops the target from reaching healthy

Symptom
Target.Timeout permanently, even with HTTP enabled on the management profile.
Cause
The health check is intrazone and goes through the policy, where the final deny matches first.
Fix
An allow rule for TCP 80 from the data subnets, in first position.

E3 · policy

application any with application-default matches nothing

Symptom
Inspection rules at 0 hits from day one, with no explanation.
Cause
With no specific applications there is no default port to derive the service from.
Fix
service any, or list explicit applications.

E4 · policy

New rules are born below the final deny

Symptom
You add a correct rule and it never matches.
Cause
set rulebase security rules appends to the end of the rulebase.
Fix
move ... top or before <cleanup>, and verify the order before committing.

E5 · CLI

The zone path changes with multi-vsys

Symptom
Invalid syntax when creating the zone, then cascading failures from not a valid reference.
Cause
With multi-vsys off the node vsys does not exist at the root.
Fix
set zone with multi-vsys off; set vsys vsys1 zone with multi-vsys on.

E6 · hybrid

AWS gives the first host of the /30 to the gateway, not to the customer

Symptom
IPsec comes up and BGP never leaves Connect.
Cause
In the /30 inside address of the tunnel, the .1 belongs to the virtual private gateway and the .2 to the customer gateway.
Fix
Read the attributes AWS returns instead of working the addresses out by hand.

E7 · hybrid

A final any/any rule also kills intrazone traffic

Symptom
BGP does not establish over the /30 of the tunnel.
Cause
A rule from any to any is of type universal: it matches intrazone as well as interzone.
Fix
An explicit intrazone rule for the tunnel inside IPs, above the deny.

E8 · Terraform

The ingress inline blocks revoke whatever you add by hand

Symptom
Management access is lost after a apply that was not touching that resource.
Cause
With inline blocks, Terraform is authoritative over the entire security group.
Fix
Make the access variable a list and keep the IPs in code. Verify with terraform plan -detailed-exitcode.

E9 · CLI

An 80-column PTY corrupts long commands

Symptom
Invalid syntax intermittently on commands that are correct.
Cause
The CLI wraps by inserting a space in the middle: ethernet1/1 arrives as ethern et1/1.
Fix
Enlarge the PTY before spawning, or lower it with edit <nodo> to shorten each line.

E10 · logs

direction equal backward shows the oldest

Symptom
Test traffic does not show up in the log even though the rule accumulates hits.
Cause
It returns the oldest entries, and the log is dominated by health checks.
Fix
show log traffic rule equal <NOMBRE>

LimitationsWhat this design does not solve

East-west between app subnets is not inspected

In each app subnet's route table, the local route of the VPC beats the default towards the endpoint on longest-prefix match. Traffic between workloads goes direct, and the rule covering it stays at zero hits.

Adding routes more specific than the local one is not enough: each direction would enter through its own zone's endpoint and be inspected by a different firewall. With a stateful device that does not work, and stickiness does not save it, because when source and destination swap, the hash changes. The real options are sending east-west through a single endpoint —losing zone independence— or accepting that this path is not inspected and removing the rule, so as not to promise something that does not happen.

No MSS clamp on tunnel interfaces

if you also terminate a VPN against the same device, bear in mind that adjust-tcp-mss does not exist under tunnel units in PAN-OS 11.1: the node only appears on ethernet subinterfaces. Without a clamp, ping goes through and large TCP transfers hang. The alternative is to clamp on the ingress LAN interfaces, at the cost of affecting all their traffic.

The workloads depend on inspection in order to provision themselves

Their internet egress goes through the endpoint, the GWLB and the firewall. While inspection is not working they have no internet: cloud-init cannot reach the metadata service and the user_data never runs, with no error visible in the console at all. That is why the end-to-end test comes at the end of the procedure and not in the middle.