Subnet, CIDR, Routing #

Understanding subnetting concepts, CIDR notation, and routing rules is a fundamental skill every cloud engineer must master. Every time we design a Virtual Private Cloud (VPC), divide IP address space for Kubernetes containers, or connect office networks to the cloud, we’ll always deal with CIDR mathematical notation and subnet division decisions. Poor IP address planning often leads to running out of IP addresses when auto-scaling kicks in, or routing collisions that force us to rebuild the entire network infrastructure from scratch. This article thoroughly unpacks how to read CIDR notation, strategies for planning subnetting in production environments, and how Route Tables intelligently determine data traffic flow direction.

CIDR Notation Basics: The Math Behind IP Addressing #

CIDR (Classless Inter-Domain Routing) is an IP addressing methodology that replaced the traditional IP class division system (Class A, B, C), which was considered inefficient in utilizing IP address space. CIDR notation always consists of a base IP address followed by a slash and a binary number called the prefix length: IP_Address/Prefix_Length.

To read CIDR notation, we must understand the basic structure of an IPv4 address, which consists of 32 bits of binary numbers divided into 4 octets (8 bits each). The number after the slash (prefix) determines how many leading bits are static (network portion) and how many remaining bits are dynamic, allocated to machines (host portion).

How to Calculate IP Capacity from CIDR Notation:

  1. Notation: 10.0.0.0/16
     → Prefix /16 means the first 16 bits are locked for network identity.
     → Remaining host bits: 32 bits - 16 bits = 16 bits.
     → Total IPs: 2^16 = 65,536 IP addresses.
     → IP Range: 10.0.0.0 to 10.0.255.255.

  2. Notation: 10.0.1.0/24
     → Prefix /24 means the first 24 bits are locked.
     → Remaining host bits: 32 bits - 24 bits = 8 bits.
     → Total IPs: 2^8 = 256 IP addresses.
     → IP Range: 10.0.1.0 to 10.0.1.255.

For easier network planning, the table below provides a quick reference of CIDR sizes commonly used in the cloud:

Binary Math: How Computers Read Subnet Masks #

At the hardware and cloud hypervisor level, computers don’t recognize the decimal numbers we read (like 10.0.1.0). Computers see the entire IP address as a 32-digit binary string.

The subnet mask (prefix) acts as a binary filter. When we set /24 notation, we’re telling the system to write the binary number 1 24 times consecutively, followed by 0 until reaching 32 bits.

Binary Representation of the /24 Subnet Mask:
  Decimal: 255.255.255.0
  Binary:  11111111.11111111.11111111.00000000
           |<─────── 24 network bits ───────>|  8 host bits

When a data packet enters the virtual router with a specific destination IP, the router performs a Bitwise AND gate operation between the destination IP and the subnet mask to determine whether the destination is on the same local network or must be thrown out through the gateway.

Bitwise AND Operation at the Virtual Router:
  Destination IP: 10.0.1.55  →  00001010.00000000.00000001.00110111
  Subnet Mask: /24           →  11111111.11111111.11111111.00000000  (AND)
  ─────────────────────────────────────────────────────────────────────
  Resulting Network Address:     00001010.00000000.00000001.00000000  (10.0.1.0)

If the bitwise AND result equals the local subnet’s network address (10.0.1.0), the router knows the data packet can be sent directly to the destination VM locally without passing through an external gateway. If the result differs, the router consults the Route Table to throw the packet to an external gateway.

Reserved IP Allocation in Cloud Subnets #

One important difference between traditional physical networks and cloud networks is the existence of reserved IP addresses. In a physical local network, only 2 IP addresses can’t be used: the network address (first IP) and the broadcast address (last IP).

However, in cloud environments, providers reserve 5 first and last IP addresses in every subnet for their internal infrastructure operations and routing.

Reserved IP Visualization on Subnet 10.0.1.0/24 (Total 256 IPs):

  10.0.1.0   → Network Address - Cannot be used.
  10.0.1.1   → Internal VPC Router Gateway Address - Reserved.
  10.0.1.2   → Internal DNS Resolver Address (Route 53 / Cloud DNS) - Reserved.
  10.0.1.3   → Spare IP for Future Needs - Reserved.
  10.0.1.255 → Network Broadcast Address - Cannot be used.

  IPs actually usable for VMs: 10.0.1.4 to 10.0.1.254 (251 IPs)

If we deploy a subnet with CIDR /28 (16 total IPs), we only have 11 usable IPs (16 - 5 = 11). If this subnet is used for an application with active auto-scaling, that IP capacity runs out instantly, preventing new VM instances from serving requests.

Subnet Segmentation Strategy: Production Network Tiering #

When designing a secure VPC architecture for production environments, we’re advised to apply the 3-Tier Subnetting pattern spread across at least 2 or 3 Availability Zones (AZs) to prevent single points of failure.

flowchart TD
    subgraph VPC["Main VPC: 10.0.0.0/16"]
        subgraph AZ_1a["Availability Zone 1 (AZ-1a)"]
            Pub1["Public Subnet 1<br>10.0.1.0/24"]
            Priv1["Private Subnet 1<br>10.0.10.0/24"]
            Data1["Data Subnet 1<br>10.0.20.0/24"]
        end
        subgraph AZ_1b["Availability Zone 2 (AZ-1b)"]
            Pub2["Public Subnet 2<br>10.0.2.0/24"]
            Priv2["Private Subnet 2<br>10.0.11.0/24"]
            Data2["Data Subnet 2<br>10.0.21.0/24"]
        end
    end

1. Public Subnet Tier (DMZ) #

This subnet connects directly to the Internet Gateway (IGW) and is allocated for resources that must be directly accessible from the outside internet.

• Resources: Application Load Balancer (ALB), NAT Gateway, Bastion Host.
• Routing: 0.0.0.0/0 directed to igw-xxxxxx.

2. Private Subnet Tier (Application) #

This subnet is isolated from the outside internet and only has a one-way outbound route (outbound-only) through a NAT Gateway for downloading OS dependencies or external API integrations.

• Resources: Application Servers (NodeJS, Go, Java), Container Tasks, EKS Workers.
• Routing: 0.0.0.0/0 directed to nat-xxxxxx.

3. Data Subnet Tier (Database) #

This subnet has the highest isolation level. It has no route to the outside internet at all, neither inbound nor outbound. Communication is limited locally to serving the compute layer.

• Resources: RDS Postgres/MySQL, Redis Cache Clusters, NoSQL Databases.
• Routing: Only has a local route (e.g., 10.0.0.0/16 local).

How Route Tables Work: Longest Prefix Match Wins #

The VPC router makes data packet routing decisions using a special rule called Longest Prefix Match. When a data packet has a specific destination IP address, the router checks every rule row in the Route Table and selects the rule with the largest (most specific) prefix mask value.

Let’s simulate how this routing decision is made inside a VPC:

Installed Route Table Rules:
  Row 1: 10.0.0.0/16   → local
  Row 2: 10.0.5.0/24   → peering-connection-xyz
  Row 3: 0.0.0.0/0     → internet-gateway

Case A: Packet Destination is 10.0.5.23 #

• The router matches this IP against Row 1 (/16), Row 2 (/24), and Row 3 (/0).
• Because /24 (Row 2) is longer and more specific than /16 (Row 1), Row 2 wins.
• The packet is sent to peering-connection-xyz.

Case B: Packet Destination is 10.0.20.15 #

• This IP matches Row 1 (/16) and Row 3 (/0).
• Because /16 (Row 1) is more specific than /0 (Row 3), Row 1 wins.
• The packet is processed locally inside the VPC safely.

Cross-AZ and Cross-Region Routing #

When designing high availability across Availability Zones (Multi-AZ), network traffic routing must account for data transfer charge implications and disaster tolerance (fault tolerance).

1. The Multi-AZ NAT Gateway Problem (Anti-Pattern) #

A very common design mistake is deploying a single NAT Gateway in AZ-1a, then connecting Route Tables from both AZ-1a and AZ-1b private subnets to that single NAT Gateway.

flowchart TD
    subgraph VPC["Main VPC"]
        subgraph AZ_1a["Availability Zone 1a"]
            NatGW["NAT Gateway 1"]
            PrivSub1["Private Subnet 1"] -->|Routing| NatGW
        end
        subgraph AZ_1b["Availability Zone 1b"]
            PrivSub2["Private Subnet 2"] -->|Cross-AZ Routing| NatGW
        end
    end
    
    NatGW --> Internet["Internet (Outbound)"]

Why is the architecture above dangerous?

• Single Point of Failure (SPOF): If AZ-1a suffers a power outage or hardware crash, NAT Gateway 1 dies. As a result, the healthy AZ-1b private subnet loses all outbound internet connectivity.
• Cross-AZ Transfer Costs: Traffic from Private Subnet 2 in AZ-1b must physically cross Availability Zones to reach the NAT Gateway in AZ-1a. This triggers Cross-AZ Data Transfer Fees of $0.01 per GB for every outgoing traffic.

2. Redundant Design Solution: One NAT Gateway per AZ #

The best design is building one NAT Gateway in each Availability Zone, and creating separate Route Tables for each private subnet so they always use the local NAT Gateway in the same zone.

Agnostic CIDR Design Planning Across Networks #

One of the most fatal beginner mistakes is setting the same CIDR for every VPC they create (for example, using the default 10.0.0.0/16 for all Dev, Staging, and Prod VPCs).

As the business grows, we’ll definitely need cross-environment integration, like creating a VPC Peering connection between the Dev and Staging VPCs, or connecting the Prod VPC to the head office network via a VPN Tunnel. If our CIDRs overlap, network routes conflict and can’t connect.

// ANTI-PATTERN: Using colliding CIDRs (Overlap)
  Production VPC:  10.0.0.0/16
  Staging VPC:     10.0.0.0/16   ← COLLISION! Can never be connected.

// CORRECT: Planning unique IP blocks systematically from the start
  Production VPC:  10.0.0.0/16   (IPs: 10.0.0.0 - 10.0.255.255)
  Staging VPC:     10.1.0.0/16   (IPs: 10.1.0.0 - 10.1.255.255)
  Development VPC: 10.2.0.0/16   (IPs: 10.2.0.0 - 10.2.255.255)
  On-Premise Office: 192.168.0.0/16

Code Example: Automating Network Calculation with Terraform #

To avoid manual IP calculation errors, we can use Terraform’s built-in functions like cidrsubnet() to dynamically and cleanly divide VPC CIDRs:

# ✓ CORRECT: Use the cidrsubnet() function to automate dynamic IP division

variable "vpc_cidr" {
  default = "10.0.0.0/16"
}

variable "availability_zones" {
  type    = list(string)
  default = ["ap-southeast-1a", "ap-southeast-1b"]
}

resource "aws_vpc" "main" {
  cidr_block = var.vpc_cidr
  tags = {
    Name = "automation-vpc"
  }
}

# Creating Public Subnets dynamically
resource "aws_subnet" "public" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  # cidrsubnet(iprange, newbits, netnum)
  # Result: index 0 -> 10.0.0.0/24, index 1 -> 10.0.1.0/24
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone = element(var.availability_zones, count.index)

  tags = {
    Name = "dynamic-public-subnet-${count.index}"
  }
}

# Creating Private Subnets dynamically
resource "aws_subnet" "private" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  # Using a netnum offset to avoid overlap
  # Result: index 0 -> 10.0.10.0/24, index 1 -> 10.0.11.0/24
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + 10)
  availability_zone = element(var.availability_zones, count.index)

  tags = {
    Name = "dynamic-private-subnet-${count.index}"
  }
}

Summary #

• CIDR notation IP/N determines IP address size — /16 gives 65,536 IPs (ideal VPC size), while /24 gives 256 IPs (ideal subnet size).
• Computers process subnet masks in binary — performing Bitwise AND operations to match whether the destination IP is local or remote.
• Cloud providers reserve 5 IP addresses in every subnet — the first IP, the last IP, and the 3 IPs after the start for DNS, router, and internal functions.
• Apply the 3-Tier Subnetting pattern — firmly separate Public, Private, and Data Subnets for optimal security.
• Route tables use the Longest Prefix Match rule — the route with the largest (most specific) netmask length is always prioritized by the router.
• NAT Gateways must be deployed redundantly per AZ — avoiding single points of failure and minimizing Cross-AZ data transfer fees.
• Plan CIDRs so they don’t overlap — ensure unique IP allocation across all VPCs and on-premise so they can be connected via VPN or Peering later.
• Use Infrastructure as Code (IaC) automation functions — leverage Terraform’s cidrsubnet() function to divide IP address space consistently and safely.

← Previous: Virtual Network (VPC / VNet)   Next: Public vs Private Network →