Configuration Management December 1, 2025 ⏱ 7 min read

Dynamic Inventory in Ansible: AWS, GCP, and Azure

Replace static inventory files with dynamic inventory scripts that automatically discover your cloud infrastructure across AWS, GCP, and Azure.

ansibleawsgcpazuredynamic-inventoryautomation

Static inventory files become a maintenance nightmare at scale. When instances spin up and down automatically, hardcoded host lists are always out of date. Dynamic inventory pulls your infrastructure directly from cloud APIs, so Ansible always knows what exists.

How Dynamic Inventory Works

Instead of a static file:

# Old way: static inventory
[webservers]
web1.example.com
web2.example.com
web3.example.com

[databases]
db1.example.com

You use a script or plugin that queries your cloud provider:

# Dynamic inventory queries cloud API
ansible-inventory -i aws_ec2.yml --list

# Returns current instances with their metadata
{
  "tag_Environment_production": {
    "hosts": ["10.0.1.100", "10.0.1.101", "10.0.2.50"]
  },
  "tag_Role_webserver": {
    "hosts": ["10.0.1.100", "10.0.1.101"]
  },
  "_meta": {
    "hostvars": {
      "10.0.1.100": {
        "ansible_host": "10.0.1.100",
        "ec2_instance_type": "t3.medium",
        "ec2_region": "us-east-1"
      }
    }
  }
}

AWS EC2 Dynamic Inventory

Installation

# Install AWS collection
ansible-galaxy collection install amazon.aws

# Install boto3 (AWS SDK)
pip install boto3 botocore

Configuration

# inventory/aws_ec2.yml
plugin: amazon.aws.aws_ec2

# AWS authentication (or use environment variables / IAM role)
aws_access_key_id: "{{ lookup('env', 'AWS_ACCESS_KEY_ID') }}"
aws_secret_access_key: "{{ lookup('env', 'AWS_SECRET_ACCESS_KEY') }}"

# Regions to query
regions:
  - us-east-1
  - us-west-2
  - eu-west-1

# Only include running instances
filters:
  instance-state-name: running

# Use private IP for SSH (within VPC)
hostnames:
  - private-ip-address
  # Or use public IP:
  # - ip-address
  # Or DNS:
  # - dns-name

# Create groups from tags and attributes
keyed_groups:
  # Group by Environment tag: tag_Environment_production
  - key: tags.Environment
    prefix: tag_Environment
    separator: "_"
  
  # Group by Role tag: tag_Role_webserver
  - key: tags.Role
    prefix: tag_Role
    separator: "_"
  
  # Group by instance type: instance_type_t3_medium
  - key: instance_type
    prefix: instance_type
    separator: "_"
  
  # Group by region: region_us_east_1
  - key: placement.region
    prefix: region
    separator: "_"
  
  # Group by VPC: vpc_id_vpc_12345678
  - key: vpc_id
    prefix: vpc_id

# Add all instances to these groups
groups:
  # All EC2 instances
  ec2: true
  
  # Production web servers (conditional)
  production_web: tags.Environment == 'production' and tags.Role == 'webserver'

# Compose variables for each host
compose:
  ansible_host: private_ip_address
  ansible_user: "'ec2-user' if 'amzn' in image_id else 'ubuntu'"
  instance_name: tags.Name | default('unnamed')

Usage

# List all discovered hosts
ansible-inventory -i inventory/aws_ec2.yml --list

# Graph view
ansible-inventory -i inventory/aws_ec2.yml --graph

# Run playbook
ansible-playbook -i inventory/aws_ec2.yml playbooks/deploy.yml

# Target specific group
ansible-playbook -i inventory/aws_ec2.yml playbooks/deploy.yml --limit tag_Role_webserver

IAM Permissions Required

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeTags",
        "ec2:DescribeRegions"
      ],
      "Resource": "*"
    }
  ]
}

GCP Compute Dynamic Inventory

Installation

ansible-galaxy collection install google.cloud
pip install google-auth requests

Configuration

# inventory/gcp.yml
plugin: google.cloud.gcp_compute

# Authentication
auth_kind: serviceaccount
service_account_file: /path/to/service-account.json
# Or use application default credentials:
# auth_kind: application

# Projects to query
projects:
  - my-project-id
  - another-project-id

# Zones (or use regions for all zones in region)
zones:
  - us-central1-a
  - us-central1-b
  - us-east1-b

# Only running instances
filters:
  - status = RUNNING

# Use internal IP
hostnames:
  - private_ip

# Create groups
keyed_groups:
  # By label: label_env_production
  - key: labels.env
    prefix: label_env
  
  # By machine type: machinetype_n1_standard_2
  - key: machineType | basename
    prefix: machinetype
  
  # By zone: zone_us_central1_a
  - key: zone | basename
    prefix: zone
  
  # By network: network_default
  - key: networkInterfaces[0].network | basename
    prefix: network

groups:
  gcp: true
  production: labels.env == 'production'

compose:
  ansible_host: networkInterfaces[0].networkIP
  ansible_user: "'ubuntu'"
  gcp_name: name

Service Account Setup

# Create service account
gcloud iam service-accounts create ansible-inventory \
  --display-name="Ansible Inventory"

# Grant permissions
gcloud projects add-iam-policy-binding my-project \
  --member="serviceAccount:[email protected]" \
  --role="roles/compute.viewer"

# Create key
gcloud iam service-accounts keys create ~/gcp-ansible.json \
  --iam-account=ansible-inventory@my-project.iam.gserviceaccount.com

Azure Dynamic Inventory

Installation

ansible-galaxy collection install azure.azcollection
pip install azure-identity azure-mgmt-compute azure-mgmt-network

Configuration

# inventory/azure_rm.yml
plugin: azure.azcollection.azure_rm

# Authentication (or use environment variables)
auth_source: auto  # Uses DefaultAzureCredential
# Or explicitly:
# auth_source: cli
# Or service principal:
# client_id: xxx
# secret: xxx
# tenant: xxx
# subscription_id: xxx

# Include these resource groups
include_vm_resource_groups:
  - production-rg
  - staging-rg

# Only running VMs
default_host_filters:
  - powerstate == 'running'

# Use private IP
hostvar_expressions:
  ansible_host: private_ipv4_addresses[0]

# Create groups
keyed_groups:
  # By resource group: resourcegroup_production_rg
  - key: resource_group
    prefix: resourcegroup
    separator: "_"
  
  # By location: location_eastus
  - key: location
    prefix: location
  
  # By tags: tag_Environment_production
  - key: tags.Environment
    prefix: tag_Environment

# Conditional groups
conditional_groups:
  webservers: "'webserver' in tags.Role"
  databases: "'database' in tags.Role"

# Plain groups
plain_host_names: true

Azure Authentication

# Using Azure CLI (simplest for development)
az login

# Service Principal for CI/CD
export AZURE_CLIENT_ID="xxx"
export AZURE_SECRET="xxx"
export AZURE_TENANT="xxx"
export AZURE_SUBSCRIPTION_ID="xxx"

Multi-Cloud Inventory

Combine multiple inventory sources:

inventory/
├── aws_ec2.yml
├── gcp.yml
├── azure_rm.yml
└── static.yml       # For on-prem or edge cases

# Use directory as inventory source
ansible-playbook -i inventory/ playbooks/site.yml

# Ansible automatically loads all files

ansible.cfg Configuration

[defaults]
inventory = ./inventory

[inventory]
enable_plugins = amazon.aws.aws_ec2, google.cloud.gcp_compute, azure.azcollection.azure_rm, ini

Caching for Performance

Cloud API queries can be slow. Enable caching:

# inventory/aws_ec2.yml
plugin: amazon.aws.aws_ec2

# Cache settings
cache: true
cache_plugin: jsonfile
cache_connection: /tmp/ansible_inventory_cache
cache_timeout: 300  # 5 minutes

# Clear cache
rm -rf /tmp/ansible_inventory_cache

# Or invalidate in ansible.cfg
[inventory]
cache_timeout = 0

Custom Inventory Scripts

For custom infrastructure or legacy systems:

#!/usr/bin/env python3
# inventory/custom_inventory.py

import json
import sys
import requests

def get_inventory():
    """Fetch inventory from custom API"""
    response = requests.get('https://cmdb.example.com/api/hosts')
    hosts = response.json()
    
    inventory = {
        '_meta': {'hostvars': {}},
        'all': {'children': ['webservers', 'databases']},
        'webservers': {'hosts': []},
        'databases': {'hosts': []}
    }
    
    for host in hosts:
        hostname = host['fqdn']
        
        # Add to appropriate group
        if host['role'] == 'web':
            inventory['webservers']['hosts'].append(hostname)
        elif host['role'] == 'db':
            inventory['databases']['hosts'].append(hostname)
        
        # Add host variables
        inventory['_meta']['hostvars'][hostname] = {
            'ansible_host': host['ip_address'],
            'ansible_user': host.get('ssh_user', 'admin'),
            'datacenter': host['datacenter'],
            'environment': host['environment']
        }
    
    return inventory

def get_host(hostname):
    """Return variables for a specific host"""
    inventory = get_inventory()
    return inventory['_meta']['hostvars'].get(hostname, {})

if __name__ == '__main__':
    if len(sys.argv) == 2 and sys.argv[1] == '--list':
        print(json.dumps(get_inventory(), indent=2))
    elif len(sys.argv) == 3 and sys.argv[1] == '--host':
        print(json.dumps(get_host(sys.argv[2]), indent=2))
    else:
        print("Usage: {} --list or {} --host <hostname>".format(sys.argv[0], sys.argv[0]))
        sys.exit(1)

# Make executable and test
chmod +x inventory/custom_inventory.py
./inventory/custom_inventory.py --list

# Use with Ansible
ansible-playbook -i inventory/custom_inventory.py playbooks/site.yml

Filtering and Targeting

Filter in Inventory Config

# aws_ec2.yml - only production instances
filters:
  instance-state-name: running
  "tag:Environment": production

Filter at Runtime

# Target specific group
ansible-playbook -i inventory/ site.yml --limit tag_Environment_production

# Combine groups
ansible-playbook -i inventory/ site.yml --limit 'tag_Role_webserver:&region_us_east_1'

# Exclude
ansible-playbook -i inventory/ site.yml --limit 'all:!tag_Environment_development'

# Pattern matching
ansible-playbook -i inventory/ site.yml --limit 'web*'

Group Variables with Dynamic Inventory

Dynamic groups work with group_vars/:

inventory/
├── aws_ec2.yml
└── group_vars/
    ├── all.yml
    ├── tag_Environment_production.yml
    ├── tag_Environment_staging.yml
    ├── tag_Role_webserver.yml
    └── region_us_east_1.yml

# group_vars/tag_Environment_production.yml
nginx_worker_processes: 4
enable_monitoring: true
log_level: warn

# group_vars/tag_Role_webserver.yml
nginx_enabled: true
app_port: 8080

Debugging Dynamic Inventory

# Test inventory plugin
ansible-inventory -i aws_ec2.yml --list --yaml

# Show graph
ansible-inventory -i aws_ec2.yml --graph

# Debug specific host
ansible-inventory -i aws_ec2.yml --host 10.0.1.100

# Verbose output
ansible-inventory -i aws_ec2.yml --list -vvv

Common Patterns

Bastion/Jump Host

# group_vars/all.yml
ansible_ssh_common_args: '-o ProxyJump=bastion.example.com'

# Or per-region
# group_vars/region_us_east_1.yml
ansible_ssh_common_args: '-o ProxyJump=bastion-use1.example.com'

Different SSH Users by AMI

# aws_ec2.yml compose section
compose:
  ansible_host: private_ip_address
  ansible_user: >-
    {%- if 'ubuntu' in tags.get('OS', '') -%}
    ubuntu
    {%- elif 'amazon' in image_id -%}
    ec2-user
    {%- else -%}
    admin
    {%- endif -%}

Auto-Scaling Groups

# aws_ec2.yml - group by ASG
keyed_groups:
  - key: 'tags["aws:autoscaling:groupName"]'
    prefix: asg
    separator: "_"

When NOT to Use Dynamic Inventory

Small, static environments — static files are simpler
Air-gapped environments — no API access
Testing — use static inventory for reproducibility
Extremely fast iteration — API calls add latency

Key Takeaways

Dynamic inventory eliminates manual host management — instances come and go automatically
Use tags religiously — they become your Ansible groups
Enable caching — cloud APIs are slow
Combine multiple sources — AWS + GCP + static in one inventory directory
Test with ansible-inventory --list before running playbooks
Use compose for host variables — derive ansible_host, ansible_user from instance data

Dynamic inventory transforms Ansible from a configuration tool into a true cloud automation platform. Once set up, you’ll never maintain static inventory files again.