Ansible Architecture, Agentless SSH Mechanics & Configuration Precedence

Most configuration management tools (such as Puppet, Chef, or SaltStack) begin with a client-server daemon setup: installing a persistent background service (puppet-agent, salt-minion) on every managed server, configuring certificates, and establishing long-running TCP sockets back to a central master server.

Ansible completely flips this paradigm: Ansible is 100% AGENTLESS.

There is no background daemon listening on target servers, no agent to upgrade when Ansible releases a new version, and no persistent master connection to maintain. Ansible operates purely via a Push Model over existing standard network protocols: OpenSSH for Linux/Unix and WinRM / PowerShell for Windows.

Topic 1: Control Node vs. Managed Nodes Architecture

An Ansible automation topology consists of two distinct host classifications:

1. The Control Node

• The machine where Ansible is installed and executed.
• Must run a Linux or Unix-like operating system (Ubuntu, RHEL, Debian, macOS, or Windows Subsystem for Linux — native Windows cannot be a Control Node).
• Requires Python 3.9+ and OpenSSH client installed.
• Houses your Ansible Playbooks, Inventories, Roles, and ansible.cfg project configurations.

2. Managed Nodes

• The target servers, cloud instances (AWS EC2, GCP Compute Engine), network routers, or containers being automated.
• Target Linux nodes require only two prerequisites:
  1. An SSH daemon (sshd) listening on port 22 with SSH public key access.
  2. A standard Python 3 interpreter installed at /usr/bin/python3 (or /usr/bin/python).
  3. Sudo privileges for tasks requiring elevated root permissions (become: true).

+----------------------------------------+
|          CONTROL NODE                  |
|  - Playbooks (site.yml)                |
|  - Inventory (hosts.ini)               |
|  - ansible.cfg & Vault Secrets         |
+----------------------------------------+
                   |
     SSH (Port 22) | OpenSSH / SFTP
                   v
+----------------------------------------+
|          MANAGED NODE                  |
|  - /usr/bin/sshd                       |
|  - /usr/bin/python3                    |
|  - NO PERMANENT ANSIBLE DAEMON!        |
+----------------------------------------+

Topic 2: Under the Hood: Payload Compilation & Execution Lifecycle

When you execute an Ansible task against a target node, Ansible does NOT transmit raw YAML or execute shell strings blindly. Instead, Ansible follows a precise 6-stage execution lifecycle:

1. Task Compilation: The Ansible Engine reads the task definition in your playbook (e.g., ansible.builtin.package: name=nginx state=present) and combines it with active host variables.
2. Python Payload Generation: Ansible generates a standalone, self-contained Python script payload containing the module code and parameter dictionary.
3. SSH Transfer: Ansible opens an SSH session to the target host and copies the Python payload into a temporary remote directory (by default ~/.ansible/tmp/ansible-tmp-<timestamp>/).
4. Remote Execution: Ansible executes the script via the target host’s Python interpreter (/usr/bin/python3 /tmp/.../ansible_payload.py).
5. JSON Return Payload: The Python script executes locally on the target, inspects or modifies system state, and outputs a structured JSON response back over SSH stdout:

   {
     "changed": true,
     "msg": "package nginx installed successfully",
     "rc": 0
   }
   

6. Automatic Cleanup: Ansible automatically deletes the temporary Python payload and directory from the target host disk.

Topic 3: SSH Performance Tuning (ControlPersist & Pipelining)

Because Ansible relies on SSH for every single task execution, unoptimized SSH configurations can introduce latency when managing hundreds of servers. Enterprise DevOps engineers tune two critical SSH parameters:

1. OpenSSH ControlPersist (SSH Multiplexing)

By default, establishing a new SSH connection requires a TCP handshake, key exchange, and authentication negotiation (~200ms per connection). ControlPersist keeps master SSH sockets open in the background. Subsequent tasks reuse the existing open socket instantly:

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s

2. SSH Pipelining (pipelining = True)

In standard operation, Ansible performs two SSH operations per task: one sftp/scp file transfer to write the Python payload file to remote disk, and one ssh command to execute it.

When Pipelining is enabled (pipelining = True), Ansible pipes the Python script directly to the remote Python interpreter stdin over the SSH connection without creating a temporary payload file on disk. This eliminates file I/O operations and speeds up playbook execution by 30% to 50%.

[!WARNING] Pipelining & sudo requiretty: Pipelining requires that requiretty is disabled in /etc/sudoers on target managed nodes. On modern OS distributions (Ubuntu, RHEL 7+), requiretty is disabled by default.

Topic 4: The 4-Level ansible.cfg Precedence Hierarchy

Ansible reads its global configuration from ansible.cfg. When configuring Ansible, there are four possible locations where ansible.cfg can exist. Ansible searches in a strict precedence order — FIRST MATCH WINS:

 ┌─────────────────────────────────────────────────────────────┐
 │ 1. $ANSIBLE_CONFIG (Environment Variable)                  │ ← Highest Precedence
 ├─────────────────────────────────────────────────────────────┤
 │ 2. ./ansible.cfg (Current Working Directory)               │ ← RECOMMENDED FOR PROJECTS
 ├─────────────────────────────────────────────────────────────┤
 │ 3. ~/.ansible.cfg (User Home Directory)                    │
 ├─────────────────────────────────────────────────────────────┤
 │ 4. /etc/ansible/ansible.cfg (Global System Package Default)│ ← Lowest Precedence
 └─────────────────────────────────────────────────────────────┘

Enterprise Production ansible.cfg Template:

Always commit a tailored ./ansible.cfg file inside the root of your playbook repository:

[defaults]
# Default inventory path
inventory = ./inventories/production/hosts.ini

# Number of parallel process forks (default is 5; increase for large fleets)
forks = 20

# Default remote SSH user
remote_user = devops

# Disable host key checking (useful in dynamic cloud environments where IPs cycle)
host_key_checking = False

# Enable clean YAML stdout logging formatting instead of dense JSON
stdout_callback = yaml

# Roles search path
roles_path = ./roles

# Timeout for SSH connections (in seconds)
timeout = 30

[privilege_escalation]
become = True
become_method = sudo
become_user = root
become_ask_pass = False

[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=15m

Topic 5: Ad-Hoc Commands for Fleet Diagnostics

An Ad-Hoc Command uses the ansible CLI binary to execute a single task module across target hosts without creating a written YAML playbook. Ad-hoc commands are perfect for one-off admin tasks, health checks, and emergency server restarts.

Syntax Structure:

ansible <pattern> -i <inventory> -m <module> -a "<module_arguments>"

Essential Ad-Hoc Command Examples:

# 1. Test SSH connectivity across all webservers
ansible webservers -i hosts.ini -m ping

# 2. Gather hardware and OS facts from all hosts matching a pattern
ansible "app*:db*" -i hosts.ini -m setup -a "filter=ansible_memtotal_mb"

# 3. Check memory utilization using the command module
ansible all -i hosts.ini -m command -a "free -h"

# 4. Force restart nginx service using sudo privilege (-b / --become)
ansible webservers -i hosts.ini -b -m service -a "name=nginx state=restarted"

# 5. Copy an emergency file to all target nodes
ansible all -i hosts.ini -b -m copy -a "src=/tmp/banner.txt dest=/etc/motd mode=0644"

# 6. Verify user creation across all database nodes
ansible dbservers -i hosts.ini -b -m user -a "name=dbadmin state=present shell=/bin/bash"

Topic 6: command vs. shell vs. raw Modules (Avoiding Common Pitfalls)

Ansible provides three modules for running command-line strings. Choosing the wrong module is a common source of bugs:

# Making command/shell modules idempotent using 'creates' or 'removes'
- name: Extract archive only if destination directory does not exist
  ansible.builtin.command: tar -xzf /tmp/app.tar.gz -C /opt/app
  args:
    creates: /opt/app/bin/executable  # Skips task if this file already exists!

Common mistake: Debugging a slow playbook by adding forks, when the real cost is a new SSH handshake per task. Without ControlPersist and pipelining = True in ansible.cfg, a 40-task play against 50 hosts pays that handshake 2,000 times — and the fix is four lines of configuration rather than more parallelism.