Context
Ansible merges variables from several places before a task runs. A value written in a group file may be overridden by a child group, host variable, play variable, or extra variable. When an unexpected setting appears, editing the first matching file can leave the real winning source untouched.
Before starting, record the exact playbook command, inventory arguments, working directory, selected host, and Ansible version. Use a harmless nonsecret variable such as a service port for diagnosis. Inventory output can contain sensitive data, so avoid dumping all host variables into shared logs.
1. Reproduce inventory selection
ansible-inventory -i inventory/review.yml --graph
ansible-playbook -i inventory/review.yml site.yml --list-hosts
ansible --version
The paths are illustrative. Match the same inventory source order and playbook directory as the failing run. Configuration discovery and relative group_vars or host_vars locations can differ between a workstation and CI. A command using another inventory is not a reliable reproduction simply because it selects a similarly named host.
Read parser warnings. A source that failed to load can change both the host set and its variables. Dynamic inventory plugins can add another source of values, so include their configuration and cache policy in the evidence.
2. Follow the host's groups and overrides
Map the selected host through all parent and child groups. Read the relevant all, group, and host variable files. Ansible flattens inventory variables to the host, and inventory precedence follows group hierarchy and host-specific values. For groups at the same level, loading and merge order can matter.
Also inspect variables outside inventory. Role defaults, play variables, included variable files, task variables, and command-line extra variables have their own precedence. An inventory trace cannot explain a value that is later overridden by the playbook invocation.
3. Use a narrow diagnostic play
An explicitly illustrative play reports only a safe port value for one test host:
- name: Inspect a nonsecret configuration value
hosts: review_one
gather_facts: false
tasks:
- name: Show selected service port
ansible.builtin.debug:
msg: "service_port={{ service_port }}"
Run this against a controlled host group with the same relevant variable sources. Do not substitute a credential variable or debug the entire hostvars object. If the normal playbook overrides the value later, place an equivalent nonsecret check at the appropriate task boundary in the review branch.
Interpret the precedence evidence
| Observation | Possible winning source | Next inspection |
|---|---|---|
| Host differs from all peers in one group | Host override | host_vars and plugin host values |
| All child-group hosts differ | Child group configuration | Group hierarchy and group_vars |
| CI differs from local execution | Invocation or source order | Inventory flags, directory and extra vars |
| Value changes during the play | Play/task precedence | Included vars, role parameters and task scope |
For an explicitly illustrative failure, service_port is set in a parent group but overridden for one host in host_vars. Updating the parent repeatedly will not correct that host. The right decision is whether the exception remains valid, then editing its actual owner.
Verify and maintain a variable ownership SOP
Correct the intended source and rerun the narrow diagnostic, then inspect the relevant play's harmless rendered or read-only result. Confirm other groups retain their expected values. Avoid resolving every conflict with extra variables, because that can create an invocation-dependent configuration nobody sees in the repository.
Document each important variable's owner, default location, allowed exceptions, and inventory-source order. Keep representative nonsecret diagnostics in troubleshooting notes. Review overrides periodically, especially when hosts move between groups or CI changes how it invokes Ansible.
Explore Clouvenix ansible automation for implementation and operational handover.