From 92e6a3eab0f09847804f666fa50e9789860b4564 Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 25 Jan 2026 21:39:42 +0530 Subject: [PATCH 1/7] Updates made by ansible with play: Demo - Network Backup and Restore Workflow --- .../Differential_Backup_Documentation.md | 253 ++++++++++++++++++ roles/backup/README.md | 31 +++ roles/backup/meta/argument_specs.yml | 7 + .../2026-01-25_ios_54.190.208.146.txt | 151 +++++++++++ roles/backup/tasks/backup.yaml | 17 +- roles/backup/tasks/cli_backup.yaml | 1 + roles/backup/tasks/differential_scm.yaml | 107 ++++++++ roles/backup/tasks/main.yaml | 2 + roles/backup/tasks/network.yaml | 1 + roles/backup/tasks/publish.yaml | 10 +- roles/backup/tasks/retrieve.yaml | 43 ++- roles/backup/tasks/validation.yaml | 2 + .../tmp_backup/network_automation_tools | 1 + 13 files changed, 622 insertions(+), 4 deletions(-) create mode 100644 roles/backup/Differential_Backup_Documentation.md create mode 100644 roles/backup/network-backup-demo/tmp/network-backup-demo/2026-01-25_ios_54.190.208.146.txt create mode 100644 roles/backup/tasks/differential_scm.yaml create mode 160000 roles/restore/tmp_backup/network_automation_tools diff --git a/roles/backup/Differential_Backup_Documentation.md b/roles/backup/Differential_Backup_Documentation.md new file mode 100644 index 0000000..6a11c9d --- /dev/null +++ b/roles/backup/Differential_Backup_Documentation.md @@ -0,0 +1,253 @@ +# Differential Backup Documentation + +## Overview + +The differential backup feature in the `network.backup.backup` role allows you to create backups **only when actual configuration changes are detected**, ignoring metadata and timestamp differences. This reduces storage overhead and SCM noise by preventing unnecessary commits and pull requests when only timestamps or metadata have changed. + +## How It Works + +### 1. Backup Type Selection + +The differential backup is controlled by the `type` parameter: + +```yaml +- name: Create Network Backup + ansible.builtin.include_role: + name: network.backup.backup + vars: + type: "diff" # Options: "full", "incremental", or "diff" + data_store: + scm: + origin: + # ... SCM configuration +``` + +- **`type: "diff"`**: Enables differential backup - only publishes if config actually changed +- **`type: "full"`** (default): Always creates and publishes backup, regardless of changes + +### 2. Backup Process Flow + +When `type: "diff"` is set, the following process occurs: + +``` +1. Retrieve previous backup from SCM (if exists) + ↓ +2. Create current backup from device + ↓ +3. Normalize both backups (remove timestamps/metadata) + ↓ +4. Compare normalized content + ↓ +5. If different → backup_has_changes = true → Publish + If identical → backup_has_changes = false → Skip publish +``` + +## Normalization Process + +The normalization process removes metadata and timestamp lines that don't represent actual configuration changes. This ensures that only meaningful configuration differences trigger a new backup. + +### What Gets Ignored (Removed During Normalization) + +The following lines are **removed** from backup files before comparison: + +#### Cisco IOS/IOS-XE +- `!Command: show running-config` +- `!Running configuration last done at: ` +- `!Time: ` +- `!NVRAM config last updated at: ` +- `!No configuration change since last restart` + +#### Cisco NX-OS +- `!Command: show running-config` +- `!Time: ` +- `!Running configuration last done at: ` +- `!NVRAM config last updated at: ` + +#### Cisco IOS-XR +- `Building configuration...` +- `!! IOS XR Configuration ` +- `!! Last configuration change at by ` + +#### Arista EOS +- `! Command: show running-config` +- `! device: ` +- `! boot system flash:/` + +#### General Metadata +- Empty lines (whitespace-only lines) +- Comment lines that contain only timestamps or metadata + +### What Gets Compared (Actual Configuration) + +After normalization, the following configuration elements are compared: + +- **Interface configurations** +- **Routing protocols** +- **Access control lists (ACLs)** +- **VLANs** +- **User accounts** +- **SNMP configuration** +- **System settings** (hostname, domain, etc.) +- **Any actual configuration commands** + +## Examples + +### Example 1: No Configuration Change (Backup Skipped) + +**Previous Backup:** +``` +!Time: Mon Dec 12 18:19:15 2025 +hostname router1 +interface GigabitEthernet0/0 + ip address 192.168.1.1 255.255.255.0 +``` + +**Current Backup:** +``` +!Time: Mon Dec 12 18:30:45 2025 +hostname router1 +interface GigabitEthernet0/0 + ip address 192.168.1.1 255.255.255.0 +``` + +**Normalized Comparison:** +``` +Previous: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.1 255.255.255.0 +Current: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.1 255.255.255.0 +``` + +**Result:** ✅ **Identical** → `backup_has_changes = false` → **Publish skipped** + +### Example 2: Configuration Change Detected (Backup Published) + +**Previous Backup:** +``` +!Time: Mon Dec 12 18:19:15 2025 +hostname router1 +interface GigabitEthernet0/0 + ip address 192.168.1.1 255.255.255.0 +``` + +**Current Backup:** +``` +!Time: Mon Dec 12 18:30:45 2025 +hostname router1 +interface GigabitEthernet0/0 + ip address 192.168.1.2 255.255.255.0 +``` + +**Normalized Comparison:** +``` +Previous: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.1 255.255.255.0 +Current: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.2 255.255.255.0 +``` + +**Result:** ❌ **Different** → `backup_has_changes = true` → **Publish triggered** + +### Example 3: First Backup (Always Published) + +**Previous Backup:** Does not exist + +**Result:** ✅ **No previous backup** → `backup_has_changes = true` → **Publish triggered** (first backup always created) + +## Implementation Details + +### Normalization Command + +The normalization is performed using `sed` and `grep` commands: + +```bash +sed -E \ + -e '/^!Command:/d' \ + -e '/^!Running configuration last done at:/d' \ + -e '/^!Time:/d' \ + -e '/^!NVRAM config last updated at:/d' \ + -e '/^!No configuration change since last restart/d' \ + "backup_file.txt" | grep -v '^[[:space:]]*$' +``` + +This command: +1. Removes timestamp/metadata lines using `sed` +2. Removes empty lines using `grep` +3. Preserves all actual configuration content + +### Comparison Logic + +The comparison is performed in the `differential_scm.yaml` task file: + +```yaml +- name: Compare normalized backups + ansible.builtin.set_fact: + backup_has_changes: "{{ normalized_previous != normalized_current }}" +``` + +- If `normalized_previous == normalized_current`: No changes detected +- If `normalized_previous != normalized_current`: Changes detected + +### Publish Decision + +The publish task only runs when `backup_has_changes` is `true`: + +```yaml +- name: Include build tasks + ansible.builtin.include_tasks: publish.yaml + when: + - data_store.scm.origin is defined + - backup_has_changes | default(true) + run_once: true +``` + +## Benefits + +1. **Reduced SCM Noise**: No unnecessary commits or pull requests for timestamp-only changes +2. **Storage Efficiency**: Only stores backups when configuration actually changes +3. **Idempotency**: Running the playbook multiple times with the same config won't create new PRs +4. **Clear Change History**: Git history only shows actual configuration changes + +## Limitations + +1. **Platform-Specific Metadata**: The normalization patterns are designed for common Cisco and Arista metadata. Custom metadata formats may not be filtered. +2. **Whitespace Changes**: Significant whitespace changes in configuration (beyond empty lines) will be detected as changes. +3. **Order-Dependent**: If device output order changes but configuration is identical, it may be detected as a change (rare). + +## Troubleshooting + +### Backup Always Shows Changes + +If backups always show changes even when configuration hasn't changed: + +1. **Check normalization patterns**: Verify that all timestamp/metadata lines are being removed +2. **Inspect normalized content**: Use debug tasks to see what's being compared: + ```yaml + - name: Debug normalized content + ansible.builtin.debug: + msg: "Previous: {{ normalized_previous }}\nCurrent: {{ normalized_current }}" + ``` + +### Backup Never Shows Changes + +If backups never show changes even when configuration changed: + +1. **Verify backup type**: Ensure `type: "diff"` is set +2. **Check file paths**: Verify previous backup file exists and is being read correctly +3. **Check normalization**: Ensure normalization isn't removing actual configuration lines + +## Best Practices + +1. **Use `type: "diff"` for automated backups**: Prevents unnecessary commits in scheduled backup jobs +2. **Use `type: "full"` for manual backups**: Ensures backup is always created when manually triggered +3. **Monitor backup_has_changes**: Use the debug message to understand when backups are skipped +4. **Review normalized content**: Periodically verify that normalization is working correctly for your device types + +## Related Files + +- `/roles/backup/tasks/differential_scm.yaml`: Differential backup logic +- `/roles/backup/tasks/backup.yaml`: Main backup orchestration +- `/roles/backup/tasks/cli_backup.yaml`: Backup file creation +- `/roles/backup/tasks/publish.yaml`: SCM publishing logic + +## See Also + +- [Network Backup Role README](README.md) +- [Ansible Network Backup Collection Documentation](https://docs.ansible.com/ansible/latest/collections/network/backup/) + diff --git a/roles/backup/README.md b/roles/backup/README.md index 24fb5b7..e17eea7 100644 --- a/roles/backup/README.md +++ b/roles/backup/README.md @@ -17,6 +17,8 @@ This role supports full and differential backups, storing them locally or in a r - Backs up the configuration **only if** there are changes compared to the last saved version. - Works with both local and Git-based data stores. - Helps reduce storage and SCM noise by saving only when diff exists. +- **Ignores timestamps and metadata** - only detects actual configuration changes. +- See [Differential Backup Documentation](Differential_Backup_Documentation.md) for detailed explanation of how it works. --- @@ -34,8 +36,11 @@ This role supports full and differential backups, storing them locally or in a r | `data_store.scm.origin.path` | Directory path inside the repo to save backup | `str` | No | N/A | | `data_store.scm.origin.ssh_key_file` | Path to the SSH private key file for Git authentication | `str` | Yes (if using SCM SSH) | N/A | | `data_store.scm.origin.ssh_key_content` | The content of the SSH private key | `str` | Yes (if using SCM SSH) | N/A | +| `type` | Type of backup to perform. Options: `"full"`, `"incremental"`, or `"diff"` | `str` | No | `"full"` | > Note: Either `data_store.local` or `data_store.scm` must be provided. +> +> **Differential Backup (`type: "diff"`)**: When enabled, the role compares the current backup with the previous backup after normalizing (removing timestamps/metadata). Only publishes if actual configuration changes are detected. See [Differential Backup Documentation](Differential_Backup_Documentation.md) for details. --- @@ -136,6 +141,32 @@ This role supports full and differential backups, storing them locally or in a r path: "backups/{{ ansible_date_time.date }}/{{ inventory_hostname }}" ``` +### Create Differential Backup (Only Publish if Config Changed) + +```yaml +- name: Create Network Backup and Push to GitHub + hosts: network + gather_facts: false + tasks: + - name: Create Network Backup + ansible.builtin.include_role: + name: network.backup.backup + vars: + type: "diff" # Enable differential backup + data_store: + scm: + origin: + user: + name: "your_name" + email: "your_email@example.com" + url: "git@github.com:youruser/your-backup-repo.git" + ssh_key_file: "/path/to/ssh/key" + filename: "{{ ansible_date_time.date }}_{{ inventory_hostname }}.txt" + path: "backups/{{ ansible_date_time.date }}/{{ inventory_hostname }}" +``` + +> **Note**: With `type: "diff"`, the backup will only be published to SCM if actual configuration changes are detected. Timestamps and metadata differences are ignored. See [Differential Backup Documentation](Differential_Backup_Documentation.md) for more details. + ## License GNU General Public License v3.0 or later. diff --git a/roles/backup/meta/argument_specs.yml b/roles/backup/meta/argument_specs.yml index 04cee10..0b5d0d3 100644 --- a/roles/backup/meta/argument_specs.yml +++ b/roles/backup/meta/argument_specs.yml @@ -45,6 +45,13 @@ argument_specs: type: dict required: false options: + parent_directory: + type: str + required: false + description: + - Parent directory where the Git repository will be cloned (e.g., /tmp or role_path). + - If not specified, defaults to role_path. + - Best practice: Use temp directory (e.g., /tmp) for isolated operations. origin: type: dict required: true diff --git a/roles/backup/network-backup-demo/tmp/network-backup-demo/2026-01-25_ios_54.190.208.146.txt b/roles/backup/network-backup-demo/tmp/network-backup-demo/2026-01-25_ios_54.190.208.146.txt new file mode 100644 index 0000000..b19eda8 --- /dev/null +++ b/roles/backup/network-backup-demo/tmp/network-backup-demo/2026-01-25_ios_54.190.208.146.txt @@ -0,0 +1,151 @@ +Building configuration... + +Current configuration : 3474 bytes +! +! Last configuration change at 08:54:22 UTC Wed Jan 7 2026 by cisco +! +version 17.13 +service timestamps debug datetime msec +service timestamps log datetime msec +platform qfp utilization monitor load 80 +platform punt-keepalive disable-kernel-core +platform sslvpn use-pd +platform console serial +! +hostname ciscoios +! +boot-start-marker +boot-end-marker +! +! +no aaa new-model +! +! +! +! +! +! +! +! +! +! +! +! +ip domain name ansible.com +! +! +! +login on-success log +! +! +subscriber templating +! +pae +! +! +crypto pki trustpoint SLA-TrustPoint + enrollment pkcs12 + revocation-check crl + hash sha256 +! +! +crypto pki certificate chain SLA-TrustPoint + certificate ca 01 + 30820321 30820209 A0030201 02020101 300D0609 2A864886 F70D0101 0B050030 + 32310E30 0C060355 040A1305 43697363 6F312030 1E060355 04031317 43697363 + 6F204C69 63656E73 696E6720 526F6F74 20434130 1E170D31 33303533 30313934 + 3834375A 170D3338 30353330 31393438 34375A30 32310E30 0C060355 040A1305 + 43697363 6F312030 1E060355 04031317 43697363 6F204C69 63656E73 696E6720 + 526F6F74 20434130 82012230 0D06092A 864886F7 0D010101 05000382 010F0030 + 82010A02 82010100 A6BCBD96 131E05F7 145EA72C 2CD686E6 17222EA1 F1EFF64D + CBB4C798 212AA147 C655D8D7 9471380D 8711441E 1AAF071A 9CAE6388 8A38E520 + 1C394D78 462EF239 C659F715 B98C0A59 5BBB5CBD 0CFEBEA3 700A8BF7 D8F256EE + 4AA4E80D DB6FD1C9 60B1FD18 FFC69C96 6FA68957 A2617DE7 104FDC5F EA2956AC + 7390A3EB 2B5436AD C847A2C5 DAB553EB 69A9A535 58E9F3E3 C0BD23CF 58BD7188 + 68E69491 20F320E7 948E71D7 AE3BCC84 F10684C7 4BC8E00F 539BA42B 42C68BB7 + C7479096 B4CB2D62 EA2F505D C7B062A4 6811D95B E8250FC4 5D5D5FB8 8F27D191 + C55F0D76 61F9A4CD 3D992327 A8BB03BD 4E6D7069 7CBADF8B DF5F4368 95135E44 + DFC7C6CF 04DD7FD1 02030100 01A34230 40300E06 03551D0F 0101FF04 04030201 + 06300F06 03551D13 0101FF04 05300301 01FF301D 0603551D 0E041604 1449DC85 + 4B3D31E5 1B3E6A17 606AF333 3D3B4C73 E8300D06 092A8648 86F70D01 010B0500 + 03820101 00507F24 D3932A66 86025D9F E838AE5C 6D4DF6B0 49631C78 240DA905 + 604EDCDE FF4FED2B 77FC460E CD636FDB DD44681E 3A5673AB 9093D3B1 6C9E3D8B + D98987BF E40CBD9E 1AECA0C2 2189BB5C 8FA85686 CD98B646 5575B146 8DFC66A8 + 467A3DF4 4D565700 6ADF0F0D CF835015 3C04FF7C 21E878AC 11BA9CD2 55A9232C + 7CA7B7E6 C1AF74F6 152E99B7 B1FCF9BB E973DE7F 5BDDEB86 C71E3B49 1765308B + 5FB0DA06 B92AFE7F 494E8A9E 07B85737 F3A58BE1 1A48A229 C37C1E69 39F08678 + 80DDCD16 D6BACECA EEBC7CF9 8428787B 35202CDC 60E4616A B623CDBD 230E3AFB + 418616A9 4093E049 4D10AB75 27E86F73 932E35B5 8862FDAE 0275156F 719BB2F0 + D697DF7F 28 + quit +! +! +license udi pid C8000V sn 92PP4OSI7L2 +memory free low-watermark processor 201711 +diagnostic bootup level minimal +! +! +! +username ansible privilege 15 secret 9 $9$Pu1UNg4muGCHcE$36tqY13fHOI4AlHHcGi2P/QKuaNEBQDF8TLsguhvdog +username cisco privilege 15 secret 9 $9$v77TR2bkbQgU6k$XacI6rYoxjQcfxG.O1vkzBTSK0q.idrf7lzSHZYSw3M +! +redundancy +! +! +! +! +! +! +! +! +interface Loopback998 + description ios_device is the best + no ip address + shutdown +! +interface GigabitEthernet1 + ip address dhcp + negotiation auto +! +interface GigabitEthernet2 + no ip address + shutdown + negotiation auto +! +interface GigabitEthernet3 + no ip address + shutdown + negotiation auto +! +interface GigabitEthernet4 + no ip address + shutdown + negotiation auto +! +ip forward-protocol nd +! +no ip http server +ip http secure-server +ip ssh bulk-mode 131072 +! +! +! +! +! +control-plane +! +! +line con 0 + stopbits 1 +line aux 0 +line vty 0 4 + login local + transport input ssh +! +! +! +! +! +! +! +end \ No newline at end of file diff --git a/roles/backup/tasks/backup.yaml b/roles/backup/tasks/backup.yaml index 57cc04a..50db5da 100644 --- a/roles/backup/tasks/backup.yaml +++ b/roles/backup/tasks/backup.yaml @@ -2,36 +2,51 @@ - name: Build Local Backup Dir Path ansible.builtin.include_tasks: path.yaml when: data_store.scm.origin is not defined + tags: always - name: Include retrieve tasks ansible.builtin.include_tasks: retrieve.yaml when: data_store['scm']['origin'] is defined run_once: true + tags: always - name: Get scm url ansible.builtin.set_fact: network_backup_path_root: "{{ role_path }}/{{ data_store.scm.origin.url.split('/')[-1] | regex_replace('\\.git$', '') }}" when: data_store['scm']['origin'] is defined + tags: always - name: Get file name ansible.builtin.set_fact: network_backup_path: "{{ network_backup_path_root }}/{{ data_store.scm.origin.path | default(role_path, true) }}" when: data_store['scm']['origin'] is defined + tags: always - name: Get timestamp ansible.builtin.set_fact: timestamp: "{{ lookup('pipe', 'date +%Y-%m-%d_%H-%M-%S') }}" + tags: always - name: Set default filename ansible.builtin.set_fact: network_backup_filename: >- {{ data_store.scm.origin.filename | default(inventory_hostname ~ '_' ~ timestamp ~ '.txt', true) }} when: network_backup_filename is undefined + tags: always - name: Include tasks ansible.builtin.include_tasks: network.yaml + tags: always + +- name: Check for differential backup (SCM only) + ansible.builtin.include_tasks: differential_scm.yaml + when: data_store['scm']['origin'] is defined + tags: always - name: Include build tasks ansible.builtin.include_tasks: publish.yaml - when: data_store.scm.origin is defined + when: + - data_store.scm.origin is defined + - backup_has_changes | default(true) run_once: true + tags: always diff --git a/roles/backup/tasks/cli_backup.yaml b/roles/backup/tasks/cli_backup.yaml index 6f45e47..07651f3 100644 --- a/roles/backup/tasks/cli_backup.yaml +++ b/roles/backup/tasks/cli_backup.yaml @@ -4,3 +4,4 @@ dir_path: "{{ network_backup_path }}" filename: "{{ network_backup_filename | default(inventory_hostname) }}" register: network_backup + tags: always diff --git a/roles/backup/tasks/differential_scm.yaml b/roles/backup/tasks/differential_scm.yaml new file mode 100644 index 0000000..4d0cc6e --- /dev/null +++ b/roles/backup/tasks/differential_scm.yaml @@ -0,0 +1,107 @@ +--- +# Differential backup logic for SCM (Git) data stores +# Compares current backup with previous backup, normalizing timestamps/metadata +# Only publishes if there are actual configuration changes + +- name: Set default backup type to full if not specified + ansible.builtin.set_fact: + backup_type: "{{ type | default('full') }}" + tags: always + +- name: Check if previous backup file exists + ansible.builtin.stat: + path: "{{ network_backup_path }}/{{ network_backup_filename }}" + register: previous_backup_stat + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + tags: always + +- name: Normalize previous backup file (remove timestamps and metadata) + ansible.builtin.shell: | + sed -E \ + -e '/^!Command:/d' \ + -e '/^!Running configuration last done at:/d' \ + -e '/^!Time:/d' \ + -e '/^!NVRAM config last updated at:/d' \ + -e '/^!No configuration change since last restart/d' \ + "{{ network_backup_path }}/{{ network_backup_filename }}" | grep -v '^[[:space:]]*$' + register: normalized_previous_result + changed_when: false + failed_when: false + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - previous_backup_stat.stat.exists | default(false) + tags: always + +- name: Normalize current backup file (remove timestamps and metadata) + ansible.builtin.shell: | + sed -E \ + -e '/^!Command:/d' \ + -e '/^!Running configuration last done at:/d' \ + -e '/^!Time:/d' \ + -e '/^!NVRAM config last updated at:/d' \ + -e '/^!No configuration change since last restart/d' \ + "{{ network_backup_path }}/{{ network_backup_filename }}" | grep -v '^[[:space:]]*$' + register: normalized_current_result + changed_when: false + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + tags: always + +- name: Set normalized content facts (when previous backup exists) + ansible.builtin.set_fact: + normalized_previous: "{{ normalized_previous_result.stdout | default('') }}" + normalized_current: "{{ normalized_current_result.stdout | default('') }}" + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - previous_backup_stat.stat.exists | default(false) + tags: always + +- name: Set normalized content facts (when no previous backup exists) + ansible.builtin.set_fact: + normalized_current: "{{ normalized_current_result.stdout | default('') }}" + normalized_previous: "" + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - not (previous_backup_stat.stat.exists | default(false)) + tags: always + +- name: Compare normalized backups + ansible.builtin.set_fact: + backup_has_changes: "{{ normalized_previous != normalized_current }}" + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - previous_backup_stat.stat.exists | default(false) + tags: always + +- name: Set backup_has_changes to true if no previous backup exists + ansible.builtin.set_fact: + backup_has_changes: true + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - not (previous_backup_stat.stat.exists | default(false)) + tags: always + +- name: Set backup_has_changes to true for full backup type + ansible.builtin.set_fact: + backup_has_changes: true + when: + - data_store['scm']['origin'] is defined + - backup_type != "diff" + tags: always + +- name: Display differential backup result + ansible.builtin.debug: + msg: "Differential backup: {{ 'Changes detected - will publish' if backup_has_changes else 'No changes detected - skipping publish' }}" + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + tags: always + diff --git a/roles/backup/tasks/main.yaml b/roles/backup/tasks/main.yaml index c2514ed..aef7987 100644 --- a/roles/backup/tasks/main.yaml +++ b/roles/backup/tasks/main.yaml @@ -1,6 +1,8 @@ --- - name: Include tasks ansible.builtin.include_tasks: validation.yaml + tags: always - name: Run the platform specific tasks ansible.builtin.include_tasks: "backup.yaml" + tags: always diff --git a/roles/backup/tasks/network.yaml b/roles/backup/tasks/network.yaml index 7464287..08434db 100644 --- a/roles/backup/tasks/network.yaml +++ b/roles/backup/tasks/network.yaml @@ -1,3 +1,4 @@ --- - name: Invoke backup task ansible.builtin.include_tasks: cli_backup.yaml + tags: always diff --git a/roles/backup/tasks/publish.yaml b/roles/backup/tasks/publish.yaml index 6fbc755..e2968d9 100644 --- a/roles/backup/tasks/publish.yaml +++ b/roles/backup/tasks/publish.yaml @@ -3,12 +3,14 @@ ansible.builtin.set_fact: time: "{{ lookup('pipe', 'date \"+%Y-%m-%d-%H-%M\"') }}" run_once: true + tags: always - name: Create default tag ansible.builtin.set_fact: default_tag: annotation: "{{ time }}" message: "backup_on: {{ time }}" + tags: always - name: Set default tag ansible.builtin.set_fact: @@ -16,10 +18,11 @@ when: - tag is defined - tag == "default" + tags: always - name: Publish the changes with tag ansible.scm.git_publish: - path: "{{ network_backup_path }}" + path: "{{ network_backup_path_root }}" token: "{{ data_store.scm.origin.get('token') if data_store.scm.origin.get('token') else omit }}" user: "{{ data_store['scm']['origin']['user'] | d({}) }}" tag: "{{ tag }}" @@ -27,18 +30,21 @@ ssh_key_file: "{{ data_store.scm.origin.get('ssh_key_file') if data_store.scm.origin.get('ssh_key_file') else omit }}" ssh_key_content: "{{ data_store.scm.origin.get('ssh_key_content') if data_store.scm.origin.get('ssh_key_content') else omit }}" when: tag is defined + tags: always - name: Publish the changes ansible.scm.git_publish: - path: "{{ network_backup_path }}" + path: "{{ network_backup_path_root }}" token: "{{ data_store.scm.origin.get('token') if data_store.scm.origin.get('token') else omit }}" user: "{{ data_store['scm']['origin']['user'] | d({}) }}" timeout: 120 ssh_key_file: "{{ data_store.scm.origin.get('ssh_key_file') if data_store.scm.origin.get('ssh_key_file') else omit }}" ssh_key_content: "{{ data_store.scm.origin.get('ssh_key_content') if data_store.scm.origin.get('ssh_key_content') else omit }}" when: tag is not defined + tags: always - name: Remove cloned repository directory ansible.builtin.file: path: "{{ network_backup_path_root }}" state: absent + tags: always diff --git a/roles/backup/tasks/retrieve.yaml b/roles/backup/tasks/retrieve.yaml index 4c02e0c..6cde9b8 100644 --- a/roles/backup/tasks/retrieve.yaml +++ b/roles/backup/tasks/retrieve.yaml @@ -2,9 +2,11 @@ - name: Set Default Path ansible.builtin.set_fact: default_path: "{{ role_path }}" + tags: always - name: Perform retrieve with provided credentials when: data_store.scm.origin.token is defined + tags: always block: - name: Retrieve the host vars with provided token access ansible.scm.git_retrieve: @@ -13,14 +15,37 @@ token: "{{ data_store['scm']['origin']['token'] }}" parent_directory: "{{ data_store.scm.parent_directory | default(default_path) }}" register: resource_manager_result + tags: always - name: Update data store path ansible.builtin.set_fact: network_backup_repository: "{{ resource_manager_result }}" + tags: always - name: Perform retrieve with default settings or SSH key when: data_store.scm.origin.token is undefined + tags: always block: + - name: Set repository path + ansible.builtin.set_fact: + repo_local_path: "{{ data_store.scm.parent_directory | default(default_path) }}/{{ data_store['scm']['origin']['url'].split('/')[-1] | regex_replace('\\.git$', '') }}" + tags: always + + - name: Check if repository directory already exists + ansible.builtin.stat: + path: "{{ repo_local_path }}" + register: repo_dir_stat + tags: always + + - name: Pull latest changes if repository exists + ansible.builtin.command: + cmd: git -C {{ repo_local_path }} pull origin main + register: git_pull_result + changed_when: "'Already up to date' not in git_pull_result.stdout" + failed_when: false + when: repo_dir_stat.stat.exists | default(false) + tags: always + - name: Retrieve host vars with default access or SSH key ansible.scm.git_retrieve: origin: @@ -29,11 +54,27 @@ ssh_key_content: "{{ data_store['scm']['origin'].ssh_key_content | default(omit) }}" parent_directory: "{{ data_store.scm.parent_directory | default(default_path) }}" register: resource_manager_result + failed_when: false + when: not (repo_dir_stat.stat.exists | default(false)) + tags: always - - name: Update data store path + - name: Set repository path when directory already exists + ansible.builtin.set_fact: + network_backup_repository: + path: "{{ repo_local_path }}" + when: repo_dir_stat.stat.exists | default(false) + tags: always + + - name: Update data store path from git_retrieve result ansible.builtin.set_fact: network_backup_repository: "{{ resource_manager_result }}" + when: + - resource_manager_result is defined + - resource_manager_result.path is defined + - not (repo_dir_stat.stat.exists | default(false)) + tags: always - name: Update Inventory Path ansible.builtin.set_fact: network_backup_path: "{{ network_backup_repository['path'] }}" + tags: always diff --git a/roles/backup/tasks/validation.yaml b/roles/backup/tasks/validation.yaml index 4598f4d..b05a844 100644 --- a/roles/backup/tasks/validation.yaml +++ b/roles/backup/tasks/validation.yaml @@ -8,7 +8,9 @@ - "nxos" - "iosxr" - "vyos" + tags: always - name: Conditional test ansible.builtin.include_tasks: "unsupported_platform.yaml" when: ansible_network_os.split('.')[-1] not in supported_platforms + tags: always diff --git a/roles/restore/tmp_backup/network_automation_tools b/roles/restore/tmp_backup/network_automation_tools new file mode 160000 index 0000000..cce4a3d --- /dev/null +++ b/roles/restore/tmp_backup/network_automation_tools @@ -0,0 +1 @@ +Subproject commit cce4a3d4ff5f9fbc801aa8d747e2713f90e04cde From 784c7824ea9210e1c4e8cc936de8678c4296801f Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 25 Jan 2026 21:54:44 +0530 Subject: [PATCH 2/7] Updates made by ansible with play: Demo - Network Backup and Restore Workflow From 7ef9f8c521f44060d7999ed1a0a8508d1786f414 Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 25 Jan 2026 21:57:06 +0530 Subject: [PATCH 3/7] Updates made by ansible with play: Demo - Network Backup and Restore Workflow From 7549667106bba6b8e4141935ec039f45a529234d Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 25 Jan 2026 21:58:21 +0530 Subject: [PATCH 4/7] Updates made by ansible with play: Demo - Network Backup and Restore Workflow From 439fb822242826caea6d106b8227a2e8d80074b2 Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 25 Jan 2026 22:14:36 +0530 Subject: [PATCH 5/7] Updates made by ansible with play: Demo - Network Backup and Restore Workflow --- logs.log | 12 ++++++++++++ .../2026-01-25_ios_54.190.208.146.txt | 0 2 files changed, 12 insertions(+) create mode 100644 logs.log rename roles/backup/network-backup-demo/{tmp => backup}/network-backup-demo/2026-01-25_ios_54.190.208.146.txt (100%) diff --git a/logs.log b/logs.log new file mode 100644 index 0000000..a17a801 --- /dev/null +++ b/logs.log @@ -0,0 +1,12 @@ +2026-01-25 22:12:41,674 p=57299 u=rohit n=ansible INFO| ansible-playbook [core 2.19.4] + config file = None + configured module search path = ['/Users/rohit/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] + ansible python module location = /Users/rohit/venvs/ansible312/lib/python3.12/site-packages/ansible + ansible collection location = /Users/rohit/.ansible/collections:/usr/share/ansible/collections + executable location = /Users/rohit/venvs/ansible312/bin/ansible-playbook + python version = 3.12.12 (main, Oct 9 2025, 11:07:00) [Clang 17.0.0 (clang-1700.0.13.3)] (/Users/rohit/venvs/ansible312/bin/python3.12) + jinja version = 3.1.6 + pyyaml version = 6.0.3 (with libyaml v0.2.5) +2026-01-25 22:12:41,674 p=57299 u=rohit n=ansible INFO| No config file found; using defaults +2026-01-25 22:12:41,674 p=57299 u=rohit n=ansible ERROR| [ERROR]: the playbook: demo_backup_restore.yml could not be found + diff --git a/roles/backup/network-backup-demo/tmp/network-backup-demo/2026-01-25_ios_54.190.208.146.txt b/roles/backup/network-backup-demo/backup/network-backup-demo/2026-01-25_ios_54.190.208.146.txt similarity index 100% rename from roles/backup/network-backup-demo/tmp/network-backup-demo/2026-01-25_ios_54.190.208.146.txt rename to roles/backup/network-backup-demo/backup/network-backup-demo/2026-01-25_ios_54.190.208.146.txt From e9e41da4c38b214bd3533ed79069da3b6f8f81f6 Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 1 Feb 2026 05:58:32 +0100 Subject: [PATCH 6/7] update tasks Signed-off-by: rohitthakur2590 --- .../2026-01-25_ios_54.190.208.146.txt | 151 ------------------ roles/backup/tasks/backup.yaml | 22 ++- roles/backup/tasks/differential_scm.yaml | 40 ++--- .../tasks/differential_scm_read_previous.yaml | 55 +++++++ roles/backup/tasks/retrieve.yaml | 2 + roles/restore/tasks/cli_restore.yaml | 15 +- roles/restore/tasks/cli_restore_config.yaml | 2 + roles/restore/tasks/common/validation.yaml | 2 + roles/restore/tasks/main.yaml | 2 + roles/restore/tasks/network.yaml | 3 + roles/restore/tasks/prepare/ios.yaml | 4 + roles/restore/tasks/restore.yaml | 9 +- roles/restore/tasks/retrieve.yaml | 6 +- 13 files changed, 133 insertions(+), 180 deletions(-) delete mode 100644 roles/backup/network-backup-demo/backup/network-backup-demo/2026-01-25_ios_54.190.208.146.txt create mode 100644 roles/backup/tasks/differential_scm_read_previous.yaml diff --git a/roles/backup/network-backup-demo/backup/network-backup-demo/2026-01-25_ios_54.190.208.146.txt b/roles/backup/network-backup-demo/backup/network-backup-demo/2026-01-25_ios_54.190.208.146.txt deleted file mode 100644 index b19eda8..0000000 --- a/roles/backup/network-backup-demo/backup/network-backup-demo/2026-01-25_ios_54.190.208.146.txt +++ /dev/null @@ -1,151 +0,0 @@ -Building configuration... - -Current configuration : 3474 bytes -! -! Last configuration change at 08:54:22 UTC Wed Jan 7 2026 by cisco -! -version 17.13 -service timestamps debug datetime msec -service timestamps log datetime msec -platform qfp utilization monitor load 80 -platform punt-keepalive disable-kernel-core -platform sslvpn use-pd -platform console serial -! -hostname ciscoios -! -boot-start-marker -boot-end-marker -! -! -no aaa new-model -! -! -! -! -! -! -! -! -! -! -! -! -ip domain name ansible.com -! -! -! -login on-success log -! -! -subscriber templating -! -pae -! -! -crypto pki trustpoint SLA-TrustPoint - enrollment pkcs12 - revocation-check crl - hash sha256 -! -! -crypto pki certificate chain SLA-TrustPoint - certificate ca 01 - 30820321 30820209 A0030201 02020101 300D0609 2A864886 F70D0101 0B050030 - 32310E30 0C060355 040A1305 43697363 6F312030 1E060355 04031317 43697363 - 6F204C69 63656E73 696E6720 526F6F74 20434130 1E170D31 33303533 30313934 - 3834375A 170D3338 30353330 31393438 34375A30 32310E30 0C060355 040A1305 - 43697363 6F312030 1E060355 04031317 43697363 6F204C69 63656E73 696E6720 - 526F6F74 20434130 82012230 0D06092A 864886F7 0D010101 05000382 010F0030 - 82010A02 82010100 A6BCBD96 131E05F7 145EA72C 2CD686E6 17222EA1 F1EFF64D - CBB4C798 212AA147 C655D8D7 9471380D 8711441E 1AAF071A 9CAE6388 8A38E520 - 1C394D78 462EF239 C659F715 B98C0A59 5BBB5CBD 0CFEBEA3 700A8BF7 D8F256EE - 4AA4E80D DB6FD1C9 60B1FD18 FFC69C96 6FA68957 A2617DE7 104FDC5F EA2956AC - 7390A3EB 2B5436AD C847A2C5 DAB553EB 69A9A535 58E9F3E3 C0BD23CF 58BD7188 - 68E69491 20F320E7 948E71D7 AE3BCC84 F10684C7 4BC8E00F 539BA42B 42C68BB7 - C7479096 B4CB2D62 EA2F505D C7B062A4 6811D95B E8250FC4 5D5D5FB8 8F27D191 - C55F0D76 61F9A4CD 3D992327 A8BB03BD 4E6D7069 7CBADF8B DF5F4368 95135E44 - DFC7C6CF 04DD7FD1 02030100 01A34230 40300E06 03551D0F 0101FF04 04030201 - 06300F06 03551D13 0101FF04 05300301 01FF301D 0603551D 0E041604 1449DC85 - 4B3D31E5 1B3E6A17 606AF333 3D3B4C73 E8300D06 092A8648 86F70D01 010B0500 - 03820101 00507F24 D3932A66 86025D9F E838AE5C 6D4DF6B0 49631C78 240DA905 - 604EDCDE FF4FED2B 77FC460E CD636FDB DD44681E 3A5673AB 9093D3B1 6C9E3D8B - D98987BF E40CBD9E 1AECA0C2 2189BB5C 8FA85686 CD98B646 5575B146 8DFC66A8 - 467A3DF4 4D565700 6ADF0F0D CF835015 3C04FF7C 21E878AC 11BA9CD2 55A9232C - 7CA7B7E6 C1AF74F6 152E99B7 B1FCF9BB E973DE7F 5BDDEB86 C71E3B49 1765308B - 5FB0DA06 B92AFE7F 494E8A9E 07B85737 F3A58BE1 1A48A229 C37C1E69 39F08678 - 80DDCD16 D6BACECA EEBC7CF9 8428787B 35202CDC 60E4616A B623CDBD 230E3AFB - 418616A9 4093E049 4D10AB75 27E86F73 932E35B5 8862FDAE 0275156F 719BB2F0 - D697DF7F 28 - quit -! -! -license udi pid C8000V sn 92PP4OSI7L2 -memory free low-watermark processor 201711 -diagnostic bootup level minimal -! -! -! -username ansible privilege 15 secret 9 $9$Pu1UNg4muGCHcE$36tqY13fHOI4AlHHcGi2P/QKuaNEBQDF8TLsguhvdog -username cisco privilege 15 secret 9 $9$v77TR2bkbQgU6k$XacI6rYoxjQcfxG.O1vkzBTSK0q.idrf7lzSHZYSw3M -! -redundancy -! -! -! -! -! -! -! -! -interface Loopback998 - description ios_device is the best - no ip address - shutdown -! -interface GigabitEthernet1 - ip address dhcp - negotiation auto -! -interface GigabitEthernet2 - no ip address - shutdown - negotiation auto -! -interface GigabitEthernet3 - no ip address - shutdown - negotiation auto -! -interface GigabitEthernet4 - no ip address - shutdown - negotiation auto -! -ip forward-protocol nd -! -no ip http server -ip http secure-server -ip ssh bulk-mode 131072 -! -! -! -! -! -control-plane -! -! -line con 0 - stopbits 1 -line aux 0 -line vty 0 4 - login local - transport input ssh -! -! -! -! -! -! -! -end \ No newline at end of file diff --git a/roles/backup/tasks/backup.yaml b/roles/backup/tasks/backup.yaml index 50db5da..875fb3d 100644 --- a/roles/backup/tasks/backup.yaml +++ b/roles/backup/tasks/backup.yaml @@ -10,9 +10,9 @@ run_once: true tags: always -- name: Get scm url +- name: Get scm url - Use actual repo path from retrieve.yaml ansible.builtin.set_fact: - network_backup_path_root: "{{ role_path }}/{{ data_store.scm.origin.url.split('/')[-1] | regex_replace('\\.git$', '') }}" + network_backup_path_root: "{{ network_backup_path | default(role_path ~ '/' ~ (data_store.scm.origin.url.split('/')[-1] | regex_replace('\\.git$', ''))) }}" when: data_store['scm']['origin'] is defined tags: always @@ -34,6 +34,13 @@ when: network_backup_filename is undefined tags: always +- name: Read previous backup for differential comparison (before current backup is created) + ansible.builtin.include_tasks: differential_scm_read_previous.yaml + when: + - data_store['scm']['origin'] is defined + - type | default('full') == "diff" + tags: always + - name: Include tasks ansible.builtin.include_tasks: network.yaml tags: always @@ -50,3 +57,14 @@ - backup_has_changes | default(true) run_once: true tags: always + +- name: Cleanup cloned repository when publish is skipped + ansible.builtin.file: + path: "{{ network_backup_path_root }}" + state: absent + when: + - data_store.scm.origin is defined + - not (backup_has_changes | default(true)) + run_once: true + delegate_to: localhost + tags: always diff --git a/roles/backup/tasks/differential_scm.yaml b/roles/backup/tasks/differential_scm.yaml index 4d0cc6e..16f7afd 100644 --- a/roles/backup/tasks/differential_scm.yaml +++ b/roles/backup/tasks/differential_scm.yaml @@ -8,6 +8,9 @@ backup_type: "{{ type | default('full') }}" tags: always +# Previous backup was already read in differential_scm_read_previous.yaml +# before the current backup was created (to avoid overwriting the file) +# Check if previous backup exists (for conditional logic) - name: Check if previous backup file exists ansible.builtin.stat: path: "{{ network_backup_path }}/{{ network_backup_filename }}" @@ -17,22 +20,12 @@ - backup_type == "diff" tags: always -- name: Normalize previous backup file (remove timestamps and metadata) - ansible.builtin.shell: | - sed -E \ - -e '/^!Command:/d' \ - -e '/^!Running configuration last done at:/d' \ - -e '/^!Time:/d' \ - -e '/^!NVRAM config last updated at:/d' \ - -e '/^!No configuration change since last restart/d' \ - "{{ network_backup_path }}/{{ network_backup_filename }}" | grep -v '^[[:space:]]*$' - register: normalized_previous_result - changed_when: false - failed_when: false +- name: Debug normalized previous backup result (from pre-read) + ansible.builtin.debug: + msg: "Normalized previous backup result (read before current backup): {{ normalized_previous_backup | default('NOT SET - previous backup may not exist') }}" when: - data_store['scm']['origin'] is defined - backup_type == "diff" - - previous_backup_stat.stat.exists | default(false) tags: always - name: Normalize current backup file (remove timestamps and metadata) @@ -40,6 +33,7 @@ sed -E \ -e '/^!Command:/d' \ -e '/^!Running configuration last done at:/d' \ + -e '/^! Last configuration change at /d' \ -e '/^!Time:/d' \ -e '/^!NVRAM config last updated at:/d' \ -e '/^!No configuration change since last restart/d' \ @@ -51,33 +45,31 @@ - backup_type == "diff" tags: always -- name: Set normalized content facts (when previous backup exists) - ansible.builtin.set_fact: - normalized_previous: "{{ normalized_previous_result.stdout | default('') }}" - normalized_current: "{{ normalized_current_result.stdout | default('') }}" +- name: Debug normalized current backup result + ansible.builtin.debug: + msg: "Normalized current backup result: {{ normalized_current_result.stdout }}" when: - data_store['scm']['origin'] is defined - backup_type == "diff" - - previous_backup_stat.stat.exists | default(false) tags: always -- name: Set normalized content facts (when no previous backup exists) +- name: Set normalized content facts (use pre-read previous backup) ansible.builtin.set_fact: + normalized_previous: "{{ normalized_previous_backup | default('') }}" normalized_current: "{{ normalized_current_result.stdout | default('') }}" - normalized_previous: "" when: - data_store['scm']['origin'] is defined - backup_type == "diff" - - not (previous_backup_stat.stat.exists | default(false)) tags: always -- name: Compare normalized backups +- name: Compare normalized backups (using pre-read previous backup) ansible.builtin.set_fact: backup_has_changes: "{{ normalized_previous != normalized_current }}" when: - data_store['scm']['origin'] is defined - backup_type == "diff" - - previous_backup_stat.stat.exists | default(false) + - normalized_previous_backup is defined + - normalized_previous_backup != "" tags: always - name: Set backup_has_changes to true if no previous backup exists @@ -86,7 +78,7 @@ when: - data_store['scm']['origin'] is defined - backup_type == "diff" - - not (previous_backup_stat.stat.exists | default(false)) + - normalized_previous_backup is not defined or normalized_previous_backup == "" tags: always - name: Set backup_has_changes to true for full backup type diff --git a/roles/backup/tasks/differential_scm_read_previous.yaml b/roles/backup/tasks/differential_scm_read_previous.yaml new file mode 100644 index 0000000..c84cc5f --- /dev/null +++ b/roles/backup/tasks/differential_scm_read_previous.yaml @@ -0,0 +1,55 @@ +--- +# Read previous backup BEFORE current backup is created +# This prevents the current backup from overwriting the previous backup file +# before we can read it for comparison + +- name: Set default backup type to full if not specified + ansible.builtin.set_fact: + backup_type: "{{ type | default('full') }}" + tags: always + +- name: Check if previous backup file exists + ansible.builtin.stat: + path: "{{ network_backup_path }}/{{ network_backup_filename }}" + register: previous_backup_stat + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + tags: always + +- name: Normalize previous backup file (remove timestamps and metadata) + ansible.builtin.shell: | + sed -E \ + -e '/^!Command:/d' \ + -e '/^!Running configuration last done at:/d' \ + -e '/^! Last configuration change at /d' \ + -e '/^!Time:/d' \ + -e '/^!NVRAM config last updated at:/d' \ + -e '/^!No configuration change since last restart/d' \ + "{{ network_backup_path }}/{{ network_backup_filename }}" | grep -v '^[[:space:]]*$' + register: normalized_previous_result + changed_when: false + failed_when: false + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - previous_backup_stat.stat.exists | default(false) + tags: always + +- name: Store normalized previous backup for later comparison + ansible.builtin.set_fact: + normalized_previous_backup: "{{ normalized_previous_result.stdout | default('') }}" + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - previous_backup_stat.stat.exists | default(false) + tags: always + +- name: Set normalized_previous to empty if no previous backup exists + ansible.builtin.set_fact: + normalized_previous_backup: "" + when: + - data_store['scm']['origin'] is defined + - backup_type == "diff" + - not (previous_backup_stat.stat.exists | default(false)) + tags: always diff --git a/roles/backup/tasks/retrieve.yaml b/roles/backup/tasks/retrieve.yaml index 6cde9b8..3e5a426 100644 --- a/roles/backup/tasks/retrieve.yaml +++ b/roles/backup/tasks/retrieve.yaml @@ -76,5 +76,7 @@ - name: Update Inventory Path ansible.builtin.set_fact: + # Use the actual repository path from git_retrieve or existing repo + # This path is used by backup.yaml to set network_backup_path_root network_backup_path: "{{ network_backup_repository['path'] }}" tags: always diff --git a/roles/restore/tasks/cli_restore.yaml b/roles/restore/tasks/cli_restore.yaml index 6d2f280..fd0f080 100644 --- a/roles/restore/tasks/cli_restore.yaml +++ b/roles/restore/tasks/cli_restore.yaml @@ -2,6 +2,7 @@ - name: Get current timestamp ansible.builtin.set_fact: timestamp: "{{ lookup('pipe', 'date +%Y%m%d%H%M%S') }}" + tags: always - name: Set content specific facts ansible.builtin.set_fact: @@ -11,16 +12,18 @@ delete_path: "{{ role_path }}/tasks/delete" health_check_path: "{{ role_path }}/tasks/health_checks" delete_backup_from_dest: true + tags: always - name: Check if file copy is possible ansible.builtin.include_tasks: "{{ health_check_path }}/{{ network_os }}.yaml" + tags: always - name: Copy file from src to a network device ansible.netcommon.net_put: src: "{{ network_restore_backup_path }}/{{ network_backup_restore_filename }}" dest: "{{ file_name }}.txt" when: network_os in ['iosxr', 'nxos', 'eos', 'junos'] - + tags: always - name: Copy file using system SCP CLI as a workaround for ios quirk and libssh SCP ansible.builtin.command: > @@ -32,21 +35,31 @@ delegate_to: localhost when: network_os == "ios" changed_when: true + tags: always - name: Prepare appliance for a restore operation ansible.builtin.include_tasks: "{{ prepare_path }}/{{ network_os }}.yaml" + tags: always + +- name: Set default appliance_dir per network OS if not provided + ansible.builtin.set_fact: + appliance_dir: "{{ appliance_dir | default('flash://' if network_os == 'ios' else 'flash:' if network_os == 'iosxr' else '/var/tmp' if network_os == 'junos' else 'flash:' if network_os == 'nxos' else 'flash:' if network_os == 'eos' else 'flash:') }}" + tags: always - name: Restore operation for {{ network_os }} ansible.netcommon.cli_restore: filename: "{{ file_name }}.txt" path: "{{ appliance_dir }}" + tags: always - name: Delete backup from appliance ansible.builtin.include_tasks: "{{ delete_path }}/{{ network_os }}.yaml" when: delete_backup_from_dest + tags: always - name: Remove locally cloned repo ansible.builtin.file: path: "{{ network_restore_backup_repo }}" state: absent when: data_store.scm.origin.path is defined + tags: always \ No newline at end of file diff --git a/roles/restore/tasks/cli_restore_config.yaml b/roles/restore/tasks/cli_restore_config.yaml index 57f6824..432595f 100644 --- a/roles/restore/tasks/cli_restore_config.yaml +++ b/roles/restore/tasks/cli_restore_config.yaml @@ -2,6 +2,8 @@ - name: Set content specific facts ansible.builtin.set_fact: cli_restore_path: "{{ role_path }}/cli_restore" + tags: always - name: Invoke restore operation ansible.builtin.include_tasks: "{{ cli_restore_path }}/{{ network_os }}.yaml" + tags: always \ No newline at end of file diff --git a/roles/restore/tasks/common/validation.yaml b/roles/restore/tasks/common/validation.yaml index cd0bd2c..ac4ed7f 100644 --- a/roles/restore/tasks/common/validation.yaml +++ b/roles/restore/tasks/common/validation.yaml @@ -7,7 +7,9 @@ - "eos" - "nxos" - "iosxr" + tags: always - name: Conditional test ansible.builtin.include_tasks: "common/unsupported_platform.yaml" when: ansible_network_os.split('.')[-1] not in supported_platforms + tags: always \ No newline at end of file diff --git a/roles/restore/tasks/main.yaml b/roles/restore/tasks/main.yaml index f45aa21..720dd51 100644 --- a/roles/restore/tasks/main.yaml +++ b/roles/restore/tasks/main.yaml @@ -1,6 +1,8 @@ --- - name: Include tasks ansible.builtin.include_tasks: common/validation.yaml + tags: always - name: Run the platform specific tasks ansible.builtin.include_tasks: "restore.yaml" + tags: always \ No newline at end of file diff --git a/roles/restore/tasks/network.yaml b/roles/restore/tasks/network.yaml index cc87c5d..db03e3c 100644 --- a/roles/restore/tasks/network.yaml +++ b/roles/restore/tasks/network.yaml @@ -9,11 +9,14 @@ - "ios" - "junos" network_os: "{{ ansible_network_os.split('.')[2] }}" + tags: always - name: Invoke restore ansible.builtin.include_tasks: "cli_restore.yaml" when: network_os not in supported_cli_restore + tags: always - name: Invoke cli specific backup task ansible.builtin.include_tasks: "cli_restore_config.yaml" when: network_os not in supported_plugin_restore + tags: always \ No newline at end of file diff --git a/roles/restore/tasks/prepare/ios.yaml b/roles/restore/tasks/prepare/ios.yaml index be86349..75162e4 100644 --- a/roles/restore/tasks/prepare/ios.yaml +++ b/roles/restore/tasks/prepare/ios.yaml @@ -4,6 +4,10 @@ msg: - "Task to prepare appliance for restore operation" +- name: Set IOS directory for restore operation + ansible.builtin.set_fact: + appliance_dir: "flash://" + - name: Overwrite startup config - archive cisco.ios.ios_config: lines: diff --git a/roles/restore/tasks/restore.yaml b/roles/restore/tasks/restore.yaml index 5441e1d..12b57bd 100644 --- a/roles/restore/tasks/restore.yaml +++ b/roles/restore/tasks/restore.yaml @@ -3,30 +3,37 @@ ansible.builtin.set_fact: network_restore_backup_path: "./tmp_backup" when: data_store.local is not defined + tags: always - name: Set default host vars path ansible.builtin.set_fact: network_restore_backup_path: "{{ data_store['local']['path'] }}" when: data_store['local'] is defined + tags: always - name: Set default host vars filename ansible.builtin.set_fact: network_backup_restore_filename: "{{ data_store['local']['filename'] }}" when: data_store['local'] is defined + tags: always - name: Set default host vars path ansible.builtin.set_fact: - network_backup_restore_relative_path: "{{ data_store['scm']['origin']['path'] }}" + network_restore_backup_relative_path: "{{ data_store['scm']['origin']['path'] }}" when: data_store['scm'] is defined + tags: always - name: Set default host vars filename ansible.builtin.set_fact: network_backup_restore_filename: "{{ data_store['scm']['origin']['filename'] }}" when: data_store['scm'] is defined + tags: always - name: Retrieve a repository from a distant location and make it available locally ansible.builtin.include_tasks: retrieve.yaml when: data_store.scm is defined + tags: always - name: Include tasks ansible.builtin.include_tasks: network.yaml + tags: always \ No newline at end of file diff --git a/roles/restore/tasks/retrieve.yaml b/roles/restore/tasks/retrieve.yaml index 5c39f3b..92aa2fb 100644 --- a/roles/restore/tasks/retrieve.yaml +++ b/roles/restore/tasks/retrieve.yaml @@ -6,14 +6,18 @@ token: "{{ data_store.scm.origin.get('token') if data_store.scm.origin.get('token') else omit }}" ssh_key_file: "{{ data_store.scm.origin.get('ssh_key_file') if data_store.scm.origin.get('ssh_key_file') else omit }}" ssh_key_content: "{{ data_store.scm.origin.get('ssh_key_content') if data_store.scm.origin.get('ssh_key_content') else omit }}" - parent_directory: "{{ role_path }}/{{ network_restore_backup_path }}" + # Use parent_directory from data_store.scm if provided, otherwise default to role_path + parent_directory: "{{ data_store.scm.parent_directory | default(role_path ~ '/' ~ network_restore_backup_path) }}" changed_when: false register: resource_manager_result + tags: always - name: Update Inventory Path ansible.builtin.set_fact: network_restore_backup_repo: "{{ resource_manager_result['path'] }}" + tags: always - name: Update Inventory Path ansible.builtin.set_fact: network_restore_backup_path: "{{ resource_manager_result['path'] }}/{{ data_store['scm']['origin']['path'] }}" + tags: always \ No newline at end of file From be88e14b5733b1f3cce967750277e92819c59a8c Mon Sep 17 00:00:00 2001 From: rohitthakur2590 Date: Sun, 1 Feb 2026 06:04:01 +0100 Subject: [PATCH 7/7] update tasks Signed-off-by: rohitthakur2590 --- .../Differential_Backup_Documentation.md | 253 ------------------ roles/backup/README.md | 6 +- 2 files changed, 1 insertion(+), 258 deletions(-) delete mode 100644 roles/backup/Differential_Backup_Documentation.md diff --git a/roles/backup/Differential_Backup_Documentation.md b/roles/backup/Differential_Backup_Documentation.md deleted file mode 100644 index 6a11c9d..0000000 --- a/roles/backup/Differential_Backup_Documentation.md +++ /dev/null @@ -1,253 +0,0 @@ -# Differential Backup Documentation - -## Overview - -The differential backup feature in the `network.backup.backup` role allows you to create backups **only when actual configuration changes are detected**, ignoring metadata and timestamp differences. This reduces storage overhead and SCM noise by preventing unnecessary commits and pull requests when only timestamps or metadata have changed. - -## How It Works - -### 1. Backup Type Selection - -The differential backup is controlled by the `type` parameter: - -```yaml -- name: Create Network Backup - ansible.builtin.include_role: - name: network.backup.backup - vars: - type: "diff" # Options: "full", "incremental", or "diff" - data_store: - scm: - origin: - # ... SCM configuration -``` - -- **`type: "diff"`**: Enables differential backup - only publishes if config actually changed -- **`type: "full"`** (default): Always creates and publishes backup, regardless of changes - -### 2. Backup Process Flow - -When `type: "diff"` is set, the following process occurs: - -``` -1. Retrieve previous backup from SCM (if exists) - ↓ -2. Create current backup from device - ↓ -3. Normalize both backups (remove timestamps/metadata) - ↓ -4. Compare normalized content - ↓ -5. If different → backup_has_changes = true → Publish - If identical → backup_has_changes = false → Skip publish -``` - -## Normalization Process - -The normalization process removes metadata and timestamp lines that don't represent actual configuration changes. This ensures that only meaningful configuration differences trigger a new backup. - -### What Gets Ignored (Removed During Normalization) - -The following lines are **removed** from backup files before comparison: - -#### Cisco IOS/IOS-XE -- `!Command: show running-config` -- `!Running configuration last done at: ` -- `!Time: ` -- `!NVRAM config last updated at: ` -- `!No configuration change since last restart` - -#### Cisco NX-OS -- `!Command: show running-config` -- `!Time: ` -- `!Running configuration last done at: ` -- `!NVRAM config last updated at: ` - -#### Cisco IOS-XR -- `Building configuration...` -- `!! IOS XR Configuration ` -- `!! Last configuration change at by ` - -#### Arista EOS -- `! Command: show running-config` -- `! device: ` -- `! boot system flash:/` - -#### General Metadata -- Empty lines (whitespace-only lines) -- Comment lines that contain only timestamps or metadata - -### What Gets Compared (Actual Configuration) - -After normalization, the following configuration elements are compared: - -- **Interface configurations** -- **Routing protocols** -- **Access control lists (ACLs)** -- **VLANs** -- **User accounts** -- **SNMP configuration** -- **System settings** (hostname, domain, etc.) -- **Any actual configuration commands** - -## Examples - -### Example 1: No Configuration Change (Backup Skipped) - -**Previous Backup:** -``` -!Time: Mon Dec 12 18:19:15 2025 -hostname router1 -interface GigabitEthernet0/0 - ip address 192.168.1.1 255.255.255.0 -``` - -**Current Backup:** -``` -!Time: Mon Dec 12 18:30:45 2025 -hostname router1 -interface GigabitEthernet0/0 - ip address 192.168.1.1 255.255.255.0 -``` - -**Normalized Comparison:** -``` -Previous: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.1 255.255.255.0 -Current: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.1 255.255.255.0 -``` - -**Result:** ✅ **Identical** → `backup_has_changes = false` → **Publish skipped** - -### Example 2: Configuration Change Detected (Backup Published) - -**Previous Backup:** -``` -!Time: Mon Dec 12 18:19:15 2025 -hostname router1 -interface GigabitEthernet0/0 - ip address 192.168.1.1 255.255.255.0 -``` - -**Current Backup:** -``` -!Time: Mon Dec 12 18:30:45 2025 -hostname router1 -interface GigabitEthernet0/0 - ip address 192.168.1.2 255.255.255.0 -``` - -**Normalized Comparison:** -``` -Previous: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.1 255.255.255.0 -Current: hostname router1\ninterface GigabitEthernet0/0\n ip address 192.168.1.2 255.255.255.0 -``` - -**Result:** ❌ **Different** → `backup_has_changes = true` → **Publish triggered** - -### Example 3: First Backup (Always Published) - -**Previous Backup:** Does not exist - -**Result:** ✅ **No previous backup** → `backup_has_changes = true` → **Publish triggered** (first backup always created) - -## Implementation Details - -### Normalization Command - -The normalization is performed using `sed` and `grep` commands: - -```bash -sed -E \ - -e '/^!Command:/d' \ - -e '/^!Running configuration last done at:/d' \ - -e '/^!Time:/d' \ - -e '/^!NVRAM config last updated at:/d' \ - -e '/^!No configuration change since last restart/d' \ - "backup_file.txt" | grep -v '^[[:space:]]*$' -``` - -This command: -1. Removes timestamp/metadata lines using `sed` -2. Removes empty lines using `grep` -3. Preserves all actual configuration content - -### Comparison Logic - -The comparison is performed in the `differential_scm.yaml` task file: - -```yaml -- name: Compare normalized backups - ansible.builtin.set_fact: - backup_has_changes: "{{ normalized_previous != normalized_current }}" -``` - -- If `normalized_previous == normalized_current`: No changes detected -- If `normalized_previous != normalized_current`: Changes detected - -### Publish Decision - -The publish task only runs when `backup_has_changes` is `true`: - -```yaml -- name: Include build tasks - ansible.builtin.include_tasks: publish.yaml - when: - - data_store.scm.origin is defined - - backup_has_changes | default(true) - run_once: true -``` - -## Benefits - -1. **Reduced SCM Noise**: No unnecessary commits or pull requests for timestamp-only changes -2. **Storage Efficiency**: Only stores backups when configuration actually changes -3. **Idempotency**: Running the playbook multiple times with the same config won't create new PRs -4. **Clear Change History**: Git history only shows actual configuration changes - -## Limitations - -1. **Platform-Specific Metadata**: The normalization patterns are designed for common Cisco and Arista metadata. Custom metadata formats may not be filtered. -2. **Whitespace Changes**: Significant whitespace changes in configuration (beyond empty lines) will be detected as changes. -3. **Order-Dependent**: If device output order changes but configuration is identical, it may be detected as a change (rare). - -## Troubleshooting - -### Backup Always Shows Changes - -If backups always show changes even when configuration hasn't changed: - -1. **Check normalization patterns**: Verify that all timestamp/metadata lines are being removed -2. **Inspect normalized content**: Use debug tasks to see what's being compared: - ```yaml - - name: Debug normalized content - ansible.builtin.debug: - msg: "Previous: {{ normalized_previous }}\nCurrent: {{ normalized_current }}" - ``` - -### Backup Never Shows Changes - -If backups never show changes even when configuration changed: - -1. **Verify backup type**: Ensure `type: "diff"` is set -2. **Check file paths**: Verify previous backup file exists and is being read correctly -3. **Check normalization**: Ensure normalization isn't removing actual configuration lines - -## Best Practices - -1. **Use `type: "diff"` for automated backups**: Prevents unnecessary commits in scheduled backup jobs -2. **Use `type: "full"` for manual backups**: Ensures backup is always created when manually triggered -3. **Monitor backup_has_changes**: Use the debug message to understand when backups are skipped -4. **Review normalized content**: Periodically verify that normalization is working correctly for your device types - -## Related Files - -- `/roles/backup/tasks/differential_scm.yaml`: Differential backup logic -- `/roles/backup/tasks/backup.yaml`: Main backup orchestration -- `/roles/backup/tasks/cli_backup.yaml`: Backup file creation -- `/roles/backup/tasks/publish.yaml`: SCM publishing logic - -## See Also - -- [Network Backup Role README](README.md) -- [Ansible Network Backup Collection Documentation](https://docs.ansible.com/ansible/latest/collections/network/backup/) - diff --git a/roles/backup/README.md b/roles/backup/README.md index e17eea7..e8a3355 100644 --- a/roles/backup/README.md +++ b/roles/backup/README.md @@ -18,7 +18,6 @@ This role supports full and differential backups, storing them locally or in a r - Works with both local and Git-based data stores. - Helps reduce storage and SCM noise by saving only when diff exists. - **Ignores timestamps and metadata** - only detects actual configuration changes. -- See [Differential Backup Documentation](Differential_Backup_Documentation.md) for detailed explanation of how it works. --- @@ -38,9 +37,6 @@ This role supports full and differential backups, storing them locally or in a r | `data_store.scm.origin.ssh_key_content` | The content of the SSH private key | `str` | Yes (if using SCM SSH) | N/A | | `type` | Type of backup to perform. Options: `"full"`, `"incremental"`, or `"diff"` | `str` | No | `"full"` | -> Note: Either `data_store.local` or `data_store.scm` must be provided. -> -> **Differential Backup (`type: "diff"`)**: When enabled, the role compares the current backup with the previous backup after normalizing (removing timestamps/metadata). Only publishes if actual configuration changes are detected. See [Differential Backup Documentation](Differential_Backup_Documentation.md) for details. --- @@ -175,4 +171,4 @@ See [LICENSE](https://www.gnu.org/licenses/gpl-3.0.txt) to see the full text. ## Author Information -- Ansible Network Content Team \ No newline at end of file +- Ansible Network Content Team