Understanding the Architecture That Powers Network Automation

Look, I get it. You’ve probably heard loads about pyATS being this amazing network testing framework, but nobody’s properly explained what you’re actually working with under the hood. I see this all the time with my students – they jump straight into writing tests without understanding how the whole thing fits together, then wonder why their code’s a mess six months later.

Cisco pyATS Blog Index

So let me sort this out for you properly. The cisco pyats ecosystem isn’t just some Python library you pip install and off you go. It’s actually a proper ecosystem – think of it like a professional workshop where each tool has its place, but they all work together to help you build something decent.

Right, so here’s what I’m going to show you. We’ll break down exactly how this cisco pyats ecosystem actually works, what each bit does, and how they all connect together. By the end, you’ll understand enough to make proper decisions about your automation projects instead of just copying and pasting code from GitHub and hoping for the best.

What You’re Actually Working With

First thing – I made this mistake myself when I started. I thought pyATS was just another testing library like unittest or something. Wrong. It’s a complete automation ecosystem that Cisco built from the ground up for network testing.

The architecture’s got three main layers, and honestly, once you get your head around this, everything else makes loads more sense. You’ve got your core framework at the bottom – that’s your basic toolbox. Then you’ve got the SDK and libraries layer (they used to call this Genie) which makes everything network-aware. Finally, you’ve got your upper layer business logic where all your actual automation lives.

Now, the clever bit is that the core framework is completely generic. You could use it to test anything – web applications, databases, whatever. But when you add the libraries layer, that’s when it becomes properly network-focused. The libraries understand devices, protocols, command outputs – all the stuff that makes networks different from everything else.

This separation’s brilliant because it means you can write tests that work across different vendors without having to worry about command syntax or connection details. The abstraction’s built right into the architecture.

The Core Framework – Your Basic Toolkit

Right, let’s dig into the actual components. There are four main ones you’ll use constantly, plus a load of supporting bits that work behind the scenes.

AEtest – Where Your Tests Live

AEtest stands for Automation Easy Testing, and it’s basically your test harness. If you’ve used unittest or pytest before, this’ll feel familiar – that was intentional.

Here’s something I wish someone had told me ages ago: AEtest uses this block-based approach where you’ve got CommonSetup, TestCases, and CommonCleanup. I kept trying to do everything in the test methods themselves. Complete waste of time. The CommonSetup is where you connect to your devices, and if you don’t get that right, everything else falls apart.

# This took me weeks to get right when I started
from pyats import aetest

class CommonSetup(aetest.CommonSetup):
    @aetest.subsection
    def connect_to_devices(self, testbed):
        # I spent hours debugging connection issues because
        # I kept trying to connect in the wrong places
        for device in testbed:
            device.connect()

class NetworkConnectivityTest(aetest.Testcase):
    @aetest.test
    def test_interface_status(self, testbed):
        # Only now can you actually test your interfaces
        for device in testbed:
            interface = device.parse("show interface")
            print(interface)

class CommonCleanup(aetest.CommonCleanup):
    @aetest.subsection
    def disconnect_devices(self, testbed):
        # Clean up properly - I've seen too many labs
        # become unusable because people skip this
        for device in testbed:
            device.disconnect()

The framework handles test discovery and execution order automatically. You just focus on writing test logic. Install it with pip install pyats[full] or get just AEtest with pip install pyats.aetest.

Easypy – Job Orchestration That Actually Works

Easypy’s your orchestration engine. You can run individual AEtest scripts directly, but Easypy gives you proper job management for production environments.

What I didn’t realise for months was how much Easypy does behind the scenes. It standardises logging, reporting, and archiving. After running a job, you get a zipped archive in ~/.pyats/archive/YY-MM with everything – logs, results, configuration snapshots. Proper enterprise-grade stuff.

# job.py - Simple Easypy job
from pyats.easypy import run

def main():
    # Run interface tests on my Manchester lab
    run(testscript='interface_tests.py',
        testbed='manchester_lab.yaml',
        environment='production')
    
    # Run routing tests on same testbed
    run(testscript='routing_tests.py',
        testbed='manchester_lab.yaml')

The brilliant thing about Easypy jobs is you can pass unknown keyword arguments and they get propagated to your testscript. Makes your tests loads more flexible.

Testbed and Topology – Your Network’s Digital Twin

The testbed defines your network in YAML format. Without a proper testbed, your tests can’t connect to anything. Simple as that.

Now, the topology module does two main things that took me ages to understand:

1. It defines and describes testbed metadata using YAML, then loads it into testbed objects
2. It lets you query topology information via object attributes instead of API calls

testbed:
  name: manchester_lab_network
  
devices:
  lab-router1:
    type: router
    os: iosxe
    connections:
      default:
        protocol: ssh
        ip: 172.16.1.10
        username: dickie1
        password: '%ENV{LAB_PASSWORD}'
    
  lab-switch1:
    type: switch
    os: iosxe
    connections:
      default:
        protocol: ssh
        ip: 172.16.1.20
        username: dickie1
        password: '%ENV{LAB_PASSWORD}'

The testbed abstraction means you can use the same test with different testbeds for dev, staging, and production. I’ve got the same test running against three different environments just by changing the testbed file.

Clean Framework – Getting Your Devices Back to Normal

The Clean framework automates device cleanup and recovery. This is absolutely crucial for test environments because you need devices to return to a known good state between test runs.

Clean handles everything from basic configuration cleanup to complete device reimaging. I’ve seen too many test environments become completely unusable because people didn’t implement proper cleanup procedures. Don’t be that person.

Supporting Components – The Hidden Infrastructure

There’s loads of stuff working behind the scenes that you don’t directly interact with, but it’s worth knowing about because it affects how you design tests.

The Bits You Don’t See But Need to Know About

Asynchronous library – Handles parallel execution and concurrent operations. This is what lets pyATS connect to multiple devices simultaneously. You don’t use this directly, but it’s why your tests can run so much faster when you’re testing large networks.

Data structures – These aren’t just standard Python dictionaries. They’re optimised for the kind of hierarchical, relationship-heavy data you get from network devices.

TCL integration – Sounds weird, but it’s there for backwards compatibility with existing Cisco testing infrastructure. Most new development doesn’t use it.

Logging system – Much more sophisticated than standard Python logging. Designed to handle the volume and complexity of logs from network testing, with proper correlation between test steps and device interactions.

Result objects – Handle test result collection and aggregation. They understand relationships between test sections and provide detailed analysis of test outcomes.

Reporter – Generates comprehensive HTML reports with detailed logs, device interactions, and test results. Absolutely essential for troubleshooting failed tests.

Utilities – Various helper functions for common network testing tasks like IP address manipulation and configuration parsing.

Robot framework support – This is interesting. pyATS integrates with the ROBOT framework, which lets you write tests using English-like keywords instead of Python code. Makes test creation accessible to people who aren’t programmers.

Manifest – Tracks test dependencies and requirements. Becomes important in complex environments with multiple test suites.

The SDK and Libraries – Where the Network Magic Happens

Right, this is where things get properly interesting. The SDK and libraries are what make pyATS network-aware instead of just another generic testing framework.

Genie – The Network Intelligence Engine

Genie’s where pyATS gets its network-specific intelligence. Without Genie, you’d just have a testing framework. With Genie, you get a network automation platform that actually understands your devices.

The Metaparser – Protocol-Agnostic Parsing

Here’s something that confused me for months: Genie’s metaparser is a protocol-agnostic parsing engine. What that means is you can create one parsing structure that works regardless of whether you’re using CLI, NETCONF, or RESTCONF to talk to your devices.

The metaparser class provides the base for building parsers. The brilliant thing is you get consistent data structures regardless of how you’re pulling data from devices. SSH commands, NETCONF queries – doesn’t matter, the data structure’s the same.

Device Abstraction – Write Once, Run Everywhere

Genie abstracts device differences so you can write tests that work across multiple vendors. Tell Genie to get interface information, and it figures out whether to use “show interfaces” on Cisco or “show interfaces terse” on Juniper.

This goes deeper than just command translation. Genie understands relationships between different pieces of network data and presents them in a unified way regardless of the underlying device differences.

Schema Engine – Defining Your Data Structure

The schema engine provides utility classes for building proper data structures:

• Any() – Wildcard that accepts any value, useful for dynamic data like interface names
• Optional() – Values that aren’t required
• And() – Values must pass all validation requirements
• Or() – Values must pass at least one validation requirement
• Default() – Provides default values when data isn’t specified

# Schema for parsing "show license summary" 
# This took me ages to get right
from genie.metaparser import MetaParser
from genie.metaparser.util.schemaengine import Any

class ShowLicenseSummarySchema(MetaParser):
    """Schema for show license summary"""
    schema = {
        'license_usage': {
            Any(): {
                'entitlement': str,
                'count': str,
                'status': str,
            }
        }
    }

The Any() class gives you flexibility on things like license names, with type annotations for each key-value pair.

Command Parsing – From Raw Output to Structured Data

Instead of parsing command output with regular expressions (which is absolute hell), Genie converts CLI output into structured Python data.

# Without Genie - manual parsing (this was a nightmare)
output = device.execute('show ip interface brief')
# Now you'd write regex patterns to extract data
# Error-prone and vendor-specific
# I spent weeks debugging regex patterns

# With Genie - structured data (this changed everything for me)
parsed_output = device.parse('show ip interface brief')
# Returns a structured dictionary you can query easily
for interface, data in parsed_output['interface'].items():
    if data['status'] == 'up':
        print(f"Interface {interface} is operational")

Genie includes parsers for thousands of commands across multiple vendors. These aren’t simple parsers either – they understand context and relationships within the data.

Vendor Agnostic Automation – Why This Matters

The vendor-agnostic capability comes from Genie’s parsing libraries. Genie provides parsers that interpret and transform raw command outputs from different network devices into structured data formats.

This gives you a unified approach to network automation regardless of the underlying hardware or software vendor. You can write scripts that work across Cisco, Juniper, Arista without worrying about command syntax differences.

It promotes more maintainable testing scripts by deferring operational data parsing to back-end libraries. It harmonises parsing output between CLI, XML, and YANG interfaces and enforces just enough structure to give you consistent behaviour across different interface types.

How Everything Works Together

Understanding individual components is useful, but the real power comes from understanding how they work together in the cisco pyats ecosystem.

What Actually Happens When You Run a Test

Here’s what happens behind the scenes:

1. Testbed Loading – pyATS loads your testbed definition using the topology module and creates device objects
2. Connection Establishment – Connection management establishes connections using appropriate protocols
3. Test Discovery – AEtest discovers and organises your test methods
4. Test Execution – Your tests run, using Genie for device communication and metaparser for structured data
5. Result Collection – Results are collected using result objects and aggregated by the reporter
6. Archive Generation – Easypy generates comprehensive archives with logs, results, and supporting data

Each component plays a specific role, but they’re designed to work together seamlessly. The data flows through well-defined interfaces between components.

Data Flow and Integration

The components integrate through well-defined data flows:

• Testbeds define device characteristics that Genie uses for device abstraction
• Genie parsers convert CLI output into structured data using the metaparser engine
• AEtest provides the framework for organising test logic that works with Genie data
• Easypy orchestrates everything and collects results using the reporter system

This integration means you get consistent behaviour across the entire testing workflow.

Real-World Architecture Patterns That Actually Work

After working with pyATS in loads of different environments, I’ve seen several patterns that work well in practice.

Single Script Pattern – Keep It Simple

For basic scenarios, put everything in a single AEtest script. Works well for simple connectivity testing or configuration validation.

# Single script for basic connectivity testing
class ConnectivityTest(aetest.Testcase):
    @aetest.test
    def test_ping_connectivity(self):
        # Test basic connectivity between devices
        pass
    
    @aetest.test  
    def test_interface_status(self):
        # Test interface operational status
        pass

Don’t overcomplicate things. I’ve seen people create elaborate frameworks for tasks that could be handled with a single script.

Modular Test Suite Pattern – When You Need to Scale

For complex scenarios, use multiple test scripts organised by function:

tests/
├── connectivity/
│   ├── interface_tests.py
│   └── reachability_tests.py
├── protocols/
│   ├── bgp_tests.py
│   └── ospf_tests.py
└── job.py

Each script focuses on a specific area. Makes the test suite easier to maintain and extend when you’ve got different team members responsible for different aspects.

Library Pattern – Enterprise Reusability

For enterprise environments, create reusable test libraries:

# test_library.py - Reusable test functions
def verify_interface_status(device, interface):
    """Reusable interface verification function"""
    output = device.parse('show interfaces')
    return output[interface]['oper_status'] == 'up'

# Multiple test scripts can import and use these functions

This promotes code reuse and consistency across your test suite. Particularly valuable when you’ve got multiple teams working on different aspects of network automation.

Performance and Scalability – What You Need to Know

The cisco pyats ecosystem is designed for enterprise-scale testing, but you need to understand the performance characteristics.

Connection Management and Pooling

The Cisco pyATS Ecosystem uses connection pooling to efficiently manage device connections. You’re not constantly establishing and tearing down SSH sessions, which significantly improves performance.

However, be mindful of device connection limits. Most network devices have limits on concurrent SSH sessions. I learned this the hard way when my tests started failing randomly – turned out I was hitting connection limits.

Parallel Execution

The asynchronous library supports parallel test execution, which can dramatically reduce test runtime for large networks. However, parallel execution requires careful design to avoid conflicts between tests.

Ensure tests don’t interfere with each other when running in parallel. Be careful about shared resources and ensure proper cleanup between tests.

Memory Usage

Genie parsers store structured data in memory, which can consume significant resources for large-scale testing. Understanding memory usage patterns helps you design efficient tests.

Common Mistakes I See All the Time

I’ve seen engineers make several common mistakes when working with the pyATS architecture.

Overcomplicating Simple Tests

Not every test needs to use every component. If you’re just checking interface status on a few devices, a simple AEtest script with basic Genie parsing is sufficient. Don’t build elaborate frameworks when simple solutions will do.

Ignoring Connection Management

Failing to properly handle connections leads to unreliable tests. Always use proper connection establishment and cleanup procedures. The connection management system exists for a reason.

Mixing Concerns in Test Design

Keep test logic separate from infrastructure code. Your tests should focus on validation logic, not connection management or parsing details. The architecture provides clear separation of concerns – respect it.

Poor Testbed Design

A poorly designed testbed makes everything else more difficult. Spend time designing a clean, maintainable testbed structure. Use environment variables for sensitive information and organise device definitions logically.

Integration with Existing Tools

The cisco pyats ecosystem integrates with existing tools and workflows. You can use pyATS within CI/CD pipelines, integrate with monitoring systems, or combine it with configuration management tools.

Most integration scenarios involve:

• Triggering pyATS tests from external systems using job orchestration capabilities
• Consuming pyATS test results in other tools through the reporter system
• Using pyATS data in reporting or monitoring systems via result objects

This integration capability makes pyATS a powerful component in broader automation strategies.

Development and Debugging Tools

PyATS includes several tools that help with development and debugging.

Interactive Shell for Development

You can use pyATS in an interactive Python shell for development and troubleshooting:

# Interactive pyATS session for development
from pyats.topology import loader
testbed = loader.load('manchester_lab.yaml')
device = testbed.devices['lab-router1']
device.connect()
output = device.parse('show version')

This is invaluable for developing and testing parser queries or troubleshooting connection issues. I spend loads of time in interactive sessions when developing new tests.

Comprehensive Logging

PyATS provides sophisticated logging that helps you understand what’s happening during test execution. You can adjust logging levels to get more or less detail as needed.

The logging system correlates device interactions with test steps, making it much easier to troubleshoot issues when they occur.

Mock Device Support

For development, pyATS supports mock devices that simulate real network equipment. This lets you develop and test automation without needing access to physical hardware.

Mock devices are particularly useful for training environments and when you’re developing tests for equipment you don’t have direct access to.

Wrapping Up

The cisco pyats ecosystem represents a mature, well-architected approach to network testing and automation. Understanding the architecture helps you use it more effectively and build maintainable, scalable test solutions.

Start with the basics – AEtest for test structure, Genie for device communication, and proper testbed design. Once you’ve mastered these fundamentals, you can leverage advanced components like the metaparser engine, ROBOT framework integration, and enterprise orchestration capabilities.

Remember, the architecture’s designed to handle complexity, but that doesn’t mean you should make things complex unnecessarily. Start simple, understand how the components work together, and gradually add sophistication as your requirements grow.

Most importantly, understand that the Cisco pyATS Ecosystem isn’t just a collection of tools – it’s an integrated system where each component is designed to work with the others. When you respect that architecture and use the components as intended, you get reliable, maintainable automation that scales effectively.

Don’t be like me when I started – jumping straight into advanced features without understanding the fundamentals. Take the time to understand the architecture properly, and you’ll save yourself months of debugging and frustration later on.

Learn more about Cisco DevNet automation best practices