Your Network’s Digital Blueprint

Right we are going to jump in to what is a pyATS testbed, so everyone’s got pyATS installed now? Good. Because what I’m about to show you next is the bit that absolutely everyone gets wrong first time. And I mean everyone. Including me. Especially me, actually.

Cisco pyATS Blog Index

Testbeds. Your pyats testbed configuration. Look, I’m not going to lie to you – this is where most people give up on pyATS. Not because it’s difficult, but because they rush through it, mess it up, then spend weeks thinking pyATS is broken when actually their YAML file’s got a bloody space in the wrong place.

I remember my first testbed. Spent three entire days convinced my router was faulty. Kept getting connection errors. Turns out I’d put a tab instead of spaces for indentation. A bloody tab! Felt like a complete muppet when I figured it out.

But here’s the thing – get testbeds right, and suddenly pyATS becomes this incredibly powerful tool that can talk to your entire network. Get them wrong, and you’ll be pulling your hair out wondering why nothing works.

So let’s do this properly, yeah? No shortcuts, no rushing. We’ll cover the YAML basics that’ll trip you up, how to actually connect to devices without going mental, and all the gotchas that nobody tells you about.

YAML – The Format That’ll Drive You Mad

First things first – YAML. Don’t know what it stands for? “YAML Ain’t Markup Language.” Yeah, recursive acronym. Programmers think they’re clever sometimes (and to be honest they are).

Point is, pyATS uses YAML for testbed files. And YAML is… well, it’s lovely when it works and absolutely infuriating when it doesn’t.

Here’s a simple example:

---
name: Richard Killeen
age: 45
location: Manchester
skills:
  - Python
  - Network Automation
  - Getting confused by YAML
contact:
  email: richard@richardkilleen.co.uk
  website: richardkilleen.co.uk

Looks simple, right? Wrong. YAML is incredibly picky about indentation. And I mean incredibly picky. One space wrong and the whole thing’s invalid.

YAML Gotchas That’ll Ruin Your Day

Listen, I’ve made every YAML mistake possible. So learn from my pain:

Tabs vs Spaces: YAML uses spaces. Not tabs. Never tabs. I can’t tell you how many times I’ve seen people use tabs because that’s what they’re used to in other languages. Don’t do it. Use spaces. Always spaces.

Consistent Indentation: Two spaces, four spaces, doesn’t matter. Pick one and stick with it. I use two spaces because I’m lazy and it’s less typing.

Colons Need Spaces: This one got me for ages. You write key:value and wonder why it doesn’t work. It needs to be key: value with a space after the colon. Every bloody time.

Lists and Dictionaries: You can’t mix them on the same level. Took me a few day’s to understand this properly.

Actually, let me tell you about my worst YAML disaster. I was building a testbed for about 50 devices. Big network, right? Spent an entire morning typing it all out. Went to test it, nothing worked. Couldn’t figure out why.

Turns out I’d copied and pasted the first device entry to make the others, but I’d left some of the original indentation. So half my devices were properly indented and half weren’t. The YAML was valid enough that it didn’t throw an error, but pyATS couldn’t parse it properly.

Three hours of debugging. Three bloody hours. All because I was too lazy to type each device entry from scratch.

What Actually Is a Testbed?

Right, so what’s a pyats testbed configuration actually for? Think of it as your network’s address book. No, better than that – it’s like your network’s entire phone directory and instruction manual rolled into one.

It tells pyATS:

• Where your devices are (IP addresses, ports)
• How to talk to them (SSH, telnet, REST APIs)
• What credentials to use
• What type of devices they are
• How they connect to each other

Without a proper testbed, pyATS is like… well, imagine trying to navigate Manchester without knowing any street names or having a map. You’re not getting very far.

Here’s why testbeds are brilliant when you get them right:

Consistency: Write one testbed, use it everywhere. No more hardcoding IP addresses in your scripts like some kind of animal.

Flexibility: Got a lab environment and a production environment? Different testbed files. Same scripts. Magic.

Scalability: Add devices to your testbed and all your existing scripts can immediately work with them. It’s beautiful when it works.

Your First Testbed (The Simple Version)

Let’s start with something basic. One device. Can’t go too wrong with one device, right?

---
devices:
  Lab-Router1:
    alias: 'Manchester Lab Router'
    type: 'router'
    os: 'iosxe'
    platform: 'csr1000v'
    credentials:
      default:
        username: dickie1
        password: Manchester123!
    connections:
      cli:
        protocol: ssh
        ip: 172.16.1.10
        port: 22
        arguments:
          connection_timeout: 360

Let me break this down because every bit matters:

devices: Top-level section. Every testbed needs this. No devices section, no testbed.

Lab-Router1: Device name. Make it meaningful because you’ll be typing this constantly. I learned this the hard way when I called devices things like “r1” and “r2” then couldn’t remember which was which six months later.

alias: Human description. Optional but useful. Future you will thank present you for writing proper descriptions.

type: Router, switch, firewall, whatever. Not strictly required but helps with organisation.

os: Operating system. This is crucial. Get this wrong and pyATS won’t know which parsers to use. Your commands won’t parse properly and you’ll think pyATS is broken when it’s actually your testbed.

platform: Hardware platform. Also important for parser selection. CSR1000v, Catalyst 9300, ASR1000, whatever.

credentials: Login details. The ‘default’ section is what pyATS uses unless you tell it otherwise.

connections: How to actually connect. We’re using SSH here because it’s 2025 and we’re not savages.

Now, about that timeout setting. I always set connection_timeout to 360 seconds. Six minutes. Seems excessive? It’s not. I’ve seen networks where it takes three minutes just to authenticate through all the TACACS servers and radius proxies and whatever other authentication nonsense corporate networks have.

Getting Connections Right

Connection configuration is where most people mess up their pyats testbed configuration. There are loads of ways to connect, and each one has its own special way of failing.

SSH – The Sensible Choice

SSH is what you want 99% of the time:

connections:
  cli:
    protocol: ssh
    ip: 172.16.1.10
    port: 22
    arguments:
      connection_timeout: 360

That arguments section? Optional, but I always include it. Default timeout is 30 seconds which is nowhere near enough for real networks.

Telnet – For When SSH Isn’t Working

Sometimes you’re stuck with telnet. Usually in labs where someone couldn’t be bothered to configure SSH properly:

connections:
  cli:
    protocol: telnet
    ip: 172.16.1.10
    port: 23

Telnet’s actually easier than SSH in some ways. No key negotiation, no encryption nonsense. Just connects. Not secure, mind you, but simple.

REST APIs – For the Modern World

Got devices with REST APIs? RESTCONF? Good for you:

connections:
  rest:
    class: rest.connector.Rest
    ip: 172.16.1.10
    port: 443
    credentials:
      rest:
        username: dickie1
        password: Manchester123!

Notice how REST connections have their own credentials section? This confused me for ages. Why can’t I just use the default credentials? Because SSH and REST might use completely different authentication systems. Makes sense when you think about it.

Multiple Connection Types

You can define multiple ways to connect to the same device:

connections:
  cli:
    protocol: ssh
    ip: 172.16.1.10
    port: 22
  rest:
    class: rest.connector.Rest
    ip: 172.16.1.10
    port: 443

Then in your scripts, you choose which connection to use. Handy for devices that support both CLI and API access.

Actually, let me tell you about the time I discovered you could do this. I’d been writing separate testbeds for CLI and REST testing. Completely separate files. Then I found out you could put both connection types in the same testbed file.

Mind blown. I felt like such an idiot. All that duplicated effort for nothing.

Credentials – Don’t Store Passwords Like an Idiot

Right, credentials. This is where people do stupid things like commit passwords to Git repositories. Don’t be one of those people.

Basic Credentials (The Insecure Way)

credentials:
  default:
    username: dickie1
    password: Manchester123!

This works, but anyone who can read your testbed file can see your passwords. Not ideal.

Multiple Credential Sets

Some devices need different credentials for different things:

credentials:
  default:
    username: dickie1
    password: Manchester123!
  enable:
    password: EnablePassword456!
  tacacs:
    username: tacacs_user
    password: TacacsPass789!

PyATS knows when to use which credentials. Usually.

Environment Variables (The Less Stupid Way)

Much better approach:

credentials:
  default:
    username: "%ENV{DEVICE_USERNAME}"
    password: "%ENV{DEVICE_PASSWORD}"

Then in your shell:

export DEVICE_USERNAME="dickie1"
export DEVICE_PASSWORD="Manchester123!"

Now your passwords aren’t sitting in files where everyone can see them.

Secret Strings (The Slightly Less Insecure Way)

PyATS can encode passwords:

pyats create testbed file --path devices.xlsx --output testbed.yaml --encode-password

This encodes passwords in your testbed. Not proper encryption, but better than plain text. Security through obscurity, basically. Not great, but better than nothing.

Look, here’s the thing about credentials. I’ve seen production networks where the testbed files contained actual production passwords, checked into Git, visible to everyone with access to the repository.

Don’t do this. Just don’t. Use environment variables, use secret management tools, use whatever you need to use, but don’t hardcode production passwords in files.

Creating Testbeds from Real Data

Most networks already have device inventories somewhere. Instead of manually typing every device like some kind of masochist, you can convert existing data.

From Excel or CSV

Create a spreadsheet with these columns:

• hostname
• ip
• username
• password
• protocol
• os

Then convert it:

pyats create testbed file --path=devices.xlsx --output=testbed.yaml

PyATS does all the hard work. Saves loads of time.

I remember the first time I discovered this. I’d been hand-typing testbed files for day’s. Suddenly found out I could import from Excel. All that wasted time! Could’ve been doing useful things instead of typing YAML like a mug.

From Network Management Systems

Using Cisco DNA Centre? NetBox? Some other inventory system? You can pull device data via APIs and build testbeds automatically.

Much better than maintaining three different device lists – one in your monitoring system, one in your documentation, one in your testbeds. That’s madness. Use your existing source of truth.

When Things Go Wrong (And They Will)

Real networks aren’t nice clean lab environments. Here’s what’ll break and how to fix it.

Legacy SSH Nonsense

Older devices often have SSH problems:

Unable to negotiate with 172.16.1.10 port 22: no matching key exchange method found.
Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1

Brilliant. Your device only supports ancient SSH algorithms. Fix it like this:

connections:
  cli:
    protocol: ssh
    ip: 172.16.1.10
    port: 22
    ssh_options: "-o KexAlgorithms=+diffie-hellman-group-exchange-sha1 -o HostKeyAlgorithms=+ssh-rsa"

Tells SSH to use the old insecure algorithms that legacy devices support. Not ideal from a security perspective, but sometimes you’ve got no choice.

Terminal Weirdness

PyATS automatically runs commands when it connects:

• no logging console – stops syslog messages messing up your output
• terminal width 511 – prevents lines wrapping
• Various other terminal settings

Sometimes you don’t want this. Maybe you’re connecting to a device where these commands don’t work or cause problems. Disable them:

connections:
  cli:
    protocol: ssh
    ip: 172.16.1.10
    port: 22
    arguments:
      connection_timeout: 360
      init_exec_commands: []
      init_config_commands: []

Empty lists disable the automatic commands.

Timeout Issues

Default connection timeout is often too short. Corporate networks with multiple authentication servers, VPN tunnels, whatever – they take time. Always set a sensible timeout:

arguments:
  connection_timeout: 360

Six minutes. Should be enough for most things. If it takes longer than six minutes to authenticate, you’ve got bigger problems.

Validating Your Testbed Before You Go Mental

Before using your pyats testbed configuration, validate it. Trust me on this. Nothing worse than discovering syntax errors during important test runs.

YAML Syntax Check

First, basic YAML validation:

yamllint testbed.yaml

If yamllint doesn’t output anything, your YAML is syntactically correct. If it shows errors, fix them first.

PyATS Validation

PyATS has its own validation:

pyats validate testbed testbed.yaml

This checks more than just YAML syntax:

Loading testbed file: testbed.yaml
--------------------------------------------------------------------------------

Testbed Name: Manchester_Lab

Testbed Devices:
└── Lab-Router1 [iosxe/csr1000v]

Warning Messages
----------------
-Device 'Lab-Router1' has no interface definitions

The warning about missing interfaces is normal. You only need interface definitions if you’re testing connectivity between devices.

Actually Testing Connections

Best validation is trying to connect:

from pyats.topology import loader

testbed = loader.load('testbed.yaml')
device = testbed.devices['Lab-Router1']

try:
    device.connect()
    print("Connection successful!")
    device.disconnect()
except Exception as e:
    print(f"Failed: {e}")

If this works, your testbed’s properly configured.

Actually, let me tell you about the worst validation failure I’ve ever seen. A work chum of mine built a testbed for about 200 devices. Massive network. Spent weeks getting all the device details, IP addresses, credentials, everything.

Never tested it.

Deployed it to production. Nothing worked. Not a single device. Turns out he’d got the YAML indentation wrong for the entire devices section. Every single device was invalid.

Took two days to fix. Two bloody days. All because he didn’t spend five minutes validating his testbed first.

Don’t be that person.

Multiple Devices and Real Networks

Right, so you’ve got one device working. Good. Real networks have loads of devices. Here’s how to handle that without losing your sanity.

Multiple Device Testbed

---
devices:
  Manchester-Core-01:
    alias: 'Manchester Core Switch 1'
    type: 'switch'
    os: 'iosxe'
    platform: 'c9300'
    credentials:
      default:
        username: dickie1
        password: Manchester123!
    connections:
      cli:
        protocol: ssh
        ip: 172.16.1.20
        port: 22

  Manchester-Core-02:
    alias: 'Manchester Core Switch 2'
    type: 'switch'
    os: 'iosxe'
    platform: 'c9300'
    credentials:
      default:
        username: dickie1
        password: Manchester123!
    connections:
      cli:
        protocol: ssh
        ip: 172.16.1.21
        port: 22

  Liverpool-Router-01:
    alias: 'Liverpool Branch Router'
    type: 'router'
    os: 'iosxe'
    platform: 'isr4431'
    credentials:
      default:
        username: dickie1
        password: Liverpool456!
    connections:
      cli:
        protocol: ssh
        ip: 172.16.2.10
        port: 22

Notice how each device has unique details. Real networks aren’t uniform. Different locations, different device types, sometimes different credentials.

Device Groups

For complex networks, you can group devices:

devices:
  Manchester-Core-01:
    # ... device config ...
    groups: ['core', 'manchester']
    
  Manchester-Core-02:
    # ... device config ...
    groups: ['core', 'manchester']
    
  Liverpool-Router-01:
    # ... device config ...
    groups: ['branch', 'liverpool']

Then you can run tests against specific groups. Test all core devices, test all Manchester devices, whatever you need.

This is brilliant for large networks. Instead of running tests against everything, you can be selective.

Building Your pyATS Testbed Programmatically

Sometimes you need to create testbeds in code instead of YAML files. Especially useful when pulling data from databases or APIs.

Building in Code

from pyats.topology import Testbed, Device

# Create testbed
testbed = Testbed('Dynamic_Lab')

# Create device
device = Device(
    'Lab-Router1',
    os='iosxe',
    platform='csr1000v',
    type='router'
)

# Add credentials
device.credentials = {
    'default': {
        'username': 'dickie1',
        'password': 'Manchester123!'
    }
}

# Add connections
device.connections = {
    'cli': {
        'protocol': 'ssh',
        'ip': '172.16.1.10',
        'port': 22
    }
}

# Add to testbed
testbed.add_device(device)

This is handy when you’re pulling device info from databases or inventory systems.

From Database

import sqlite3
from pyats.topology import Testbed, Device

def create_testbed_from_db():
    testbed = Testbed('Production_Network')
    
    # Query devices from database
    conn = sqlite3.connect('network_inventory.db')
    cursor = conn.cursor()
    
    cursor.execute("""
        SELECT hostname, ip_address, os_type, platform, username, password 
        FROM devices 
        WHERE status = 'active'
    """)
    
    for row in cursor.fetchall():
        hostname, ip, os_type, platform, username, password = row
        
        device = Device(hostname, os=os_type, platform=platform)
        device.credentials = {
            'default': {
                'username': username,
                'password': password
            }
        }
        device.connections = {
            'cli': {
                'protocol': 'ssh',
                'ip': ip,
                'port': 22
            }
        }
        
        testbed.add_device(device)
    
    return testbed

Much better than manually maintaining YAML files when you’ve got hundreds of devices.

Debugging Connection Problems

When your pyats testbed configuration doesn’t work, here’s how to figure out what’s going wrong.

Start Simple

Before blaming pyATS, check basic connectivity:

ping 172.16.1.10
telnet 172.16.1.10 22

If these don’t work, fix your network first. No point debugging pyATS when your device isn’t even reachable.

Check Connection Logs

PyATS creates detailed logs. Look for files like connection_Lab-Router1.txt:

2024-07-27 14:03:06,027: %UNICON-INFO: +++ Lab-Router1 logfile ./connection_Lab-Router1.txt +++
2024-07-27 14:03:06,681: %UNICON-INFO: +++ connection to spawn: ssh -l dickie1 172.16.1.10 -p 22 +++
2024-07-27 14:03:06,681: %UNICON-INFO: connection to Lab-Router1 (dickie1@172.16.1.10) Password:
Lab-Router1#

This shows exactly what pyATS tried to do and where it succeeded or failed. Much better than guessing.

Common Failures

Authentication problems: Wrong username or password. Account locked. Check your credentials.

SSH negotiation failures: Legacy device with old SSH algorithms. Use ssh_options.

Timeout errors: Network’s slow, authentication takes time. Increase connection_timeout.

Parser errors: Wrong os or platform settings. pyATS doesn’t know how to parse your device output.

Interactive Testing

Test connections interactively:

from pyats.topology import loader

testbed = loader.load('testbed.yaml')
device = testbed.devices['Lab-Router1']

try:
    device.connect()
    print("Connected!")
    
    # Try a command
    output = device.execute('show version')
    print("Command worked!")
    
except Exception as e:
    print(f"Failed: {e}")
    
finally:
    if device.connected:
        device.disconnect()

Immediate feedback on what’s working and what isn’t.

I can’t tell you how many times I’ve seen people spend hours debugging scripts when the real problem was just a wrong IP address in their testbed. Start simple, test the basics first.

Production Best Practices

After building loads of pyats testbed configuration files for production networks, here’s what actually works:

Organise Your Files

Don’t put everything in one massive testbed. Split by:

• Environment (lab, staging, production)
• Location (manchester, liverpool, london)
• Function (core, access, dmz)

testbeds/
├── lab/
│   ├── manchester-lab.yaml
│   └── liverpool-lab.yaml
├── production/
│   ├── manchester-prod.yaml
│   └── liverpool-prod.yaml
└── templates/
    └── device-template.yaml

Much easier to manage.

Use Version Control

Testbeds are code. Treat them like code:

• Store in Git
• Use branches for changes
• Review testbed modifications
• Tag releases

I’ve seen too many networks where testbeds just lived on someone’s laptop. That person leaves the company and suddenly nobody knows how to connect to anything. Don’t be that organisation.

Automate Updates

Don’t manually update testbeds. Pull from your source of truth:

#!/bin/bash
# Update testbeds from inventory system
python3 scripts/generate_testbed.py --source cmdb --environment prod
git add testbeds/production/
git commit -m "Update prod testbeds from CMDB"

Monitor Testbed Health

Regularly check your testbeds still work:

# testbed_health_check.py
from pyats.topology import loader

def check_testbed(testbed_file):
    testbed = loader.load(testbed_file)
    failed = []
    
    for name, device in testbed.devices.items():
        try:
            device.connect(log_stdout=False)
            device.disconnect()
            print(f"✓ {name}")
        except Exception as e:
            print(f"✗ {name}: {e}")
            failed.append(name)
    
    return len(failed) == 0

Run this regularly to catch problems early. Nothing worse than discovering your testbed’s broken when you need to run an important test.

Working Across Environments

Real automation works across multiple environments. Here’s how to handle this sanely.

Environment Variables

Use environment variables for things that change:

devices:
  Lab-Router1:
    os: 'iosxe'
    platform: 'csr1000v'
    credentials:
      default:
        username: "%ENV{LAB_USERNAME}"
        password: "%ENV{LAB_PASSWORD}"
    connections:
      cli:
        protocol: ssh
        ip: "%ENV{LAB_ROUTER_IP}"
        port: 22

Different values for different environments:

# Lab
export LAB_USERNAME="labuser"
export LAB_PASSWORD="labpass"
export LAB_ROUTER_IP="172.16.1.10"

# Production
export LAB_USERNAME="produser"  
export LAB_PASSWORD="prodpass"
export LAB_ROUTER_IP="10.1.1.10"

Same testbed file, different environments.

Testbed Templates

Create templates and populate them:

# Template
devices:
  {{device_name}}:
    os: '{{device_os}}'
    platform: '{{device_platform}}'
    credentials:
      default:
        username: '{{username}}'
        password: '{{password}}'
    connections:
      cli:
        protocol: ssh
        ip: '{{device_ip}}'
        port: 22

Generate actual testbeds:

from jinja2 import Template

template = Template(open('template.yaml').read())

devices = [
    {
        'device_name': 'Manchester-Router-01',
        'device_os': 'iosxe',
        'device_platform': 'asr1000',
        'username': 'dickie1',
        'password': 'Manchester123!',
        'device_ip': '172.16.1.10'
    }
]

for device in devices:
    content = template.render(**device)
    with open(f"testbeds/{device['device_name']}.yaml", 'w') as f:
        f.write(content)

Scales much better than hand-editing hundreds of YAML files.

Final Thoughts (Don’t Mess This Up)

Look, getting your pyats testbed configuration right is absolutely critical. It’s the foundation everything else builds on. I’ve seen automation projects completely fail because people rushed through testbed creation then spent months fighting connection issues.

What you need to remember:

• YAML is picky – use spaces, be consistent, validate everything
• Test connections manually before assuming pyATS is broken
• Don’t hardcode credentials – use environment variables or secret management
• Organise testbeds sensibly – split by environment and function
• Use version control – testbeds are code
• Pull from existing inventory systems – don’t maintain duplicate data

Common mistakes I see constantly:

• Mixing tabs and spaces in YAML (drives you mental for hours)
• Wrong OS/platform settings (commands don’t parse properly)
• Hardcoded production passwords (security nightmare)
• Massive single testbed files (impossible to maintain)
• No validation before deployment (everything breaks in production)
• Hand-typing device lists (waste of time when you could import)

Don’t be like me when I started. I created a 500-device testbed by hand, typing every IP address, every credential. Took a week. Then spent days debugging connection issues because I’d made typing errors.

Learn from my stupidity. Start small, test everything, use the tools available. And whatever you do, don’t hardcode production passwords in Git repositories. I’ve seen it happen and it’s not pretty.

Start with one device. Get it working perfectly. Then expand gradually. Test every change. Use validation tools. And for crying out loud, don’t spend three days debugging a connection issue when the problem is a missing space in your YAML file.

Next we’ll look at mock devices so you can develop automation without needing real hardware. But master testbeds first. Everything depends on this foundation.

Actually, before you move on – quick tip. When debugging connection problems, always check the connection log files first. They show exactly what pyATS tried and where it failed. Saves hours of guessing.

Right, go build some proper testbeds. Stop making excuses and start doing it properly.

---

Learn more about YAML best practices at the official YAML specification site

---