online posts 16
python 101 for network engineers

task 8.3 · chapter 8 of 8: capstone: an inventory to csv · 25 min

Two options, two sources, one CSV

objective

Give scripts/inventory.py a real command line with argparse: –source yaml|api and –out FILE, with help and errors for free. Then show that the YAML file and the API give the same CSV.

you edit, in the labscripts/inventory.py

open a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter

given
The functionparse_args() in scripts/inventory.py: the starter ignores the command line and returns the defaults
The options–source yaml or api (default yaml), –out FILE (default inventory.csv)
The sourcesload_devices(“yaml”) and load_devices(“api”) from task 8.1
Before you startr2 running again: make reset after task 8.2
new to this? step by stepbeginner
  1. Try argparse in the REPL with a parser of your own (the follow-along).
  2. In scripts/inventory.py, write parse_args() (TODO 8.3) with argparse.ArgumentParser and two add_argument() calls.
  3. python scripts/inventory.py --help, then --source api --out reports/from-api.csv, and diff the two CSVs.
  4. make check T=8.3.

more in meet the lab

1 explain

A command line, not an edit

Options instead of editing the script

Right now, to read the API instead of the YAML, you’d have to change the code. A tool other people run takes options on its command line instead, like every command you’ve used in a terminal: ls -l, ping -c 3. main() already reads args.source and args.out. Your job is the function that fills them in.

What argparse does

argparse, in the standard library, reads the words after the script’s name (sys.argv), checks them against the options you describe, and returns a Namespace: an object with one attribute per option.

You writeYou get
ArgumentParser(description=...)the parser, and the first line of --help
add_argument("--name", default=...)args.name, set to the default when the option isn’t given
choices=[...]anything else refused, with a usage line and exit code 2
help="..."that option’s line in --help

--help (and -h) is added for you. So are the error messages: a value not in choices, an option with its value missing, or an option nobody defined all stop the script before any device is touched, with exit code 2. That’s the same 2 as task 8.2’s partial failure; the message on stderr says which it was.

in one sentence Describe the options; argparse writes the parsing, the checking, the help and the errors.

Why parse_args takes argv

parse_args(argv=None) hands argv to the parser’s own parse_args(). None means “read the real command line”. A list, such as ["--source", "api"], means “parse this instead”, which is how you try it in the REPL and how the checks test it without a terminal.

2 show

The help argparse writes

Once parse_args() is done, this is all generated from your description and help strings:

$ python scripts/inventory.py --help
usage: inventory.py [-h] [--source {yaml,api}] [--out OUT]

Inventory every lab device to a CSV file.

options:
  -h, --help           show this help message and exit
  --source {yaml,api}  where the device list comes from (default: yaml, the
                       file inventory.yml)
  --out OUT            the CSV file to write (default: inventory.csv)

And a value it doesn’t allow:

$ python scripts/inventory.py --source xml
usage: inventory.py [-h] [--source {yaml,api}] [--out OUT]
inventory.py: error: argument --source: invalid choice: 'xml' (choose from 'yaml', 'api')
$ echo $?
2
3 follow along

A parser of your own, in the REPL

  1. Start the REPL, and build a small parser for an imaginary tool that pings one lab device:

    run in the terminal
    python
    
    >>> import argparse
    >>> p = argparse.ArgumentParser(prog="ping-one", description="Ping one lab device.")
    >>> p.add_argument("--device", choices=["r1", "r2", "r3", "sw1"], default="r1", help="which device (default: r1)")
    >>> p.add_argument("--count", type=int, default=3, help="how many pings (default: 3)")
    

    Each add_argument() prints a line describing the option it made. You can ignore it. prog sets the name in the usage line; in a script, argparse takes it from the file’s name.

  2. Parse an empty command line, then a full one:

    >>> p.parse_args([])
    Namespace(device='r1', count=3)
    >>> args = p.parse_args(["--device", "r3", "--count", "5"])
    >>> args.device, args.count
    ('r3', 5)
    

    count is the number 5, not the text "5": type=int converted it. Without type, every value is a string.

  3. Print the help:

    >>> p.print_help()
    usage: ping-one [-h] [--device {r1,r2,r3,sw1}] [--count COUNT]
    
    Ping one lab device.
    
    options:
      -h, --help            show this help message and exit
      --device {r1,r2,r3,sw1}
                            which device (default: r1)
      --count COUNT         how many pings (default: 3)
    
  4. Give it a device that isn’t in choices. argparse ends the program when it refuses input, so catch that here to keep the REPL open:

    >>> try:
    ...     p.parse_args(["--device", "r7"])
    ... except SystemExit as e:
    ...     print("exit code", e.code)
    ...
    usage: ping-one [-h] [--device {r1,r2,r3,sw1}] [--count COUNT]
    ping-one: error: argument --device: invalid choice: 'r7' (choose from 'r1', 'r2', 'r3', 'sw1')
    exit code 2
    

    In a script you don’t catch it: the usage line and exit code 2 are exactly what you want the user to get.

in one sentence parse_args() returns a Namespace; on bad input it prints a usage line and exits with code 2.

4 challenge

–source and –out

Write parse_args() in scripts/inventory.py: --source, which is yaml or api and nothing else (default yaml), and --out, the CSV file to write (default inventory.csv). Give the parser a description and each option a help line. Then prove the two sources agree:

run in the terminal
python scripts/inventory.py --help
python scripts/inventory.py
python scripts/inventory.py --source api --out reports/from-api.csv
diff inventory.csv reports/from-api.csv

diff prints nothing when two files are the same. (If a minute rolled over between the two runs, the uptime column differs: run them again. The check ignores uptime for that reason.)

hint

p = argparse.ArgumentParser(description="..."), two p.add_argument(...) calls, and return p.parse_args(argv). Pass argv on: that’s what lets the checks hand it a list.

hint

choices=["yaml", "api"] on --source gives you the refusal and exit code 2 for --source xml without writing an if.

show the solutionscripts/inventory.py

or pull it into your lab: make solve T=8.3, then git diff --staged shows what changed

def parse_args(argv=None):
    p = argparse.ArgumentParser(description="Inventory every lab device to a CSV file.")
    p.add_argument("--source", choices=["yaml", "api"], default="yaml",
                   help="where the device list comes from (default: yaml, the file inventory.yml)")
    p.add_argument("--out", default="inventory.csv", help="the CSV file to write (default: inventory.csv)")
    return p.parse_args(argv)
$ python scripts/inventory.py --source api --out reports/from-api.csv
wrote reports/from-api.csv: 4 of 4 devices
$ diff inventory.csv reports/from-api.csv

The same four rows from either source, because api_to_inventory() in task 7.2 reshaped the API’s JSON into the same five keys as the YAML, and everything after load_devices() works on that list. --out reports/... works because write_csv() makes the folder. Your help text can say something else: the check looks for --source and --out in --help, refuses --source xml with exit code 2, and compares the two CSVs.

When the check passes, that’s the lab. You started with len("GigabitEthernet1") at the REPL, and you’ve finished with a tool that reads an inventory from a file or an API, logs in to every device over SSH, parses two CLIs into one shape, writes a CSV, names the devices it couldn’t reach, exits with a code a pipeline can trust, and takes its settings from the command line. On the way you wrote a parser for show ip interface brief by hand and with a regex, a TextFSM template, a Device dataclass, and functions a test suite you didn’t write could call.

netlab/ is yours. Give inventory.yml other devices, each with its device_type, and their login in .env, and the same script inventories them. The capstone lesson covers what a framework such as Nornir would add next: many devices at once, an inventory system, and a result for every host.

Where to go from here:

  • Ansible 101: tasks against an inventory without writing the loop. Python 101 is recommended first.
  • NetBox: the inventory as a source of truth, with a REST API like the lab’s. Python 101 is required for it, and for network testing and ANTA.

5 check: run it, then prove it

$ make check T=8.3

when it's right, you'll see

task 8.3  checks/test_8_3.py
checks/test_8_3.py::test_help PASSED                                     [ 33%]
checks/test_8_3.py::test_bad_source PASSED                               [ 66%]
checks/test_8_3.py::test_both_sources PASSED                             [100%]
============================== 3 passed in 7.81s ===============================
✓ task 8.3 done