task 8.3 · chapter 8 of 8: capstone: an inventory to csv · 25 min
Two options, two sources, one CSV
objective
Give scripts/inventory.py a real command line with argparse: –source yaml|api and –out FILE, with help and errors for free. Then show that the YAML file and the API give the same CSV.
scripts/inventory.pyopen a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter
| The function | parse_args() in scripts/inventory.py: the starter ignores the command line and returns the defaults |
|---|---|
| The options | –source yaml or api (default yaml), –out FILE (default inventory.csv) |
| The sources | load_devices(“yaml”) and load_devices(“api”) from task 8.1 |
| Before you start | r2 running again: make reset after task 8.2 |
new to this? step by stepbeginner
- Try argparse in the REPL with a parser of your own (the follow-along).
- In
scripts/inventory.py, writeparse_args()(TODO 8.3) withargparse.ArgumentParserand twoadd_argument()calls. python scripts/inventory.py --help, then--source api --out reports/from-api.csv, anddiffthe two CSVs.make check T=8.3.
more in meet the lab
A command line, not an edit
Options instead of editing the script
Right now, to read the API instead of the YAML, you’d have to change the code. A tool other people run takes
options on its command line instead, like every command you’ve used in a terminal: ls -l,
ping -c 3. main() already reads args.source and args.out. Your job is the function that fills them in.
What argparse does
argparse, in the standard library, reads the words after the script’s name (sys.argv), checks them
against the options you describe, and returns a Namespace: an object with one attribute per option.
| You write | You get |
|---|---|
ArgumentParser(description=...) | the parser, and the first line of --help |
add_argument("--name", default=...) | args.name, set to the default when the option isn’t given |
choices=[...] | anything else refused, with a usage line and exit code 2 |
help="..." | that option’s line in --help |
--help (and -h) is added for you. So are the error messages: a value not in choices, an option with its
value missing, or an option nobody defined all stop the script before any device is touched, with exit
code 2. That’s the same 2 as task 8.2’s partial failure; the message on stderr says which it was.
in one sentence Describe the options; argparse writes the parsing, the checking, the help and the errors.
Why parse_args takes argv
parse_args(argv=None) hands argv to the parser’s own parse_args(). None means “read the real command
line”. A list, such as ["--source", "api"], means “parse this instead”, which is how you try it in the REPL
and how the checks test it without a terminal.
The help argparse writes
Once parse_args() is done, this is all generated from your description and help strings:
$ python scripts/inventory.py --help
usage: inventory.py [-h] [--source {yaml,api}] [--out OUT]
Inventory every lab device to a CSV file.
options:
-h, --help show this help message and exit
--source {yaml,api} where the device list comes from (default: yaml, the
file inventory.yml)
--out OUT the CSV file to write (default: inventory.csv)
And a value it doesn’t allow:
$ python scripts/inventory.py --source xml
usage: inventory.py [-h] [--source {yaml,api}] [--out OUT]
inventory.py: error: argument --source: invalid choice: 'xml' (choose from 'yaml', 'api')
$ echo $?
2
A parser of your own, in the REPL
Start the REPL, and build a small parser for an imaginary tool that pings one lab device:
run in the terminalpython>>> import argparse >>> p = argparse.ArgumentParser(prog="ping-one", description="Ping one lab device.") >>> p.add_argument("--device", choices=["r1", "r2", "r3", "sw1"], default="r1", help="which device (default: r1)") >>> p.add_argument("--count", type=int, default=3, help="how many pings (default: 3)")Each
add_argument()prints a line describing the option it made. You can ignore it.progsets the name in the usage line; in a script, argparse takes it from the file’s name.Parse an empty command line, then a full one:
>>> p.parse_args([]) Namespace(device='r1', count=3) >>> args = p.parse_args(["--device", "r3", "--count", "5"]) >>> args.device, args.count ('r3', 5)countis the number 5, not the text"5":type=intconverted it. Withouttype, every value is a string.Print the help:
>>> p.print_help() usage: ping-one [-h] [--device {r1,r2,r3,sw1}] [--count COUNT] Ping one lab device. options: -h, --help show this help message and exit --device {r1,r2,r3,sw1} which device (default: r1) --count COUNT how many pings (default: 3)Give it a device that isn’t in
choices. argparse ends the program when it refuses input, so catch that here to keep the REPL open:>>> try: ... p.parse_args(["--device", "r7"]) ... except SystemExit as e: ... print("exit code", e.code) ... usage: ping-one [-h] [--device {r1,r2,r3,sw1}] [--count COUNT] ping-one: error: argument --device: invalid choice: 'r7' (choose from 'r1', 'r2', 'r3', 'sw1') exit code 2In a script you don’t catch it: the usage line and exit code 2 are exactly what you want the user to get.
in one sentence parse_args() returns a Namespace; on bad input it prints a usage line and exits with code 2.
–source and –out
Write parse_args() in scripts/inventory.py: --source, which is yaml or api and nothing else
(default yaml), and --out, the CSV file to write (default inventory.csv). Give the parser a description
and each option a help line. Then prove the two sources agree:
python scripts/inventory.py --help
python scripts/inventory.py
python scripts/inventory.py --source api --out reports/from-api.csv
diff inventory.csv reports/from-api.csv
diff prints nothing when two files are the same. (If a minute rolled over between the two runs, the uptime
column differs: run them again. The check ignores uptime for that reason.)
hint
p = argparse.ArgumentParser(description="..."), two p.add_argument(...) calls, and
return p.parse_args(argv). Pass argv on: that’s what lets the checks hand it a list.
hint
choices=["yaml", "api"] on --source gives you the refusal and exit code 2 for --source xml without
writing an if.
show the solutionscripts/inventory.py
or pull it into your lab: make solve T=8.3, then git diff --staged shows what changed
def parse_args(argv=None):
p = argparse.ArgumentParser(description="Inventory every lab device to a CSV file.")
p.add_argument("--source", choices=["yaml", "api"], default="yaml",
help="where the device list comes from (default: yaml, the file inventory.yml)")
p.add_argument("--out", default="inventory.csv", help="the CSV file to write (default: inventory.csv)")
return p.parse_args(argv)
$ python scripts/inventory.py --source api --out reports/from-api.csv
wrote reports/from-api.csv: 4 of 4 devices
$ diff inventory.csv reports/from-api.csv
The same four rows from either source, because api_to_inventory() in task 7.2 reshaped the API’s JSON into
the same five keys as the YAML, and everything after load_devices() works on that list. --out reports/...
works because write_csv() makes the folder. Your help text can say something else: the check looks for
--source and --out in --help, refuses --source xml with exit code 2, and compares the two CSVs.
When the check passes, that’s the lab. You started with len("GigabitEthernet1") at the REPL, and
you’ve finished with a tool that reads an inventory from a file or an API, logs in to every device over SSH, parses two CLIs into one shape,
writes a CSV, names the devices it couldn’t reach, exits with a code a pipeline can trust, and takes its
settings from the command line. On the way you wrote a parser for show ip interface brief by hand and with
a regex, a TextFSM template, a Device dataclass, and functions a test suite you didn’t write could call.
netlab/ is yours. Give inventory.yml other devices, each with its device_type, and their login in
.env, and the same script inventories them. The
capstone lesson covers what a framework such as
Nornir would add next: many devices at once, an inventory system, and a
result for every host.
Where to go from here:
- Ansible 101: tasks against an inventory without writing the loop. Python 101 is recommended first.
- NetBox: the inventory as a source of truth, with a REST API like the lab’s. Python 101 is required for it, and for network testing and ANTA.
5 check: run it, then prove it
$ make check T=8.3when it's right, you'll see
task 8.3 checks/test_8_3.py
checks/test_8_3.py::test_help PASSED [ 33%]
checks/test_8_3.py::test_bad_source PASSED [ 66%]
checks/test_8_3.py::test_both_sources PASSED [100%]
============================== 3 passed in 7.81s ===============================
✓ task 8.3 done