task 8.1 · chapter 8 of 8: capstone: an inventory to csv · 40 min
Every device, one CSV
objective
Write the three steps scripts/inventory.py runs: load the inventory, turn one device’s show version into a row, and write the rows with csv.DictWriter. Then make the row come out the same for r3, the Pulse router.
netlab/report.pyopen a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter
| The script | scripts/inventory.py: given. It calls your functions in netlab/report.py |
|---|---|
| From earlier chapters | connect() and DeviceError (6.1, 6.2), load_inventory() (7.1), get_devices() and api_to_inventory() (7.2) |
| The inventory | inventory.yml, or the API at LAB_API_URL with LAB_API_TOKEN (both in .env) |
| The output | inventory.csv: hostname, model, version, serial, uptime, ip_interfaces |
new to this? step by stepbeginner
- Starting the lab at chapter 8?
make skip CH=8solves chapters 1 to 7 first: this task calls their functions. - In
netlab/report.py, writeload_devices(),write_csv()andfacts()(TODO 8.1a, 8.1c, 8.1b). - Try
facts()in the REPL on r1. Challenge: make it work on r3 (arista_eos) too. python scripts/inventory.py,cat inventory.csv, thenmake check T=8.1.
more in meet the lab
Inventory in, one function per step, results out
The shape of the tool
scripts/inventory.py is given, and its main() is four lines of steps: read the options, load the devices,
get a row from each, write the CSV. The steps are functions in netlab/report.py, and in this task you write
three of them:
| Function | Takes | Gives back |
|---|---|---|
load_devices(source) | "yaml" or "api" | the inventory: a list of dicts, one per device |
facts(device) | one of those dicts | one row: a dict with the six keys in COLUMNS |
write_csv(rows, path) | a list of rows, a file name | nothing: it writes the file |
Most of the work is already done. Chapter 7 loads the inventory from YAML and from the API, and chapter 6 logs in. The new parts are shaping one row and writing the CSV.
in one sentence One function per step means you can try each one in the REPL before the script runs them all.
Two device types, one row
show version, parsed by ntc-templates, gives each device type different keys. r1, r2 and sw1 are
cisco_ios: the release is version, and the serial is serial, a list (a switch stack has one per
member). r3 is arista_eos: the release is image, the serial is serial_number, a string. Only uptime is
the same on both. Normalizing is mapping each type’s keys onto one set of column names, so the CSV doesn’t
care which device a row came from.
Two columns don’t come from show version at all. The hostname is conn.base_prompt, which Netmiko works
out from the prompt (r1# gives r1), so it’s there on every device. The model is in the template’s output,
but empty on these devices, so it comes from the inventory’s model.
in one sentence Pick the column names first, then find where each one is reliable on each device type.
csv.DictWriter
csv.DictWriter(f, fieldnames=COLUMNS) writes dicts as CSV rows: writeheader() writes the column names,
writerows(rows) writes every row, each value in the column its key names. A missing key gives an empty
field; a key that isn’t in fieldnames raises ValueError. Open the file with newline="", as the
csv documentation asks, because the module writes its own line
endings.
Two rows, two device types
facts() for r1 and for r3, in the REPL (devices is load_devices("yaml")):
>>> facts(devices[0])
{'hostname': 'r1', 'model': 'CapeOS-Bolt', 'version': '0.11.2', 'serial': 'CAPEHFI5RPW', 'uptime': '10 minutes', 'ip_interfaces': 5}
>>> facts(devices[2])
{'hostname': 'r3', 'model': 'CapeOS-Pulse', 'version': '4.99.0C', 'serial': 'CAPE4S5VG44', 'uptime': '10 minutes', 'ip_interfaces': 4}
And the whole script, with the CSV it wrote:
$ python scripts/inventory.py
wrote inventory.csv: 4 of 4 devices
$ cat inventory.csv
hostname,model,version,serial,uptime,ip_interfaces
r1,CapeOS-Bolt,0.11.2,CAPEHFI5RPW,0 minutes,5
r2,CapeOS-Bolt,0.11.2,CAPED9P9655,0 minutes,5
r3,CapeOS-Pulse,4.99.0C,CAPE4S5VG44,0 minutes,4
sw1,CapeOS-Bolt,0.11.2,CAPEEXIN1H1,0 minutes,2
Your serials are different: the lab makes new ones every time it starts.
Load, write, and a row for r1
Open
netlab/report.py. Each function has a docstring that says what it should do, and araise NotImplementedErrorline where your code goes. Replace that line in each, and keep the docstring.TODO 8.1a. The inventory, from either source. Chapter 7 wrote both halves:
editnetlab/report.pyCtrl+P → report.pydef load_devices(source="yaml"): if source == "api": load_env() return api_to_inventory(get_devices(os.environ["LAB_API_URL"], os.environ["LAB_API_TOKEN"])) return load_inventory(ROOT / "inventory.yml")load_env()reads.envinto the environment, soLAB_API_URLandLAB_API_TOKENare there to read.ROOTis the repository’s folder, so the script findsinventory.ymlfrom any folder you run it in.TODO 8.1c. The CSV:
editnetlab/report.pyCtrl+P → report.pydef write_csv(rows, path): path = pathlib.Path(path) path.parent.mkdir(parents=True, exist_ok=True) with path.open("w", newline="") as f: writer = csv.DictWriter(f, fieldnames=COLUMNS) writer.writeheader() writer.writerows(rows)mkdir(parents=True, exist_ok=True)makes the folder if it isn’t there, so--out reports/x.csvworks in task 8.3, and does nothing if it is.TODO 8.1b. One device’s row. First the login, with Netmiko’s two exceptions turned into
DeviceErrorascollect()did in task 6.2. Then the two commands, anddisconnect()in afinally:so the SSH session closes even if a command fails:editnetlab/report.pyCtrl+P → report.pydef facts(device): try: conn = connect(device["host"], device["device_type"]) except NetmikoAuthenticationException: raise DeviceError(f"{device['name']}: login failed") from None except NetmikoTimeoutException: raise DeviceError(f"{device['name']}: no answer from {device['host']}") from None try: version = conn.send_command("show version", use_textfsm=True)[0] brief = conn.send_command("show ip interface brief", use_textfsm=True) hostname = conn.base_prompt finally: conn.disconnect() release, serial = version["version"], version["serial"][0] return { "hostname": hostname, "model": device["model"], "version": release, "serial": serial, "uptime": version["uptime"], "ip_interfaces": sum(1 for i in brief if i["ip_address"] != "unassigned"), }[0]aftershow versiontakes the one dict out of the list ntc-templates returns.sum(1 for ...)counts the interfaces that have an address: one for each row that passes the test.Try it on r1, the first device in the inventory. Start the REPL from the repository’s folder:
run in the terminalpython>>> from netlab.report import load_devices, facts >>> devices = load_devices("yaml") >>> facts(devices[0])You get r1’s row, as in the Show stage. If you change
report.py,exit()and start the REPL again: Python reads a module once, when you import it.
in one sentence facts() logs in, runs two commands, disconnects, and returns a dict with the same six keys every time.
The same row from r3
devices[2] is r3, on CapeOS Pulse (arista_eos). Call facts(devices[2]): it stops with a KeyError.
Make facts() give r3 a row with the same six keys, then run the whole script and look at the CSV:
python scripts/inventory.py
cat inventory.csv
hint
Look at what r3’s parsed show version has. In the REPL:
>>> from netlab.devices import connect
>>> conn = connect("172.80.111.13", "arista_eos")
>>> conn.send_command("show version", use_textfsm=True)[0]
then conn.disconnect(). Which keys hold the release and the serial?
hint
The device’s own dict says which kind it is: device["device_type"]. An if on it can pick the keys.
hint
r3’s serial is already a string. Only the cisco_ios one needs [0].
show the solutionnetlab/report.py
or pull it into your lab: make solve T=8.1, then git diff --staged shows what changed
def facts(device):
try:
conn = connect(device["host"], device["device_type"])
except NetmikoAuthenticationException:
raise DeviceError(f"{device['name']}: login failed") from None
except NetmikoTimeoutException:
raise DeviceError(f"{device['name']}: no answer from {device['host']}") from None
try:
version = conn.send_command("show version", use_textfsm=True)[0]
brief = conn.send_command("show ip interface brief", use_textfsm=True)
hostname = conn.base_prompt
except DROPPED as e:
raise DeviceError(f"{device['name']}: the session broke ({type(e).__name__})") from None
finally:
try:
conn.disconnect()
except DROPPED:
pass # already gone: nothing left to close
if device["device_type"] == "arista_eos":
release, serial = version["image"], version["serial_number"]
else:
release, serial = version["version"], version["serial"][0]
return {
"hostname": hostname,
"model": device["model"],
"version": release,
"serial": serial,
"uptime": version["uptime"],
"ip_interfaces": sum(1 for i in brief if i["ip_address"] != "unassigned"),
}
DROPPED, at the top of report.py, is the lab’s list of what Netmiko raises when a device it had
already logged in to drops the session: a read that timed out, an SSH error, a closed socket. Catching it
here means one device that falls over halfway through a command becomes a failure for that device, not the
end of the run (task 8.2 is about that). The try around disconnect() is for the same case: there’s nothing
left to close.
This is r3’s show version with the cisco_ios key, and then what it does have:
>>> version["version"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'version'
>>> version
{'model': '', 'hw_version': '', 'serial_number': 'CAPE4S5VG44', 'sys_mac': 'cae2.b268.c68e', 'image': '4.99.0C', 'total_memory': '8025148', 'free_memory': '5733076', 'uptime': '15 minutes'}
The if is on the inventory’s device_type, the same field that told Netmiko which CLI to expect, so one
field picks both the dialect and the keys. The mapping lives in one place: a third device type later is one
more elif, and nothing after facts() changes. ip_interfaces needs no if, because both templates call
the address ip_address (r3’s includes the prefix, 10.0.13.2/30, which is still not unassigned).
5 check: run it, then prove it
$ make check T=8.1when it's right, you'll see
task 8.1 checks/test_8_1.py
checks/test_8_1.py::test_inventory_csv PASSED [100%]
============================== 1 passed in 4.84s ===============================
✓ task 8.1 done