online posts 16
python 101 for network engineers

task 8.2 · chapter 8 of 8: capstone: an inventory to csv · 25 min

A router that's down, a router that isn't there

objective

Stop r2 and add a router that was never built, then make run() carry on past both: every device that answered goes in the CSV, the ones that didn’t are listed with a reason, and the script exits 2.

you edit, in the labnetlab/report.py

open a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter

given
The problemmake break T=8.2: stops r2, and adds r9 (172.80.111.19, which doesn’t exist) to inventory.yml and the API
The functionrun() in netlab/report.py: the starter’s stops at the first device that fails
The exit codemain() in scripts/inventory.py already returns 2 when run() reports failures
Afterwardsmake reset starts r2 and takes r9 out again
new to this? step by stepbeginner
  1. make break T=8.2, then python scripts/inventory.py: it stops with a traceback at r2.
  2. In netlab/report.py, make run() (TODO 8.2) catch DeviceError for each device, remember the reason, and carry on.
  3. Run the script again, then echo $?: it should be 2.
  4. make check T=8.2, then make reset.

more in meet the lab

1 explain

Carry on, and say what you couldn’t do

One dead device shouldn’t cost you the others

A real inventory always has a device that’s down, being replaced, or listed by mistake. The starter’s run() calls facts() for each device in a plain loop. When one device fails, facts() raises DeviceError, nothing catches it, and the script stops: no CSV, even for the devices that had already answered, and the devices after it never tried.

Where you put the try decides what a failure costs. Around the whole loop, the first failure ends the run. Around one device, inside the loop, it costs that device’s row and nothing else.

in one sentence Put the try around the smallest piece of work you can lose: here, one device.

Catch only what you mean

facts() raises DeviceError on purpose, for the two failures that are the network’s fault: the login was refused, or the device didn’t answer. Catch that, by name. A typo in your own code raises something else, a KeyError or a NameError, and that should still stop the script with a traceback, because it’s a bug you need to see. except Exception: would hide it as one more “couldn’t inventory”.

Failures are a result too

run() returns two things, as a tuple: the rows, and a dict of device name to the reason. main() prints each reason on stderr, standard error, and returns an exit code: 0 when every device answered, 2 when some didn’t. The shell keeps that number in $?, and anything that runs the script (cron, a CI pipeline, && in a shell) reads it to know whether to trust the CSV.

in one sentence Partial results, a list of what’s missing, and an exit code that isn’t 0: that’s a tool that tells the truth about the network.

2 show

What the break does

$ make break T=8.2

  setting up task 8.2's problem

  docker    stop r2   (as if it lost power)
  wrote     r9 at the end of inventory.yml: a device that doesn't exist
  api       r9 in the API's devices too

  make reset puts everything back.

With the starter’s run(), the script gets as far as r2:

$ python scripts/inventory.py
Traceback (most recent call last):
  File "/workspaces/python-101/scripts/inventory.py", line 37, in <module>
    sys.exit(main())
             ^^^^^^
  File "/workspaces/python-101/scripts/inventory.py", line 28, in main
    rows, failures = run(devices)
                     ^^^^^^^^^^^^
  File "/workspaces/python-101/netlab/report.py", line 102, in run
    rows.append(facts(d))
                ^^^^^^^^
  File "/workspaces/python-101/netlab/report.py", line 55, in facts
    raise DeviceError(f"{device['name']}: no answer from {device['host']}") from None
netlab.devices.DeviceError: r2: no answer from 172.80.111.12
3 follow along

Read the traceback, find r9

  1. Set up the problem, and run the script:

    run in the terminal
    make break T=8.2
    python scripts/inventory.py
    
  2. Read the traceback from the bottom up. The last line says what: DeviceError, for r2, which didn’t answer. The lines above say where, innermost last: raised in facts(), called from run(), called from main(). Nothing in between caught it.

  3. Check that no CSV was written for the devices that did answer. r1 came before r2 in the inventory:

    run in the terminal
    ls inventory.csv
    

    If it’s there, it’s from task 8.1, not from this run: look at its time with ls -l.

  4. Look at the end of the inventory, where the break added r9:

    run in the terminal
    tail -8 inventory.yml
    

    r9 is at 172.80.111.19, an address nothing on the management network answers. To the script it looks exactly like r2: a timeout. A device that’s in the inventory and was never built is a real and common problem, and the inventory script is often how you find it.

in one sentence The traceback’s bottom line names the device; the line above it names the function that raised; the frames above that are who didn’t catch it.

4 challenge

Make run() carry on

Change run() so that it calls facts() for every device, keeps the rows of the ones that answered (in inventory order), and returns a dict of the ones that didn’t: device name to the DeviceError’s message. main() does the rest. Then:

run in the terminal
python scripts/inventory.py
echo $?

The script should write three of five devices, list r2 and r9 as failures, and print 2.

hint

Move a try: inside the for loop, around the one line that calls facts(d).

hint

except DeviceError as e: gives you the exception as e, and str(e) is its message, such as r2: no answer from 172.80.111.12.

hint

Start with failures = {} next to rows = [], and set failures[d["name"]] in the except.

show the solutionnetlab/report.py

or pull it into your lab: make solve T=8.2, then git diff --staged shows what changed

def run(devices):
    rows, failures = [], {}
    for d in devices:
        try:
            rows.append(facts(d))
        except DeviceError as e:
            failures[d["name"]] = str(e)
    return rows, failures
$ time python scripts/inventory.py
wrote inventory.csv: 3 of 5 devices
  couldn't inventory r2: no answer from 172.80.111.12
  couldn't inventory r9: no answer from 172.80.111.19
python scripts/inventory.py  0.59s user 0.10s system 6% cpu 11.285 total
$ cat inventory.csv
hostname,model,version,serial,uptime,ip_interfaces
r1,CapeOS-Bolt,0.11.2,CAPEHFI5RPW,0 minutes,5
r3,CapeOS-Pulse,4.99.0C,CAPE4S5VG44,0 minutes,4
sw1,CapeOS-Bolt,0.11.2,CAPEEXIN1H1,0 minutes,2

The try around one call to facts() means a failure costs that device and nothing more: the loop goes on to the next. Catching DeviceError and not Exception keeps your own bugs visible. The run took about 11 seconds, mostly the two 5-second timeouts, one for r2 and one for r9: the loop waits for each device in turn.

The check runs run() with a device of its own that doesn’t exist, then runs the whole script and checks the CSV, the failures on stderr and the exit code. It passes with the break in place or without it. Afterwards, put the lab back, because tasks 8.1 and 8.3 need r2:

run in the terminal
make reset

5 check: run it, then prove it

$ make check T=8.2

when it's right, you'll see

task 8.2  checks/test_8_2.py
checks/test_8_2.py::test_run_carries_on PASSED                           [ 50%]
checks/test_8_2.py::test_the_script PASSED                               [100%]
============================== 2 passed in 10.66s ==============================
✓ task 8.2 done