task 8.2 · chapter 8 of 8: capstone: an inventory to csv · 25 min
A router that's down, a router that isn't there
objective
Stop r2 and add a router that was never built, then make run() carry on past both: every device that answered goes in the CSV, the ones that didn’t are listed with a reason, and the script exits 2.
netlab/report.pyopen a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter
| The problem | make break T=8.2: stops r2, and adds r9 (172.80.111.19, which doesn’t exist) to inventory.yml and the API |
|---|---|
| The function | run() in netlab/report.py: the starter’s stops at the first device that fails |
| The exit code | main() in scripts/inventory.py already returns 2 when run() reports failures |
| Afterwards | make reset starts r2 and takes r9 out again |
new to this? step by stepbeginner
make break T=8.2, thenpython scripts/inventory.py: it stops with a traceback at r2.- In
netlab/report.py, makerun()(TODO 8.2) catchDeviceErrorfor each device, remember the reason, and carry on. - Run the script again, then
echo $?: it should be 2. make check T=8.2, thenmake reset.
more in meet the lab
Carry on, and say what you couldn’t do
One dead device shouldn’t cost you the others
A real inventory always has a device that’s down, being replaced, or listed by mistake. The starter’s run()
calls facts() for each device in a plain loop. When one device fails, facts() raises DeviceError,
nothing catches it, and the script stops: no CSV, even for the devices that had already answered, and the
devices after it never tried.
Where you put the try decides what a failure costs. Around the whole loop, the first failure ends the run.
Around one device, inside the loop, it costs that device’s row and nothing else.
in one sentence Put the try around the smallest piece of work you can lose: here, one device.
Catch only what you mean
facts() raises DeviceError on purpose, for the two failures that are the network’s fault: the login was
refused, or the device didn’t answer. Catch that, by name. A typo in your own code raises something else, a
KeyError or a NameError, and that should still stop the script with a traceback, because it’s a bug you
need to see. except Exception: would hide it as one more “couldn’t inventory”.
Failures are a result too
run() returns two things, as a tuple: the rows, and a dict of device name to the reason. main() prints
each reason on stderr, standard error, and returns an exit code: 0 when every device answered, 2 when
some didn’t. The shell keeps that number in $?, and anything that runs the script (cron, a CI pipeline,
&& in a shell) reads it to know whether to trust the CSV.
in one sentence Partial results, a list of what’s missing, and an exit code that isn’t 0: that’s a tool that tells the truth about the network.
What the break does
$ make break T=8.2
setting up task 8.2's problem
docker stop r2 (as if it lost power)
wrote r9 at the end of inventory.yml: a device that doesn't exist
api r9 in the API's devices too
make reset puts everything back.
With the starter’s run(), the script gets as far as r2:
$ python scripts/inventory.py
Traceback (most recent call last):
File "/workspaces/python-101/scripts/inventory.py", line 37, in <module>
sys.exit(main())
^^^^^^
File "/workspaces/python-101/scripts/inventory.py", line 28, in main
rows, failures = run(devices)
^^^^^^^^^^^^
File "/workspaces/python-101/netlab/report.py", line 102, in run
rows.append(facts(d))
^^^^^^^^
File "/workspaces/python-101/netlab/report.py", line 55, in facts
raise DeviceError(f"{device['name']}: no answer from {device['host']}") from None
netlab.devices.DeviceError: r2: no answer from 172.80.111.12
Read the traceback, find r9
Set up the problem, and run the script:
run in the terminalmake break T=8.2 python scripts/inventory.pyRead the traceback from the bottom up. The last line says what:
DeviceError, for r2, which didn’t answer. The lines above say where, innermost last: raised infacts(), called fromrun(), called frommain(). Nothing in between caught it.Check that no CSV was written for the devices that did answer. r1 came before r2 in the inventory:
run in the terminalls inventory.csvIf it’s there, it’s from task 8.1, not from this run: look at its time with
ls -l.Look at the end of the inventory, where the break added r9:
run in the terminaltail -8 inventory.ymlr9 is at
172.80.111.19, an address nothing on the management network answers. To the script it looks exactly like r2: a timeout. A device that’s in the inventory and was never built is a real and common problem, and the inventory script is often how you find it.
in one sentence The traceback’s bottom line names the device; the line above it names the function that raised; the frames above that are who didn’t catch it.
Make run() carry on
Change run() so that it calls facts() for every device, keeps the rows of the ones that answered (in
inventory order), and returns a dict of the ones that didn’t: device name to the DeviceError’s message.
main() does the rest. Then:
python scripts/inventory.py
echo $?
The script should write three of five devices, list r2 and r9 as failures, and print 2.
hint
Move a try: inside the for loop, around the one line that calls facts(d).
hint
except DeviceError as e: gives you the exception as e, and str(e) is its message, such as
r2: no answer from 172.80.111.12.
hint
Start with failures = {} next to rows = [], and set failures[d["name"]] in the except.
show the solutionnetlab/report.py
or pull it into your lab: make solve T=8.2, then git diff --staged shows what changed
def run(devices):
rows, failures = [], {}
for d in devices:
try:
rows.append(facts(d))
except DeviceError as e:
failures[d["name"]] = str(e)
return rows, failures
$ time python scripts/inventory.py
wrote inventory.csv: 3 of 5 devices
couldn't inventory r2: no answer from 172.80.111.12
couldn't inventory r9: no answer from 172.80.111.19
python scripts/inventory.py 0.59s user 0.10s system 6% cpu 11.285 total
$ cat inventory.csv
hostname,model,version,serial,uptime,ip_interfaces
r1,CapeOS-Bolt,0.11.2,CAPEHFI5RPW,0 minutes,5
r3,CapeOS-Pulse,4.99.0C,CAPE4S5VG44,0 minutes,4
sw1,CapeOS-Bolt,0.11.2,CAPEEXIN1H1,0 minutes,2
The try around one call to facts() means a failure costs that device and nothing more: the loop goes on to
the next. Catching DeviceError and not Exception keeps your own bugs visible. The run took about 11
seconds, mostly the two 5-second timeouts, one for r2 and one for r9: the loop waits for each device in turn.
The check runs run() with a device of its own that doesn’t exist, then runs the whole script and checks the
CSV, the failures on stderr and the exit code. It passes with the break in place or without it. Afterwards,
put the lab back, because tasks 8.1 and 8.3 need r2:
make reset
5 check: run it, then prove it
$ make check T=8.2when it's right, you'll see
task 8.2 checks/test_8_2.py
checks/test_8_2.py::test_run_carries_on PASSED [ 50%]
checks/test_8_2.py::test_the_script PASSED [100%]
============================== 2 passed in 10.66s ==============================
✓ task 8.2 done