online posts 16
python 101 for network engineers

task 4.1 · chapter 4 of 8: files, modules and errors · 30 min

Saved configs to a report

objective

Read every saved config in configs/, find out which devices have an NTP server, and write the answer to reports/ntp.csv, with the functions in a module and a given script that calls them.

you edit, in the labnetlab/configs.py

open a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter

given
The configsconfigs/r1.cfg, r2.cfg, r3.cfg and sw1.cfg: each device’s saved running-config
Your modulenetlab/configs.py: has_ntp() (TODO 4.1a), load_configs() (4.1b), write_report() (4.1c)
The scriptscripts/ntp_report.py, given: python scripts/ntp_report.py writes reports/ntp.csv
Try itpython at the top of the repo, then from netlab.configs import has_ntp
new to this? step by stepbeginner
  1. Open configs/r1.cfg and find its ntp server lines. Open netlab/configs.py and find TODO 4.1a.
  2. Write has_ntp() and load_configs() from the follow-along, trying each in the REPL.
  3. Challenge: write write_report() with the csv module.
  4. python scripts/ntp_report.py, open reports/ntp.csv, then make check T=4.1.

more in meet the lab

1 explain

A module, a script, and a folder of files

Reading a file

pathlib.Path names a file, and read_text() reads all of it into one string:

from pathlib import Path
text = Path("configs/r1.cfg").read_text(encoding="utf-8")

Path("configs").glob("*.cfg") finds every .cfg file in the folder. It returns them in no particular order, so wrap it in sorted(). Each one is a Path, and path.stem is the name without .cfg: in this lab, the device’s name.

in one sentence sorted(Path(folder).glob("*.cfg")) is every config, in order; path.stem is the device.

Writing one

open(path, "w") opens a file for writing, and with closes it again when the block ends, whatever happens inside it. A CSV file goes through csv.writer, which takes one list per row and gets the commas and quoting right. Open it with newline="", as the csv module asks.

Module and script

The functions go in a module, netlab/configs.py, so the checker and other scripts can import them and call them on any folder. scripts/ntp_report.py is given: it picks the folder and the output file, calls your functions, and prints one line. It’s already written for task 4.2, so it calls load_configs_safe(), which until 4.2 calls your load_configs().

in one sentence Functions in netlab/, a thin script in scripts/ on top: the script runs, the functions get tested.

2 show

Before and after

Before any TODO is written, the script stops at the first one it reaches. Read the traceback from the bottom: the last line says which TODO, and the frame above it says where.

$ python scripts/ntp_report.py
Traceback (most recent call last):
  File "/workspaces/python-101/scripts/ntp_report.py", line 29, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/workspaces/python-101/scripts/ntp_report.py", line 20, in main
    configs, problems = load_configs_safe(folder)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/python-101/netlab/configs.py", line 48, in load_configs_safe
    return load_configs(folder), {}
           ^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/python-101/netlab/configs.py", line 22, in load_configs
    raise NotImplementedError("TODO 4.1b is still open")
NotImplementedError: TODO 4.1b is still open

And with the three functions written:

$ python scripts/ntp_report.py
wrote /workspaces/python-101/reports/ntp.csv (4 devices)
$ cat reports/ntp.csv
device,has_ntp
r1,yes
r2,no
r3,yes
sw1,yes

r2 has ntp master 3 instead: it’s the clock the others use, at 172.80.111.12, not a client of anyone.

3 follow along

has_ntp() and load_configs()

  1. Look at the data first. In the REPL, at the top of the repository:

    run in the terminal
    python
    
    >>> from pathlib import Path
    >>> text = Path("configs/r1.cfg").read_text(encoding="utf-8")
    >>> text.splitlines()[:3]
    ['Building configuration...', '', 'Current configuration : 741 bytes']
    >>> [line for line in text.splitlines() if line.startswith("ntp server")]
    ['ntp server 172.80.111.12', 'ntp server 172.80.111.99']
    

    The first lines are CapeOS’s, from saving show running-config. They don’t matter: you’re looking for lines that start with ntp server.

  2. Write has_ntp(). True when any line starts with ntp server, otherwise False:

    editnetlab/configs.pyCtrl+P → configs.py
    def has_ntp(config_text):
        """True when the config has at least one line that starts with "ntp server", otherwise False."""
        for line in config_text.splitlines():
            if line.startswith("ntp server"):
                return True
        return False
    

    startswith on each line, rather than "ntp server" in config_text, matters: a comment like ! ntp server 10.0.0.1 or an indented line inside another block isn’t a configured server, and the check tries both.

  3. Write load_configs(): every .cfg file in the folder, as a dict of device name to text:

    editnetlab/configs.pyCtrl+P → configs.py
    def load_configs(folder):
        """Every *.cfg file in the folder, as a dict of device name -> the file's text, sorted by name.
    
        Other files (a README, a .txt) are ignored. Read the files as UTF-8.
        """
        configs = {}
        for path in sorted(pathlib.Path(folder).glob("*.cfg")):
            configs[path.stem] = path.read_text(encoding="utf-8")
        return configs
    

    pathlib.Path(folder) works whether you’re given a string, "configs", or a Path already. The module imports pathlib at the top, so the name is pathlib.Path here.

  4. Try both. Leave the REPL and start a new one, so it imports your saved file (a module is imported once per session):

    >>> from netlab.configs import has_ntp, load_configs
    >>> configs = load_configs("configs")
    >>> {name: has_ntp(text) for name, text in configs.items()}
    {'r1': True, 'r2': False, 'r3': True, 'sw1': True}
    

in one sentence Read every file once into a dict; then any question about the configs is a loop over the dict.

4 challenge

write_report()

Write write_report(configs, out_path): a CSV file with the header device,has_ntp, then one row per device, sorted by name, with yes or no. Create the folder the file goes in if it isn’t there yet (reports/ isn’t, in a new codespace). Then run python scripts/ntp_report.py and open reports/ntp.csv.

hint

pathlib.Path(out_path).parent is the folder. .mkdir(parents=True, exist_ok=True) makes it, and does nothing if it’s already there.

hint

with path.open("w", newline="") as f:, then writer = csv.writer(f) and writer.writerow([...]) once for the header and once per device.

hint

sorted(configs) gives the device names in order. "yes" if has_ntp(configs[name]) else "no" is the second column.

show the solutionnetlab/configs.py

or pull it into your lab: make solve T=4.1, then git diff --staged shows what changed

def write_report(configs, out_path):
    """Write the NTP report as a CSV file with a header row and one row per device, sorted by name:

        device,has_ntp
        r1,yes
        r2,no

    Create the folder the file goes in if it isn't there yet.
    """
    out_path = pathlib.Path(out_path)
    out_path.parent.mkdir(parents=True, exist_ok=True)
    with out_path.open("w", newline="") as f:
        writer = csv.writer(f)
        writer.writerow(["device", "has_ntp"])
        for name in sorted(configs):
            writer.writerow([name, "yes" if has_ntp(configs[name]) else "no"])

The function calls has_ntp() rather than repeating its loop: one place decides what “has NTP” means. The check runs the script on a folder of six made-up configs it writes fresh each time, with a README.txt that mentions ntp server too, so load_configs() must only read *.cfg, and the rows must come out sorted.

5 check: run it, then prove it

$ make check T=4.1

when it's right, you'll see

task 4.1  checks/test_4_1.py
checks/test_4_1.py::test_has_ntp PASSED                                  [ 25%]
checks/test_4_1.py::test_load_configs PASSED                             [ 50%]
checks/test_4_1.py::test_the_script PASSED                               [ 75%]
checks/test_4_1.py::test_the_lab_folder PASSED                           [100%]
============================== 4 passed in 0.04s ===============================
✓ task 4.1 done