task 7.1 · chapter 7 of 8: yaml, json and rest · 25 min
The inventory in YAML and JSON
objective
Write load_inventory(), save_json() and load_json() in netlab/inventory.py: the lab’s devices from inventory.yml, out to a JSON file and back, unchanged. Then find the four values at the top of inventory.yml that YAML reads differently from you, and quote them.
netlab/inventory.pyinventory.ymlopen a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter
| The inventory | inventory.yml: a few site settings, then devices:, a list of r1, r2, r3 and sw1 |
|---|---|
| Each device | five keys: name, host, device_type, role, model |
| The libraries | PyYAML 6.0 (import yaml) and the standard library’s json, both imported already |
| The rule | yaml.safe_load, never yaml.load: the check looks |
new to this? step by stepbeginner
cat inventory.yml, then in the REPL:import yaml,doc = yaml.safe_load(open("inventory.yml")),doc["devices"][0].- Write
load_inventory()(TODO 7.1a),save_json()(7.1b) andload_json()(7.1c), each with awith open(...). - Challenge (TODO 7.1d): print
docwithoutdevices, find the four values that aren’t what the file says, and quote them in inventory.yml. make check T=7.1.
more in meet the lab
Two spellings of one list
YAML and JSON
Both write down the same shapes: lists and dicts of strings, numbers, booleans and nothing. In YAML, key: value is a dict entry, - starts a list item, and indentation (spaces only) shows what’s inside what. JSON
writes the same thing with {}, [] and double quotes. Loaded into Python, they’re identical.
in one sentence Load a YAML or a JSON file, and you get the same lists and dicts.
safe_load
yaml.safe_load(f) builds only plain values. yaml.load() with PyYAML’s full loader can build any Python
object a file asks for, which means a YAML file can run code on your machine. An inventory needs only plain
values, so there’s never a reason to use anything else.
json.dump and json.load
json.dump(data, f, indent=2) writes Python values to an open file as JSON, indented so a person can read it;
json.load(f) reads them back. (With an s on the end, dumps and loads work on strings instead of files.)
What YAML guesses
YAML decides a value’s type from how it looks, and PyYAML follows YAML 1.1’s rules, which guess more than you’d expect: some words are booleans, a leading zero means octal, and colons can make a number. When a value has to stay a string, put it in quotes.
in one sentence If it must be a string, quote it.
One device, three ways
>>> import yaml, json
>>> doc = yaml.safe_load(open("inventory.yml"))
>>> doc.keys()
dict_keys(['site', 'country', 'rack', 'backup', 'devices'])
>>> doc["devices"][2]
{'name': 'r3', 'host': '172.80.111.13', 'device_type': 'arista_eos', 'role': 'router', 'model': 'CapeOS-Pulse'}
>>> print(json.dumps(doc["devices"][2], indent=2))
{
"name": "r3",
"host": "172.80.111.13",
"device_type": "arista_eos",
"role": "router",
"model": "CapeOS-Pulse"
}
And the starter’s check, before you’ve started: one assertion, and two functions still to write.
________________________ test_the_surprises_are_quoted _________________________
checks/test_7_1.py:13: in test_the_surprises_are_quoted
assert doc["country"] == "NO", f"country loads as {doc['country']!r}: Norway's code should stay the string 'NO'"
E AssertionError: country loads as False: Norway's code should stay the string 'NO'
E assert False == 'NO'
______________________________ test_lab_inventory ______________________________
TODO 7.1a is still open: write load_inventory() in netlab/inventory.py, then check again
____________________________ test_unseen_round_trip ____________________________
TODO 7.1a is still open: write load_inventory() in netlab/inventory.py, then check again
3 failed, 1 passed in 0.08s
Load, save, load again
Open
netlab/inventory.pyand find TODO 7.1a. The devices are the list under the file’s top-leveldevices:key:editnetlab/inventory.pyCtrl+P → inventory.pydef load_inventory(path): """...the docstring stays as it is...""" with open(path) as f: return yaml.safe_load(f)["devices"]TODO 7.1b writes the list to a JSON file, opened for writing with
"w":editnetlab/inventory.pyCtrl+P → inventory.pydef save_json(devices, path): with open(path, "w") as f: json.dump(devices, f, indent=2) f.write("\n")json.dumpdoesn’t end the file with a newline; the last line adds one, the way text files on Linux end.TODO 7.1c reads it back. That one’s yours:
with open(...), and thejsonfunction that reads from a file.Try the round trip in the REPL:
>>> from netlab.inventory import load_inventory, save_json, load_json >>> devices = load_inventory("inventory.yml") >>> save_json(devices, "scratch/inventory.json") >>> load_json("scratch/inventory.json") == devices TrueOpen
scratch/inventory.jsonin the editor: the same four devices, in JSON.
in one sentence YAML in, JSON out, JSON in: the same list of dicts all the way around.
Four values YAML reads its own way
TODO 7.1d is in inventory.yml itself. The block above devices: has four values that don’t load the way
they read. Load the file in the REPL, look at what each of country, rack and backup became, and quote
the four so YAML keeps them as the strings the comments describe. Don’t change what they say, only how
they’re written.
hint
Print everything except the devices: {k: v for k, v in doc.items() if k != "devices"}. Compare each value
with the line in the file. Look at the keys of backup as well as its values.
hint
Lesson 7 has a table of what YAML 1.1 does to NO, on, 0123 and 2:30, and why.
hint
A key can be quoted too: "on": sunday.
show the solutioninventory.yml
or pull it into your lab: make solve T=7.1, then git diff --staged shows what changed
site: lab
country: "NO" # Norway
rack: "0123" # the label on the rack
backup:
"on": sunday # which day the configs are backed up
at: "2:30" # and when
>>> yaml.safe_load("country: NO\nrack: 0123\nbackup:\n on: sunday\n at: 2:30\n")
{'country': False, 'rack': 83, 'backup': {True: 'sunday', 'at': 150}}
>>> yaml.safe_load('country: "NO"\nrack: "0123"\nbackup:\n "on": sunday\n at: "2:30"\n')
{'country': 'NO', 'rack': '0123', 'backup': {'on': 'sunday', 'at': '2:30'}}
NO and on are YAML 1.1 booleans, 0123 is octal (83), and 2:30 is base 60 (2×60 + 30 = 150). Quoting
makes each a string. And load_json():
def load_json(path):
with open(path) as f:
return json.load(f)
None of the four affects the devices, which is why the round trip passed before you fixed them. That’s the
danger: the file loads without an error, and the wrong value turns up later, in whatever reads country.
5 check: run it, then prove it
$ make check T=7.1when it's right, you'll see
task 7.1 checks/test_7_1.py
checks/test_7_1.py::test_the_surprises_are_quoted PASSED [ 25%]
checks/test_7_1.py::test_lab_inventory PASSED [ 50%]
checks/test_7_1.py::test_safe_load_only PASSED [ 75%]
checks/test_7_1.py::test_unseen_round_trip PASSED [100%]
============================== 4 passed in 0.10s ===============================
✓ task 7.1 done