<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>Python 101 for network engineers · Network Doodles</title><link>https://networkdoodles.com/lessons/python-101/</link><description>Enough Python to automate a network: run code and keep packages in a venv, turn show output into lists and dicts, read files and survive bad ones, parse with regex and TextFSM, log in with Netmiko, read YAML, JSON and a REST API, and ship a small inventory tool. Eight lessons, each with a hands-on lab chapter.</description><language>en-us</language><managingEditor>David Henderson</managingEditor><lastBuildDate>Mon, 16 Nov 2026 09:00:00 -0500</lastBuildDate><atom:link href="https://networkdoodles.com/lessons/python-101/index.xml" rel="self" type="application/rss+xml"/><item><title>Capstone: an Inventory to CSV (Python 101, Lesson 8)</title><link>https://networkdoodles.com/lessons/python-101/inventory-to-csv/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/inventory-to-csv/</guid><category>python</category><category>network-automation</category><category>howto</category><category>capeos</category><category>netmiko</category><description>Build a small network tool in Python: inventory in, Netmiko and TextFSM per device, csv.DictWriter out, carry on past failed devices, exit codes and argparse.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p>A useful network script has the same shape whatever it does: it reads an inventory, does one job per device,
and writes the results where a person or another tool can use them. Give each step its own function, catch
failures <strong>per device</strong> so one dead router doesn&rsquo;t stop the rest, and finish with an <strong>exit code</strong> that says
whether everything worked. The capstone puts that shape around lessons 2 to 7, and ends with a CSV of every
device&rsquo;s version and serial number.</p>
<p>The tool is <code>scripts/inventory.py</code> in the <a href="/labs/python-101/">lab</a>. This lesson walks through the decisions in
it: three small functions instead of one long script, two device types&rsquo; <code>show version</code> in one row,
<code>csv.DictWriter</code>, what happens when r2 is switched off, and two options with <code>argparse</code>. Every output below is
from the lab: Python 3.11, Netmiko 4.8 and ntc-templates 9.3, against CapeOS.</p>
<hr>
<h2 id="what-shape-should-a-small-network-tool-have">What shape should a small network tool have?</h2>
<p>Inventory in, one function per step, results out. This is the script&rsquo;s <code>main()</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">main</span><span class="p">(</span><span class="n">argv</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">args</span> <span class="o">=</span> <span class="n">parse_args</span><span class="p">(</span><span class="n">argv</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">devices</span> <span class="o">=</span> <span class="n">load_devices</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">source</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">rows</span><span class="p">,</span> <span class="n">failures</span> <span class="o">=</span> <span class="n">run</span><span class="p">(</span><span class="n">devices</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">write_csv</span><span class="p">(</span><span class="n">rows</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">out</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;wrote </span><span class="si">{</span><span class="n">args</span><span class="o">.</span><span class="n">out</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">)</span><span class="si">}</span><span class="s2"> of </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">devices</span><span class="p">)</span><span class="si">}</span><span class="s2"> devices&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">reason</span> <span class="ow">in</span> <span class="n">failures</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span></span><span class="line"><span class="cl">        <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;  couldn&#39;t inventory </span><span class="si">{</span><span class="n">reason</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="mi">2</span> <span class="k">if</span> <span class="n">failures</span> <span class="k">else</span> <span class="mi">0</span>
</span></span></code></pre></div><p>It reads as a list of steps, and each step is a function in <code>netlab/report.py</code> or the script itself:</p>
<table>
  <thead>
      <tr>
          <th>Function</th>
          <th>Takes</th>
          <th>Gives back</th>
          <th>Network job</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>parse_args()</code></td>
          <td>the command line</td>
          <td>which source, which file</td>
          <td>none: the tool&rsquo;s settings</td>
      </tr>
      <tr>
          <td><code>load_devices(source)</code></td>
          <td><code>&quot;yaml&quot;</code> or <code>&quot;api&quot;</code></td>
          <td>a list of device dicts</td>
          <td>read the inventory (lesson 7)</td>
      </tr>
      <tr>
          <td><code>facts(device)</code></td>
          <td>one device dict</td>
          <td>one row, as a dict</td>
          <td>log in, <code>show version</code>, <code>show ip interface brief</code></td>
      </tr>
      <tr>
          <td><code>run(devices)</code></td>
          <td>the device list</td>
          <td>the rows, and the failures</td>
          <td><code>facts()</code> for every device</td>
      </tr>
      <tr>
          <td><code>write_csv(rows, path)</code></td>
          <td>the rows, a file name</td>
          <td>nothing: it writes the file</td>
          <td>the report</td>
      </tr>
  </tbody>
</table>
<p>Why not one long script? Because a function with one job can be tried on its own. You can call <code>facts()</code> on
r3 in the REPL and look at the dict before any CSV exists, and the lab&rsquo;s checker does the same: it calls
<code>run()</code> with a device list it made up, including a device that doesn&rsquo;t exist. A script with no functions can
only be tested by running all of it against the network.</p>
<p>Each piece is also replaceable. When the inventory moves from a YAML file to an API, only <code>load_devices()</code>
changes. It&rsquo;s the same idea that makes <a href="/posts/netboxasinventory/">NetBox as your Ansible inventory</a> work: the
tool asks for devices, not for a particular file.</p>
<hr>
<h2 id="why-doesnt-show-version-give-the-same-keys-on-every-device">Why doesn&rsquo;t show version give the same keys on every device?</h2>
<p>Because ntc-templates has one template per platform and command, each written for one CLI, and each names its
fields after what that CLI prints. Here are r1 (CapeOS Bolt, which Netmiko knows as <code>cisco_ios</code>) and r3
(CapeOS Pulse, <code>arista_eos</code>), parsed with <code>parse_output()</code> from <a href="/lessons/python-101/regex-and-textfsm/">lesson 5</a>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show version&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/version-r1.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">())</span>
</span></span><span class="line"><span class="cl"><span class="p">[{</span><span class="s1">&#39;software_image&#39;</span><span class="p">:</span> <span class="s1">&#39;CO-BOLT&#39;</span><span class="p">,</span> <span class="s1">&#39;version&#39;</span><span class="p">:</span> <span class="s1">&#39;0.11.2&#39;</span><span class="p">,</span> <span class="s1">&#39;release&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;rommon&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;hostname&#39;</span><span class="p">:</span> <span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime&#39;</span><span class="p">:</span> <span class="s1">&#39;0 minutes&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime_years&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime_weeks&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime_days&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime_hours&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime_minutes&#39;</span><span class="p">:</span> <span class="s1">&#39;0&#39;</span><span class="p">,</span> <span class="s1">&#39;reload_reason&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;running_image&#39;</span><span class="p">:</span> <span class="s1">&#39;capeos-bolt.bin&#39;</span><span class="p">,</span> <span class="s1">&#39;hardware&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;serial&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;CAPEHFI5RPW&#39;</span><span class="p">],</span> <span class="s1">&#39;config_register&#39;</span><span class="p">:</span> <span class="s1">&#39;0x2102&#39;</span><span class="p">,</span> <span class="s1">&#39;mac_address&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;restarted&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">}]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;arista_eos&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show version&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/version-r3-pulse.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">())</span>
</span></span><span class="line"><span class="cl"><span class="p">[{</span><span class="s1">&#39;model&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;hw_version&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;serial_number&#39;</span><span class="p">:</span> <span class="s1">&#39;CAPE4S5VG44&#39;</span><span class="p">,</span> <span class="s1">&#39;sys_mac&#39;</span><span class="p">:</span> <span class="s1">&#39;cae2.b268.c68e&#39;</span><span class="p">,</span> <span class="s1">&#39;image&#39;</span><span class="p">:</span> <span class="s1">&#39;4.99.0C&#39;</span><span class="p">,</span> <span class="s1">&#39;total_memory&#39;</span><span class="p">:</span> <span class="s1">&#39;8025148&#39;</span><span class="p">,</span> <span class="s1">&#39;free_memory&#39;</span><span class="p">:</span> <span class="s1">&#39;6362980&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime&#39;</span><span class="p">:</span> <span class="s1">&#39;0 minutes&#39;</span><span class="p">}]</span>
</span></span></code></pre></div><p>Line up what the CSV wants, and only one key is the same on both:</p>
<table>
  <thead>
      <tr>
          <th>CSV column</th>
          <th><code>cisco_ios</code> (r1, r2, sw1)</th>
          <th><code>arista_eos</code> (r3)</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>version</td>
          <td><code>version</code>: <code>'0.11.2'</code></td>
          <td><code>image</code>: <code>'4.99.0C'</code></td>
      </tr>
      <tr>
          <td>serial</td>
          <td><code>serial</code>: a <strong>list</strong>, <code>['CAPEHFI5RPW']</code></td>
          <td><code>serial_number</code>: a string</td>
      </tr>
      <tr>
          <td>uptime</td>
          <td><code>uptime</code></td>
          <td><code>uptime</code></td>
      </tr>
      <tr>
          <td>hostname</td>
          <td><code>hostname</code></td>
          <td>not there</td>
      </tr>
      <tr>
          <td>model</td>
          <td><code>hardware</code>: an empty list</td>
          <td><code>model</code>: an empty string</td>
      </tr>
  </tbody>
</table>
<p>The <code>cisco_ios</code> serial is a list because a stack of switches reports one serial per member, so the template
always gives a list, even of one. And neither template finds a model on these devices: the field is there, and
empty. That&rsquo;s common. Not every device prints every fact where a template looks for it, so a tool has to
decide where each column comes from, and sometimes the answer is &ldquo;not from the device&rdquo;.</p>
<hr>
<h2 id="how-do-you-turn-two-device-types-into-one-row">How do you turn two device types into one row?</h2>
<p>Pick one name for each column, and map each device type&rsquo;s keys onto it in one place. This is the heart of
<code>facts()</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl">    <span class="k">if</span> <span class="n">device</span><span class="p">[</span><span class="s2">&#34;device_type&#34;</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;arista_eos&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">release</span><span class="p">,</span> <span class="n">serial</span> <span class="o">=</span> <span class="n">version</span><span class="p">[</span><span class="s2">&#34;image&#34;</span><span class="p">],</span> <span class="n">version</span><span class="p">[</span><span class="s2">&#34;serial_number&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">    <span class="k">else</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">release</span><span class="p">,</span> <span class="n">serial</span> <span class="o">=</span> <span class="n">version</span><span class="p">[</span><span class="s2">&#34;version&#34;</span><span class="p">],</span> <span class="n">version</span><span class="p">[</span><span class="s2">&#34;serial&#34;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;hostname&#34;</span><span class="p">:</span> <span class="n">hostname</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;model&#34;</span><span class="p">:</span> <span class="n">device</span><span class="p">[</span><span class="s2">&#34;model&#34;</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;version&#34;</span><span class="p">:</span> <span class="n">release</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;serial&#34;</span><span class="p">:</span> <span class="n">serial</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;uptime&#34;</span><span class="p">:</span> <span class="n">version</span><span class="p">[</span><span class="s2">&#34;uptime&#34;</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;ip_interfaces&#34;</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">brief</span> <span class="k">if</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;ip_address&#34;</span><span class="p">]</span> <span class="o">!=</span> <span class="s2">&#34;unassigned&#34;</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span></code></pre></div><p>This is <strong>normalizing</strong>: turning data that means the same thing, spelled differently, into one shape. The <code>if</code> is
on <code>device_type</code>, the inventory field that told Netmiko which CLI to expect in
<a href="/lessons/python-101/netmiko/">lesson 6</a>. The other columns come from wherever they&rsquo;re reliable:</p>
<ul>
<li><strong>hostname</strong> is <code>conn.base_prompt</code>, which Netmiko works out from the prompt when it logs in: <code>r1#</code> gives
<code>'r1'</code>. Every device has a prompt; only one template has a <code>hostname</code> key.</li>
<li><strong>model</strong> is <code>device[&quot;model&quot;]</code>, from the inventory, because <code>show version</code> doesn&rsquo;t report one here. The
inventory is where you&rsquo;d record it anyway: it&rsquo;s what you bought.</li>
<li><strong>ip_interfaces</strong> counts the rows of <code>show ip interface brief</code> whose address isn&rsquo;t <code>unassigned</code>. Both
templates call that field <code>ip_address</code>, though Pulse&rsquo;s includes the prefix (<code>10.0.13.2/30</code>).</li>
</ul>
<p>The result, for one device of each type:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">facts</span><span class="p">(</span><span class="n">devices</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;hostname&#39;</span><span class="p">:</span> <span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;model&#39;</span><span class="p">:</span> <span class="s1">&#39;CapeOS-Bolt&#39;</span><span class="p">,</span> <span class="s1">&#39;version&#39;</span><span class="p">:</span> <span class="s1">&#39;0.11.2&#39;</span><span class="p">,</span> <span class="s1">&#39;serial&#39;</span><span class="p">:</span> <span class="s1">&#39;CAPEHFI5RPW&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime&#39;</span><span class="p">:</span> <span class="s1">&#39;10 minutes&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_interfaces&#39;</span><span class="p">:</span> <span class="mi">5</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">facts</span><span class="p">(</span><span class="n">devices</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;hostname&#39;</span><span class="p">:</span> <span class="s1">&#39;r3&#39;</span><span class="p">,</span> <span class="s1">&#39;model&#39;</span><span class="p">:</span> <span class="s1">&#39;CapeOS-Pulse&#39;</span><span class="p">,</span> <span class="s1">&#39;version&#39;</span><span class="p">:</span> <span class="s1">&#39;4.99.0C&#39;</span><span class="p">,</span> <span class="s1">&#39;serial&#39;</span><span class="p">:</span> <span class="s1">&#39;CAPE4S5VG44&#39;</span><span class="p">,</span> <span class="s1">&#39;uptime&#39;</span><span class="p">:</span> <span class="s1">&#39;10 minutes&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_interfaces&#39;</span><span class="p">:</span> <span class="mi">4</span><span class="p">}</span>
</span></span></code></pre></div><p>Same keys, same order, two CLIs. Everything after <code>facts()</code> can treat every device alike.</p>
<hr>
<h2 id="how-does-csvdictwriter-write-the-rows">How does csv.DictWriter write the rows?</h2>
<p><strong>CSV</strong>, comma-separated values, is a text file with one record per line and commas between the fields; any
spreadsheet opens it. <a href="/lessons/python-101/files-and-errors/">Lesson 4</a> used the <code>csv</code> module&rsquo;s plain writer,
which takes each row as a list. <code>csv.DictWriter</code> takes each row as a <strong>dict</strong>, which is what <code>facts()</code> returns,
and puts each value in its column by key:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">w</span> <span class="o">=</span> <span class="n">csv</span><span class="o">.</span><span class="n">DictWriter</span><span class="p">(</span><span class="n">buf</span><span class="p">,</span> <span class="n">fieldnames</span><span class="o">=</span><span class="n">COLUMNS</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">w</span><span class="o">.</span><span class="n">writeheader</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="mi">52</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">w</span><span class="o">.</span><span class="n">writerow</span><span class="p">(</span><span class="n">facts</span><span class="p">(</span><span class="n">devices</span><span class="p">[</span><span class="mi">2</span><span class="p">]))</span>
</span></span><span class="line"><span class="cl"><span class="mi">50</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">w</span><span class="o">.</span><span class="n">writerow</span><span class="p">({</span><span class="s2">&#34;hostname&#34;</span><span class="p">:</span> <span class="s2">&#34;r9&#34;</span><span class="p">})</span>
</span></span><span class="line"><span class="cl"><span class="mi">9</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">getvalue</span><span class="p">())</span>
</span></span><span class="line"><span class="cl"><span class="n">hostname</span><span class="p">,</span><span class="n">model</span><span class="p">,</span><span class="n">version</span><span class="p">,</span><span class="n">serial</span><span class="p">,</span><span class="n">uptime</span><span class="p">,</span><span class="n">ip_interfaces</span>
</span></span><span class="line"><span class="cl"><span class="n">r3</span><span class="p">,</span><span class="n">CapeOS</span><span class="o">-</span><span class="n">Pulse</span><span class="p">,</span><span class="mf">4.99.0</span><span class="n">C</span><span class="p">,</span><span class="n">CAPE4S5VG44</span><span class="p">,</span><span class="mi">10</span> <span class="n">minutes</span><span class="p">,</span><span class="mi">4</span>
</span></span><span class="line"><span class="cl"><span class="n">r9</span><span class="p">,,,,,</span>
</span></span></code></pre></div><p>(<code>buf</code> is an <code>io.StringIO</code>, a file that lives in memory, so the REPL can print what would have gone to disk.
The numbers are how many characters each call wrote.)</p>
<p><code>fieldnames</code> fixes the columns and their order, and <code>writeheader()</code> writes them as the first line. A missing
key gives an empty field, as r9 shows. A key that isn&rsquo;t a column is an error, which catches typos:
<code>w.writerow({&quot;hostname&quot;: &quot;r9&quot;, &quot;vendor&quot;: &quot;x&quot;})</code> stops with
<code>ValueError: dict contains fields not in fieldnames: 'vendor'</code>.</p>
<p>The lab&rsquo;s <code>write_csv()</code> opens the file with <code>path.open(&quot;w&quot;, newline=&quot;&quot;)</code> and calls <code>writeheader()</code> and
<code>writerows(rows)</code>, which writes a whole list in one go. <code>newline=&quot;&quot;</code> is the detail to remember: the
<a href="https://docs.python.org/3/library/csv.html">csv module&rsquo;s documentation</a> asks for it on every file you give a
csv writer, because the module writes its own line endings and the file would otherwise translate them again.</p>
<hr>
<h2 id="what-should-a-script-do-when-one-device-doesnt-answer">What should a script do when one device doesn&rsquo;t answer?</h2>
<p>Carry on, and say so at the end. A network always has a device that&rsquo;s down, being replaced, or in the
inventory by mistake, and a tool that stops at the first one gets run once. The starter&rsquo;s <code>run()</code> is a plain
loop, <code>rows.append(facts(d))</code> for each device. With r2 switched off:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/inventory.py
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/scripts/inventory.py&#34;, line 37, in &lt;module&gt;
</span></span><span class="line"><span class="cl">    sys.exit(main())
</span></span><span class="line"><span class="cl">             ^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/scripts/inventory.py&#34;, line 28, in main
</span></span><span class="line"><span class="cl">    rows, failures = run(devices)
</span></span><span class="line"><span class="cl">                     ^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/netlab/report.py&#34;, line 102, in run
</span></span><span class="line"><span class="cl">    rows.append(facts(d))
</span></span><span class="line"><span class="cl">                ^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/netlab/report.py&#34;, line 55, in facts
</span></span><span class="line"><span class="cl">    raise DeviceError(f&#34;{device[&#39;name&#39;]}: no answer from {device[&#39;host&#39;]}&#34;) from None
</span></span><span class="line"><span class="cl">netlab.devices.DeviceError: r2: no answer from 172.80.111.12
</span></span></code></pre></div><p>Read it from the bottom, as lesson 4 showed: <code>facts()</code> raised <code>DeviceError</code> for r2, nothing caught it, and it
went up through <code>run()</code> and <code>main()</code> until Python stopped. r1 had answered, but no CSV was written, and r3 and
sw1 were never tried. The fix is a <code>try</code> <strong>inside</strong> the loop, around one device:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl">    <span class="n">rows</span><span class="p">,</span> <span class="n">failures</span> <span class="o">=</span> <span class="p">[],</span> <span class="p">{}</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">devices</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">facts</span><span class="p">(</span><span class="n">d</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">        <span class="k">except</span> <span class="n">DeviceError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">failures</span><span class="p">[</span><span class="n">d</span><span class="p">[</span><span class="s2">&#34;name&#34;</span><span class="p">]]</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">rows</span><span class="p">,</span> <span class="n">failures</span>
</span></span></code></pre></div><p>Where the <code>try</code> sits decides what a failure costs. Around the whole loop, one dead router ends the run. Around
one device, it costs that device&rsquo;s row and nothing else. It catches only <code>DeviceError</code>, the &ldquo;couldn&rsquo;t log in
or couldn&rsquo;t reach it&rdquo; that <code>facts()</code> raises on purpose. A typo in your own code raises something else, and
should still stop the script with a traceback, because you want to see it.</p>
<p><code>failures</code> maps a device&rsquo;s name to the reason, so the report can say exactly what happened. With r2 stopped and
r9 in the inventory, a router that was never built:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ time python scripts/inventory.py
</span></span><span class="line"><span class="cl">wrote inventory.csv: 3 of 5 devices
</span></span><span class="line"><span class="cl">  couldn&#39;t inventory r2: no answer from 172.80.111.12
</span></span><span class="line"><span class="cl">  couldn&#39;t inventory r9: no answer from 172.80.111.19
</span></span><span class="line"><span class="cl">python scripts/inventory.py  0.59s user 0.10s system 6% cpu 11.285 total
</span></span></code></pre></div><p>Three rows in the CSV, and two lines naming what&rsquo;s missing and why. &ldquo;No answer&rdquo; is Netmiko&rsquo;s
<code>NetmikoTimeoutException</code>, turned into one line in <code>facts()</code> the way <code>collect()</code> did it in lesson 6.</p>
<hr>
<h2 id="what-is-an-exit-code-and-why-does-the-script-exit-2">What is an exit code, and why does the script exit 2?</h2>
<p>Every program hands the shell a number when it finishes, its <strong>exit code</strong> (or exit status): 0 means it worked,
anything else means something didn&rsquo;t. The shell keeps the last one in <code>$?</code>, and <code>&amp;&amp;</code>, a shell script&rsquo;s <code>if</code>,
cron and every CI pipeline decide what to do next from it. <a href="/lessons/linux-cli-101/">Linux CLI 101</a> covers it
from the shell&rsquo;s side. In Python, <code>main()</code> returns the code and the script&rsquo;s last line hands it over:
<code>sys.exit(main())</code>.</p>
<table>
  <thead>
      <tr>
          <th>Situation</th>
          <th>Exit code</th>
          <th>Who decides it</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>every device answered</td>
          <td>0</td>
          <td><code>main()</code></td>
      </tr>
      <tr>
          <td>some devices failed; the rest are in the CSV</td>
          <td>2</td>
          <td><code>main()</code></td>
      </tr>
      <tr>
          <td>a bad option, such as <code>--source xml</code></td>
          <td>2</td>
          <td>argparse, before any device is touched</td>
      </tr>
      <tr>
          <td>a bug: an exception nobody caught</td>
          <td>1</td>
          <td>Python</td>
      </tr>
  </tbody>
</table>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/inventory.py --source xml; echo $?
</span></span><span class="line"><span class="cl">usage: inventory.py [-h] [--source {yaml,api}] [--out OUT]
</span></span><span class="line"><span class="cl">inventory.py: error: argument --source: invalid choice: &#39;xml&#39; (choose from &#39;yaml&#39;, &#39;api&#39;)
</span></span><span class="line"><span class="cl">2
</span></span></code></pre></div><p>Why 2 and not 1? Python gives an uncaught exception 1, so 1 means &ldquo;the tool broke&rdquo;; 2 here means &ldquo;the tool
worked, part of the network didn&rsquo;t&rdquo;. The catch is that argparse also exits 2 for a bad option, so a caller
can&rsquo;t tell those apart from the number alone. For this tool that&rsquo;s acceptable: a bad option never writes a
CSV, and the message says which it was. There&rsquo;s no universal rule; <code>grep</code>, for example, exits 0 for &ldquo;found&rdquo;,
1 for &ldquo;not found&rdquo; and 2 for an error.</p>
<p>Notice where the output goes, too. The summary goes to <strong>stdout</strong> (standard output) and the failures to
<strong>stderr</strong> (standard error), with <code>file=sys.stderr</code>. A terminal shows both. Redirect the script to a file
with <code>&gt;</code> and only the summary goes there: the failures stay on screen, where a person sees them.</p>
<hr>
<h2 id="how-does-argparse-turn-a-command-line-into-options">How does argparse turn a command line into options?</h2>
<p><code>argparse</code>, in the standard library, reads the words after the script&rsquo;s name, checks them, and gives them
back as one object with an attribute per option. This is the lab&rsquo;s <code>parse_args()</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">parse_args</span><span class="p">(</span><span class="n">argv</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">p</span> <span class="o">=</span> <span class="n">argparse</span><span class="o">.</span><span class="n">ArgumentParser</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s2">&#34;Inventory every lab device to a CSV file.&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">p</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">&#34;--source&#34;</span><span class="p">,</span> <span class="n">choices</span><span class="o">=</span><span class="p">[</span><span class="s2">&#34;yaml&#34;</span><span class="p">,</span> <span class="s2">&#34;api&#34;</span><span class="p">],</span> <span class="n">default</span><span class="o">=</span><span class="s2">&#34;yaml&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                   <span class="n">help</span><span class="o">=</span><span class="s2">&#34;where the device list comes from (default: yaml, the file inventory.yml)&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">p</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">&#34;--out&#34;</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="s2">&#34;inventory.csv&#34;</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">&#34;the CSV file to write (default: inventory.csv)&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">p</span><span class="o">.</span><span class="n">parse_args</span><span class="p">(</span><span class="n">argv</span><span class="p">)</span>
</span></span></code></pre></div><p>Each <code>add_argument()</code> names an option and its rules: <code>default</code> when it isn&rsquo;t given, <code>choices</code> for the only
values allowed, <code>help</code> for the help text. <code>parse_args()</code> returns a <code>Namespace</code>, and <code>args.source</code> and
<code>args.out</code> are the values. <code>argv=None</code> means the real command line; a list lets you try it in the REPL
(here <code>inv</code> is <code>scripts/inventory.py</code>, loaded as a module):</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">inv</span><span class="o">.</span><span class="n">parse_args</span><span class="p">([])</span>
</span></span><span class="line"><span class="cl"><span class="n">Namespace</span><span class="p">(</span><span class="n">source</span><span class="o">=</span><span class="s1">&#39;yaml&#39;</span><span class="p">,</span> <span class="n">out</span><span class="o">=</span><span class="s1">&#39;inventory.csv&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">inv</span><span class="o">.</span><span class="n">parse_args</span><span class="p">([</span><span class="s2">&#34;--source&#34;</span><span class="p">,</span> <span class="s2">&#34;api&#34;</span><span class="p">,</span> <span class="s2">&#34;--out&#34;</span><span class="p">,</span> <span class="s2">&#34;reports/x.csv&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="n">Namespace</span><span class="p">(</span><span class="n">source</span><span class="o">=</span><span class="s1">&#39;api&#39;</span><span class="p">,</span> <span class="n">out</span><span class="o">=</span><span class="s1">&#39;reports/x.csv&#39;</span><span class="p">)</span>
</span></span></code></pre></div><p>What you get for free is the reason to use it rather than reading <code>sys.argv</code> yourself. It refuses bad input
with a usage line and exit code 2 before your code runs, as <code>--source xml</code> showed, and <code>--help</code> prints a
page built from the <code>description</code> and each option&rsquo;s <code>help</code>, without a line of code from you.</p>
<p>The <a href="https://docs.python.org/3/library/argparse.html">argparse documentation</a> has the rest: positional
arguments, on/off flags with <code>action=&quot;store_true&quot;</code>, and types.</p>
<hr>
<h2 id="what-would-nornir-add-to-a-tool-like-this">What would Nornir add to a tool like this?</h2>
<p>The same shape, with the next parts already written. <a href="https://nornir.readthedocs.io/">Nornir</a> is a Python
automation framework built around this loop: an inventory of hosts, a task that runs on each one, and a result
per host.</p>
<table>
  <thead>
      <tr>
          <th></th>
          <th>This script</th>
          <th>Nornir</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Devices at once</td>
          <td>one at a time, in a <code>for</code> loop</td>
          <td>many at once, in <strong>threads</strong> (several connections waiting in parallel)</td>
      </tr>
      <tr>
          <td>Inventory</td>
          <td>a list of dicts from YAML or the API</td>
          <td>hosts, groups and defaults, from files or plugins (NetBox has one)</td>
      </tr>
      <tr>
          <td>Results</td>
          <td><code>rows</code> and a <code>failures</code> dict you build</td>
          <td>a result per host, with the failed hosts listed for you</td>
      </tr>
  </tbody>
</table>
<p>The first is the one you&rsquo;d notice. A clean run in the lab takes between 6 and 12 seconds for four devices,
most of it waiting for SSH and prompts, and each device that doesn&rsquo;t answer adds its whole 5-second timeout,
because the loop waits for each device in turn. With a few hundred devices that&rsquo;s too slow, and running
connections in parallel safely is a subject of its own.</p>
<p>It&rsquo;s easier to learn once you&rsquo;ve written the loop it replaces: <code>facts()</code> is what Nornir calls a task, <code>run()</code>
is what its runner does, and <code>failures</code> is its failed hosts. Nornir&rsquo;s documentation starts with a tutorial.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong>The script stops at the first dead device.</strong> The <code>try</code>/<code>except</code> is around the loop, or missing. Put it
inside the loop, around one device.</p>
<p><strong><code>KeyError: 'version'</code> on r3.</strong> The <code>cisco_ios</code> key was used on an <code>arista_eos</code> device. Print each type&rsquo;s
parsed <code>show version</code> in the REPL before you map keys.</p>
<p><strong>The serial column shows <code>['CAPEHFI5RPW']</code>.</strong> The <code>cisco_ios</code> template gives a list. Take <code>[0]</code>.</p>
<p><strong>The model column is empty.</strong> It came from <code>show version</code>, where these devices leave it blank. Take it from
the inventory.</p>
<p><strong><code>ValueError: dict contains fields not in fieldnames</code>.</strong> A key in the row isn&rsquo;t in <code>COLUMNS</code>, often a typo.
The error names the key.</p>
<p><strong>Every failure is caught, your own bugs included.</strong> <code>except Exception:</code> in <code>run()</code> turns a typo in <code>facts()</code>
into &ldquo;couldn&rsquo;t inventory&rdquo; for every device, with no traceback to find it by. Catch <code>DeviceError</code>.</p>
<p><strong>The script exits 0 when devices failed.</strong> The script calls <code>main()</code> without <code>sys.exit()</code>, or <code>main()</code>
doesn&rsquo;t return the code. Run <code>echo $?</code> straight after the script to see.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-8">Chapter 8 of the lab</a> has you write <code>load_devices()</code>, <code>facts()</code> and
<code>write_csv()</code>, and run the script against r1, r2, r3 and sw1. Then <code>make break T=8.2</code> stops r2 and adds a
router that doesn&rsquo;t exist, and you make <code>run()</code> carry on and the script exit 2. Last, <code>--source</code> and <code>--out</code>
with argparse, and proof that the YAML and the API give the same CSV.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>That&rsquo;s Python 101: from <code>2 + 2</code> at the REPL to a tool that logs in to every device in an inventory, reads two
CLIs, and reports what it found and what it couldn&rsquo;t reach. Where next depends on what you want to automate:</p>
<ul>
<li><a href="/lessons/ansible-101/">Ansible 101</a> runs tasks against an inventory without you writing the loop. Python 101
is recommended first: YAML&rsquo;s data shapes, Jinja2&rsquo;s expressions and Ansible&rsquo;s tracebacks all read like the
Python you now know.</li>
<li><span class="link-soon">NetBox</span> makes the inventory a source of truth, with a REST API like the lab&rsquo;s <code>api</code>.
Python 101 is required for it, as it is for <span class="link-soon">network testing</span> and
<span class="link-soon">ANTA</span>.</li>
</ul>
]]></content:encoded></item><item><title>YAML, JSON and a First REST Call: One Inventory, Two Sources (Python 101, Lesson 7)</title><link>https://networkdoodles.com/lessons/python-101/yaml-json-rest/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/yaml-json-rest/</guid><category>python</category><category>network-automation</category><category>yaml</category><category>netbox</category><category>capeos</category><description>YAML and JSON as the same lists and dicts, safe_load and YAML's surprises like the Norway problem, then a REST call with requests, a token and status codes.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p>YAML and JSON are two ways of writing down the same thing: lists and dicts of strings, numbers, booleans and
nothing (<code>None</code>). <code>yaml.safe_load()</code> turns a YAML file into those Python values, <code>json.load()</code> and
<code>json.dumps()</code> read and write JSON, and a REST API sends JSON back when you ask it with <code>requests.get()</code>. Two
cautions carry most of the lesson: use <code>safe_load</code>, never <code>load</code>, and quote any YAML value that could be read
as something other than a string, because <code>NO</code> is <code>False</code> and <code>2:30</code> is <code>150</code>.</p>
<p>Lesson 6 wrote the list of devices into the script. Real inventories live outside the code, in a file people
edit or in a system such as NetBox that a script asks over HTTP. This lesson reads the lab&rsquo;s inventory both
ways and gets the same list of dicts from each. The outputs are from the <a href="/labs/python-101/">lab</a>: PyYAML
6.0.3 and requests 2.34.2 on Python 3.11.</p>
<hr>
<h2 id="why-are-yaml-and-json-the-same-data">Why are YAML and JSON the same data?</h2>
<p>Because both describe the two shapes <a href="/lessons/python-101/strings-lists-dicts/">lesson 2</a> built everything
from: a <strong>list</strong> (an ordered sequence) and a <strong>dict</strong> (keys and values). Here&rsquo;s one device from the lab&rsquo;s
<code>inventory.yml</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">devices</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">r3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">host</span><span class="p">:</span><span class="w"> </span><span class="m">172.80.111.13</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">device_type</span><span class="p">:</span><span class="w"> </span><span class="l">arista_eos</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">role</span><span class="p">:</span><span class="w"> </span><span class="l">router</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">model</span><span class="p">:</span><span class="w"> </span><span class="l">CapeOS-Pulse</span><span class="w">
</span></span></span></code></pre></div><p><code>devices:</code> is a key whose value is a list; each <code>- </code> starts an item; each item is a dict of five keys.
Indentation shows what belongs to what. Load it, and print that device as JSON:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">yaml</span><span class="o">,</span> <span class="nn">json</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">doc</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s2">&#34;inventory.yml&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">doc</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">dict_keys</span><span class="p">([</span><span class="s1">&#39;site&#39;</span><span class="p">,</span> <span class="s1">&#39;country&#39;</span><span class="p">,</span> <span class="s1">&#39;rack&#39;</span><span class="p">,</span> <span class="s1">&#39;backup&#39;</span><span class="p">,</span> <span class="s1">&#39;devices&#39;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">doc</span><span class="p">[</span><span class="s2">&#34;devices&#34;</span><span class="p">][</span><span class="mi">2</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;r3&#39;</span><span class="p">,</span> <span class="s1">&#39;host&#39;</span><span class="p">:</span> <span class="s1">&#39;172.80.111.13&#39;</span><span class="p">,</span> <span class="s1">&#39;device_type&#39;</span><span class="p">:</span> <span class="s1">&#39;arista_eos&#39;</span><span class="p">,</span> <span class="s1">&#39;role&#39;</span><span class="p">:</span> <span class="s1">&#39;router&#39;</span><span class="p">,</span> <span class="s1">&#39;model&#39;</span><span class="p">:</span> <span class="s1">&#39;CapeOS-Pulse&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">doc</span><span class="p">[</span><span class="s2">&#34;devices&#34;</span><span class="p">][</span><span class="mi">2</span><span class="p">],</span> <span class="n">indent</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r3&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;host&#34;</span><span class="p">:</span> <span class="s2">&#34;172.80.111.13&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;device_type&#34;</span><span class="p">:</span> <span class="s2">&#34;arista_eos&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;router&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;model&#34;</span><span class="p">:</span> <span class="s2">&#34;CapeOS-Pulse&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>Three spellings of one dict: YAML, Python and JSON. The differences are only in how they&rsquo;re written:</p>
<table>
  <thead>
      <tr>
          <th></th>
          <th>YAML</th>
          <th>JSON</th>
          <th>Python</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>dict</td>
          <td><code>key: value</code>, indented</td>
          <td><code>{&quot;key&quot;: &quot;value&quot;}</code></td>
          <td><code>{'key': 'value'}</code></td>
      </tr>
      <tr>
          <td>list</td>
          <td><code>- item</code> lines</td>
          <td><code>[&quot;a&quot;, &quot;b&quot;]</code></td>
          <td><code>['a', 'b']</code></td>
      </tr>
      <tr>
          <td>string</td>
          <td>usually unquoted</td>
          <td>always <code>&quot;double quotes&quot;</code></td>
          <td><code>'quotes'</code></td>
      </tr>
      <tr>
          <td>true / false</td>
          <td><code>true</code>, <code>false</code> (and more, below)</td>
          <td><code>true</code>, <code>false</code></td>
          <td><code>True</code>, <code>False</code></td>
      </tr>
      <tr>
          <td>nothing</td>
          <td><code>null</code> or <code>~</code></td>
          <td><code>null</code></td>
          <td><code>None</code></td>
      </tr>
      <tr>
          <td>comments</td>
          <td><code># like this</code></td>
          <td>none</td>
          <td><code># like this</code></td>
      </tr>
  </tbody>
</table>
<p>YAML is written for people, which is why Ansible playbooks and most inventories use it (the
<a href="/posts/understandingyaml/">YAML post</a> covers its syntax in more depth). JSON
(<a href="https://www.rfc-editor.org/rfc/rfc8259">RFC 8259</a>) is stricter and simpler to parse, which is why APIs send it.</p>
<hr>
<h2 id="how-do-you-read-and-write-json">How do you read and write JSON?</h2>
<p>The <code>json</code> module is in the standard library. <code>json.dumps()</code> turns Python values into a JSON string
(<em>dump string</em>), and <code>json.loads()</code> turns a string back. Watch the booleans and <code>None</code> change spelling:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="s1">&#39;{&#34;name&#34;: &#34;r1&#34;, &#34;snmp&#34;: true, &#34;serial&#34;: null}&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;snmp&#39;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> <span class="s1">&#39;serial&#39;</span><span class="p">:</span> <span class="kc">None</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">({</span><span class="s2">&#34;enabled&#34;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> <span class="s2">&#34;serial&#34;</span><span class="p">:</span> <span class="kc">None</span><span class="p">})</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;{&#34;enabled&#34;: true, &#34;serial&#34;: null}&#39;</span>
</span></span></code></pre></div><p>Without the <code>s</code>, <code>json.dump(data, f)</code> writes to an open file and <code>json.load(f)</code> reads one, the way
<a href="/lessons/python-101/files-and-errors/">lesson 4</a> opened files with <code>with open(...)</code>. <code>indent=2</code> puts one
value per line, indented, for a person to read; without it the whole file is one line. A JSON round trip,
writing a list of dicts and reading it back, gives you exactly what you started with, which is what task 7.1
checks.</p>
<hr>
<h2 id="why-yamlsafe_load-and-never-yamlload">Why yaml.safe_load and never yaml.load?</h2>
<p>Because YAML can carry <strong>tags</strong>, instructions that say what kind of object to build, and PyYAML&rsquo;s full loader
will build any Python object a tag names, including calling a function. This one-line document asks for
<code>os.getcwd()</code> to be run:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s2">&#34;!!python/object/apply:os.getcwd []&#34;</span><span class="p">,</span> <span class="n">Loader</span><span class="o">=</span><span class="n">yaml</span><span class="o">.</span><span class="n">UnsafeLoader</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;/workspaces/python-101&#39;</span>
</span></span></code></pre></div><p>It ran. <code>getcwd</code> is harmless; a file that names <code>os.system</code> instead runs a shell command on your machine, as
you, the moment you load it. <code>safe_load</code> only builds plain values (strings, numbers, booleans, <code>None</code>, lists
and dicts) and refuses everything else:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&gt;&gt;&gt; yaml.safe_load(&#34;!!python/object/apply:os.getcwd []&#34;)
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">yaml.constructor.ConstructorError: could not determine a constructor for the tag &#39;tag:yaml.org,2002:python/object/apply:os.getcwd&#39;
</span></span><span class="line"><span class="cl">  in &#34;&lt;unicode string&gt;&#34;, line 1, column 1:
</span></span><span class="line"><span class="cl">    !!python/object/apply:os.getcwd []
</span></span><span class="line"><span class="cl">    ^
</span></span></code></pre></div><p>An inventory is plain values, so <code>safe_load</code> loses nothing. PyYAML 5.1 started warning about <code>yaml.load(f)</code>
without a loader, and PyYAML 6 refuses it: it stops with <code>TypeError: load() missing 1 required positional argument: 'Loader'</code>, so you have to
pick a loader. Pick <code>safe_load</code>. The PyYAML project explains the history in
<a href="https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load%28input%29-Deprecation">its note on yaml.load</a>.</p>
<hr>
<h2 id="why-does-yaml-read-no-as-false">Why does YAML read NO as False?</h2>
<p>Because YAML guesses a value&rsquo;s type from how it looks, and PyYAML follows the rules of YAML 1.1, from 2005,
which guess generously. The lab&rsquo;s starter <code>inventory.yml</code> has four innocent-looking lines at the top:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">country</span><span class="p">:</span><span class="w"> </span><span class="kc">NO</span><span class="w">                 </span><span class="c"># Norway</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">rack</span><span class="p">:</span><span class="w"> </span><span class="m">0123</span><span class="w">                  </span><span class="c"># the label on the rack</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">backup</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">on</span><span class="p">:</span><span class="w"> </span><span class="l">sunday               </span><span class="w"> </span><span class="c"># which day the configs are backed up</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">at</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="p">:</span><span class="m">30</span><span class="w">                  </span><span class="c"># and when</span><span class="w">
</span></span></span></code></pre></div><p>And this is what they load as:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="s2">&#34;country: NO</span><span class="se">\n</span><span class="s2">rack: 0123</span><span class="se">\n</span><span class="s2">backup:</span><span class="se">\n</span><span class="s2">  on: sunday</span><span class="se">\n</span><span class="s2">  at: 2:30</span><span class="se">\n</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;country&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span> <span class="s1">&#39;rack&#39;</span><span class="p">:</span> <span class="mi">83</span><span class="p">,</span> <span class="s1">&#39;backup&#39;</span><span class="p">:</span> <span class="p">{</span><span class="kc">True</span><span class="p">:</span> <span class="s1">&#39;sunday&#39;</span><span class="p">,</span> <span class="s1">&#39;at&#39;</span><span class="p">:</span> <span class="mi">150</span><span class="p">}}</span>
</span></span></code></pre></div><p>None of the four survived:</p>
<table>
  <thead>
      <tr>
          <th>You wrote</th>
          <th>YAML 1.1 reads</th>
          <th>Why</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>NO</code></td>
          <td><code>False</code></td>
          <td><a href="https://yaml.org/type/bool.html">booleans</a> include <code>yes</code>, <code>no</code>, <code>on</code>, <code>off</code>, in lower, title or upper case</td>
      </tr>
      <tr>
          <td><code>on:</code> (a key)</td>
          <td><code>True</code></td>
          <td>the same rule, applied to a key: the dict now has the key <code>True</code>, not <code>'on'</code></td>
      </tr>
      <tr>
          <td><code>0123</code></td>
          <td><code>83</code></td>
          <td>a leading <code>0</code> means <a href="https://yaml.org/type/int.html">octal</a>, base 8: 1×64 + 2×8 + 3</td>
      </tr>
      <tr>
          <td><code>2:30</code></td>
          <td><code>150</code></td>
          <td>numbers with colons are base 60 (<em>sexagesimal</em>), for times: 2×60 + 30</td>
      </tr>
  </tbody>
</table>
<p>The first is famous enough to have a name, <strong>the Norway problem</strong>: a list of country codes in which Norway,
<code>NO</code>, turns into <code>False</code>. A few more from the same rules:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="s2">&#34;version: 1.10</span><span class="se">\n</span><span class="s2">vlan: 010</span><span class="se">\n</span><span class="s2">enabled: yes</span><span class="se">\n</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;version&#39;</span><span class="p">:</span> <span class="mf">1.1</span><span class="p">,</span> <span class="s1">&#39;vlan&#39;</span><span class="p">:</span> <span class="mi">8</span><span class="p">,</span> <span class="s1">&#39;enabled&#39;</span><span class="p">:</span> <span class="kc">True</span><span class="p">}</span>
</span></span></code></pre></div><p><code>1.10</code> is a <a href="https://yaml.org/type/float.html">float</a>, so the trailing zero goes and a software version
becomes 1.1. <code>010</code> is octal 8, a VLAN you didn&rsquo;t mean. <code>yes</code> is <code>True</code>, which there was probably what you
wanted.</p>
<p><a href="https://yaml.org/spec/1.2.2/">YAML 1.2</a>, from 2009, dropped most of these guesses: only <code>true</code> and <code>false</code> are
booleans, and there&rsquo;s no base 60. But PyYAML, and so Ansible and most Python tools, still read YAML 1.1. The
fix works in both: <strong>quote anything that must stay a string</strong>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="s1">&#39;country: &#34;NO&#34;</span><span class="se">\n</span><span class="s1">rack: &#34;0123&#34;</span><span class="se">\n</span><span class="s1">backup:</span><span class="se">\n</span><span class="s1">  &#34;on&#34;: sunday</span><span class="se">\n</span><span class="s1">  at: &#34;2:30&#34;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;country&#39;</span><span class="p">:</span> <span class="s1">&#39;NO&#39;</span><span class="p">,</span> <span class="s1">&#39;rack&#39;</span><span class="p">:</span> <span class="s1">&#39;0123&#39;</span><span class="p">,</span> <span class="s1">&#39;backup&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;on&#39;</span><span class="p">:</span> <span class="s1">&#39;sunday&#39;</span><span class="p">,</span> <span class="s1">&#39;at&#39;</span><span class="p">:</span> <span class="s1">&#39;2:30&#39;</span><span class="p">}}</span>
</span></span></code></pre></div><p>Quoting <code>on</code> works as a key too. Addresses such as <code>172.80.111.13</code> are safe unquoted: with three dots they
can&rsquo;t be a number, so YAML keeps them as strings.</p>
<hr>
<h2 id="what-is-a-rest-api">What is a REST API?</h2>
<p>An <strong>API</strong> (application programming interface) is a way for one program to ask another for data. A <strong>REST
API</strong> does it over HTTP, the protocol of the web: every thing it holds has a URL, and you act on it with the
HTTP <strong>methods</strong> your browser already uses. The lab&rsquo;s <code>api</code> service holds the same inventory, at
<code>http://api:8000/api/devices/</code>:</p>
<table>
  <thead>
      <tr>
          <th>Part</th>
          <th>In <code>http://api:8000/api/devices/1/</code></th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>scheme</td>
          <td><code>http</code>: plain HTTP, fine inside a lab; a real API uses <code>https</code></td>
      </tr>
      <tr>
          <td>host and port</td>
          <td><code>api:8000</code>: the lab&rsquo;s API container, port 8000</td>
      </tr>
      <tr>
          <td>path</td>
          <td><code>/api/devices/</code> is the list of devices; <code>/api/devices/1/</code> is the one with ID 1</td>
      </tr>
  </tbody>
</table>
<p>The methods say what you want done (<a href="https://www.rfc-editor.org/rfc/rfc9110">RFC 9110</a> defines them):</p>
<table>
  <thead>
      <tr>
          <th>Method</th>
          <th>Means</th>
          <th>In this lab</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>GET</code></td>
          <td>read</td>
          <td>yes</td>
      </tr>
      <tr>
          <td><code>POST</code></td>
          <td>create</td>
          <td><code>405 Method Not Allowed</code>: the lab&rsquo;s API is read-only</td>
      </tr>
      <tr>
          <td><code>PUT</code>, <code>PATCH</code></td>
          <td>replace, change</td>
          <td>405</td>
      </tr>
      <tr>
          <td><code>DELETE</code></td>
          <td>delete</td>
          <td>405</td>
      </tr>
  </tbody>
</table>
<p>The answer starts with a <strong>status code</strong>, three digits that say how it went. The first digit is the class:
2xx worked, 4xx is a problem with your request, 5xx is a problem on the server.</p>
<table>
  <thead>
      <tr>
          <th>Code</th>
          <th>Means</th>
          <th>You get it from the lab&rsquo;s API when</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>200</td>
          <td>OK</td>
          <td>the request worked</td>
      </tr>
      <tr>
          <td>403</td>
          <td>Forbidden</td>
          <td>the token is missing or wrong</td>
      </tr>
      <tr>
          <td>404</td>
          <td>Not Found</td>
          <td>there&rsquo;s no device with that ID</td>
      </tr>
      <tr>
          <td>405</td>
          <td>Method Not Allowed</td>
          <td>you tried to change something</td>
      </tr>
  </tbody>
</table>
<p><strong>Headers</strong> carry everything around the data: what format it&rsquo;s in (<code>Content-Type: application/json</code>), and who
you are. The lab&rsquo;s API wants a <strong>token</strong>, a long random string that stands in for a password, in an
<code>Authorization: Token &lt;token&gt;</code> header. That&rsquo;s the form NetBox has used for years; NetBox 4.6 deprecates it in
favor of newer tokens sent as <code>Authorization: Bearer ...</code>, but the idea is the same. <code>curl -i</code>, from <a href="/lessons/linux-cli-101/">Linux CLI 101</a>, shows the
status line and the headers. Without a token:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ curl -i http://api:8000/api/devices/1/
</span></span><span class="line"><span class="cl">HTTP/1.0 403 Forbidden
</span></span><span class="line"><span class="cl">Server: lab-api/1.0 Python/3.11.2
</span></span><span class="line"><span class="cl">Date: Wed, 11 Nov 2026 05:40:46 GMT
</span></span><span class="line"><span class="cl">Content-Type: application/json
</span></span><span class="line"><span class="cl">Content-Length: 64
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">{
</span></span><span class="line"><span class="cl">  &#34;detail&#34;: &#34;Authentication credentials were not provided.&#34;
</span></span><span class="line"><span class="cl">}
</span></span></code></pre></div><p>The token is in <code>.env</code> as <code>LAB_API_TOKEN</code>, next to the device password, for the same reason: a token is a
password, and it stays out of the code.</p>
<hr>
<h2 id="how-do-you-call-an-api-from-python">How do you call an API from Python?</h2>
<p>With <a href="https://requests.readthedocs.io/">requests</a>, the library most Python code uses for HTTP.
<code>requests.get()</code> takes the URL, the headers as a dict, and a <code>timeout</code> in seconds:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">requests</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;http://api:8000/api/devices/&#34;</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Authorization&#34;</span><span class="p">:</span> <span class="s2">&#34;Token 0123456789abcdef0123456789abcdef01234567&#34;</span><span class="p">},</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r</span>
</span></span><span class="line"><span class="cl"><span class="o">&lt;</span><span class="n">Response</span> <span class="p">[</span><span class="mi">200</span><span class="p">]</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r</span><span class="o">.</span><span class="n">status_code</span><span class="p">,</span> <span class="n">r</span><span class="o">.</span><span class="n">headers</span><span class="p">[</span><span class="s2">&#34;Content-Type&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="mi">200</span><span class="p">,</span> <span class="s1">&#39;application/json&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">data</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">data</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">dict_keys</span><span class="p">([</span><span class="s1">&#39;count&#39;</span><span class="p">,</span> <span class="s1">&#39;next&#39;</span><span class="p">,</span> <span class="s1">&#39;previous&#39;</span><span class="p">,</span> <span class="s1">&#39;results&#39;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">data</span><span class="p">[</span><span class="s2">&#34;count&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="mi">4</span>
</span></span></code></pre></div><p><code>r.json()</code> parses the body with the <code>json</code> module and gives you Python values. (In the lab&rsquo;s code, the token
comes from <code>os.environ[&quot;LAB_API_TOKEN&quot;]</code>; it&rsquo;s written out here so you can see the header.)</p>
<p>Always pass a <code>timeout</code>. Without one, requests never gives up waiting for a server that has stopped answering,
and a script that hangs is harder to debug than one that fails.</p>
<p><code>count</code>, <code>next</code>, <code>previous</code> and <code>results</code> are NetBox&rsquo;s way of <strong>paging</strong>: an API with ten thousand devices
doesn&rsquo;t send them in one answer. <code>results</code> holds one page, <code>count</code> the total, and <code>next</code> the URL of the
following page, or <code>None</code> on the last one. With two to a page, and the token read from <code>.env</code> this time:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">os</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.env</span> <span class="kn">import</span> <span class="n">load_env</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">load_env</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">h</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&#34;Authorization&#34;</span><span class="p">:</span> <span class="sa">f</span><span class="s2">&#34;Token </span><span class="si">{</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s1">&#39;LAB_API_TOKEN&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">page</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;http://api:8000/api/devices/&#34;</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">h</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;limit&#34;</span><span class="p">:</span> <span class="mi">2</span><span class="p">},</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">page</span><span class="p">[</span><span class="s2">&#34;count&#34;</span><span class="p">],</span> <span class="n">page</span><span class="p">[</span><span class="s2">&#34;next&#34;</span><span class="p">],</span> <span class="n">page</span><span class="p">[</span><span class="s2">&#34;previous&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="s1">&#39;http://api:8000/api/devices/?limit=2&amp;offset=2&#39;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="p">[</span><span class="n">d</span><span class="p">[</span><span class="s2">&#34;name&#34;</span><span class="p">]</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">page</span><span class="p">[</span><span class="s2">&#34;results&#34;</span><span class="p">]]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;r2&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p>The lab&rsquo;s four devices fit in the default page of 50, so the course&rsquo;s code reads <code>results</code> once. A script for a
real NetBox follows <code>next</code> until it&rsquo;s <code>None</code>. Paging, other kinds of authentication and API schemas are
material for a later course on data formats and APIs.</p>
<hr>
<h2 id="what-happens-when-the-token-is-wrong">What happens when the token is wrong?</h2>
<p>The API answers, with a 403 and a JSON body that says why. And requests doesn&rsquo;t treat that as an error:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">bad</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;http://api:8000/api/devices/&#34;</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Authorization&#34;</span><span class="p">:</span> <span class="s2">&#34;Token nope&#34;</span><span class="p">},</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">bad</span><span class="o">.</span><span class="n">status_code</span><span class="p">,</span> <span class="n">bad</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="mi">403</span><span class="p">,</span> <span class="p">{</span><span class="s1">&#39;detail&#39;</span><span class="p">:</span> <span class="s1">&#39;Invalid token&#39;</span><span class="p">})</span>
</span></span></code></pre></div><p>No exception. A script that goes straight on to <code>bad.json()[&quot;results&quot;]</code> fails a line later with
<code>KeyError: 'results'</code>, which says nothing about tokens. <code>raise_for_status()</code> turns any 4xx or 5xx status into
an exception at the point where it happened:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&gt;&gt;&gt; bad.raise_for_status()
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  File &#34;&lt;stdin&gt;&#34;, line 1, in &lt;module&gt;
</span></span><span class="line"><span class="cl">  File &#34;/home/vscode/.venvs/lab/lib/python3.11/site-packages/requests/models.py&#34;, line 1167, in raise_for_status
</span></span><span class="line"><span class="cl">    raise HTTPError(http_error_msg, response=self)
</span></span><span class="line"><span class="cl">requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://api:8000/api/devices/
</span></span></code></pre></div><p>When nothing answers at all, there&rsquo;s no status code to check: requests raises <code>ConnectionError</code> itself. Here,
on a port where nothing listens, the last line of a long traceback says:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">requests.exceptions.ConnectionError: HTTPConnectionPool(host=&#39;api&#39;, port=8999): Max retries exceeded with url: /api/devices/ (Caused by NewConnectionError(&#34;HTTPConnection(host=&#39;api&#39;, port=8999): Failed to establish a new connection: [Errno 111] Connection refused&#34;))
</span></span></code></pre></div><p>Both are kinds of <code>requests.RequestException</code>, so one <code>except</code> can catch every way an HTTP call fails, and
turn it into one line that names the cause, as lesson 6 did for a refused SSH login. That&rsquo;s task 7.2.</p>
<hr>
<h2 id="why-does-the-apis-json-look-so-different-from-the-yaml">Why does the API&rsquo;s JSON look so different from the YAML?</h2>
<p>Because the API is shaped like NetBox, and NetBox nests. Here&rsquo;s r1 as the API returns it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">data</span><span class="p">[</span><span class="s2">&#34;results&#34;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;id&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;url&#39;</span><span class="p">:</span> <span class="s1">&#39;http://api:8000/api/devices/1/&#39;</span><span class="p">,</span> <span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;device_type&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;model&#39;</span><span class="p">:</span> <span class="s1">&#39;CapeOS-Bolt&#39;</span><span class="p">,</span> <span class="s1">&#39;manufacturer&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;Network Doodles&#39;</span><span class="p">,</span> <span class="s1">&#39;slug&#39;</span><span class="p">:</span> <span class="s1">&#39;network-doodles&#39;</span><span class="p">}},</span> <span class="s1">&#39;role&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;Router&#39;</span><span class="p">,</span> <span class="s1">&#39;slug&#39;</span><span class="p">:</span> <span class="s1">&#39;router&#39;</span><span class="p">},</span> <span class="s1">&#39;platform&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;cisco_ios&#39;</span><span class="p">,</span> <span class="s1">&#39;slug&#39;</span><span class="p">:</span> <span class="s1">&#39;cisco_ios&#39;</span><span class="p">},</span> <span class="s1">&#39;serial&#39;</span><span class="p">:</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;site&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;LAB&#39;</span><span class="p">,</span> <span class="s1">&#39;slug&#39;</span><span class="p">:</span> <span class="s1">&#39;lab&#39;</span><span class="p">},</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;value&#39;</span><span class="p">:</span> <span class="s1">&#39;active&#39;</span><span class="p">,</span> <span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="s1">&#39;Active&#39;</span><span class="p">},</span> <span class="s1">&#39;primary_ip&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;address&#39;</span><span class="p">:</span> <span class="s1">&#39;172.80.111.11/24&#39;</span><span class="p">,</span> <span class="s1">&#39;family&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;value&#39;</span><span class="p">:</span> <span class="mi">4</span><span class="p">,</span> <span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="s1">&#39;IPv4&#39;</span><span class="p">}}}</span>
</span></span></code></pre></div><p>The role isn&rsquo;t a string, it&rsquo;s a dict with a display <code>name</code> and a <code>slug</code>, the short lower-case form used in
URLs and code. The model is inside <code>device_type</code>, the address is inside <code>primary_ip</code> and has its prefix
length. And <code>device_type</code> means something else here: in NetBox it&rsquo;s the hardware model, while Netmiko&rsquo;s
<code>device_type</code> is NetBox&rsquo;s <strong>platform</strong>. Each of those is an object of its own in NetBox, and
<a href="https://netboxlabs.com/docs/netbox/integrations/rest-api/">NetBox&rsquo;s REST API docs</a> say related objects are
included as &ldquo;nested brief representations&rdquo;.</p>
<p>To use it, you reshape it into the flat dict the rest of the code expects, one key at a time:</p>
<table>
  <thead>
      <tr>
          <th>Inventory key</th>
          <th>Where it is in the API&rsquo;s JSON</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>name</code></td>
          <td><code>name</code></td>
      </tr>
      <tr>
          <td><code>host</code></td>
          <td><code>primary_ip.address</code>, without the <code>/24</code></td>
      </tr>
      <tr>
          <td><code>device_type</code></td>
          <td><code>platform.slug</code></td>
      </tr>
      <tr>
          <td><code>role</code></td>
          <td><code>role.slug</code></td>
      </tr>
      <tr>
          <td><code>model</code></td>
          <td><code>device_type.model</code></td>
      </tr>
  </tbody>
</table>
<p>Do that, and the two sources agree exactly. With the lab&rsquo;s finished <code>netlab/inventory.py</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">api_to_inventory</span><span class="p">(</span><span class="n">get_devices</span><span class="p">(</span><span class="s2">&#34;http://api:8000&#34;</span><span class="p">,</span> <span class="s2">&#34;0123456789abcdef0123456789abcdef01234567&#34;</span><span class="p">))</span> <span class="o">==</span> <span class="n">load_inventory</span><span class="p">(</span><span class="s2">&#34;inventory.yml&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="kc">True</span>
</span></span></code></pre></div><p>That <code>==</code> compares two lists of four dicts, key by key. One function reads a file, the other asks a server,
and the code that uses the result can&rsquo;t tell which it came from. <a href="/posts/netboxasinventory/">NetBox as your Ansible
inventory</a> does the same thing for Ansible, and the <span class="link-soon">NetBox course</span>
builds a real one.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>yaml.load(f)</code> raises <code>TypeError</code> about <code>Loader</code>.</strong> PyYAML 6 wants a loader named. Use <code>yaml.safe_load(f)</code>.</p>
<p><strong>A value comes back as <code>False</code>, <code>True</code> or a number you didn&rsquo;t write.</strong> YAML 1.1 read it as a boolean, an octal
or a base-60 number. Quote it.</p>
<p><strong>A key is <code>True</code> instead of <code>'on'</code>.</strong> The same rule, on a key: <code>d[&quot;on&quot;]</code> raises <code>KeyError</code>. Quote the key.</p>
<p><strong><code>yaml.scanner.ScannerError</code> or <code>ParserError</code>.</strong> Usually indentation: a tab, or one item indented differently
from its neighbors. YAML allows spaces only.</p>
<p><strong><code>KeyError: 'results'</code> after an API call.</strong> The API returned an error body, not a list. Call
<code>raise_for_status()</code> before reading the JSON.</p>
<p><strong>A 403 with the right token.</strong> Check the header&rsquo;s exact form: <code>Authorization: Token &lt;token&gt;</code>, with the word
<code>Token</code> and one space.</p>
<p><strong>The script hangs.</strong> No <code>timeout=</code>. Add one to every request.</p>
<p><strong><code>requests.exceptions.JSONDecodeError</code>.</strong> The body isn&rsquo;t JSON, often an HTML error page from a proxy or the
wrong URL. Look at <code>r.status_code</code> and <code>r.text</code>.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-7">Chapter 7 of the lab</a> has you load <code>inventory.yml</code> with <code>safe_load</code>, write it to
JSON and read it back, and find and quote the four values in the file that YAML reads differently from you.
Then you call the lab&rsquo;s API with the token, turn a 403 into one clear line, and reshape the nested JSON into
the same list of dicts the YAML gives.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>You can now read an inventory from a file or an API and log in to every device in it.
<a href="/lessons/python-101/inventory-to-csv/">Lesson 8</a> puts the two together: inventory in, <code>show version</code> from
every device, one CSV row each, and a clear list of the devices it couldn&rsquo;t reach.</p>
]]></content:encoded></item><item><title>Talking to Devices with Netmiko: SSH from Python (Python 101, Lesson 6)</title><link>https://networkdoodles.com/lessons/python-101/netmiko/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/netmiko/</guid><category>python</category><category>network-automation</category><category>netmiko</category><category>textfsm</category><category>capeos</category><description>Log in to routers from Python with Netmiko: ConnectHandler, send_command with TextFSM, config changes, passwords from the environment, and login errors handled.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p><strong>Netmiko</strong> is a Python library that logs in to network devices over SSH and runs CLI commands for you. One
call, <code>ConnectHandler()</code>, opens the session and gets it ready; <code>send_command()</code> runs a show command and
returns its output, already parsed into a list of dicts if you add <code>use_textfsm=True</code>; <code>send_config_set()</code>
pushes configuration lines. The username and password come from environment variables, never from the code,
and a failed login or a device that doesn&rsquo;t answer arrives as an exception you can catch.</p>
<p>Until now the course has parsed show output saved in files. This lesson gets it from live devices, and takes
a first look at <strong>objects</strong> and <strong>classes</strong>, because the connection Netmiko gives you is one. Every output
below comes from the <a href="/labs/python-101/">lab</a>: Netmiko 4.8.0 against r1 (CapeOS Bolt) and r3 (CapeOS Pulse).</p>
<hr>
<h2 id="what-does-netmiko-do-that-plain-ssh-doesnt">What does Netmiko do that plain SSH doesn&rsquo;t?</h2>
<p>SSH itself only carries bytes: you type, the device sends characters back. The Python library that speaks SSH
is <a href="https://www.paramiko.org/">paramiko</a>, and Netmiko is built on top of it. With paramiko alone you&rsquo;d have to
solve the problems a person at a terminal solves without thinking:</p>
<table>
  <thead>
      <tr>
          <th>Problem</th>
          <th>What a person does</th>
          <th>What Netmiko does</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><strong>The prompt.</strong> When has the output finished?</td>
          <td>waits until <code>r1#</code> comes back</td>
          <td>learns the prompt at login, and reads until it sees it again</td>
      </tr>
      <tr>
          <td><strong>Paging.</strong> Long output stops at <code>--More--</code></td>
          <td>presses space</td>
          <td>turns paging off at login (<code>terminal length 0</code>)</td>
      </tr>
      <tr>
          <td><strong>Line wrapping.</strong> Wide lines break</td>
          <td>widens the window</td>
          <td>sets the terminal width (<code>terminal width 511</code>)</td>
      </tr>
      <tr>
          <td><strong>Privilege.</strong> Some commands need <code>enable</code></td>
          <td>types <code>enable</code> and the secret</td>
          <td><code>conn.enable()</code>, with <code>secret=</code> given at login</td>
      </tr>
      <tr>
          <td><strong>Config mode.</strong> Changes need <code>configure terminal</code></td>
          <td>types it, then <code>end</code></td>
          <td><code>send_config_set()</code> enters and leaves for you</td>
      </tr>
  </tbody>
</table>
<p>You can watch it happen. Netmiko writes everything it sends and receives to a file if you give it
<code>session_log=&quot;session.log&quot;</code>. This is the log of a login to r1, one <code>show clock</code>, and the logout:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#terminal width 511
</span></span><span class="line"><span class="cl">r1#terminal length 0
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#show clock
</span></span><span class="line"><span class="cl">05:51:53.965 UTC Wed Nov 11 2026
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#exit
</span></span></code></pre></div><p>The first four lines are Netmiko&rsquo;s <strong>session preparation</strong>, the setup it runs before your first command. The
lab&rsquo;s login lands straight at the privileged <code>r1#</code> prompt, so there&rsquo;s no <code>enable</code> here: <code>conn.check_enable_mode()</code>
returns <code>True</code>. On a device that logs you in at <code>r1&gt;</code>, you&rsquo;d pass <code>secret=</code> to <code>ConnectHandler()</code> and call
<code>conn.enable()</code>.</p>
<p><a href="https://github.com/carlmontanari/scrapli">Scrapli</a> does the same job with a different design, including a
way to run many sessions at once. This course uses <a href="https://github.com/ktbyers/netmiko">Netmiko</a>, the one most
network automation examples are written with.</p>
<hr>
<h2 id="why-does-a-capeos-router-say-cisco_ios">Why does a CapeOS router say cisco_ios?</h2>
<p>Netmiko needs to know what kind of CLI it&rsquo;s talking to, because the commands that turn paging off and the
shape of the prompt differ from one network OS to another. You tell it with <strong><code>device_type</code></strong>, a name from
<a href="https://github.com/ktbyers/netmiko/blob/develop/PLATFORMS.md">Netmiko&rsquo;s list of platforms</a>, such as
<code>cisco_ios</code>, <code>arista_eos</code> or <code>juniper_junos</code>.</p>
<p><a href="/posts/capeos-lab-network-os/">CapeOS</a>, the lab&rsquo;s network OS, has two CLI dialects. <strong>Bolt</strong> (r1, r2 and sw1)
behaves like the CLI that Netmiko and ntc-templates call <code>cisco_ios</code>; <strong>Pulse</strong> (r3) behaves like the one they
call <code>arista_eos</code>. They behave closely enough that both libraries work against it unchanged. So in this lab
the <code>device_type</code> is what picks the dialect:</p>
<table>
  <thead>
      <tr>
          <th>Device</th>
          <th>CapeOS dialect</th>
          <th><code>device_type</code></th>
          <th>Netmiko&rsquo;s class</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>r1, r2, sw1</td>
          <td>Bolt</td>
          <td><code>cisco_ios</code></td>
          <td><code>CiscoIosSSH</code></td>
      </tr>
      <tr>
          <td>r3</td>
          <td>Pulse</td>
          <td><code>arista_eos</code></td>
          <td><code>AristaSSH</code></td>
      </tr>
  </tbody>
</table>
<p>That doesn&rsquo;t make CapeOS either of those products: it&rsquo;s an open-source OS of its own, and <code>device_type</code> names
the CLI each dialect behaves like. The same name also picks the ntc-templates template (lesson 5) for parsed
output, so a wrong <code>device_type</code> can log in fine and still prepare the session and parse for the wrong CLI.</p>
<hr>
<h2 id="how-do-you-log-in-and-run-a-command">How do you log in and run a command?</h2>
<p><code>ConnectHandler()</code> takes the device&rsquo;s details as <strong>keyword arguments</strong>, <code>name=value</code> pairs, so the order
doesn&rsquo;t matter and each value says what it is. It opens the SSH session, runs the session preparation, and
gives you back the connection:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">os</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netmiko</span> <span class="kn">import</span> <span class="n">ConnectHandler</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.env</span> <span class="kn">import</span> <span class="n">load_env</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">load_env</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">ConnectHandler</span><span class="p">(</span><span class="n">device_type</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="n">username</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_USER&#34;</span><span class="p">],</span> <span class="n">password</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_PASSWORD&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span><span class="o">.</span><span class="n">find_prompt</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;r1#&#39;</span>
</span></span></code></pre></div><p><code>host</code> can be a name or an address: in the lab, <code>r1</code> resolves to <code>172.80.111.11</code> on the management network.
<code>find_prompt()</code> asks the device for its prompt, the quickest way to prove the login worked.</p>
<p><code>send_command()</code> runs one command and returns everything the device printed, as a single string:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="n">Interface</span>              <span class="n">IP</span><span class="o">-</span><span class="n">Address</span>      <span class="n">OK</span><span class="err">?</span> <span class="n">Method</span> <span class="n">Status</span>                <span class="n">Protocol</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet0</span>       <span class="mf">172.80.111.11</span>   <span class="n">YES</span> <span class="n">DHCP</span>   <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet1</span>       <span class="mf">192.168.10.1</span>    <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet2</span>       <span class="mf">10.0.12.1</span>       <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet3</span>       <span class="mf">10.0.13.1</span>       <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet4</span>       <span class="n">unassigned</span>      <span class="n">YES</span> <span class="n">unset</span>  <span class="n">administratively</span> <span class="n">down</span> <span class="n">down</span>
</span></span><span class="line"><span class="cl"><span class="n">Loopback0</span>              <span class="mf">10.255.0.1</span>      <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span></code></pre></div><p>That&rsquo;s the text lessons 2 to 5 parsed by hand, with <code>split()</code>, a regex and TextFSM. Now it comes from the
router instead of a file. <code>send_command()</code> waits for the prompt to come back before it returns, for up to 10
seconds by default (<code>read_timeout=</code>); a slow command such as a large <code>show running-config</code> may need more.</p>
<p><code>conn.disconnect()</code> logs out. A device allows only so many SSH sessions at once (its VTY lines), and a script
that forgets to disconnect holds them until they time out.</p>
<hr>
<h2 id="what-does-use_textfsmtrue-give-you">What does use_textfsm=True give you?</h2>
<p>Add <code>use_textfsm=True</code> and Netmiko hands the output to ntc-templates before returning it. You get the same
list of dicts as <a href="/lessons/python-101/regex-and-textfsm/">lesson 5</a>&rsquo;s <code>parse_output()</code>, with no parsing code
of your own:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">rows</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">use_textfsm</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">type</span><span class="p">(</span><span class="n">rows</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="o">&lt;</span><span class="k">class</span> <span class="err">&#39;</span><span class="nc">list</span><span class="s1">&#39;&gt;, 6)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">rows</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;192.168.10.1&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;proto&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>Six interfaces, six dicts. Ask r3, the Pulse router, the same question with <code>device_type=&quot;arista_eos&quot;</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r3</span> <span class="o">=</span> <span class="n">ConnectHandler</span><span class="p">(</span><span class="n">device_type</span><span class="o">=</span><span class="s2">&#34;arista_eos&#34;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s2">&#34;r3&#34;</span><span class="p">,</span> <span class="n">username</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_USER&#34;</span><span class="p">],</span> <span class="n">password</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_PASSWORD&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r3</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">use_textfsm</span><span class="o">=</span><span class="kc">True</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;Ethernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;10.0.13.2/30&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;mtu&#39;</span><span class="p">:</span> <span class="s1">&#39;1500&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>Same command, different template, and the keys differ: <code>protocol</code> instead of <code>proto</code>, an <code>mtu</code> column, and the
address with its prefix length, because that&rsquo;s how Pulse prints it. Each template&rsquo;s keys come from the
template&rsquo;s author, not from a standard. Code that runs against two device types should use the keys they share
(<code>interface</code>, <code>status</code>) or translate one shape into the other.</p>
<p>If there&rsquo;s no template for a platform and command, <code>use_textfsm=True</code> gives you back the plain string, as if
you hadn&rsquo;t asked. Lesson 5 found that ntc-templates has none for <code>show ntp associations</code> on <code>cisco_ios</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">out</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ntp associations&#34;</span><span class="p">,</span> <span class="n">use_textfsm</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">type</span><span class="p">(</span><span class="n">out</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&lt;</span><span class="k">class</span> <span class="err">&#39;</span><span class="nc">str</span><span class="s1">&#39;&gt;</span>
</span></span></code></pre></div><p>No error, no warning. A <code>for</code> loop over that string walks it one character at a time, which is a confusing
way to find out.</p>
<hr>
<h2 id="how-do-you-change-the-configuration">How do you change the configuration?</h2>
<p><code>send_config_set()</code> takes a list of configuration lines. It enters configuration mode, sends each line,
leaves again, and returns the whole conversation. Back on r1&rsquo;s connection:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">send_config_set</span><span class="p">([</span><span class="s2">&#34;interface GigabitEthernet4&#34;</span><span class="p">,</span> <span class="s2">&#34;description spare, configured from Python&#34;</span><span class="p">]))</span>
</span></span><span class="line"><span class="cl"><span class="n">configure</span> <span class="n">terminal</span>
</span></span><span class="line"><span class="cl"><span class="n">Configuration</span> <span class="n">mode</span><span class="p">:</span> <span class="n">one</span> <span class="n">command</span> <span class="n">per</span> <span class="n">line</span><span class="o">.</span> <span class="n">Finish</span> <span class="k">with</span> <span class="s1">&#39;end&#39;</span> <span class="ow">or</span> <span class="n">Ctrl</span><span class="o">-</span><span class="n">Z</span><span class="o">.</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="p">)</span><span class="c1">#interface GigabitEthernet4</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="o">-</span><span class="k">if</span><span class="p">)</span><span class="c1">#description spare, configured from Python</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="o">-</span><span class="k">if</span><span class="p">)</span><span class="c1">#end</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="c1">#</span>
</span></span></code></pre></div><p><code>show interfaces description</code> afterwards lists <code>Gi4</code> with its new description.</p>
<p>Read that output, because by default Netmiko doesn&rsquo;t judge it. r1 has no <code>GigabitEthernet9</code>, and this is
what comes back for it, with no exception:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">send_config_set</span><span class="p">([</span><span class="s2">&#34;interface GigabitEthernet9&#34;</span><span class="p">]))</span>
</span></span><span class="line"><span class="cl"><span class="n">configure</span> <span class="n">terminal</span>
</span></span><span class="line"><span class="cl"><span class="n">Configuration</span> <span class="n">mode</span><span class="p">:</span> <span class="n">one</span> <span class="n">command</span> <span class="n">per</span> <span class="n">line</span><span class="o">.</span> <span class="n">Finish</span> <span class="k">with</span> <span class="s1">&#39;end&#39;</span> <span class="ow">or</span> <span class="n">Ctrl</span><span class="o">-</span><span class="n">Z</span><span class="o">.</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="p">)</span><span class="c1">#interface GigabitEthernet9</span>
</span></span><span class="line"><span class="cl">                     <span class="o">^</span>
</span></span><span class="line"><span class="cl"><span class="o">%</span> <span class="n">Invalid</span> <span class="nb">input</span> <span class="n">detected</span> <span class="n">at</span> <span class="s1">&#39;^&#39;</span> <span class="n">marker</span><span class="o">.</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="p">)</span><span class="c1">#end</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="c1">#</span>
</span></span></code></pre></div><p>The refusal is in the returned string, and your code has to look for it. A change that works is in the
running configuration only, as if you&rsquo;d typed it; saving it is a separate command. Pushing configuration from
a loop is fast in both directions, which is why tools such as <a href="/lessons/ansible-101/">Ansible</a> wrap it in
checks, dry runs and diffs.</p>
<hr>
<h2 id="where-should-the-password-live">Where should the password live?</h2>
<p>Not in the code. A script gets copied, pasted into chat, committed to Git and shared, and a password written
in it goes everywhere the script goes. Git keeps it in the history even after you delete the line.
<a href="/posts/donthardcodepasswords/">Don&rsquo;t hardcode passwords</a> covers the same idea for Ansible.</p>
<p>The usual answer is an <strong>environment variable</strong>: a named value the shell hands to every program it starts
(Linux CLI 101 covers <code>export</code> and <code>env</code>). Python reads them from <code>os.environ</code>, a dict-like object:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_USER&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;admin&#39;</span>
</span></span></code></pre></div><p>To save typing <code>export</code> in every terminal, the values go in a file called <strong><code>.env</code></strong>, one <code>KEY=value</code> per line,
in the project folder. The lab&rsquo;s has the login and the API&rsquo;s token in it. <code>make setup</code> copies it from
<code>.env.example</code>, and <code>.gitignore</code> lists <code>.env</code>, so Git never picks it up. The lab&rsquo;s <code>netlab/env.py</code> has a small
<code>load_env()</code> that reads <code>.env</code> into <code>os.environ</code>, the same thing the
<a href="https://pypi.org/project/python-dotenv/">python-dotenv</a> package does; a variable already set in your shell
wins over the file.</p>
<p>So the code names the variable, and the value lives somewhere else:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">password</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_PASSWORD&#34;</span><span class="p">]</span>
</span></span></code></pre></div><p>The lab&rsquo;s check for task 6.1 tests this both ways. It fails if the lab&rsquo;s password appears anywhere in your
code, and it logs in with a deliberately wrong <code>LAB_PASSWORD</code> in the environment, expecting the login to fail.
If your code still gets in, the password didn&rsquo;t come from the environment.</p>
<hr>
<h2 id="what-happens-when-the-login-fails">What happens when the login fails?</h2>
<p>Netmiko raises an <strong>exception</strong>, the same mechanism as the <code>FileNotFoundError</code> in
<a href="/lessons/python-101/files-and-errors/">lesson 4</a>. A wrong password gives this, trimmed to its last part:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&gt;&gt;&gt; ConnectHandler(device_type=&#34;cisco_ios&#34;, host=&#34;r1&#34;, username=&#34;admin&#34;, password=&#34;wrong&#34;)
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">paramiko.ssh_exception.AuthenticationException: Authentication failed.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">During handling of the above exception, another exception occurred:
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">netmiko.exceptions.NetmikoAuthenticationException: Authentication to device failed.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Common causes of this problem are:
</span></span><span class="line"><span class="cl">1. Invalid username and password
</span></span><span class="line"><span class="cl">2. Incorrect SSH-key file
</span></span><span class="line"><span class="cl">3. Connecting to the wrong device
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Device settings: cisco_ios r1:22
</span></span></code></pre></div><p>Two tracebacks, read from the bottom up: paramiko failed to authenticate, and Netmiko caught that and raised
its own, clearer exception. A device that doesn&rsquo;t answer at all, here an address with nothing on it, gives a
different one after <code>conn_timeout</code> seconds (10 by default; the lab uses 5), with paramiko&rsquo;s
<em>Unable to connect to port 22 on 172.80.111.19</em> underneath:</p>
<p><code>netmiko.exceptions.NetmikoTimeoutException: TCP connection to device failed.</code></p>
<table>
  <thead>
      <tr>
          <th>Exception</th>
          <th>Means</th>
          <th>Usual cause</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>NetmikoAuthenticationException</code></td>
          <td>SSH connected, the login was refused</td>
          <td>wrong username or password, wrong <code>.env</code></td>
      </tr>
      <tr>
          <td><code>NetmikoTimeoutException</code></td>
          <td>nothing answered on port 22</td>
          <td>the device is down, wrong address, a firewall</td>
      </tr>
  </tbody>
</table>
<p>Both live in <code>netmiko.exceptions</code>, so you can catch each one with <code>try</code>/<code>except</code> and say what happened in
one line that names the device:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">netmiko.exceptions</span> <span class="kn">import</span> <span class="n">NetmikoAuthenticationException</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">conn</span> <span class="o">=</span> <span class="n">ConnectHandler</span><span class="p">(</span><span class="n">device_type</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="n">username</span><span class="o">=</span><span class="n">user</span><span class="p">,</span> <span class="n">password</span><span class="o">=</span><span class="n">password</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">except</span> <span class="n">NetmikoAuthenticationException</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;r1: login failed&#34;</span><span class="p">)</span>
</span></span></code></pre></div><p>The person running the script reads one line instead of two screens of traceback, and it tells them what to
fix.</p>
<hr>
<h2 id="what-is-an-object-and-what-is-a-class">What is an object, and what is a class?</h2>
<p>Look at what <code>ConnectHandler()</code> returned:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span>
</span></span><span class="line"><span class="cl"><span class="o">&lt;</span><span class="n">netmiko</span><span class="o">.</span><span class="n">cisco</span><span class="o">.</span><span class="n">cisco_ios</span><span class="o">.</span><span class="n">CiscoIosSSH</span> <span class="nb">object</span> <span class="n">at</span> <span class="mh">0xffff9c629cd0</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span><span class="o">.</span><span class="n">base_prompt</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;r1&#39;</span>
</span></span></code></pre></div><p><code>conn</code> is an <strong>object</strong>: a value that holds data and also has functions of its own. The data are its
<strong>attributes</strong>, read with a dot: <code>conn.base_prompt</code> is the prompt without the <code>#</code>. The functions are its
<strong>methods</strong>, called with a dot and parentheses: <code>conn.send_command(...)</code>, <code>conn.disconnect()</code>. You&rsquo;ve used
methods since lesson 2, because a string is an object too: <code>line.split()</code> is the <code>split</code> method of one
string.</p>
<p>A <strong>class</strong> is the kind of object, the blueprint every object of that kind is made from. <code>conn</code> is a
<code>CiscoIosSSH</code>; <code>r3</code> above is an <code>AristaSSH</code>. They&rsquo;re different classes with the same method names, so
<code>send_command()</code> works on both, and each does the right thing for its CLI. That&rsquo;s why the code doesn&rsquo;t
change between Bolt and Pulse: only <code>device_type</code> does, and <code>ConnectHandler()</code> picks the class from it.</p>
<p>You can make classes of your own. This course needs only one, and it&rsquo;s the simplest kind: a <strong>dataclass</strong>,
a class that only holds a few named values. The lab&rsquo;s inventory of devices uses one:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">dataclasses</span> <span class="kn">import</span> <span class="n">dataclass</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd">@dataclass</span>
</span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">Device</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
</span></span><span class="line"><span class="cl">    <span class="n">host</span><span class="p">:</span> <span class="nb">str</span>
</span></span><span class="line"><span class="cl">    <span class="n">device_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&#34;cisco_ios&#34;</span>
</span></span></code></pre></div><p>The lines inside list the <strong>fields</strong>, each with a type (<code>str</code>, a string) that Python records but doesn&rsquo;t
enforce. <code>= &quot;cisco_ios&quot;</code> is a default, like a function&rsquo;s default argument. The <code>@dataclass</code> line above the
class (a <strong>decorator</strong>) writes the rest for you: a way to create one, a readable printout, and <code>==</code> that
compares the fields:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">d</span> <span class="o">=</span> <span class="n">Device</span><span class="p">(</span><span class="s2">&#34;r3&#34;</span><span class="p">,</span> <span class="s2">&#34;172.80.111.13&#34;</span><span class="p">,</span> <span class="s2">&#34;arista_eos&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">d</span>
</span></span><span class="line"><span class="cl"><span class="n">Device</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;r3&#39;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s1">&#39;172.80.111.13&#39;</span><span class="p">,</span> <span class="n">device_type</span><span class="o">=</span><span class="s1">&#39;arista_eos&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">Device</span><span class="p">(</span><span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="s2">&#34;172.80.111.11&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">Device</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s1">&#39;172.80.111.11&#39;</span><span class="p">,</span> <span class="n">device_type</span><span class="o">=</span><span class="s1">&#39;cisco_ios&#39;</span><span class="p">)</span>
</span></span></code></pre></div><p>Why not a dict? <code>d.host</code> is shorter than <code>d[&quot;host&quot;]</code>, making one with a field missing fails at once (a
<code>TypeError</code>) where a dict would quietly lack the key, and the printout says what the thing is. The
<a href="https://docs.python.org/3/library/dataclasses.html">dataclasses documentation</a> has the rest. Classes that
build on other classes (<strong>inheritance</strong>) wait for ANTA, where every test you
<span class="link-soon">write yourself</span> is a class built on ANTA&rsquo;s <code>AntaTest</code>.</p>
<hr>
<h2 id="how-does-one-script-talk-to-every-device">How does one script talk to every device?</h2>
<p>With a list of <code>Device</code>s and a loop: connect, run the command, disconnect, keep the result in a dict under the
device&rsquo;s name. That&rsquo;s <code>collect()</code>, the function task 6.2 asks for, and the lab&rsquo;s <code>scripts/collect.py</code> prints a
line per device from what it returns:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/collect.py
</span></span><span class="line"><span class="cl">r1   6 IP interfaces, 5 up
</span></span><span class="line"><span class="cl">r2   6 IP interfaces, 5 up
</span></span><span class="line"><span class="cl">sw1  10 IP interfaces, 4 up
</span></span><span class="line"><span class="cl">r3   4 IP interfaces, 4 up
</span></span></code></pre></div><p>Three Bolt devices and a Pulse router, through one function, because each <code>Device</code> carries its own
<code>device_type</code>. The <code>disconnect()</code> goes in a <code>finally:</code> block, so the session is closed even when the command
fails. With r2&rsquo;s password changed (the lab&rsquo;s <code>make break T=6.2</code>), the same script stops at r2 with one line,
and exits 1 so a shell script or a CI job can tell it failed:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/collect.py
</span></span><span class="line"><span class="cl">error: r2: login failed (check LAB_USER and LAB_PASSWORD)
</span></span></code></pre></div><p>The devices are done one at a time, and the first failure stops the run. Lesson 8 carries on past failures;
Nornir, in a later course, runs the same work on many devices at once.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>KeyError: 'LAB_PASSWORD'</code>.</strong> The variable isn&rsquo;t set: <code>.env</code> is missing (<code>make setup</code> makes it), or
<code>load_env()</code> wasn&rsquo;t called before <code>os.environ</code> was read.</p>
<p><strong>The password is in the code &ldquo;for now&rdquo;.</strong> It ends up in Git and in every copy of the script. Read it from the
environment from the first line.</p>
<p><strong><code>NetmikoAuthenticationException</code>, though <code>ssh</code> by hand works.</strong> Compare <code>.env</code> with what you typed, username
included.</p>
<p><strong><code>NetmikoTimeoutException</code>.</strong> Nothing answered on port 22. Check the address and that the device is up before
suspecting the code.</p>
<p><strong>The wrong <code>device_type</code>.</strong> The login may work, but the session preparation and the template are for another
CLI. Use the name your inventory gives that device.</p>
<p><strong><code>use_textfsm=True</code> returns a string.</strong> No template for that platform and command. Check <code>type()</code> before
looping.</p>
<p><strong>Sessions left open, and unread config output.</strong> Put <code>disconnect()</code> in a <code>finally:</code> block, and look for
<code>% Invalid</code> in what <code>send_config_set()</code> returns.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-6">Chapter 6 of the lab</a> has you write <code>connect()</code> and <code>interface_table()</code> with the
login read from <code>.env</code>, and run a script that prints r1&rsquo;s interfaces. Then you write the <code>Device</code> dataclass
and <code>collect()</code>, log in to r1, r2, sw1 and r3 (two device types through one function), and turn a refused
login into one clear line instead of a traceback.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>So far the list of devices is written in the script. <a href="/lessons/python-101/yaml-json-rest/">Lesson 7</a> moves it
out, into a YAML inventory file and then a REST API, and shows that both end up as the same list of dicts.</p>
]]></content:encoded></item><item><title>Parsing Show Output: Regular Expressions and TextFSM (Python 101, Lesson 5)</title><link>https://networkdoodles.com/lessons/python-101/regex-and-textfsm/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/regex-and-textfsm/</guid><category>python</category><category>network-automation</category><category>textfsm</category><category>howto</category><category>capeos</category><description>Turn show ip interface brief into data with a regular expression and with TextFSM and ntc-templates, see how a template is picked, and write your own.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p><code>show</code> output is a table laid out for people, and the job of a parser is to turn it into data: a list of
dicts, one per row, with the same keys every time. A <strong>regular expression</strong> describes the shape of a line
and pulls out the parts you name. A <strong>TextFSM template</strong> does the same for a whole output, with a few rules
for which lines to skip, and <a href="https://github.com/networktocode/ntc-templates">ntc-templates</a> is a library of
979 of them, picked by platform and command. Use a regex for one value or a one-off; use ntc-templates when it
has a template for your command; write a template of your own when it doesn&rsquo;t.</p>
<p><a href="/lessons/python-101/strings-lists-dicts/">Lesson 2</a> did this by hand with <code>split()</code>, and you saw how much
care one two-word status needed. This lesson does it properly, twice, on the same <code>show ip interface brief</code>
from r1, and the <a href="/labs/python-101/">lab</a>&rsquo;s chapter 5 checks that both ways give exactly the same answer.</p>
<hr>
<h2 id="why-is-screen-scraping-fragile">Why is screen-scraping fragile?</h2>
<p><strong>Screen-scraping</strong> means reading data out of text that was laid out for a person to look at. Here&rsquo;s r1&rsquo;s
<code>show ip interface brief</code>, which you know from <a href="/lessons/networking-101/">Networking 101</a>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Interface              IP-Address      OK? Method Status                Protocol
</span></span><span class="line"><span class="cl">GigabitEthernet0       172.80.111.11   YES DHCP   up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet1       192.168.10.1    YES manual up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet2       10.0.12.1       YES manual up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet3       10.0.13.1       YES manual up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet4       unassigned      YES unset  administratively down down
</span></span><span class="line"><span class="cl">Loopback0              10.255.0.1      YES manual up                    up
</span></span></code></pre></div><p>A person sees six columns. Python sees a string, and <code>split()</code> sees words:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">line</span> <span class="o">=</span> <span class="s2">&#34;GigabitEthernet4       unassigned      YES unset  administratively down down&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;unset&#39;</span><span class="p">,</span> <span class="s1">&#39;administratively&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">())</span>
</span></span><span class="line"><span class="cl"><span class="mi">7</span>
</span></span></code></pre></div><p>Seven items where every other line has six, so &ldquo;the status is item 4&rdquo; is wrong on exactly the line you care
about. Every kind of change breaks a scraper a different way: a two-word value, a blank line, a header
repeated after a page break, a new column in the next software release, or the same command on another
platform with another layout.</p>
<p>What fixes it is <strong>structured data</strong>: values with names, like
<code>{&quot;interface&quot;: &quot;GigabitEthernet4&quot;, &quot;status&quot;: &quot;administratively down&quot;}</code>. Code that reads <code>row[&quot;status&quot;]</code>
doesn&rsquo;t care which column it came from. The best source of structured data is the device itself, over an
API, and lesson 7 does that. For the many commands that only print text, a parser is the place where the
fragile part lives, once, tested, so the rest of your code never sees it.</p>
<hr>
<h2 id="what-is-a-regular-expression">What is a regular expression?</h2>
<p>A <strong>regular expression</strong>, or <strong>regex</strong>, is a small pattern language for text. Python&rsquo;s is the
<a href="https://docs.python.org/3/library/re.html"><code>re</code> module</a>. Most characters match themselves, and a few have
meanings:</p>
<table>
  <thead>
      <tr>
          <th>Pattern</th>
          <th>Matches</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>\S</code></td>
          <td>one character that isn&rsquo;t whitespace</td>
      </tr>
      <tr>
          <td><code>\s</code></td>
          <td>one whitespace character: a space, a tab</td>
      </tr>
      <tr>
          <td><code>\d</code></td>
          <td>one digit</td>
      </tr>
      <tr>
          <td><code>+</code></td>
          <td>one or more of the thing before it: <code>\S+</code> is a word, <code>\s+</code> is a gap</td>
      </tr>
      <tr>
          <td><code>^</code> and <code>$</code></td>
          <td>the start and the end of the line</td>
      </tr>
      <tr>
          <td><code>up|down</code></td>
          <td>either one</td>
      </tr>
      <tr>
          <td><code>\.</code></td>
          <td>a real dot (a bare <code>.</code> means any character)</td>
      </tr>
      <tr>
          <td><code>( )</code></td>
          <td>a <strong>group</strong>: the part you want back</td>
      </tr>
  </tbody>
</table>
<p>Patterns are written as <strong>raw strings</strong>, <code>r&quot;...&quot;</code>, so that Python leaves the backslashes alone for <code>re</code> to
read. An address is <code>\d+\.\d+\.\d+\.\d+</code>; an interface name followed by a gap is <code>\S+\s+</code>.</p>
<h3 id="what-do-groups-give-you">What do groups give you?</h3>
<p>A group marks the part of the match you want to keep. <code>re.match()</code> returns a <strong>match object</strong>, and
<code>.group(1)</code> is the first group&rsquo;s text:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">re</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;(\S+)\s+(\S+)&#34;</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="n">m</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">)</span>
</span></span></code></pre></div><p>Numbers get confusing by the fourth group, so name them. <code>(?P&lt;name&gt;...)</code> is a <strong>named group</strong>, and
<code>groupdict()</code> hands the whole match back as a dict, which is already the shape you want:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">version</span> <span class="o">=</span> <span class="s2">&#34;CapeOS Software (CO-BOLT), Version 0.11.2, LAB USE ONLY&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;\((?P&lt;image&gt;[^)]+)\), Version (?P&lt;version&gt;[^,]+)&#34;</span><span class="p">,</span> <span class="n">version</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span><span class="p">[</span><span class="s2">&#34;image&#34;</span><span class="p">],</span> <span class="n">m</span><span class="p">[</span><span class="s2">&#34;version&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="s1">&#39;CO-BOLT&#39;</span><span class="p">,</span> <span class="s1">&#39;0.11.2&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span><span class="o">.</span><span class="n">groupdict</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;image&#39;</span><span class="p">:</span> <span class="s1">&#39;CO-BOLT&#39;</span><span class="p">,</span> <span class="s1">&#39;version&#39;</span><span class="p">:</span> <span class="s1">&#39;0.11.2&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p><code>[^)]+</code> means &ldquo;one or more characters that aren&rsquo;t <code>)</code>&rdquo;, a common way to say &ldquo;up to the next bracket&rdquo;.</p>
<h3 id="match-search-or-findall"><code>match</code>, <code>search</code> or <code>findall</code>?</h3>
<table>
  <thead>
      <tr>
          <th>Function</th>
          <th>Looks</th>
          <th>Returns</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>re.match(p, s)</code></td>
          <td>only at the start of <code>s</code></td>
          <td>a match object, or <code>None</code></td>
      </tr>
      <tr>
          <td><code>re.search(p, s)</code></td>
          <td>anywhere in <code>s</code>, first match</td>
          <td>a match object, or <code>None</code></td>
      </tr>
      <tr>
          <td><code>re.findall(p, s)</code></td>
          <td>everywhere in <code>s</code></td>
          <td>a list: strings for one group, tuples for several</td>
      </tr>
  </tbody>
</table>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;Version (\S+),&#34;</span><span class="p">,</span> <span class="n">version</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="kc">None</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;Version (\S+),&#34;</span><span class="p">,</span> <span class="n">version</span><span class="p">)</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;0.11.2&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">text</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/brief-r1.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(\S+)\s+(\d+\.\d+\.\d+\.\d+)&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">,</span> <span class="n">re</span><span class="o">.</span><span class="n">M</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[(</span><span class="s1">&#39;GigabitEthernet0&#39;</span><span class="p">,</span> <span class="s1">&#39;172.80.111.11&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;192.168.10.1&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;GigabitEthernet2&#39;</span><span class="p">,</span> <span class="s1">&#39;10.0.12.1&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;GigabitEthernet3&#39;</span><span class="p">,</span> <span class="s1">&#39;10.0.13.1&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;Loopback0&#39;</span><span class="p">,</span> <span class="s1">&#39;10.255.0.1&#39;</span><span class="p">)]</span>
</span></span></code></pre></div><p><code>re.match</code> says <code>None</code> because the line doesn&rsquo;t <em>start</em> with <code>Version</code>. <code>re.M</code> (multiline) makes <code>^</code> mean the
start of every line, not only the start of the text; without it, the same <code>findall</code> for unassigned
interfaces finds nothing:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(\S+)\s+unassigned&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(?P&lt;interface&gt;\S+)\s+unassigned&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">,</span> <span class="n">re</span><span class="o">.</span><span class="n">M</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p>A <code>None</code> you didn&rsquo;t expect is the classic regex failure: <code>.group(1)</code> on it raises <code>AttributeError: 'NoneType' object has no attribute 'group'</code>. Check the match before you use it.</p>
<hr>
<h2 id="how-does-a-regex-parse-a-whole-table">How does a regex parse a whole table?</h2>
<p>One pattern for one line, with a named group per column, applied to every line of the output. The status
column is the hard part, so it spells out the three values it can have:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">pattern</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(?P&lt;interface&gt;\S+)\s+(?P&lt;ip_address&gt;\S+)\s+(?P&lt;ok&gt;YES|NO)\s+(?P&lt;method&gt;\S+)\s+(?P&lt;status&gt;up|down|administratively down)\s+(?P&lt;protocol&gt;\S+)\s*$&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">pattern</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="n">line</span><span class="p">)</span><span class="o">.</span><span class="n">groupdict</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;ok&#39;</span><span class="p">:</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;method&#39;</span><span class="p">:</span> <span class="s1">&#39;unset&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;down&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">pattern</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="s2">&#34;Interface              IP-Address      OK? Method Status                Protocol&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="kc">None</span>
</span></span></code></pre></div><p><code>re.compile()</code> turns the pattern into an object once, so a loop doesn&rsquo;t rebuild it for every line. Because
the pattern is strict (<code>YES|NO</code> in the third column, a known status), the header line doesn&rsquo;t match, and
neither does a blank line: the loop keeps the lines that match and skips the rest. That&rsquo;s the whole of a
regex parser, and in the lab it&rsquo;s about ten lines.</p>
<p>The strictness cuts both ways. On r3, which runs CapeOS Pulse, the same command prints a different table
(<code>Ethernet1  10.0.13.2/30  up  up  1500</code>, no <code>OK?</code> column), and the Bolt pattern matches none of it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.parsers</span> <span class="kn">import</span> <span class="n">BRIEF_LINE</span><span class="p">,</span> <span class="n">parse_brief_regex</span><span class="p">,</span> <span class="n">parse_brief_ntc</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">pulse</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/pulse-brief-r3.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_brief_regex</span><span class="p">(</span><span class="n">pulse</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span></code></pre></div><p>That&rsquo;s the right result for a pattern written for one layout. It&rsquo;s also the reason a hand-written regex
doesn&rsquo;t scale to every command on every platform you run.</p>
<hr>
<h2 id="what-is-textfsm">What is TextFSM?</h2>
<p><a href="https://github.com/google/textfsm/wiki/TextFSM">TextFSM</a> is a Python module, written at Google, that parses
text with a <strong>template</strong>: a small file that names the values you want and gives the rules for finding them. FSM
stands for <em>finite state machine</em>: the template has <strong>states</strong>, and the parser is in one state at a time,
which lets it treat the header, the body and the footer of an output differently. This is a complete template
for two columns of the table above:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Value INTERFACE (\S+)
</span></span><span class="line"><span class="cl">Value STATUS (up|down|administratively down)
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Start
</span></span><span class="line"><span class="cl">  ^${INTERFACE}\s+\S+\s+(YES|NO)\s+\S+\s+${STATUS}\s -&gt; Record
</span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">textfsm</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;scratch/brief_two.textfsm&#34;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">fsm</span> <span class="o">=</span> <span class="n">textfsm</span><span class="o">.</span><span class="n">TextFSM</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">fsm</span><span class="o">.</span><span class="n">header</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;INTERFACE&#39;</span><span class="p">,</span> <span class="s1">&#39;STATUS&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">fsm</span><span class="o">.</span><span class="n">ParseText</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[[</span><span class="s1">&#39;GigabitEthernet0&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet2&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet3&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;Loopback0&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">]]</span>
</span></span></code></pre></div><p>A template has two parts:</p>
<ul>
<li><strong>Values</strong>, at the top: <code>Value NAME (regex)</code>, one per column. The regex in brackets says what that value
looks like. A value can have options before its name: <code>Required</code> (a row without it is dropped), <code>Filldown</code>
(keep it for the next rows, for a value printed once above several lines), <code>List</code> (collect several matches
into a list) and <code>Key</code>.</li>
<li><strong>States</strong>, after a blank line: a name on its own line, then <strong>rules</strong>, indented, each a regex that starts
with <code>^</code>. <code>${INTERFACE}</code> inside a rule stands for that value&rsquo;s regex, and whatever it matches is saved
under that name. Every template has a <code>Start</code> state.</li>
</ul>
<p>TextFSM reads the output a line at a time and tries the current state&rsquo;s rules in order. The first rule that
matches wins; a line that matches no rule is ignored. After the <code>-&gt;</code> comes the <strong>action</strong>: <code>Record</code> saves the
values collected so far as one row and clears them; a state name, like <code>-&gt; Servers</code>, moves to that state; and
<code>-&gt; Error</code> stops the parse with an error, which is how a template says &ldquo;I&rsquo;ve seen a line I don&rsquo;t understand&rdquo;.
At the end of the text, TextFSM records whatever is left. A literal <code>$</code> at the end of a rule is written <code>$$</code>,
because a single <code>$</code> starts a value&rsquo;s name.</p>
<p>The result is a list of rows in <code>fsm.header</code>&rsquo;s order. Zip the two together, and lower-case the names, and
you have the same list of dicts the regex gave you.</p>
<hr>
<h2 id="how-does-ntc-templates-pick-a-template">How does ntc-templates pick a template?</h2>
<p>Network to Code&rsquo;s <strong>ntc-templates</strong> is a package of TextFSM templates for common commands, community written
and tested, 979 of them in version 9.3.0. You don&rsquo;t name a template. You say which <strong>platform</strong> the output
came from and which <strong>command</strong> made it, and <code>parse_output()</code> finds the template and runs it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">ntc_templates.parse</span> <span class="kn">import</span> <span class="n">parse_output</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">text</span><span class="p">)[</span><span class="mi">4</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">,</span> <span class="s1">&#39;proto&#39;</span><span class="p">:</span> <span class="s1">&#39;down&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>Its field names are its own (<code>proto</code>, not <code>protocol</code>), and it doesn&rsquo;t turn <code>unassigned</code> into <code>None</code>. The
lab&rsquo;s <code>parse_brief_ntc()</code> maps those two so its rows equal the regex&rsquo;s exactly. The platform is <code>cisco_ios</code>
because <a href="/posts/capeos-lab-network-os/">CapeOS</a> Bolt prints this command the way that platform does; lesson 6 says more about why the lab
uses those names.</p>
<p>The choice is made by a file in the package, <code>templates/index</code>. Each line is a template file, a hostname
pattern, a platform and a command pattern:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">cisco_ios_show_ip_interface_brief.textfsm, .*, cisco_ios, sh[[ow]] ip int[[erface]] br[[ief]]
</span></span></code></pre></div><p><code>sh[[ow]]</code> means <code>sh</code>, <code>sho</code> or <code>show</code>, so the abbreviations you type on a device work too:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;sh ip int br&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">text</span><span class="p">)[</span><span class="mi">4</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">,</span> <span class="s1">&#39;proto&#39;</span><span class="p">:</span> <span class="s1">&#39;down&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>The index has 139 entries for <code>cisco_ios</code> and 47 for <code>arista_eos</code>. Pulse output goes to the <code>arista_eos</code>
template, which knows its columns, MTU included:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;arista_eos&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">pulse</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;Ethernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;10.0.13.2/30&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;mtu&#39;</span><span class="p">:</span> <span class="s1">&#39;1500&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">pulse</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span></code></pre></div><p>The second line is a trap worth remembering: the wrong platform gives an empty list, not an error. And when
there&rsquo;s no template at all, you get an exception. ntc-templates 9.3.0 has nothing for <code>show ntp associations</code>
on <code>cisco_ios</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&gt;&gt;&gt; parse_output(platform=&#34;cisco_ios&#34;, command=&#34;show ntp associations&#34;, data=open(&#34;checks/data/ntp-r1.txt&#34;).read())
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">ntc_templates.parse.ParsingException: Unable to parse command &#34;show ntp associations&#34; on platform cisco_ios - No template found for attributes: &#34;{&#39;Command&#39;: &#39;show ntp associations&#39;, &#39;Platform&#39;: &#39;cisco_ios&#39;}&#34;
</span></span></code></pre></div><p>That&rsquo;s the lab&rsquo;s task 5.2: you write that template yourself.</p>
<hr>
<h2 id="regex-or-textfsm-which-one-when">Regex or TextFSM: which one, when?</h2>
<table>
  <thead>
      <tr>
          <th>You want</th>
          <th>Use</th>
          <th>Why</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>one value from one output: a version, an uptime, a serial</td>
          <td>a regex, <code>re.search</code></td>
          <td>one line of code; a template would be overkill</td>
      </tr>
      <tr>
          <td>a table from a command ntc-templates covers</td>
          <td><code>parse_output()</code></td>
          <td>someone has already written and tested it, across releases</td>
      </tr>
      <tr>
          <td>a table from a command it doesn&rsquo;t cover</td>
          <td>your own TextFSM template</td>
          <td>states, <code>Required</code> and <code>-&gt; Error</code> handle headers, footers and surprises</td>
      </tr>
      <tr>
          <td>data from a device that can give it to you structured</td>
          <td>its API or JSON output</td>
          <td>no parsing at all, and nothing to break</td>
      </tr>
  </tbody>
</table>
<p>Regex is the tool you&rsquo;ll use most often, for small things. TextFSM earns its place on tables, and on outputs
with sections, like <code>show interfaces</code>, where values printed once at the top apply to the lines below. In lesson
6, Netmiko&rsquo;s <code>use_textfsm=True</code> calls ntc-templates for you, straight after the command runs, so most of the
time <code>parse_output()</code> is one argument away.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>AttributeError: 'NoneType' object has no attribute 'group'</code>.</strong> The regex didn&rsquo;t match, and <code>re.match</code> or
<code>re.search</code> returned <code>None</code>. Print the line you gave it; check whether it needed <code>search</code> rather than <code>match</code>.</p>
<p><strong><code>findall</code> returns <code>[]</code> on a whole output.</strong> <code>^</code> or <code>$</code> in the pattern without <code>re.M</code>.</p>
<p><strong>The header comes back as a row.</strong> The pattern is too loose: <code>\S+</code> matches <code>Interface</code> as happily as
<code>GigabitEthernet0</code>. Make one column strict, like <code>YES|NO</code>, or skip the header with a state.</p>
<p><strong>A two-word value split in half.</strong> <code>\S+</code> stops at the space in <code>administratively down</code>. Spell out the
values the column can have.</p>
<p><strong>An empty list from <code>parse_output()</code>, no error.</strong> The platform&rsquo;s wrong for this output, so a template ran and
nothing matched. Check what the device really is.</p>
<p><strong><code>ParsingException: ... No template found</code>.</strong> ntc-templates has nothing for that platform and command. Check
the index, or write the template.</p>
<p><strong><code>TextFSMError: State Error raised</code>.</strong> A line hit an <code>-&gt; Error</code> rule. The message gives the rule&rsquo;s line
number in the template and the input line it didn&rsquo;t expect.</p>
<p><strong>A different answer after an upgrade.</strong> A new ntc-templates release can rename a field. That&rsquo;s why the lab
pins 9.3.0.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-5">Chapter 5 of the lab</a> has you write <code>BRIEF_LINE</code> and <code>parse_brief_regex()</code>, then
<code>parse_brief_ntc()</code> with ntc-templates, and the check makes them agree on outputs they haven&rsquo;t seen. Then you
write <code>templates/ntp_associations.textfsm</code>, the template ntc-templates doesn&rsquo;t have, for r1&rsquo;s
<code>show ntp associations</code>.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>So far the show output has come from files. <a href="/lessons/python-101/netmiko/">Lesson 6</a> gets it from the devices
themselves, over SSH with Netmiko, with <code>use_textfsm=True</code> doing this lesson&rsquo;s parsing on the way back.</p>
]]></content:encoded></item><item><title>Files, Modules and Errors: Saved Configs In, a Report Out (Python 101, Lesson 4)</title><link>https://networkdoodles.com/lessons/python-101/files-and-errors/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/files-and-errors/</guid><category>python</category><category>network-automation</category><category>howto</category><category>capeos</category><description>Read and write files with pathlib, CSV and JSON, import your own module, read a traceback bottom up, and catch only the exceptions you can handle.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p>A saved config is a text file, and Python reads one in a line: <code>Path(&quot;configs/r1.cfg&quot;).read_text()</code>. The
<code>with</code> statement opens a file and closes it again whatever happens in between, and the standard library
already has the modules to write what you found as CSV or JSON. When something goes wrong, Python raises an
<strong>exception</strong> and prints a <strong>traceback</strong>: read it from the bottom up. Catch only the exceptions you know what
to do with, and let every other one stop the program.</p>
<p>This lesson builds the tool from the lab&rsquo;s chapter 4: read every saved config in a folder, say which devices
have an NTP server, write the answer to a CSV file, and keep going when one file can&rsquo;t be read. On the way it
covers modules and <code>import</code>, why the lab&rsquo;s scripts add a folder to <code>sys.path</code>, and what
<code>if __name__ == &quot;__main__&quot;</code> is for. Every output below is from the <a href="/labs/python-101/">lab</a>&rsquo;s codespace.</p>
<hr>
<h2 id="what-is-a-file-to-python">What is a file to Python?</h2>
<p>A text file, like a saved config, is bytes that stand for characters in an agreed <strong>encoding</strong>. Python finds
the file, turns the bytes into a string and hands you the string. From then on it&rsquo;s the same <code>str</code> as in
<a href="/lessons/python-101/strings-lists-dicts/">lesson 2</a>, and <code>splitlines()</code> and <code>startswith()</code> work as before.</p>
<p>The modern way to name a file is <a href="https://docs.python.org/3/library/pathlib.html"><code>pathlib</code></a>, from the
standard library. A <code>Path</code> knows it&rsquo;s a path: you build one with <code>/</code> and ask it questions, instead of gluing
strings together.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">pathlib</span> <span class="kn">import</span> <span class="n">Path</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s2">&#34;configs&#34;</span><span class="p">)</span> <span class="o">/</span> <span class="s2">&#34;r1.cfg&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">p</span><span class="o">.</span><span class="n">exists</span><span class="p">(),</span> <span class="n">p</span><span class="o">.</span><span class="n">is_file</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="kc">True</span><span class="p">,</span> <span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">p</span><span class="o">.</span><span class="n">stem</span><span class="p">,</span> <span class="n">p</span><span class="o">.</span><span class="n">suffix</span><span class="p">,</span> <span class="n">p</span><span class="o">.</span><span class="n">name</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;.cfg&#39;</span><span class="p">,</span> <span class="s1">&#39;r1.cfg&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">text</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="n">read_text</span><span class="p">(</span><span class="n">encoding</span><span class="o">=</span><span class="s2">&#34;utf-8&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">text</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()[:</span><span class="mi">3</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;Building configuration...&#39;</span><span class="p">,</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="s1">&#39;Current configuration : 741 bytes&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="p">[</span><span class="n">line</span> <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">text</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()</span> <span class="k">if</span> <span class="n">line</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&#34;ntp server&#34;</span><span class="p">)]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;ntp server 172.80.111.12&#39;</span><span class="p">,</span> <span class="s1">&#39;ntp server 172.80.111.99&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p><code>p.stem</code> is the name without the extension, and in the lab that&rsquo;s the device: <code>configs/r1.cfg</code> is r1. The
first lines are a real CapeOS save: <code>show running-config</code> starts with <code>Building configuration...</code> and a byte
count before the first <code>hostname</code>. Your code looks for the lines it wants and ignores the rest.</p>
<p>A folder is one call more. <code>glob(&quot;*.cfg&quot;)</code> finds every matching name, the way <code>ls *.cfg</code> does in the shell,
in whatever order the file system keeps them, so <code>sorted()</code> makes the order repeatable:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="p">[</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">Path</span><span class="p">(</span><span class="s2">&#34;configs&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">&#34;*.cfg&#34;</span><span class="p">))]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;locked.cfg&#39;</span><span class="p">,</span> <span class="s1">&#39;old-r9.cfg&#39;</span><span class="p">,</span> <span class="s1">&#39;r1.cfg&#39;</span><span class="p">,</span> <span class="s1">&#39;r2.cfg&#39;</span><span class="p">,</span> <span class="s1">&#39;r3.cfg&#39;</span><span class="p">,</span> <span class="s1">&#39;sw1.cfg&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p>Two of those files are there on purpose, to break things later in this lesson.</p>
<hr>
<h2 id="what-does-with-do">What does <code>with</code> do?</h2>
<p><code>read_text()</code> opens, reads and closes in one go. To read a line at a time or write in pieces, you <code>open()</code> the
file yourself, and then something has to close it. That&rsquo;s the job of <code>with</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;configs/r1.cfg&#34;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">first</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">first</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;Building configuration...</span><span class="se">\n</span><span class="s1">&#39;</span>
</span></span></code></pre></div><p><code>open()</code> gives back a <strong>file object</strong>, <code>f</code>, for as long as the indented block runs, and <code>with</code> closes it when
the block ends, even if the block raises an error. A script that opens a thousand configs and never closes
them runs out of file handles, and a file you were writing may not reach the disk. <code>with</code> is the
<a href="https://docs.python.org/3/reference/compound_stmts.html#the-with-statement">context manager</a> statement, and
anything that has to be opened and then closed again can offer it.</p>
<p>The second argument to <code>open()</code> is the <strong>mode</strong>: <code>&quot;r&quot;</code> reads (the default), <code>&quot;w&quot;</code> empties the file and writes,
and <code>&quot;a&quot;</code> adds to the end. Writing never adds a newline for you.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">out</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s2">&#34;reports/hello.txt&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">out</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">out</span><span class="o">.</span><span class="n">write_text</span><span class="p">(</span><span class="s2">&#34;r1 has NTP</span><span class="se">\n</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">11</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;reports/hello.txt&#34;</span><span class="p">,</span> <span class="s2">&#34;a&#34;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">&#34;r2 doesn&#39;t</span><span class="se">\n</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="mi">11</span>
</span></span></code></pre></div><p><code>mkdir(parents=True, exist_ok=True)</code> makes the folder if it isn&rsquo;t there and stays quiet if it is. <code>11</code> is the
number of characters written.</p>
<p>Why <code>encoding=&quot;utf-8&quot;</code> every time? Because on Python 3.11 the default comes from the operating system&rsquo;s
locale: UTF-8 in the codespace, perhaps something else on a Windows laptop or in a cron job. Saying it means
the same file reads the same way everywhere.</p>
<hr>
<h2 id="whats-in-the-standard-library-for-this">What&rsquo;s in the standard library for this?</h2>
<p>Most of what a network script does with files is already written, in the <strong>standard library</strong> that comes with
Python:</p>
<table>
  <thead>
      <tr>
          <th>Module</th>
          <th>What it&rsquo;s for</th>
          <th>In this lab</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><a href="https://docs.python.org/3/library/pathlib.html"><code>pathlib</code></a></td>
          <td>paths, folders, reading and writing whole files</td>
          <td>finding and reading <code>configs/*.cfg</code></td>
      </tr>
      <tr>
          <td><a href="https://docs.python.org/3/library/csv.html"><code>csv</code></a></td>
          <td>rows and columns, for spreadsheets</td>
          <td><code>reports/ntp.csv</code>, and the capstone&rsquo;s inventory</td>
      </tr>
      <tr>
          <td><a href="https://docs.python.org/3/library/json.html"><code>json</code></a></td>
          <td>lists and dicts as text, for APIs and other programs</td>
          <td>lesson 7&rsquo;s API, and printing a dict readably</td>
      </tr>
      <tr>
          <td><a href="https://docs.python.org/3/library/datetime.html"><code>datetime</code></a></td>
          <td>dates and times</td>
          <td>a date in a report&rsquo;s file name</td>
      </tr>
  </tbody>
</table>
<p><code>csv.writer</code> turns a list into a correctly quoted line, and <code>csv.DictReader</code> reads a file back as one dict per
row, keyed by the header:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">csv</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;reports/ntp.csv&#34;</span><span class="p">,</span> <span class="n">newline</span><span class="o">=</span><span class="s2">&#34;&#34;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">rows</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">csv</span><span class="o">.</span><span class="n">DictReader</span><span class="p">(</span><span class="n">f</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;device&#39;</span><span class="p">:</span> <span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="s1">&#39;has_ntp&#39;</span><span class="p">:</span> <span class="s1">&#39;yes&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="p">[</span><span class="n">r</span><span class="p">[</span><span class="s2">&#34;device&#34;</span><span class="p">]</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">rows</span> <span class="k">if</span> <span class="n">r</span><span class="p">[</span><span class="s2">&#34;has_ntp&#34;</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;no&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;r2&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p><code>newline=&quot;&quot;</code> is what the <code>csv</code> documentation asks for: the module handles line endings itself. <code>json.dumps()</code>
turns a dict into JSON text, here the files the finished chapter 4 code couldn&rsquo;t read, and <code>datetime</code> gives
you today in ISO 8601, the date format that sorts properly:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.configs</span> <span class="kn">import</span> <span class="n">load_configs_safe</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">configs</span><span class="p">,</span> <span class="n">problems</span> <span class="o">=</span> <span class="n">load_configs_safe</span><span class="p">(</span><span class="s2">&#34;configs&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">json</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">problems</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;locked.cfg&#34;</span><span class="p">:</span> <span class="s2">&#34;permission denied&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;old-r9.cfg&#34;</span><span class="p">:</span> <span class="s2">&#34;not UTF-8 text (byte 64)&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">datetime</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">today</span><span class="p">()</span><span class="o">.</span><span class="n">isoformat</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;2026-11-11&#39;</span>
</span></span></code></pre></div><p>Before you install a package for something, look in the
<a href="https://docs.python.org/3/library/index.html">library index</a> first.</p>
<hr>
<h2 id="what-is-a-module-and-what-does-import-do">What is a module, and what does <code>import</code> do?</h2>
<p>A <strong>module</strong> is a <code>.py</code> file. <code>import</code> runs it once and gives you its names: its functions and variables.
<code>import csv</code> finds <code>csv.py</code> in the standard library; <code>from netlab.configs import has_ntp</code> finds the lab&rsquo;s own
<code>netlab/configs.py</code> and takes one function from it. A folder of modules, like <code>netlab/</code>, is a <strong>package</strong>,
and the dot in <code>netlab.configs</code> is the folder.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">netlab.configs</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">netlab</span><span class="o">.</span><span class="n">configs</span><span class="o">.</span><span class="vm">__file__</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;/workspaces/python-101/netlab/configs.py&#39;</span>
</span></span></code></pre></div><p>That&rsquo;s why every chapter of the lab has two kinds of file. The functions live in a module, where the checker
and other scripts can import them and call them on any input. The script, <code>scripts/ntp_report.py</code>, is the thin
part on top: which folder, which output file, call the functions, print. A function you can import is a
function you can test. A script that does everything in one long run of lines can only be run.</p>
<h3 id="why-do-the-labs-scripts-change-syspath">Why do the lab&rsquo;s scripts change <code>sys.path</code>?</h3>
<p>Python finds a module by looking through a list of folders, <code>sys.path</code>, and taking the first match. The
<a href="https://docs.python.org/3/tutorial/modules.html#the-module-search-path">first entry</a> is the folder of the
script you ran, or in the REPL the current folder (<code>''</code>), followed by the standard library and the venv&rsquo;s
<code>site-packages</code>. So in a REPL started at the top of the repository, <code>import netlab</code> works. Run
<code>python scripts/ntp_report.py</code> and the first folder is <code>scripts/</code>, which has no <code>netlab</code> in it. Take two lines
out of the lab&rsquo;s script and this is what happens:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/ntp_report.py
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/scripts/ntp_report.py&#34;, line 12, in &lt;module&gt;
</span></span><span class="line"><span class="cl">    from netlab.configs import load_configs_safe, write_report  # noqa: E402
</span></span><span class="line"><span class="cl">    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">ModuleNotFoundError: No module named &#39;netlab&#39;
</span></span></code></pre></div><p>These are the two lines, near the top of every script in the lab:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">ROOT</span> <span class="o">=</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span><span class="p">(</span><span class="vm">__file__</span><span class="p">)</span><span class="o">.</span><span class="n">resolve</span><span class="p">()</span><span class="o">.</span><span class="n">parents</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">ROOT</span><span class="p">))</span>
</span></span></code></pre></div><p><code>__file__</code> is the script&rsquo;s own path, <code>.parents[1]</code> is two levels up from it (the repository), and
<code>sys.path.insert(0, ...)</code> puts that folder first. Now <code>import netlab</code> works from any folder. Bigger projects
install their package into the venv instead (<code>pip install -e .</code>), and then no script needs these lines.</p>
<hr>
<h2 id="what-is-if-__name__--__main__-for">What is <code>if __name__ == &quot;__main__&quot;</code> for?</h2>
<p>Every module has a name, <code>__name__</code>. Imported, it&rsquo;s the dotted name, <code>netlab.configs</code>. <strong>Run</strong> as a script,
Python calls it <code>&quot;__main__&quot;</code> instead. A file with one line, <code>print(&quot;__name__ is&quot;, __name__)</code>, shows both:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scratch/where.py
</span></span><span class="line"><span class="cl">__name__ is __main__
</span></span><span class="line"><span class="cl">$ python -c &#34;import scratch.where&#34;
</span></span><span class="line"><span class="cl">__name__ is scratch.where
</span></span></code></pre></div><p>That&rsquo;s what the last two lines of <code>scripts/ntp_report.py</code> test:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">&#34;__main__&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="n">main</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">))</span>
</span></span></code></pre></div><p>Run the file and <code>main()</code> runs. Import it, from a check or another script, and you get the <code>main</code> function
without anything happening: no folder read, no file written. The
<a href="https://docs.python.org/3/library/__main__.html"><code>__main__</code> documentation</a> calls it the idiom for top-level
code. <code>sys.exit()</code> hands <code>main()</code>&rsquo;s return value to the shell as the <strong>exit code</strong>, the <code>$?</code> from
<a href="/lessons/linux-cli-101/">Linux CLI 101</a>: 0 for &ldquo;all fine&rdquo;, anything else for &ldquo;look at me&rdquo;.</p>
<hr>
<h2 id="how-do-you-read-a-traceback">How do you read a traceback?</h2>
<p>When Python hits something it can&rsquo;t do, it <strong>raises an exception</strong>, an object that says what went wrong. If
nothing catches it, the program stops and prints a <strong>traceback</strong>: the chain of calls that led there. This is
the report script, with only the 4.1 functions written, on a folder with a file nobody may read:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/ntp_report.py
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/scripts/ntp_report.py&#34;, line 29, in &lt;module&gt;
</span></span><span class="line"><span class="cl">    sys.exit(main(sys.argv))
</span></span><span class="line"><span class="cl">             ^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/scripts/ntp_report.py&#34;, line 20, in main
</span></span><span class="line"><span class="cl">    configs, problems = load_configs_safe(folder)
</span></span><span class="line"><span class="cl">                        ^^^^^^^^^^^^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/netlab/configs.py&#34;, line 60, in load_configs_safe
</span></span><span class="line"><span class="cl">    return load_configs(folder), {}
</span></span><span class="line"><span class="cl">           ^^^^^^^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/workspaces/python-101/netlab/configs.py&#34;, line 27, in load_configs
</span></span><span class="line"><span class="cl">    configs[path.stem] = path.read_text(encoding=&#34;utf-8&#34;)
</span></span><span class="line"><span class="cl">                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/usr/local/lib/python3.11/pathlib.py&#34;, line 1058, in read_text
</span></span><span class="line"><span class="cl">    with self.open(mode=&#39;r&#39;, encoding=encoding, errors=errors) as f:
</span></span><span class="line"><span class="cl">         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">  File &#34;/usr/local/lib/python3.11/pathlib.py&#34;, line 1044, in open
</span></span><span class="line"><span class="cl">    return io.open(self, mode, buffering, encoding, errors, newline)
</span></span><span class="line"><span class="cl">           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span></span><span class="line"><span class="cl">PermissionError: [Errno 13] Permission denied: &#39;/workspaces/python-101/configs/locked.cfg&#39;
</span></span></code></pre></div><p>&ldquo;Most recent call last&rdquo; gives the order away: the top is where the program started, the bottom is where it
stopped. So read it <strong>from the bottom up</strong>. The last line is the answer: the exception&rsquo;s type,
<code>PermissionError</code>, and a message that names the file. Often that&rsquo;s all you need. Above it, each
<code>File ... line ... in ...</code> pair is a call that was still in progress, innermost at the bottom, with <code>^^^^</code>
under the part that was running. The bottom two frames are in <code>pathlib.py</code>, the standard library, and that
code is fine: it was asked to open a file it isn&rsquo;t allowed to. The lowest frame in <em>your</em> code is
<code>netlab/configs.py</code>, line 27, in <code>load_configs</code>. That&rsquo;s where to look. The frames above it say how you got
there: <code>main()</code> called <code>load_configs_safe()</code>, which called <code>load_configs()</code>.</p>
<p>The same reading works on Netmiko&rsquo;s tracebacks and <a href="/lessons/ansible-101/">Ansible</a>&rsquo;s: the error at the bottom, your code a few frames
up, and library code in between that you can usually skip.</p>
<hr>
<h2 id="what-does-tryexcept-do">What does <code>try</code>/<code>except</code> do?</h2>
<p>Every exception has a <strong>type</strong>, and the type says what kind of problem it was. These are the ones a script
that reads files meets:</p>
<table>
  <thead>
      <tr>
          <th>Exception</th>
          <th>What happened</th>
          <th>In the lab</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>FileNotFoundError</code></td>
          <td>no file by that name</td>
          <td><code>Path(&quot;configs/nope.cfg&quot;).read_text()</code></td>
      </tr>
      <tr>
          <td><code>PermissionError</code></td>
          <td>the file&rsquo;s there, and you may not read it</td>
          <td><code>configs/locked.cfg</code>, after <code>chmod 000</code></td>
      </tr>
      <tr>
          <td><code>IsADirectoryError</code></td>
          <td>the name is a folder, not a file</td>
          <td><code>Path(&quot;configs&quot;).read_text()</code></td>
      </tr>
      <tr>
          <td><code>UnicodeDecodeError</code></td>
          <td>the bytes aren&rsquo;t valid in the encoding you asked for</td>
          <td>a config saved in Latin-1</td>
      </tr>
      <tr>
          <td><code>ValueError</code></td>
          <td>the right type with a wrong value</td>
          <td><code>int(&quot;Gi1&quot;)</code></td>
      </tr>
  </tbody>
</table>
<p>The first three are kinds of <code>OSError</code>, the operating system saying no; the
<a href="https://docs.python.org/3/library/exceptions.html#exception-hierarchy">exception hierarchy</a> shows how they
nest. The Latin-1 one is common with configs saved by older tools. In Latin-1 an <code>é</code> is the single byte
<code>0xe9</code>; in UTF-8 it&rsquo;s two bytes, and <code>0xe9</code> followed by a space isn&rsquo;t valid, so reading as UTF-8 stops there:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">Path</span><span class="p">(</span><span class="s2">&#34;configs/old-r9.cfg&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read_text</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;/usr/local/lib/python3.11/pathlib.py&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1059</span><span class="p">,</span> <span class="ow">in</span> <span class="n">read_text</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">           <span class="o">^^^^^^^^</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;frozen codecs&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">322</span><span class="p">,</span> <span class="ow">in</span> <span class="n">decode</span>
</span></span><span class="line"><span class="cl"><span class="ne">UnicodeDecodeError</span><span class="p">:</span> <span class="s1">&#39;utf-8&#39;</span> <span class="n">codec</span> <span class="n">can</span><span class="s1">&#39;t decode byte 0xe9 in position 64: invalid continuation byte</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">Path</span><span class="p">(</span><span class="s2">&#34;configs/old-r9.cfg&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read_bytes</span><span class="p">()[</span><span class="mi">60</span><span class="p">:</span><span class="mi">80</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="sa">b</span><span class="s1">&#39; Caf</span><span class="se">\xe9</span><span class="s1"> network ^C</span><span class="se">\n</span><span class="s1">ntp&#39;</span>
</span></span></code></pre></div><p>A banner that says <em>Café network</em> is enough to stop a whole report. <code>try</code> and <code>except</code> let you decide what
happens instead. Python runs the <code>try</code> block, and if it raises an exception of the type an <code>except</code> names,
runs that <code>except</code> block and carries on after it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">Path</span><span class="p">(</span><span class="s2">&#34;configs/old-r9.cfg&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read_text</span><span class="p">(</span><span class="n">encoding</span><span class="o">=</span><span class="s2">&#34;utf-8&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span> <span class="k">except</span> <span class="ne">UnicodeDecodeError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;skipped:&#34;</span><span class="p">,</span> <span class="n">e</span><span class="o">.</span><span class="n">reason</span><span class="p">,</span> <span class="s2">&#34;at byte&#34;</span><span class="p">,</span> <span class="n">e</span><span class="o">.</span><span class="n">start</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="n">skipped</span><span class="p">:</span> <span class="n">invalid</span> <span class="n">continuation</span> <span class="n">byte</span> <span class="n">at</span> <span class="n">byte</span> <span class="mi">64</span>
</span></span></code></pre></div><p><code>as e</code> keeps the exception, so you can use what it knows. Inside a loop over files, that&rsquo;s the difference
between &ldquo;the report crashed&rdquo; and this, the lab&rsquo;s finished script on the same folder:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/ntp_report.py configs reports/ntp.csv
</span></span><span class="line"><span class="cl">wrote reports/ntp.csv (4 devices)
</span></span><span class="line"><span class="cl">skipped locked.cfg: permission denied
</span></span><span class="line"><span class="cl">skipped old-r9.cfg: not UTF-8 text (byte 64)
</span></span></code></pre></div><p>It still exits with 1, not 0, so a cron job or a CI pipeline knows something needs a look.</p>
<hr>
<h2 id="why-catch-only-what-you-can-handle">Why catch only what you can handle?</h2>
<p>Because an <code>except</code> is a promise: <em>I know what this problem is, and this is the right thing to do about it.</em> A
file you may not read, or one in the wrong encoding, is a problem with the <strong>data</strong>, and skipping it with a
message is the right answer. Other exceptions mean something else.</p>
<p>Say a folder inside <code>configs/</code> happens to be called <code>archive.cfg</code>. Reading it raises <code>IsADirectoryError</code>.
That isn&rsquo;t a bad config: it&rsquo;s a mistake in how the folder was put together, or in the code that picks the
files. Catch everything, with <code>except Exception:</code>, and the report says &ldquo;skipped archive.cfg&rdquo; and the mistake
hides. The same broad <code>except</code> catches your own bugs too: a mistyped variable name raises <code>NameError</code>, and a
catch-all turns it into one more skipped file on every run. A bare <code>except:</code> is worse still: it even catches
<kbd>Ctrl</kbd>+<kbd>C</kbd>.</p>
<p><a href="https://peps.python.org/pep-0008/#programming-recommendations">PEP 8</a>, Python&rsquo;s style guide, says to name
specific exceptions whenever you can, rather than a bare <code>except:</code>. For scripts that run against a network,
that comes down to two rules. Catch the failures you expect from the world (a file you can&rsquo;t read, a device
that doesn&rsquo;t answer, a refused login in lesson 6), report each one, and carry on. Don&rsquo;t catch what you didn&rsquo;t
plan for: let it stop the program, because the traceback is how you&rsquo;ll find it.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>FileNotFoundError</code> for a file that&rsquo;s right there.</strong> A relative path like <code>configs/r1.cfg</code> is relative to
the folder you&rsquo;re <em>in</em>, not to the script. The lab&rsquo;s scripts build their default paths from <code>ROOT</code> for that
reason.</p>
<p><strong><code>ModuleNotFoundError: No module named 'netlab'</code></strong> from a script, while the same import works in the REPL.
The script&rsquo;s folder is first on <code>sys.path</code>, not the repository&rsquo;s.</p>
<p><strong><code>UnicodeDecodeError</code> on one file in a hundred.</strong> Something saved it in Latin-1 or a Windows code page.
Report it, and fix it once at the source, rather than guessing encodings in every script.</p>
<p><strong>A CSV with a blank line between every row</strong> in a spreadsheet: the file was opened without <code>newline=&quot;&quot;</code>.</p>
<p><strong><code>open(..., &quot;w&quot;)</code> on the file you meant to add to.</strong> <code>&quot;w&quot;</code> empties it first. <code>&quot;a&quot;</code> appends.</p>
<p><strong><code>except Exception: pass</code>.</strong> The program never crashes and never tells you anything, so when a problem shows
up later there&rsquo;s nothing to read.</p>
<p><strong>Reading a traceback from the top.</strong> The top is the start of the program, the least interesting part. Start
at the last line, then find the lowest frame in your own code.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-4">Chapter 4 of the lab</a> has you write <code>has_ntp()</code>, <code>load_configs()</code> and
<code>write_report()</code> in <code>netlab/configs.py</code>, and run the given <code>scripts/ntp_report.py</code> over the saved configs of
r1, r2, r3 and sw1 to get <code>reports/ntp.csv</code>. Then <code>make break T=4.2</code> adds a locked file and a Latin-1 one, and
you make the report finish anyway, catching those two problems and nothing else.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>The configs were straightforward to search, because the line you wanted starts with the same words every
time. <code>show</code> output isn&rsquo;t like that: it&rsquo;s a table, laid out for people.
<a href="/lessons/python-101/regex-and-textfsm/">Lesson 5</a> turns that table into data, with a regular expression and
then with TextFSM, and compares the two.</p>
]]></content:encoded></item><item><title>Loops, Conditionals and Functions: Which Interfaces Are Down? (Python 101, Lesson 3)</title><link>https://networkdoodles.com/lessons/python-101/loops-and-functions/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/loops-and-functions/</guid><category>python</category><category>network-automation</category><category>howto</category><category>capeos</category><description>for loops over lists and dicts, if and elif, truthiness, range and enumerate, and small functions with return values: find the down interfaces in show output.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p>A <code>for</code> loop runs the same lines once for every item in a list. An <code>if</code> decides which items matter. A
<strong>function</strong> puts that under a name, takes the data as a <strong>parameter</strong>, and <strong>returns</strong> the answer. Together
they answer <em>which interfaces are down?</em> in six lines, for any device. Because the function returns its answer
instead of printing it, other code can use the result, and a test can check it: that&rsquo;s exactly what the lab&rsquo;s
checker does.</p>
<p><a href="/lessons/python-101/strings-lists-dicts/">Lesson 2</a> turned <code>show ip interface brief</code> into a list of dicts, one
per interface. This lesson asks that list questions: which interfaces are down, how many have each status, and
which have no address. Every session below ran in the <a href="/labs/python-101/">lab</a>&rsquo;s codespace, Python 3.11, on
r1&rsquo;s and sw1&rsquo;s real output.</p>
<hr>
<h2 id="how-does-a-for-loop-work">How does a for loop work?</h2>
<p>A <code>for</code> loop takes each item of a list in turn, gives it a name, and runs the indented lines under it with
that name. Here <code>interfaces</code> is r1&rsquo;s <code>show ip interface brief</code> as a list of dicts, made by the function you
write in chapter 2:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.parse</span> <span class="kn">import</span> <span class="n">SAMPLE</span><span class="p">,</span> <span class="n">interfaces_to_dicts</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">interfaces</span> <span class="o">=</span> <span class="n">interfaces_to_dicts</span><span class="p">(</span><span class="n">SAMPLE</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">interfaces</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">[</span><span class="s2">&#34;interface&#34;</span><span class="p">],</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet0</span> <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet1</span> <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet2</span> <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet3</span> <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet4</span> <span class="n">administratively</span> <span class="n">down</span>
</span></span><span class="line"><span class="cl"><span class="n">Loopback0</span> <span class="n">up</span>
</span></span></code></pre></div><p>Read it as <em>for each interface, call it <code>i</code>, and print two of its fields</em>. The name <code>i</code> is yours to choose;
<code>for intf in interfaces</code> works the same. The colon ends the <code>for</code> line, and the <strong>indentation</strong> (four spaces)
marks which lines belong to the loop. Python uses indentation where other languages use brackets, so it isn&rsquo;t
optional. Forget it and you get an error before anything runs:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">interfaces</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span> <span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">[</span><span class="s2">&#34;interface&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">2</span>
</span></span><span class="line"><span class="cl">    <span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">[</span><span class="s2">&#34;interface&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl">    <span class="o">^^^^^</span>
</span></span><span class="line"><span class="cl"><span class="ne">IndentationError</span><span class="p">:</span> <span class="n">expected</span> <span class="n">an</span> <span class="n">indented</span> <span class="n">block</span> <span class="n">after</span> <span class="s1">&#39;for&#39;</span> <span class="n">statement</span> <span class="n">on</span> <span class="n">line</span> <span class="mi">1</span>
</span></span></code></pre></div><p>(The <code>...</code> prompt is the REPL saying it&rsquo;s waiting for the rest of the block. A blank line ends it.)</p>
<hr>
<h2 id="how-do-you-loop-over-a-dict">How do you loop over a dict?</h2>
<p>A loop over a dict gives you its <strong>keys</strong>, in the order they were added:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="s2">&#34;host&#34;</span><span class="p">:</span> <span class="s2">&#34;172.80.111.11&#34;</span><span class="p">,</span> <span class="s2">&#34;device_type&#34;</span><span class="p">:</span> <span class="s2">&#34;cisco_ios&#34;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">r1</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="n">name</span>
</span></span><span class="line"><span class="cl"><span class="n">host</span>
</span></span><span class="line"><span class="cl"><span class="n">device_type</span>
</span></span></code></pre></div><p>Usually you want the values as well. <code>items()</code> gives you both, as pairs, and the <code>for</code> line names the two
halves:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">r1</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">key</span><span class="si">:</span><span class="s2">&lt;12</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">value</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="n">name</span>         <span class="n">r1</span>
</span></span><span class="line"><span class="cl"><span class="n">host</span>         <span class="mf">172.80.111.11</span>
</span></span><span class="line"><span class="cl"><span class="n">device_type</span>  <span class="n">cisco_ios</span>
</span></span></code></pre></div><p><code>{key:&lt;12}</code> pads the key to 12 characters, so the values line up like a <code>show</code> command&rsquo;s.</p>
<hr>
<h2 id="how-does-if-decide">How does if decide?</h2>
<p><code>if</code> runs its indented lines only when its <strong>condition</strong> is true. <code>elif</code> (<em>else if</em>) tries another condition
when the first failed, and <code>else</code> catches everything left. Python tries them top to bottom and runs the first
that matches, and only that one:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">status</span> <span class="o">=</span> <span class="s2">&#34;administratively down&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">if</span> <span class="n">status</span> <span class="o">==</span> <span class="s2">&#34;up&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;working&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span> <span class="k">elif</span> <span class="n">status</span> <span class="o">==</span> <span class="s2">&#34;administratively down&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;someone shut it&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span> <span class="k">else</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;broken&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="n">someone</span> <span class="n">shut</span> <span class="n">it</span>
</span></span></code></pre></div><p>A condition is anything that comes out <code>True</code> or <code>False</code>. These are the comparisons you&rsquo;ll use:</p>
<table>
  <thead>
      <tr>
          <th>Operator</th>
          <th>Means</th>
          <th>Example</th>
          <th>Result</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>==</code></td>
          <td>equal (two <code>=</code>; one <code>=</code> is assignment)</td>
          <td><code>&quot;up&quot; == &quot;UP&quot;</code></td>
          <td><code>False</code>: case matters</td>
      </tr>
      <tr>
          <td><code>!=</code></td>
          <td>not equal</td>
          <td><code>&quot;administratively down&quot; != &quot;up&quot;</code></td>
          <td><code>True</code></td>
      </tr>
      <tr>
          <td><code>&lt;</code>, <code>&gt;</code>, <code>&lt;=</code>, <code>&gt;=</code></td>
          <td>less than, greater than</td>
          <td><code>3 &lt; 10</code></td>
          <td><code>True</code></td>
      </tr>
      <tr>
          <td><code>in</code></td>
          <td>is contained in</td>
          <td><code>&quot;down&quot; in &quot;administratively down&quot;</code></td>
          <td><code>True</code></td>
      </tr>
      <tr>
          <td><code>is</code></td>
          <td>is this very object; use it for <code>None</code></td>
          <td><code>ip is None</code></td>
          <td><code>True</code> when there&rsquo;s no address</td>
      </tr>
  </tbody>
</table>
<p>Comparing strings compares them letter by letter, like a dictionary does, not as numbers. That catches people
out with VLAN numbers kept as text:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34;10&#34;</span> <span class="o">&lt;</span> <span class="s2">&#34;9&#34;</span>
</span></span><span class="line"><span class="cl"><span class="kc">True</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">10</span> <span class="o">&lt;</span> <span class="mi">9</span>
</span></span><span class="line"><span class="cl"><span class="kc">False</span>
</span></span></code></pre></div><p><code>and</code>, <code>or</code> and <code>not</code> combine conditions. An interface works only when its status <strong>and</strong> its protocol are
both <code>up</code>, so it&rsquo;s broken when either one isn&rsquo;t:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">i</span> <span class="o">=</span> <span class="n">interfaces</span><span class="p">[</span><span class="mi">4</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">]</span> <span class="o">!=</span> <span class="s2">&#34;up&#34;</span> <span class="ow">or</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;protocol&#34;</span><span class="p">]</span> <span class="o">!=</span> <span class="s2">&#34;up&#34;</span>
</span></span><span class="line"><span class="cl"><span class="kc">True</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;down&#34;</span>
</span></span><span class="line"><span class="cl"><span class="kc">False</span>
</span></span></code></pre></div><p>The second line is the trap: GigabitEthernet4 is down, but its status isn&rsquo;t the word <code>&quot;down&quot;</code>. Ask <em>is it up?</em>
and treat everything else as not working, and you catch <code>down</code>, <code>administratively down</code> and a status that&rsquo;s up
with the protocol down, all with one test.</p>
<hr>
<h2 id="what-counts-as-true">What counts as true?</h2>
<p><code>if</code> doesn&rsquo;t need a <code>True</code> or <code>False</code>. It accepts any value, and Python decides whether that value counts as
true. This is called <strong>truthiness</strong>. Empty things and zero count as false; everything else counts as true:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">bool</span><span class="p">(</span><span class="s2">&#34;&#34;</span><span class="p">),</span> <span class="nb">bool</span><span class="p">(</span><span class="s2">&#34;down&#34;</span><span class="p">),</span> <span class="nb">bool</span><span class="p">([]),</span> <span class="nb">bool</span><span class="p">([</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">bool</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="nb">bool</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="kc">False</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span>
</span></span></code></pre></div><p>So <code>if interfaces:</code> means <em>if the list has anything in it</em>, and an empty list goes to the <code>else</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">if</span> <span class="p">[]:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;there are some&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span> <span class="k">else</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;none at all&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="n">none</span> <span class="n">at</span> <span class="nb">all</span>
</span></span></code></pre></div><p>That&rsquo;s handy, and it has one sharp edge: <code>None</code>, <code>&quot;&quot;</code>, <code>0</code> and <code>[]</code> all look the same to <code>if</code>. When you mean
<em>there&rsquo;s no address</em>, say so with <code>if ip is None:</code>. When you mean <em>nothing at all</em>, <code>if not ip:</code> is fine. The
rules are in the docs under <a href="https://docs.python.org/3/library/stdtypes.html#truth-value-testing">truth value testing</a>.</p>
<hr>
<h2 id="what-do-range-and-enumerate-give-you">What do range and enumerate give you?</h2>
<p><code>range()</code> counts. It gives the numbers from a start up to, but not including, a stop, which is the same rule
as slices:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">vlan</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">13</span><span class="p">):</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="n">vlan</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="mi">10</span>
</span></span><span class="line"><span class="cl"><span class="mi">11</span>
</span></span><span class="line"><span class="cl"><span class="mi">12</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
</span></span></code></pre></div><p>With one number it starts at 0. <code>list()</code> turns the range into a list so the REPL shows all of it.</p>
<p><code>enumerate()</code> gives you each item together with its position, so you don&rsquo;t keep a counter yourself. It starts
at 0 unless you tell it otherwise:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">devices</span> <span class="o">=</span> <span class="p">[{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;router&#34;</span><span class="p">},</span> <span class="p">{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r2&#34;</span><span class="p">,</span> <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;router&#34;</span><span class="p">},</span> <span class="p">{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;sw1&#34;</span><span class="p">,</span> <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;switch&#34;</span><span class="p">},</span> <span class="p">{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r3&#34;</span><span class="p">,</span> <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;router&#34;</span><span class="p">}]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">n</span><span class="p">,</span> <span class="n">d</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">devices</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">n</span><span class="si">}</span><span class="s2">. </span><span class="si">{</span><span class="n">d</span><span class="p">[</span><span class="s1">&#39;name&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="mf">1.</span> <span class="n">r1</span>
</span></span><span class="line"><span class="cl"><span class="mf">2.</span> <span class="n">r2</span>
</span></span><span class="line"><span class="cl"><span class="mf">3.</span> <span class="n">sw1</span>
</span></span><span class="line"><span class="cl"><span class="mf">4.</span> <span class="n">r3</span>
</span></span></code></pre></div><p>Knowing the position matters when an item&rsquo;s meaning depends on its neighbors. In a sorted VLAN list, a
number starts a new range when it isn&rsquo;t one more than the number before it. The position lets you look at the
next one:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span> <span class="o">=</span> <span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">20</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">vlans</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">):</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="n">vlans</span><span class="p">[</span><span class="n">n</span><span class="p">],</span> <span class="n">vlans</span><span class="p">[</span><span class="n">n</span> <span class="o">+</span> <span class="mi">1</span><span class="p">],</span> <span class="n">vlans</span><span class="p">[</span><span class="n">n</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span> <span class="o">==</span> <span class="n">vlans</span><span class="p">[</span><span class="n">n</span><span class="p">]</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="mi">10</span> <span class="mi">11</span> <span class="kc">True</span>
</span></span><span class="line"><span class="cl"><span class="mi">11</span> <span class="mi">12</span> <span class="kc">True</span>
</span></span><span class="line"><span class="cl"><span class="mi">12</span> <span class="mi">20</span> <span class="kc">False</span>
</span></span></code></pre></div><hr>
<h2 id="how-do-you-count-with-a-dict">How do you count with a dict?</h2>
<p>Keep a dict whose keys are the things you&rsquo;re counting, and add one each time you meet one. <code>get(key, 0)</code> gives
the count so far, or 0 the first time:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">by_role</span> <span class="o">=</span> <span class="p">{}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">devices</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">by_role</span><span class="p">[</span><span class="n">d</span><span class="p">[</span><span class="s2">&#34;role&#34;</span><span class="p">]]</span> <span class="o">=</span> <span class="n">by_role</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">d</span><span class="p">[</span><span class="s2">&#34;role&#34;</span><span class="p">],</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">by_role</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;router&#39;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">&#39;switch&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">}</span>
</span></span></code></pre></div><p>Without <code>get()</code>, the first <code>+= 1</code> on a key that isn&rsquo;t there yet fails with the <code>KeyError</code> from lesson 2.</p>
<hr>
<h2 id="what-is-a-function">What is a function?</h2>
<p>A <strong>function</strong> is a named block of code that takes some inputs and gives back a result. <code>def</code> defines it.
The names in its brackets are its <strong>parameters</strong>: when you call it, each one is set to the value you passed,
the <strong>argument</strong>. <code>return</code> hands the result back to whoever called it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">def</span> <span class="nf">describe</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">status</span><span class="o">=</span><span class="s2">&#34;up&#34;</span><span class="p">):</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="k">return</span> <span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2"> is </span><span class="si">{</span><span class="n">status</span><span class="si">}</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">describe</span><span class="p">(</span><span class="s2">&#34;Gi1&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;Gi1 is up&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">describe</span><span class="p">(</span><span class="s2">&#34;Gi4&#34;</span><span class="p">,</span> <span class="s2">&#34;administratively down&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;Gi4 is administratively down&#39;</span>
</span></span></code></pre></div><p><code>status=&quot;up&quot;</code> is a <strong>default argument</strong>: leave it out and it&rsquo;s <code>&quot;up&quot;</code>. <code>name</code> has no default, so you must pass
it, and Python says exactly what&rsquo;s missing if you don&rsquo;t:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">describe</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">TypeError</span><span class="p">:</span> <span class="n">describe</span><span class="p">()</span> <span class="n">missing</span> <span class="mi">1</span> <span class="n">required</span> <span class="n">positional</span> <span class="n">argument</span><span class="p">:</span> <span class="s1">&#39;name&#39;</span>
</span></span></code></pre></div><p>A function only runs when it&rsquo;s called. Defining it, the <code>def</code> block, makes the name and nothing else.</p>
<hr>
<h2 id="why-return-the-answer-instead-of-printing-it">Why return the answer instead of printing it?</h2>
<p>Because a printed answer is for a person, and a returned answer is for the next line of code. Here are two
functions that count working interfaces. They differ in their last line:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">def</span> <span class="nf">count_up</span><span class="p">(</span><span class="n">interfaces</span><span class="p">):</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">total</span> <span class="o">=</span> <span class="mi">0</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">interfaces</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>         <span class="k">if</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;up&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>             <span class="n">total</span> <span class="o">=</span> <span class="n">total</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="k">return</span> <span class="n">total</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">count_up</span><span class="p">(</span><span class="n">interfaces</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">5</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">def</span> <span class="nf">count_up_printed</span><span class="p">(</span><span class="n">interfaces</span><span class="p">):</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">total</span> <span class="o">=</span> <span class="mi">0</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">interfaces</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>         <span class="k">if</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;up&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>             <span class="n">total</span> <span class="o">=</span> <span class="n">total</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="nb">print</span><span class="p">(</span><span class="n">total</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">n</span> <span class="o">=</span> <span class="n">count_up_printed</span><span class="p">(</span><span class="n">interfaces</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">5</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">n</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">TypeError</span><span class="p">:</span> <span class="n">unsupported</span> <span class="n">operand</span> <span class="nb">type</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="k">for</span> <span class="o">+</span><span class="p">:</span> <span class="s1">&#39;NoneType&#39;</span> <span class="ow">and</span> <span class="s1">&#39;int&#39;</span>
</span></span></code></pre></div><p>Both show <code>5</code> on the screen. Only the first gives the 5 back. A function with no <code>return</code> returns <code>None</code>, so <code>n</code>
is <code>None</code>, and the next line has nothing to add 1 to. <code>print()</code> itself is a function that returns <code>None</code>, which
you can see for yourself:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">result</span> <span class="o">=</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;hi&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">hi</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="kc">None</span>
</span></span></code></pre></div><p>The rule for this course: functions in <code>netlab/</code> return; only scripts print.</p>
<hr>
<h2 id="which-interfaces-are-down">Which interfaces are down?</h2>
<p>Now all of it together. This is <code>down_interfaces()</code> from the lab&rsquo;s <code>netlab/interfaces.py</code>, the function you
write in task 3.1:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">down_interfaces</span><span class="p">(</span><span class="n">interfaces</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">down</span> <span class="o">=</span> <span class="p">[]</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">interfaces</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">if</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">]</span> <span class="o">!=</span> <span class="s2">&#34;up&#34;</span> <span class="ow">or</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;protocol&#34;</span><span class="p">]</span> <span class="o">!=</span> <span class="s2">&#34;up&#34;</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">down</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">i</span><span class="p">[</span><span class="s2">&#34;interface&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">down</span>
</span></span></code></pre></div><p>Start with an empty list, walk the interfaces, append the name of each one that isn&rsquo;t fully up, and return the
list. On r1, and on the lab&rsquo;s switch sw1, whose ports 3 to 8 have nothing plugged in:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.interfaces</span> <span class="kn">import</span> <span class="n">down_interfaces</span><span class="p">,</span> <span class="n">count_by_status</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">down_interfaces</span><span class="p">(</span><span class="n">interfaces</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">text</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/brief-sw1.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">sw1</span> <span class="o">=</span> <span class="n">interfaces_to_dicts</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">sw1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">10</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">down_interfaces</span><span class="p">(</span><span class="n">sw1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet3&#39;</span><span class="p">,</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;GigabitEthernet5&#39;</span><span class="p">,</span> <span class="s1">&#39;GigabitEthernet6&#39;</span><span class="p">,</span> <span class="s1">&#39;GigabitEthernet7&#39;</span><span class="p">,</span> <span class="s1">&#39;GigabitEthernet8&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">count_by_status</span><span class="p">(</span><span class="n">sw1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;up&#39;</span><span class="p">:</span> <span class="mi">4</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">:</span> <span class="mi">6</span><span class="p">}</span>
</span></span></code></pre></div><p>(<code>open(...).read()</code> reads a file into one string: <a href="/lessons/python-101/files-and-errors/">lesson 4</a> covers
files.) The function doesn&rsquo;t know or care which device the list came from, or whether it came from a device at
all. It takes data and returns data, which is what makes the next section possible.</p>
<p>You&rsquo;ll also meet a shorter spelling of the same loop, a <strong>list comprehension</strong>: the loop and the <code>if</code> inside
square brackets, giving a list. It&rsquo;s worth recognizing, though the long form is easier to read while you&rsquo;re
learning:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="p">[</span><span class="n">i</span><span class="p">[</span><span class="s2">&#34;interface&#34;</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">interfaces</span> <span class="k">if</span> <span class="n">i</span><span class="p">[</span><span class="s2">&#34;ip_address&#34;</span><span class="p">]</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">]</span>
</span></span></code></pre></div><hr>
<h2 id="why-write-small-functions">Why write small functions?</h2>
<p>Because you can test a small function, and a script that does everything at once you mostly can&rsquo;t. A
<strong>test</strong> is code that calls your function with an input whose answer is known and checks that the answer comes
back. This is one of the lab&rsquo;s checks for <code>down_interfaces()</code>, from <code>checks/test_3_1.py</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">test_up_but_protocol_down</span><span class="p">():</span>
</span></span><span class="line"><span class="cl">    <span class="n">rows</span> <span class="o">=</span> <span class="p">[{</span><span class="s2">&#34;interface&#34;</span><span class="p">:</span> <span class="s2">&#34;Gi1&#34;</span><span class="p">,</span> <span class="s2">&#34;status&#34;</span><span class="p">:</span> <span class="s2">&#34;up&#34;</span><span class="p">,</span> <span class="s2">&#34;protocol&#34;</span><span class="p">:</span> <span class="s2">&#34;down&#34;</span><span class="p">}]</span>
</span></span><span class="line"><span class="cl">    <span class="k">assert</span> <span class="n">call</span><span class="p">(</span><span class="n">down_interfaces</span><span class="p">,</span> <span class="n">rows</span><span class="p">)</span> <span class="o">==</span> <span class="p">[</span><span class="s2">&#34;Gi1&#34;</span><span class="p">],</span> <span class="s2">&#34;status up but protocol down still isn&#39;t working&#34;</span>
</span></span></code></pre></div><p><code>assert</code> means <em>this must be true</em>; if it isn&rsquo;t, the test fails with the message after the comma. No device
had to be in that state for the test to run: the test made the data up. Other checks make up a fresh
<code>show ip interface brief</code> on every run, so code that only works on r1&rsquo;s output doesn&rsquo;t pass. You can do the same
in the REPL:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">down_interfaces</span><span class="p">([{</span><span class="s2">&#34;interface&#34;</span><span class="p">:</span> <span class="s2">&#34;Gi1&#34;</span><span class="p">,</span> <span class="s2">&#34;status&#34;</span><span class="p">:</span> <span class="s2">&#34;up&#34;</span><span class="p">,</span> <span class="s2">&#34;protocol&#34;</span><span class="p">:</span> <span class="s2">&#34;down&#34;</span><span class="p">}])</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;Gi1&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">down_interfaces</span><span class="p">([])</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span></code></pre></div><p>A function that logs in, runs the command, parses, decides and prints in one go can only be tested against a
real device in exactly the right state. Split it into small functions, each taking data and returning data,
and every piece can be checked on its own, in milliseconds. The same idea scales up:
<span class="link-soon">ANTA lesson 6</span> unit-tests a custom network test on recorded switch output,
and the <span class="link-soon">network testing</span> series tests whole networks before they&rsquo;re built.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>IndentationError</code>.</strong> A line after a <code>:</code> isn&rsquo;t indented, or the indentation is mixed. Use four spaces, and
let the editor do it.</p>
<p><strong>Only <code>&quot;down&quot;</code> counted as down.</strong> <code>status == &quot;down&quot;</code> misses <code>administratively down</code> and an interface whose
status is up with the protocol down. Test for what <em>working</em> looks like, and treat everything else as broken.</p>
<p><strong><code>status == &quot;down&quot; or &quot;administratively down&quot;</code>.</strong> This reads like English and is always true. Python sees
two conditions: <code>status == &quot;down&quot;</code>, and the string <code>&quot;administratively down&quot;</code>, which isn&rsquo;t empty, so it counts
as true. In the REPL it returns <code>'administratively down'</code>. Write the comparison out twice, or use
<code>status in (&quot;down&quot;, &quot;administratively down&quot;)</code>.</p>
<p><strong><code>TypeError: ... 'NoneType' and 'int'</code>.</strong> A function printed its answer instead of returning it, so the
caller got <code>None</code>.</p>
<p><strong><code>KeyError</code> inside a counting loop.</strong> <code>counts[key] += 1</code> on a key that doesn&rsquo;t exist yet. Use
<code>counts.get(key, 0) + 1</code>.</p>
<p><strong>Sorting changes the caller&rsquo;s list.</strong> <code>vlans.sort()</code> sorts the list in place, so the list someone passed in is
now in a different order. <code>sorted(vlans)</code> returns a sorted copy and leaves the original alone.</p>
<p><strong><code>return</code> inside the loop.</strong> A <code>return</code> indented one level too far returns on the first item, and the
function looks as if it only ever sees one interface. <code>return</code> goes after the loop, level with <code>for</code>.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-3">Chapter 3 of the lab</a> has you write <code>down_interfaces()</code> and <code>count_by_status()</code>
in <code>netlab/interfaces.py</code>, checked on interface lists the checker makes up. Then the challenge: <code>vlan_ranges()</code>,
which turns <code>[10, 11, 12, 20]</code> into <code>&quot;10-12,20&quot;</code>, the way a trunk&rsquo;s allowed VLAN list is written.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>So far the data has been typed in or kept in a variable. <a href="/lessons/python-101/files-and-errors/">Lesson 4</a>
reads it from files: a folder of saved configs, a module of your own, a traceback read from the bottom up, and
a file that can&rsquo;t be read, handled without stopping everything else.</p>
]]></content:encoded></item><item><title>Strings, Lists and Dicts: From a Line of show Output to Data (Python 101, Lesson 2)</title><link>https://networkdoodles.com/lessons/python-101/strings-lists-dicts/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/strings-lists-dicts/</guid><category>python</category><category>network-automation</category><category>howto</category><category>capeos</category><description>Python's data types for network engineers: split and strip show output, f-strings, lists, dicts and get(), and why lists of dicts are YAML and JSON.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p>Everything a device prints reaches Python as one thing: a <strong>string</strong>, a piece of text. Python&rsquo;s string methods
(<code>split</code>, <code>strip</code>, <code>partition</code> and slicing) cut it into pieces, a <strong>dict</strong> gives each piece a name, and an
<strong>f-string</strong> puts the pieces back into a line. A <strong>list</strong> of those dicts is exactly the shape a YAML or JSON
file holds, so the same few types carry you from <code>show</code> output to inventories and APIs.</p>
<p>This lesson works on two real lines from the <a href="/labs/python-101/">lab</a>&rsquo;s router r1: the first line of
<code>show version</code>, and a row of <code>show ip interface brief</code>. By the end, each one is a dict you can ask questions
of, and a dict you can turn back into a line. Every REPL session below is Python 3.11 in the lab&rsquo;s codespace.</p>
<hr>
<h2 id="what-kinds-of-values-does-python-have">What kinds of values does Python have?</h2>
<p>A <strong>value</strong> is a piece of data: a word, a number, a yes or no. Every value has a <strong>type</strong>, which decides what
you can do with it. Five types cover almost everything in this course:</p>
<table>
  <thead>
      <tr>
          <th>Type</th>
          <th>What it holds</th>
          <th>Network example</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>str</code> (string)</td>
          <td>text, in quotes</td>
          <td><code>&quot;GigabitEthernet1&quot;</code>, <code>&quot;administratively down&quot;</code></td>
      </tr>
      <tr>
          <td><code>int</code> (integer)</td>
          <td>a whole number</td>
          <td><code>24</code> (a prefix length), <code>4094</code> (the highest VLAN)</td>
      </tr>
      <tr>
          <td><code>float</code></td>
          <td>a number with a decimal point</td>
          <td><code>1.5</code> (a delay in milliseconds)</td>
      </tr>
      <tr>
          <td><code>bool</code> (Boolean)</td>
          <td><code>True</code> or <code>False</code></td>
          <td>is the interface up?</td>
      </tr>
      <tr>
          <td><code>None</code></td>
          <td>nothing: no value at all</td>
          <td>the address of an unassigned interface</td>
      </tr>
  </tbody>
</table>
<p><code>type()</code> tells you which one you have:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">type</span><span class="p">(</span><span class="s2">&#34;up&#34;</span><span class="p">),</span> <span class="nb">type</span><span class="p">(</span><span class="mi">24</span><span class="p">),</span> <span class="nb">type</span><span class="p">(</span><span class="mf">1.5</span><span class="p">),</span> <span class="nb">type</span><span class="p">(</span><span class="kc">True</span><span class="p">),</span> <span class="nb">type</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="o">&lt;</span><span class="k">class</span> <span class="err">&#39;</span><span class="nc">str</span><span class="s1">&#39;&gt;, &lt;class &#39;</span><span class="nb">int</span><span class="s1">&#39;&gt;, &lt;class &#39;</span><span class="nb">float</span><span class="s1">&#39;&gt;, &lt;class &#39;</span><span class="nb">bool</span><span class="s1">&#39;&gt;, &lt;class &#39;</span><span class="n">NoneType</span><span class="s1">&#39;&gt;)</span>
</span></span></code></pre></div><p>The type matters because Python won&rsquo;t guess. <code>&quot;24&quot;</code> in quotes is text that happens to contain digits, and
adding a number to text is an error:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34;24&#34;</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">TypeError</span><span class="p">:</span> <span class="n">can</span> <span class="n">only</span> <span class="n">concatenate</span> <span class="nb">str</span> <span class="p">(</span><span class="ow">not</span> <span class="s2">&#34;int&#34;</span><span class="p">)</span> <span class="n">to</span> <span class="nb">str</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">int</span><span class="p">(</span><span class="s2">&#34;24&#34;</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="mi">25</span>
</span></span></code></pre></div><p><code>int()</code> turns text into a whole number, and refuses anything that isn&rsquo;t one. A software version looks like a
number, but it isn&rsquo;t:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">int</span><span class="p">(</span><span class="s2">&#34;0.11&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">ValueError</span><span class="p">:</span> <span class="n">invalid</span> <span class="n">literal</span> <span class="k">for</span> <span class="nb">int</span><span class="p">()</span> <span class="k">with</span> <span class="n">base</span> <span class="mi">10</span><span class="p">:</span> <span class="s1">&#39;0.11&#39;</span>
</span></span></code></pre></div><p>That&rsquo;s why versions, interface names and addresses stay strings in this course. Python&rsquo;s
<a href="https://docs.python.org/3/library/stdtypes.html">built-in types</a> page lists everything each type can do.</p>
<hr>
<h2 id="what-is-a-variable">What is a variable?</h2>
<p>A <strong>variable</strong> is a name for a value. <code>=</code> attaches the name on the left to the value on the right:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;GigabitEthernet1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&#34;Gig&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="kc">True</span>
</span></span></code></pre></div><p>Think of it as a label tied to the value, not a box the value is copied into. Two names can be tied to the
same list, and a change made through one shows through the other:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">a</span> <span class="o">=</span> <span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">b</span> <span class="o">=</span> <span class="n">a</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">b</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">12</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">a</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">]</span>
</span></span></code></pre></div><p>Numbers and strings can&rsquo;t be changed at all, so this never surprises you with them. <code>y = y + 1</code> makes a new
number and moves the label <code>y</code> to it; <code>x</code> still names the old one:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">x</span> <span class="o">=</span> <span class="mi">5</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">y</span> <span class="o">=</span> <span class="n">x</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">y</span> <span class="o">=</span> <span class="n">y</span> <span class="o">+</span> <span class="mi">1</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">x</span>
</span></span><span class="line"><span class="cl"><span class="mi">5</span>
</span></span></code></pre></div><p>The same goes for string methods. <code>name.upper()</code> gives you a new string and leaves <code>name</code> as it was:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;GIGABITETHERNET1&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;GigabitEthernet1&#39;</span>
</span></span></code></pre></div><p>If you want to keep the result, give it a name: <code>upper_name = name.upper()</code>.</p>
<hr>
<h2 id="how-do-you-cut-a-line-of-show-output-into-pieces">How do you cut a line of show output into pieces?</h2>
<p>With <code>split()</code>. Called with nothing in the parentheses, it splits on any run of spaces and throws the spaces
away, which is exactly what a column-aligned <code>show</code> table needs. Called with a character, it splits on that
character and keeps everything else, spaces included. Here&rsquo;s the first line of r1&rsquo;s <code>show version</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">line</span> <span class="o">=</span> <span class="s2">&#34;CapeOS Software (CO-BOLT), Version 0.11.2, LAB USE ONLY&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parts</span> <span class="o">=</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">&#34;,&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parts</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;CapeOS Software (CO-BOLT)&#39;</span><span class="p">,</span> <span class="s1">&#39; Version 0.11.2&#39;</span><span class="p">,</span> <span class="s1">&#39; LAB USE ONLY&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">parts</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">3</span>
</span></span></code></pre></div><p>Three pieces, and the second and third start with the space that followed each comma. <code>strip()</code> removes
spaces from both ends, and <code>removeprefix()</code> removes a fixed bit of text from the front (it arrived in Python
3.9, with <a href="https://peps.python.org/pep-0616/">PEP 616</a>):</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parts</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;Version 0.11.2&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parts</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span><span class="o">.</span><span class="n">removeprefix</span><span class="p">(</span><span class="s2">&#34;Version&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39; 0.11.2&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parts</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span><span class="o">.</span><span class="n">removeprefix</span><span class="p">(</span><span class="s2">&#34;Version&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;0.11.2&#39;</span>
</span></span></code></pre></div><p>Each method returns a new string, so you can chain them, left to right. The first piece holds two things, the
software&rsquo;s name and the image in parentheses. <code>partition()</code> splits a string once, at the first match, and gives
back three parts: before, the separator, and after:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">first</span> <span class="o">=</span> <span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">first</span><span class="o">.</span><span class="n">partition</span><span class="p">(</span><span class="s2">&#34;(&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="s1">&#39;CapeOS Software &#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">,</span> <span class="s1">&#39;CO-BOLT)&#39;</span><span class="p">)</span>
</span></span></code></pre></div><p>The string methods you&rsquo;ll use most on CLI output:</p>
<table>
  <thead>
      <tr>
          <th>Method</th>
          <th>What it does</th>
          <th>Example</th>
          <th>Result</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>split()</code></td>
          <td>split on runs of spaces</td>
          <td><code>&quot;Gi1   up  up&quot;.split()</code></td>
          <td><code>['Gi1', 'up', 'up']</code></td>
      </tr>
      <tr>
          <td><code>split(&quot;,&quot;)</code></td>
          <td>split on one character</td>
          <td><code>&quot;10.0.12.1/30&quot;.split(&quot;/&quot;)</code></td>
          <td><code>['10.0.12.1', '30']</code></td>
      </tr>
      <tr>
          <td><code>strip()</code></td>
          <td>remove spaces from both ends</td>
          <td><code>&quot;  r1#  &quot;.strip()</code></td>
          <td><code>'r1#'</code></td>
      </tr>
      <tr>
          <td><code>strip(&quot;()&quot;)</code></td>
          <td>remove those characters from both ends</td>
          <td><code>&quot;(CO-BOLT)&quot;.strip(&quot;()&quot;)</code></td>
          <td><code>'CO-BOLT'</code></td>
      </tr>
      <tr>
          <td><code>partition(&quot;(&quot;)</code></td>
          <td>split once, keep the separator</td>
          <td>as above</td>
          <td>three strings</td>
      </tr>
      <tr>
          <td><code>startswith(&quot;show&quot;)</code></td>
          <td>does it start with this?</td>
          <td><code>&quot;show ip int brief&quot;.startswith(&quot;show&quot;)</code></td>
          <td><code>True</code></td>
      </tr>
      <tr>
          <td><code>removeprefix(&quot;Version&quot;)</code></td>
          <td>drop this text from the front, if it&rsquo;s there</td>
          <td>as above</td>
          <td>a new string</td>
      </tr>
      <tr>
          <td><code>lower()</code>, <code>upper()</code></td>
          <td>change the case</td>
          <td><code>name.lower()</code></td>
          <td><code>'gigabitethernet1'</code></td>
      </tr>
  </tbody>
</table>
<p>The full list is in the docs under <a href="https://docs.python.org/3/library/stdtypes.html#string-methods">string methods</a>.</p>
<hr>
<h2 id="how-do-you-pick-out-part-of-a-string-or-a-list">How do you pick out part of a string or a list?</h2>
<p>By position, with square brackets. Positions are counted from <strong>0</strong>, so <code>[0]</code> is the first item. Negative
positions count from the end, so <code>[-1]</code> is the last:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">2</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;Gi&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;1&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="p">[:</span><span class="mi">15</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;GigabitEthernet&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="p">[</span><span class="mi">15</span><span class="p">:]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;1&#39;</span>
</span></span></code></pre></div><p><code>[start:stop]</code> is a <strong>slice</strong>: everything from <code>start</code> up to, but not including, <code>stop</code>. Leave out <code>start</code> and
it begins at the beginning; leave out <code>stop</code> and it runs to the end. Slices work the same way on lists. Asking
for a single position that isn&rsquo;t there is an error; a slice past the end gives you whatever there is:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="p">[</span><span class="mi">10</span><span class="p">:</span><span class="mi">100</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;ernet1&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">name</span><span class="p">[</span><span class="mi">100</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">IndexError</span><span class="p">:</span> <span class="n">string</span> <span class="n">index</span> <span class="n">out</span> <span class="n">of</span> <span class="nb">range</span>
</span></span></code></pre></div><hr>
<h2 id="why-does-split-break-on-administratively-down">Why does split() break on administratively down?</h2>
<p>Because <code>split()</code> doesn&rsquo;t know what a column is. It sees spaces. This is the whole of r1&rsquo;s
<code>show ip interface brief</code>, as the lab keeps it in <code>netlab/parse.py</code> under the name <code>SAMPLE</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.parse</span> <span class="kn">import</span> <span class="n">SAMPLE</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">SAMPLE</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">Interface</span>              <span class="n">IP</span><span class="o">-</span><span class="n">Address</span>      <span class="n">OK</span><span class="err">?</span> <span class="n">Method</span> <span class="n">Status</span>                <span class="n">Protocol</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet0</span>       <span class="mf">172.80.111.11</span>   <span class="n">YES</span> <span class="n">DHCP</span>   <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet1</span>       <span class="mf">192.168.10.1</span>    <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet2</span>       <span class="mf">10.0.12.1</span>       <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet3</span>       <span class="mf">10.0.13.1</span>       <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet4</span>       <span class="n">unassigned</span>      <span class="n">YES</span> <span class="n">unset</span>  <span class="n">administratively</span> <span class="n">down</span> <span class="n">down</span>
</span></span><span class="line"><span class="cl"><span class="n">Loopback0</span>              <span class="mf">10.255.0.1</span>      <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span></code></pre></div><p><strong>Status</strong> is the state of the interface itself and <strong>Protocol</strong> is the state of the line protocol on top of
it, the two layers <a href="/lessons/networking-101/">Networking 101</a> separates. <em>administratively down</em> means someone
configured <code>shutdown</code> on it. A working interface splits into six words:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">SAMPLE</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;192.168.10.1&#39;</span><span class="p">,</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;manual&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p>GigabitEthernet4 splits into seven, because its status is two words:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span> <span class="o">=</span> <span class="n">SAMPLE</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()[</span><span class="mi">5</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;unset&#39;</span><span class="p">,</span> <span class="s1">&#39;administratively&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p>(<code>splitlines()</code> cuts text into a list of lines, so <code>[5]</code> is the sixth line, counting the header.) Code that
assumes the status is always <code>words[4]</code> reads GigabitEthernet4 as status <code>administratively</code> and protocol
<code>down</code>, and gets it wrong without an error. The fix is to count from both ends. The first four columns are
always one word each, and so is the last one, so the status is whatever is left in the middle:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span><span class="p">[:</span><span class="mi">4</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;unset&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span><span class="p">[</span><span class="mi">4</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;administratively&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34; &#34;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">words</span><span class="p">[</span><span class="mi">4</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;administratively down&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;down&#39;</span>
</span></span></code></pre></div><p><code>&quot; &quot;.join(list)</code> is <code>split()</code> in reverse: it glues a list of strings together with a space between each. On a
working interface, <code>words[4:-1]</code> is the one word <code>['up']</code>, and joining one word gives <code>'up'</code>, so the same code
works for both.</p>
<hr>
<h2 id="how-do-you-put-the-pieces-back-into-a-line">How do you put the pieces back into a line?</h2>
<p>With an <strong>f-string</strong>: a string with an <code>f</code> in front of its opening quote, where anything in <code>{curly braces}</code>
is worked out and dropped into the text. They arrived in Python 3.6 with
<a href="https://peps.python.org/pep-0498/">PEP 498</a>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">intf</span> <span class="o">=</span> <span class="s2">&#34;Gi1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlan</span> <span class="o">=</span> <span class="mi">10</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="sa">f</span><span class="s2">&#34;interface </span><span class="si">{</span><span class="n">intf</span><span class="si">}</span><span class="s2"> is in VLAN </span><span class="si">{</span><span class="n">vlan</span><span class="si">}</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;interface Gi1 is in VLAN 10&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">intf</span><span class="si">:</span><span class="s2">&lt;10</span><span class="si">}</span><span class="s2">|&#34;</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;Gi1       |&#39;</span>
</span></span></code></pre></div><p>The second one pads <code>intf</code> to 10 characters, left-aligned, which is how a <code>show</code> command lines up its columns.
An f-string turns a number into text for you, as <code>{vlan}</code> did. Gluing with <code>+</code> doesn&rsquo;t: that&rsquo;s the
<code>TypeError</code> from the first section, and <code>str(vlan)</code> would be the fix.</p>
<p><code>join()</code> is the other way to build a line, when you have a list. It also wants strings, and says so:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34;,&#34;</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="s2">&#34;10&#34;</span><span class="p">,</span> <span class="s2">&#34;11&#34;</span><span class="p">,</span> <span class="s2">&#34;12&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;10,11,12&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34;,&#34;</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">TypeError</span><span class="p">:</span> <span class="n">sequence</span> <span class="n">item</span> <span class="mi">0</span><span class="p">:</span> <span class="n">expected</span> <span class="nb">str</span> <span class="n">instance</span><span class="p">,</span> <span class="nb">int</span> <span class="n">found</span>
</span></span></code></pre></div><hr>
<h2 id="what-is-a-list">What is a list?</h2>
<p>A <strong>list</strong> is an ordered collection of values, in square brackets. <code>split()</code> returns one. The VLANs on the
lab&rsquo;s switch, sw1, are a list of numbers:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span> <span class="o">=</span> <span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">20</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="mi">10</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="mi">20</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">30</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">vlans</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">30</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">vlans</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">5</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">20</span> <span class="ow">in</span> <span class="n">vlans</span>
</span></span><span class="line"><span class="cl"><span class="kc">True</span>
</span></span></code></pre></div><p>Lists keep their order, can hold anything (strings, numbers, other lists, dicts), and can change: <code>append()</code>
adds an item to the end. <code>in</code> asks whether a value is in the list. A position past the end is an
<code>IndexError</code>, as with strings.</p>
<hr>
<h2 id="what-is-a-dict">What is a dict?</h2>
<p>A <strong>dict</strong> (dictionary) maps <strong>keys</strong> to <strong>values</strong>: you look a value up by its name instead of its
position. Keys are usually strings. A device in an inventory is a dict:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="s2">&#34;host&#34;</span><span class="p">:</span> <span class="s2">&#34;172.80.111.11&#34;</span><span class="p">,</span> <span class="s2">&#34;device_type&#34;</span><span class="p">:</span> <span class="s2">&#34;cisco_ios&#34;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span><span class="p">[</span><span class="s2">&#34;host&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;172.80.111.11&#39;</span>
</span></span></code></pre></div><p>Asking for a key that isn&rsquo;t there is an error, a <code>KeyError</code>, and its message is the missing key:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span><span class="p">[</span><span class="s2">&#34;model&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">  <span class="n">File</span> <span class="s2">&#34;&lt;stdin&gt;&#34;</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="ne">KeyError</span><span class="p">:</span> <span class="s1">&#39;model&#39;</span>
</span></span></code></pre></div><p><code>get()</code> asks more politely. It returns <code>None</code> when the key is missing (the REPL prints nothing for <code>None</code>), or
a default you choose:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;model&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;model&#34;</span><span class="p">,</span> <span class="s2">&#34;unknown&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;unknown&#39;</span>
</span></span></code></pre></div><p>Assigning to a key adds it, or replaces the value if the key is already there. <code>in</code> asks about keys, not
values:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span><span class="p">[</span><span class="s2">&#34;model&#34;</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&#34;CapeOS-Bolt&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r1</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">dict_keys</span><span class="p">([</span><span class="s1">&#39;name&#39;</span><span class="p">,</span> <span class="s1">&#39;host&#39;</span><span class="p">,</span> <span class="s1">&#39;device_type&#39;</span><span class="p">,</span> <span class="s1">&#39;model&#39;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34;host&#34;</span> <span class="ow">in</span> <span class="n">r1</span>
</span></span><span class="line"><span class="cl"><span class="kc">True</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="s2">&#34;172.80.111.11&#34;</span> <span class="ow">in</span> <span class="n">r1</span>
</span></span><span class="line"><span class="cl"><span class="kc">False</span>
</span></span></code></pre></div><p>Each key appears once. A dict keeps its keys in the order you added them, which Python has guaranteed since
3.7 (the <a href="https://docs.python.org/3/library/stdtypes.html#mapping-types-dict">dict documentation</a> says so).</p>
<table>
  <thead>
      <tr>
          <th></th>
          <th>list</th>
          <th>dict</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Brackets</td>
          <td><code>[10, 11, 12]</code></td>
          <td><code>{&quot;name&quot;: &quot;r1&quot;}</code></td>
      </tr>
      <tr>
          <td>Look up by</td>
          <td>position: <code>vlans[0]</code></td>
          <td>key: <code>r1[&quot;name&quot;]</code></td>
      </tr>
      <tr>
          <td>Missing</td>
          <td><code>IndexError</code></td>
          <td><code>KeyError</code>, or <code>get()</code> for <code>None</code></td>
      </tr>
      <tr>
          <td>Good for</td>
          <td>many of the same thing: VLANs, interfaces, devices</td>
          <td>one thing&rsquo;s named fields: a device, an interface</td>
      </tr>
  </tbody>
</table>
<hr>
<h2 id="how-does-a-line-of-show-output-become-a-dict-and-a-dict-a-line">How does a line of show output become a dict, and a dict a line?</h2>
<p>Put the last few sections together. Split a row, and give each piece a key:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span> <span class="o">=</span> <span class="n">SAMPLE</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">row</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&#34;interface&#34;</span><span class="p">:</span> <span class="n">words</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s2">&#34;ip_address&#34;</span><span class="p">:</span> <span class="n">words</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s2">&#34;status&#34;</span><span class="p">:</span> <span class="n">words</span><span class="p">[</span><span class="mi">4</span><span class="p">],</span> <span class="s2">&#34;protocol&#34;</span><span class="p">:</span> <span class="n">words</span><span class="p">[</span><span class="mi">5</span><span class="p">]}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">row</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;192.168.10.1&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">row</span><span class="p">[</span><span class="s2">&#34;status&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;up&#39;</span>
</span></span></code></pre></div><p>Now the rest of your code asks for <code>row[&quot;status&quot;]</code> and never needs to know which column it came from. If the
next release of the OS adds a column, one function changes, not every line that uses the data.</p>
<p>Some values need translating as you go. An interface with no address says <code>unassigned</code>, which is a word, not
an address. The honest Python value for <em>no address</em> is <code>None</code>, and a one-line <code>if</code> does it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span> <span class="o">=</span> <span class="n">SAMPLE</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()[</span><span class="mi">5</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">words</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;unassigned&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">address</span> <span class="o">=</span> <span class="n">words</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kc">None</span> <span class="k">if</span> <span class="n">address</span> <span class="o">==</span> <span class="s2">&#34;unassigned&#34;</span> <span class="k">else</span> <span class="n">address</span>
</span></span></code></pre></div><p>That&rsquo;s GigabitEthernet4 again, and the last line prints nothing, because the answer is <code>None</code>. Test for <code>None</code> with <code>is</code>:
<code>ip is None</code> is <code>True</code> when there&rsquo;s no address. <code>None</code> isn&rsquo;t an empty string (<code>&quot;&quot; == None</code> is <code>False</code>), and
most things you&rsquo;d do with a string fail on it: <code>len(None)</code> is a <code>TypeError</code>.</p>
<p>Going back is one f-string. The keys go in single quotes inside it, because the f-string itself uses double
quotes:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">row</span><span class="p">[</span><span class="s1">&#39;interface&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2"> (</span><span class="si">{</span><span class="n">row</span><span class="p">[</span><span class="s1">&#39;ip_address&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">) is </span><span class="si">{</span><span class="n">row</span><span class="p">[</span><span class="s1">&#39;status&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">/</span><span class="si">{</span><span class="n">row</span><span class="p">[</span><span class="s1">&#39;protocol&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;GigabitEthernet1 (192.168.10.1) is up/up&#39;</span>
</span></span></code></pre></div><hr>
<h2 id="why-is-a-list-of-dicts-the-shape-of-yaml-and-json">Why is a list of dicts the shape of YAML and JSON?</h2>
<p>Because YAML and JSON are built from the same two containers: a sequence of things (a list) and a set of
named fields (a dict). A list of dicts is a table: one dict per row, the same keys in each. A list of devices:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">devices</span> <span class="o">=</span> <span class="p">[{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;router&#34;</span><span class="p">},</span> <span class="p">{</span><span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;sw1&#34;</span><span class="p">,</span> <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;switch&#34;</span><span class="p">}]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">devices</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s2">&#34;name&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;sw1&#39;</span>
</span></span></code></pre></div><p><code>devices[1]</code> is the second dict, and <code>[&quot;name&quot;]</code> looks up a key in it. Hand that list to the <code>json</code> module from
the standard library, and to PyYAML, and each writes it in its own spelling:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">json</span><span class="o">,</span> <span class="nn">yaml</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">devices</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span>
</span></span><span class="line"><span class="cl">  <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;r1&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;router&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="p">},</span>
</span></span><span class="line"><span class="cl">  <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="s2">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;sw1&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="s2">&#34;role&#34;</span><span class="p">:</span> <span class="s2">&#34;switch&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">safe_dump</span><span class="p">({</span><span class="s2">&#34;devices&#34;</span><span class="p">:</span> <span class="n">devices</span><span class="p">},</span> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="n">devices</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">-</span> <span class="n">name</span><span class="p">:</span> <span class="n">r1</span>
</span></span><span class="line"><span class="cl">  <span class="n">role</span><span class="p">:</span> <span class="n">router</span>
</span></span><span class="line"><span class="cl"><span class="o">-</span> <span class="n">name</span><span class="p">:</span> <span class="n">sw1</span>
</span></span><span class="line"><span class="cl">  <span class="n">role</span><span class="p">:</span> <span class="n">switch</span>
</span></span></code></pre></div><p>JSON (<a href="https://www.rfc-editor.org/rfc/rfc8259">RFC 8259</a>) uses brackets and braces much like Python&rsquo;s. YAML
uses indentation and a <code>-</code> for each item in a list. The data underneath is identical. That&rsquo;s why an Ansible
inventory, a NetBox API response and the output of a TextFSM parser all feel familiar once you can read a
list of dicts: they&rsquo;re the same shape. <a href="/posts/understandingyaml/">Understanding YAML</a> walks through YAML&rsquo;s
syntax, and <a href="/lessons/python-101/yaml-json-rest/">lesson 7</a> loads both from files and from an API.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>TypeError: can only concatenate str (not &quot;int&quot;) to str</code>.</strong> You used <code>+</code> between text and a number. Use an
f-string, or <code>str()</code> on the number.</p>
<p><strong><code>ValueError: too many values to unpack</code>.</strong> A line like <code>a, b, c = line.split()</code> expects an exact number of
pieces, and this line had more. On <code>show ip interface brief</code>, that&rsquo;s <em>administratively down</em>. Slice from both
ends instead.</p>
<p><strong><code>ValueError: not enough values to unpack (expected 6, got 0)</code>.</strong> The same unpacking hit a blank line, which
splits into an empty list. Skip lines with no words in them.</p>
<p><strong>A value that&rsquo;s wrong, with no error at all.</strong> <code>words[4]</code> on an administratively down interface gives
<code>'administratively'</code>. Python can&rsquo;t tell you that&rsquo;s not what you meant. Look at the data in the REPL before you
trust a column number.</p>
<p><strong><code>KeyError: 'model'</code>.</strong> The dict doesn&rsquo;t have that key. Check the spelling (keys are case-sensitive:
<code>&quot;Host&quot;</code> isn&rsquo;t <code>&quot;host&quot;</code>), or use <code>get()</code> when the key may be missing.</p>
<p><strong><code>'unassigned'</code> where you wanted no address.</strong> The word from the CLI went straight into the data. Translate
it to <code>None</code> once, where you parse, so the rest of the code has one way to say <em>nothing</em>.</p>
<p><strong>Calling a method and losing the result.</strong> <code>name.strip()</code> on its own line changes nothing: strings can&rsquo;t be
changed. Write <code>name = name.strip()</code>.</p>
<p><strong>A list changed behind your back.</strong> <code>b = a</code> doesn&rsquo;t copy a list; both names point at the same one. Use
<code>b = list(a)</code> for a copy.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-2">Chapter 2 of the lab</a> has you write two functions in <code>netlab/parse.py</code>: the
first line of <code>show version</code> into a dict and back, then all of <code>show ip interface brief</code> into a list of dicts,
two-word statuses and unassigned addresses included. The checks run them on lines and outputs they&rsquo;ve never
seen, and task 2.1 shows you how to read pytest&rsquo;s output when one fails.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>You can turn one interface into a dict. <a href="/lessons/python-101/loops-and-functions/">Lesson 3</a> works through a
whole list of them: loops, <code>if</code>, and functions small enough to test, ending with a function that says which
interfaces are down.</p>
]]></content:encoded></item><item><title>Running Python: the REPL, Scripts, pip and venvs (Python 101, Lesson 1)</title><link>https://networkdoodles.com/lessons/python-101/running-python/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/running-python/</guid><category>python</category><category>network-automation</category><category>howto</category><category>capeos</category><description>The REPL, scripts and which Python runs them, where pip installs packages, why each project gets a venv, the externally-managed error, and pinned requirements.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p>You can run Python three ways: type lines at its prompt, the <strong>REPL</strong>, and see each answer straight away; save
the lines in a file and run the file as a <strong>script</strong>; or run a tool that someone else wrote in Python. All three
use the same program, the Python <strong>interpreter</strong>. The thing that trips beginners up isn&rsquo;t the code. It&rsquo;s
<em>which</em> Python runs it, and where the packages it needs were installed.</p>
<p>So this lesson is about the ground everything else stands on: the REPL as a calculator and an experiment
bench, a first script, packages and <code>pip</code>, and the <strong>virtual environment</strong> (venv), a folder that gives one
project its own Python and its own packages. Every command and output below comes from the
<a href="/labs/python-101/">lab</a>&rsquo;s codespace: Python 3.11 on Debian 12.</p>
<hr>
<h2 id="what-does-running-python-actually-mean">What does &ldquo;running Python&rdquo; actually mean?</h2>
<p>Python is a program, the <strong>interpreter</strong>, that reads Python code and does what it says, one statement at a
time. There&rsquo;s no separate build step, unlike C or Go: you hand it the code and it runs. That makes it quick to
try things, and it&rsquo;s a big part of why network automation settled on it.</p>
<p>In a terminal, the interpreter is called <code>python3</code>, and often <code>python</code> as well. Ask it what it is:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python --version
</span></span><span class="line"><span class="cl">Python 3.11.18
</span></span></code></pre></div><p>The first number is the language: <strong>Python 3</strong>. Python 2 stopped getting fixes in 2020, and nothing in this
course, or in the libraries it uses, runs on it. The second number, 11, is the release, and new releases add
features: this course uses nothing newer than 3.11. The last number is a bug-fix release.</p>
<hr>
<h2 id="what-is-the-repl-for">What is the REPL for?</h2>
<p>Run <code>python</code> with no file and you get its interactive prompt:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python
</span></span><span class="line"><span class="cl">Python 3.11.18 (main, Oct 28 2026, 04:42:12) [GCC 12.2.0] on linux
</span></span><span class="line"><span class="cl">Type &#34;help&#34;, &#34;copyright&#34;, &#34;credits&#34; or &#34;license&#34; for more information.
</span></span><span class="line"><span class="cl">&gt;&gt;&gt;
</span></span></code></pre></div><p>That&rsquo;s the <strong>REPL</strong>: it <strong>R</strong>eads a line, <strong>E</strong>valuates it, <strong>P</strong>rints the result and <strong>L</strong>oops back for the
next one. <code>&gt;&gt;&gt;</code> is its prompt, the way <code>r1#</code> is a router&rsquo;s. Type an expression and the answer comes straight
back:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">2</span> <span class="o">+</span> <span class="mi">2</span>
</span></span><span class="line"><span class="cl"><span class="mi">4</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">10</span> <span class="o">/</span> <span class="mi">4</span>
</span></span><span class="line"><span class="cl"><span class="mf">2.5</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">10</span> <span class="o">//</span> <span class="mi">4</span>
</span></span><span class="line"><span class="cl"><span class="mi">2</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">2</span> <span class="o">**</span> <span class="mi">10</span>
</span></span><span class="line"><span class="cl"><span class="mi">1024</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="s2">&#34;GigabitEthernet1&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="mi">16</span>
</span></span></code></pre></div><p><code>/</code> always gives a decimal, <code>//</code> throws the remainder away, and <code>**</code> is &ldquo;to the power of&rdquo;. <code>len()</code> counts, here
the letters in an interface name. You can keep values under a name and use them again:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">x</span> <span class="o">=</span> <span class="mi">24</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">32</span> <span class="o">-</span> <span class="n">x</span>
</span></span><span class="line"><span class="cl"><span class="mi">8</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="mi">2</span> <span class="o">**</span> <span class="p">(</span><span class="mi">32</span> <span class="o">-</span> <span class="n">x</span><span class="p">)</span> <span class="o">-</span> <span class="mi">2</span>
</span></span><span class="line"><span class="cl"><span class="mi">254</span>
</span></span></code></pre></div><p>That&rsquo;s the number of usable addresses in a /24, worked out from the prefix length. Python comes with a whole
library for that too, so you rarely have to do it by hand:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">ipaddress</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">ipaddress</span><span class="o">.</span><span class="n">ip_network</span><span class="p">(</span><span class="s2">&#34;192.168.10.0/24&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">num_addresses</span>
</span></span><span class="line"><span class="cl"><span class="mi">256</span>
</span></span></code></pre></div><p>Use the REPL like a network engineer uses a lab router: to try a command before it goes into a config. When
you&rsquo;re not sure what a piece of code does, paste it in and look. <code>help()</code> explains anything it knows about:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">help</span><span class="p">(</span><span class="nb">len</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">Help</span> <span class="n">on</span> <span class="n">built</span><span class="o">-</span><span class="ow">in</span> <span class="n">function</span> <span class="nb">len</span> <span class="ow">in</span> <span class="n">module</span> <span class="n">builtins</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nb">len</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="o">/</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">Return</span> <span class="n">the</span> <span class="n">number</span> <span class="n">of</span> <span class="n">items</span> <span class="ow">in</span> <span class="n">a</span> <span class="n">container</span><span class="o">.</span>
</span></span></code></pre></div><p><code>exit()</code>, or <kbd>Ctrl</kbd>+<kbd>D</kbd>, leaves the REPL. Nothing you typed is kept, which is the point of
the next section.</p>
<hr>
<h2 id="what-is-a-script">What is a script?</h2>
<p>A <strong>script</strong> is the same Python, saved in a file so it can run again. By convention the file ends in <code>.py</code>.
This is the whole of the lab&rsquo;s first one, <code>scripts/hello.py</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="s2">&#34;Hello, network!&#34;</span><span class="p">)</span>
</span></span></code></pre></div><p>And this is running it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/hello.py
</span></span><span class="line"><span class="cl">Hello, network!
</span></span></code></pre></div><p><code>python</code> reads the file from top to bottom and runs each line, as if you&rsquo;d typed them at <code>&gt;&gt;&gt;</code>. There&rsquo;s
one difference: a script prints only what you tell it to <code>print()</code>. In the REPL, <code>2 + 2</code> shows <code>4</code> because the
REPL prints every answer. In a script, <code>2 + 2</code> is worked out and quietly thrown away.</p>
<p>Everything you&rsquo;ll build in this course is a script or a function a script calls. The REPL is where you find out
how to write them.</p>
<hr>
<h2 id="which-python-is-running">Which Python is running?</h2>
<p>A computer can have several Pythons installed, and the terminal runs whichever one it finds first in the folders
listed in <code>PATH</code> (the variable <a href="/lessons/linux-cli-101/">Linux CLI 101</a> explains). <code>which</code> says which one that
is. In a new terminal in the lab&rsquo;s codespace:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ which python
</span></span><span class="line"><span class="cl">/home/vscode/.venvs/lab/bin/python
</span></span></code></pre></div><p>That&rsquo;s not a system folder. It&rsquo;s the lab&rsquo;s venv, which the next sections explain. Leave it with <code>deactivate</code>,
and the same command finds another Python:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ deactivate
</span></span><span class="line"><span class="cl">$ which python
</span></span><span class="line"><span class="cl">/usr/local/bin/python
</span></span></code></pre></div><p>And the codespace has a third one, Debian&rsquo;s own, at <code>/usr/bin/python3</code>. They aren&rsquo;t interchangeable: each has
its own set of installed packages, and a package installed for one doesn&rsquo;t exist for the others. When code
works in one terminal and fails in another with <code>ModuleNotFoundError</code>, the first thing to check is <code>which python</code>.</p>
<hr>
<h2 id="what-are-packages-and-where-does-pip-put-them">What are packages, and where does pip put them?</h2>
<p>Python comes with a large <strong>standard library</strong>: <code>ipaddress</code> above, and modules for files, CSV, JSON, dates,
HTTP, regular expressions and much more. Anything else is a <strong>package</strong> that someone published, almost always
on <a href="https://pypi.org/">PyPI</a>, the Python Package Index. Netmiko, the library this course uses to log in to
devices, is one.</p>
<p><code>pip</code> is the tool that downloads a package from PyPI and installs it. It installs it <strong>into one particular
Python</strong>, the one that runs pip, in a folder called <code>site-packages</code>. That&rsquo;s why the safe way to call it names
the Python as well:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python -m pip install netaddr
</span></span></code></pre></div><p><code>python -m pip</code> means <em>the pip that belongs to this python</em>, so the package lands where this Python will look
for it. Plain <code>pip</code> works too, as long as it&rsquo;s the pip you think it is.</p>
<p>A package usually brings others with it. The lab&rsquo;s <code>requirements.txt</code> lists six packages, and installing them
gives 26, because Netmiko needs paramiko for SSH, paramiko needs cryptography, and so on down. <code>pip list</code>
shows them all, each with its version:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ pip list
</span></span><span class="line"><span class="cl">Package            Version
</span></span><span class="line"><span class="cl">------------------ ----------
</span></span><span class="line"><span class="cl">bcrypt             5.0.0
</span></span><span class="line"><span class="cl">certifi            2026.10.23
</span></span><span class="line"><span class="cl">...
</span></span><span class="line"><span class="cl">netmiko            4.8.0
</span></span><span class="line"><span class="cl">ntc_templates      9.3.0
</span></span><span class="line"><span class="cl">...
</span></span><span class="line"><span class="cl">paramiko           5.1.0
</span></span><span class="line"><span class="cl">...
</span></span><span class="line"><span class="cl">textfsm            2.1.1
</span></span><span class="line"><span class="cl">urllib3            2.8.0
</span></span></code></pre></div><hr>
<h2 id="why-does-every-project-get-its-own-venv">Why does every project get its own venv?</h2>
<p>Because a Python can only hold <strong>one version of each package</strong>. Install everything into one shared Python and
sooner or later two projects need different versions of the same thing: an Ansible collection wants one
release of a library, the script you wrote last year wants another, and upgrading one breaks the other. The
operating system&rsquo;s own tools are written in Python too, and they need the versions <em>they</em> were tested with.</p>
<p>Debian and Ubuntu now refuse to let <code>pip</code> touch the system Python at all. This is a plain Debian 12 container
with pip installed from Debian&rsquo;s packages, running <code>pip install netaddr</code> as an ordinary user:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ pip install netaddr
</span></span><span class="line"><span class="cl">error: externally-managed-environment
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">× This environment is externally managed
</span></span><span class="line"><span class="cl">╰─&gt; To install Python packages system-wide, try apt install
</span></span><span class="line"><span class="cl">    python3-xyz, where xyz is the package you are trying to
</span></span><span class="line"><span class="cl">    install.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    If you wish to install a non-Debian-packaged Python package,
</span></span><span class="line"><span class="cl">    create a virtual environment using python3 -m venv path/to/venv.
</span></span><span class="line"><span class="cl">    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
</span></span><span class="line"><span class="cl">    sure you have python3-full installed.
</span></span><span class="line"><span class="cl">    ...
</span></span><span class="line"><span class="cl">hint: See PEP 668 for the detailed specification.
</span></span></code></pre></div><p>That&rsquo;s <a href="https://peps.python.org/pep-0668/">PEP 668</a> at work: the distribution marks its Python as <em>externally
managed</em>, meaning its packages belong to <code>apt</code>, and pip respects the mark. The message already says what to do
instead: create a virtual environment.</p>
<p>A <strong>virtual environment</strong>, or <strong>venv</strong>, is a folder that acts as a Python of its own. It points at an
installed Python for the interpreter itself, but it has its <strong>own <code>site-packages</code></strong>, so what you install there
stays there. Making one is one command, with the <code>venv</code> module from the standard library:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python3 -m venv scratch/.venv
</span></span></code></pre></div><p>Inside the folder is <code>bin/python</code> (the venv&rsquo;s Python), <code>bin/pip</code>, <code>lib/python3.11/site-packages/</code> (only pip
and setuptools, for now) and <code>pyvenv.cfg</code>, which records what it was made from. The lab&rsquo;s says:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">home = /usr/local/bin
</span></span><span class="line"><span class="cl">include-system-site-packages = false
</span></span><span class="line"><span class="cl">version = 3.11.18
</span></span><span class="line"><span class="cl">prompt = &#39;lab&#39;
</span></span><span class="line"><span class="cl">executable = /usr/local/bin/python3.11
</span></span></code></pre></div><p><code>include-system-site-packages = false</code> is the important line: nothing installed anywhere else leaks in.</p>
<p>You can always run a venv&rsquo;s Python by its path, <code>scratch/.venv/bin/python</code>. <strong>Activating</strong> the venv is a
shortcut: <code>source scratch/.venv/bin/activate</code> puts its <code>bin</code> folder first on <code>PATH</code>, so plain <code>python</code> and
<code>pip</code> mean the venv&rsquo;s. The prompt shows the venv&rsquo;s name in brackets while it&rsquo;s active, and <code>deactivate</code> puts
<code>PATH</code> back. Nothing else changes: activation is only about which <code>python</code> the shell finds.</p>
<p>The codespace&rsquo;s <code>/usr/local/bin/python</code> isn&rsquo;t marked as externally managed, so pip there would install, into a
folder in your home directory that every project using that Python shares. Use a venv anyway. The reasons above
don&rsquo;t go away because nothing stops you.</p>
<hr>
<h2 id="what-does-requirementstxt-pin-and-why-pin-at-all">What does requirements.txt pin, and why pin at all?</h2>
<p>A venv keeps one project&rsquo;s packages apart. <code>requirements.txt</code> makes them <strong>repeatable</strong>: it lists what the
project needs, so anyone can build the same venv with <code>pip install -r requirements.txt</code>. This is the lab&rsquo;s:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl"># The lab&#39;s Python packages, pinned: the same versions for everyone, every time.
</span></span><span class="line"><span class="cl">netmiko==4.8.0
</span></span><span class="line"><span class="cl">textfsm==2.1.1
</span></span><span class="line"><span class="cl">ntc-templates==9.3.0
</span></span><span class="line"><span class="cl">PyYAML==6.0.3
</span></span><span class="line"><span class="cl">requests==2.34.2
</span></span><span class="line"><span class="cl">pytest==9.2.0
</span></span></code></pre></div><p><code>==</code> <strong>pins</strong> each package to one exact version. Without it, <code>pip install netmiko</code> takes whatever&rsquo;s newest on
the day, and next month that&rsquo;s a different version from the one the code was written against. For network
automation that matters more than it sounds: ntc-templates, for example, is a library of parsers for show
commands, and a new release can change a template&rsquo;s field names. A pinned project breaks only when you change
the pin, on purpose, and test it.</p>
<p><code>pip freeze</code> prints every installed package in that same <code>name==version</code> form, the six the lab asked for and
their 20 dependencies, which is the quickest way to write a <code>requirements.txt</code> for a venv you&rsquo;ve built by hand.</p>
<hr>
<h2 id="how-is-the-labs-python-set-up">How is the lab&rsquo;s Python set up?</h2>
<p>The lab&rsquo;s dev container builds one venv when the image is built, at <code>~/.venvs/lab</code>, from the image&rsquo;s own
Python 3.11 and the pinned <code>requirements.txt</code> above. Every new terminal activates it, from a line in
<code>~/.zshrc</code>, which is why <code>which python</code> pointed there and why the prompt starts with <code>(lab)</code>. <code>make check</code> and
the other <code>make</code> commands use the same venv by its path, so they work even in a terminal where you&rsquo;ve run
<code>deactivate</code>.</p>
<p>Your own code lives in the repository, in <code>netlab/</code> and <code>scripts/</code>, never in the venv. The venv holds other
people&rsquo;s packages; the repository holds yours. Delete the venv and you can build it again in a minute from
<code>requirements.txt</code>. That&rsquo;s the difference between the two, and the reason <code>.venv/</code> folders never go into Git.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>ModuleNotFoundError: No module named 'netmiko'</code>.</strong> The package is installed, but not into the Python that&rsquo;s
running. Check <code>which python</code>. In the lab, a new terminal fixes it; elsewhere, activate the project&rsquo;s venv.</p>
<p><strong><code>pip install</code> works, and the import still fails.</strong> That <code>pip</code> belonged to another Python. Use
<code>python -m pip install</code>, so the pip and the python are the same one.</p>
<p><strong><code>sudo pip install</code>.</strong> It installs into the system&rsquo;s Python, as root, over packages the operating system
depends on. On Debian and Ubuntu it now stops with <em>externally-managed-environment</em>, unless someone added
<code>--break-system-packages</code>, which does what it says.</p>
<p><strong>One venv for everything.</strong> It works until two projects want different versions of one package. A venv is
cheap: one per project.</p>
<p><strong>No pins.</strong> The script worked last month and fails today, on code nobody changed. Something it depends on
moved. Pin versions in <code>requirements.txt</code> and upgrade them deliberately.</p>
<p><strong>Committing the venv.</strong> It&rsquo;s thousands of files, specific to one machine&rsquo;s paths and processor. Commit
<code>requirements.txt</code> instead, and list the venv in <code>.gitignore</code>.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-1">Chapter 1 of the lab</a> has you use the REPL as a calculator, write and run
<code>scripts/hello.py</code>, and find out which Python each terminal runs. Then you build a venv of your own, install a
pinned package into it, and prove that the Python outside it doesn&rsquo;t have the package.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>Python can now run your code. <a href="/lessons/python-101/strings-lists-dicts/">Lesson 2</a> is about the data that code
works on: strings, lists and dicts, starting from a line of <code>show ip interface brief</code> and ending with the shape
every YAML and JSON file you&rsquo;ll meet is built from.</p>
]]></content:encoded></item></channel></rss>