<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>Textfsm · Network Doodles</title><link>https://networkdoodles.com/tags/textfsm/</link><description>Network automation, from first playbook to agentic operations. Ansible, NetBox, MCP, Python and the occasional doodle.</description><language>en-us</language><managingEditor>David Henderson</managingEditor><lastBuildDate>Mon, 16 Nov 2026 09:00:00 -0500</lastBuildDate><atom:link href="https://networkdoodles.com/tags/textfsm/index.xml" rel="self" type="application/rss+xml"/><item><title>Talking to Devices with Netmiko: SSH from Python (Python 101, Lesson 6)</title><link>https://networkdoodles.com/lessons/python-101/netmiko/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/netmiko/</guid><category>python</category><category>network-automation</category><category>netmiko</category><category>textfsm</category><category>capeos</category><description>Log in to routers from Python with Netmiko: ConnectHandler, send_command with TextFSM, config changes, passwords from the environment, and login errors handled.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p><strong>Netmiko</strong> is a Python library that logs in to network devices over SSH and runs CLI commands for you. One
call, <code>ConnectHandler()</code>, opens the session and gets it ready; <code>send_command()</code> runs a show command and
returns its output, already parsed into a list of dicts if you add <code>use_textfsm=True</code>; <code>send_config_set()</code>
pushes configuration lines. The username and password come from environment variables, never from the code,
and a failed login or a device that doesn&rsquo;t answer arrives as an exception you can catch.</p>
<p>Until now the course has parsed show output saved in files. This lesson gets it from live devices, and takes
a first look at <strong>objects</strong> and <strong>classes</strong>, because the connection Netmiko gives you is one. Every output
below comes from the <a href="/labs/python-101/">lab</a>: Netmiko 4.8.0 against r1 (CapeOS Bolt) and r3 (CapeOS Pulse).</p>
<hr>
<h2 id="what-does-netmiko-do-that-plain-ssh-doesnt">What does Netmiko do that plain SSH doesn&rsquo;t?</h2>
<p>SSH itself only carries bytes: you type, the device sends characters back. The Python library that speaks SSH
is <a href="https://www.paramiko.org/">paramiko</a>, and Netmiko is built on top of it. With paramiko alone you&rsquo;d have to
solve the problems a person at a terminal solves without thinking:</p>
<table>
  <thead>
      <tr>
          <th>Problem</th>
          <th>What a person does</th>
          <th>What Netmiko does</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><strong>The prompt.</strong> When has the output finished?</td>
          <td>waits until <code>r1#</code> comes back</td>
          <td>learns the prompt at login, and reads until it sees it again</td>
      </tr>
      <tr>
          <td><strong>Paging.</strong> Long output stops at <code>--More--</code></td>
          <td>presses space</td>
          <td>turns paging off at login (<code>terminal length 0</code>)</td>
      </tr>
      <tr>
          <td><strong>Line wrapping.</strong> Wide lines break</td>
          <td>widens the window</td>
          <td>sets the terminal width (<code>terminal width 511</code>)</td>
      </tr>
      <tr>
          <td><strong>Privilege.</strong> Some commands need <code>enable</code></td>
          <td>types <code>enable</code> and the secret</td>
          <td><code>conn.enable()</code>, with <code>secret=</code> given at login</td>
      </tr>
      <tr>
          <td><strong>Config mode.</strong> Changes need <code>configure terminal</code></td>
          <td>types it, then <code>end</code></td>
          <td><code>send_config_set()</code> enters and leaves for you</td>
      </tr>
  </tbody>
</table>
<p>You can watch it happen. Netmiko writes everything it sends and receives to a file if you give it
<code>session_log=&quot;session.log&quot;</code>. This is the log of a login to r1, one <code>show clock</code>, and the logout:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#terminal width 511
</span></span><span class="line"><span class="cl">r1#terminal length 0
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#show clock
</span></span><span class="line"><span class="cl">05:51:53.965 UTC Wed Nov 11 2026
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#
</span></span><span class="line"><span class="cl">r1#exit
</span></span></code></pre></div><p>The first four lines are Netmiko&rsquo;s <strong>session preparation</strong>, the setup it runs before your first command. The
lab&rsquo;s login lands straight at the privileged <code>r1#</code> prompt, so there&rsquo;s no <code>enable</code> here: <code>conn.check_enable_mode()</code>
returns <code>True</code>. On a device that logs you in at <code>r1&gt;</code>, you&rsquo;d pass <code>secret=</code> to <code>ConnectHandler()</code> and call
<code>conn.enable()</code>.</p>
<p><a href="https://github.com/carlmontanari/scrapli">Scrapli</a> does the same job with a different design, including a
way to run many sessions at once. This course uses <a href="https://github.com/ktbyers/netmiko">Netmiko</a>, the one most
network automation examples are written with.</p>
<hr>
<h2 id="why-does-a-capeos-router-say-cisco_ios">Why does a CapeOS router say cisco_ios?</h2>
<p>Netmiko needs to know what kind of CLI it&rsquo;s talking to, because the commands that turn paging off and the
shape of the prompt differ from one network OS to another. You tell it with <strong><code>device_type</code></strong>, a name from
<a href="https://github.com/ktbyers/netmiko/blob/develop/PLATFORMS.md">Netmiko&rsquo;s list of platforms</a>, such as
<code>cisco_ios</code>, <code>arista_eos</code> or <code>juniper_junos</code>.</p>
<p><a href="/posts/capeos-lab-network-os/">CapeOS</a>, the lab&rsquo;s network OS, has two CLI dialects. <strong>Bolt</strong> (r1, r2 and sw1)
behaves like the CLI that Netmiko and ntc-templates call <code>cisco_ios</code>; <strong>Pulse</strong> (r3) behaves like the one they
call <code>arista_eos</code>. They behave closely enough that both libraries work against it unchanged. So in this lab
the <code>device_type</code> is what picks the dialect:</p>
<table>
  <thead>
      <tr>
          <th>Device</th>
          <th>CapeOS dialect</th>
          <th><code>device_type</code></th>
          <th>Netmiko&rsquo;s class</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>r1, r2, sw1</td>
          <td>Bolt</td>
          <td><code>cisco_ios</code></td>
          <td><code>CiscoIosSSH</code></td>
      </tr>
      <tr>
          <td>r3</td>
          <td>Pulse</td>
          <td><code>arista_eos</code></td>
          <td><code>AristaSSH</code></td>
      </tr>
  </tbody>
</table>
<p>That doesn&rsquo;t make CapeOS either of those products: it&rsquo;s an open-source OS of its own, and <code>device_type</code> names
the CLI each dialect behaves like. The same name also picks the ntc-templates template (lesson 5) for parsed
output, so a wrong <code>device_type</code> can log in fine and still prepare the session and parse for the wrong CLI.</p>
<hr>
<h2 id="how-do-you-log-in-and-run-a-command">How do you log in and run a command?</h2>
<p><code>ConnectHandler()</code> takes the device&rsquo;s details as <strong>keyword arguments</strong>, <code>name=value</code> pairs, so the order
doesn&rsquo;t matter and each value says what it is. It opens the SSH session, runs the session preparation, and
gives you back the connection:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">os</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netmiko</span> <span class="kn">import</span> <span class="n">ConnectHandler</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.env</span> <span class="kn">import</span> <span class="n">load_env</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">load_env</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">ConnectHandler</span><span class="p">(</span><span class="n">device_type</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="n">username</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_USER&#34;</span><span class="p">],</span> <span class="n">password</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_PASSWORD&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span><span class="o">.</span><span class="n">find_prompt</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;r1#&#39;</span>
</span></span></code></pre></div><p><code>host</code> can be a name or an address: in the lab, <code>r1</code> resolves to <code>172.80.111.11</code> on the management network.
<code>find_prompt()</code> asks the device for its prompt, the quickest way to prove the login worked.</p>
<p><code>send_command()</code> runs one command and returns everything the device printed, as a single string:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="n">Interface</span>              <span class="n">IP</span><span class="o">-</span><span class="n">Address</span>      <span class="n">OK</span><span class="err">?</span> <span class="n">Method</span> <span class="n">Status</span>                <span class="n">Protocol</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet0</span>       <span class="mf">172.80.111.11</span>   <span class="n">YES</span> <span class="n">DHCP</span>   <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet1</span>       <span class="mf">192.168.10.1</span>    <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet2</span>       <span class="mf">10.0.12.1</span>       <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet3</span>       <span class="mf">10.0.13.1</span>       <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span><span class="line"><span class="cl"><span class="n">GigabitEthernet4</span>       <span class="n">unassigned</span>      <span class="n">YES</span> <span class="n">unset</span>  <span class="n">administratively</span> <span class="n">down</span> <span class="n">down</span>
</span></span><span class="line"><span class="cl"><span class="n">Loopback0</span>              <span class="mf">10.255.0.1</span>      <span class="n">YES</span> <span class="n">manual</span> <span class="n">up</span>                    <span class="n">up</span>
</span></span></code></pre></div><p>That&rsquo;s the text lessons 2 to 5 parsed by hand, with <code>split()</code>, a regex and TextFSM. Now it comes from the
router instead of a file. <code>send_command()</code> waits for the prompt to come back before it returns, for up to 10
seconds by default (<code>read_timeout=</code>); a slow command such as a large <code>show running-config</code> may need more.</p>
<p><code>conn.disconnect()</code> logs out. A device allows only so many SSH sessions at once (its VTY lines), and a script
that forgets to disconnect holds them until they time out.</p>
<hr>
<h2 id="what-does-use_textfsmtrue-give-you">What does use_textfsm=True give you?</h2>
<p>Add <code>use_textfsm=True</code> and Netmiko hands the output to ntc-templates before returning it. You get the same
list of dicts as <a href="/lessons/python-101/regex-and-textfsm/">lesson 5</a>&rsquo;s <code>parse_output()</code>, with no parsing code
of your own:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">rows</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">use_textfsm</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">type</span><span class="p">(</span><span class="n">rows</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="o">&lt;</span><span class="k">class</span> <span class="err">&#39;</span><span class="nc">list</span><span class="s1">&#39;&gt;, 6)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">rows</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;192.168.10.1&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;proto&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>Six interfaces, six dicts. Ask r3, the Pulse router, the same question with <code>device_type=&quot;arista_eos&quot;</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r3</span> <span class="o">=</span> <span class="n">ConnectHandler</span><span class="p">(</span><span class="n">device_type</span><span class="o">=</span><span class="s2">&#34;arista_eos&#34;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s2">&#34;r3&#34;</span><span class="p">,</span> <span class="n">username</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_USER&#34;</span><span class="p">],</span> <span class="n">password</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_PASSWORD&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">r3</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">use_textfsm</span><span class="o">=</span><span class="kc">True</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;Ethernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;10.0.13.2/30&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;mtu&#39;</span><span class="p">:</span> <span class="s1">&#39;1500&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>Same command, different template, and the keys differ: <code>protocol</code> instead of <code>proto</code>, an <code>mtu</code> column, and the
address with its prefix length, because that&rsquo;s how Pulse prints it. Each template&rsquo;s keys come from the
template&rsquo;s author, not from a standard. Code that runs against two device types should use the keys they share
(<code>interface</code>, <code>status</code>) or translate one shape into the other.</p>
<p>If there&rsquo;s no template for a platform and command, <code>use_textfsm=True</code> gives you back the plain string, as if
you hadn&rsquo;t asked. Lesson 5 found that ntc-templates has none for <code>show ntp associations</code> on <code>cisco_ios</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">out</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">send_command</span><span class="p">(</span><span class="s2">&#34;show ntp associations&#34;</span><span class="p">,</span> <span class="n">use_textfsm</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">type</span><span class="p">(</span><span class="n">out</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&lt;</span><span class="k">class</span> <span class="err">&#39;</span><span class="nc">str</span><span class="s1">&#39;&gt;</span>
</span></span></code></pre></div><p>No error, no warning. A <code>for</code> loop over that string walks it one character at a time, which is a confusing
way to find out.</p>
<hr>
<h2 id="how-do-you-change-the-configuration">How do you change the configuration?</h2>
<p><code>send_config_set()</code> takes a list of configuration lines. It enters configuration mode, sends each line,
leaves again, and returns the whole conversation. Back on r1&rsquo;s connection:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">send_config_set</span><span class="p">([</span><span class="s2">&#34;interface GigabitEthernet4&#34;</span><span class="p">,</span> <span class="s2">&#34;description spare, configured from Python&#34;</span><span class="p">]))</span>
</span></span><span class="line"><span class="cl"><span class="n">configure</span> <span class="n">terminal</span>
</span></span><span class="line"><span class="cl"><span class="n">Configuration</span> <span class="n">mode</span><span class="p">:</span> <span class="n">one</span> <span class="n">command</span> <span class="n">per</span> <span class="n">line</span><span class="o">.</span> <span class="n">Finish</span> <span class="k">with</span> <span class="s1">&#39;end&#39;</span> <span class="ow">or</span> <span class="n">Ctrl</span><span class="o">-</span><span class="n">Z</span><span class="o">.</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="p">)</span><span class="c1">#interface GigabitEthernet4</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="o">-</span><span class="k">if</span><span class="p">)</span><span class="c1">#description spare, configured from Python</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="o">-</span><span class="k">if</span><span class="p">)</span><span class="c1">#end</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="c1">#</span>
</span></span></code></pre></div><p><code>show interfaces description</code> afterwards lists <code>Gi4</code> with its new description.</p>
<p>Read that output, because by default Netmiko doesn&rsquo;t judge it. r1 has no <code>GigabitEthernet9</code>, and this is
what comes back for it, with no exception:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">send_config_set</span><span class="p">([</span><span class="s2">&#34;interface GigabitEthernet9&#34;</span><span class="p">]))</span>
</span></span><span class="line"><span class="cl"><span class="n">configure</span> <span class="n">terminal</span>
</span></span><span class="line"><span class="cl"><span class="n">Configuration</span> <span class="n">mode</span><span class="p">:</span> <span class="n">one</span> <span class="n">command</span> <span class="n">per</span> <span class="n">line</span><span class="o">.</span> <span class="n">Finish</span> <span class="k">with</span> <span class="s1">&#39;end&#39;</span> <span class="ow">or</span> <span class="n">Ctrl</span><span class="o">-</span><span class="n">Z</span><span class="o">.</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="p">)</span><span class="c1">#interface GigabitEthernet9</span>
</span></span><span class="line"><span class="cl">                     <span class="o">^</span>
</span></span><span class="line"><span class="cl"><span class="o">%</span> <span class="n">Invalid</span> <span class="nb">input</span> <span class="n">detected</span> <span class="n">at</span> <span class="s1">&#39;^&#39;</span> <span class="n">marker</span><span class="o">.</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="p">(</span><span class="n">config</span><span class="p">)</span><span class="c1">#end</span>
</span></span><span class="line"><span class="cl"><span class="n">r1</span><span class="c1">#</span>
</span></span></code></pre></div><p>The refusal is in the returned string, and your code has to look for it. A change that works is in the
running configuration only, as if you&rsquo;d typed it; saving it is a separate command. Pushing configuration from
a loop is fast in both directions, which is why tools such as <a href="/lessons/ansible-101/">Ansible</a> wrap it in
checks, dry runs and diffs.</p>
<hr>
<h2 id="where-should-the-password-live">Where should the password live?</h2>
<p>Not in the code. A script gets copied, pasted into chat, committed to Git and shared, and a password written
in it goes everywhere the script goes. Git keeps it in the history even after you delete the line.
<a href="/posts/donthardcodepasswords/">Don&rsquo;t hardcode passwords</a> covers the same idea for Ansible.</p>
<p>The usual answer is an <strong>environment variable</strong>: a named value the shell hands to every program it starts
(Linux CLI 101 covers <code>export</code> and <code>env</code>). Python reads them from <code>os.environ</code>, a dict-like object:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_USER&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;admin&#39;</span>
</span></span></code></pre></div><p>To save typing <code>export</code> in every terminal, the values go in a file called <strong><code>.env</code></strong>, one <code>KEY=value</code> per line,
in the project folder. The lab&rsquo;s has the login and the API&rsquo;s token in it. <code>make setup</code> copies it from
<code>.env.example</code>, and <code>.gitignore</code> lists <code>.env</code>, so Git never picks it up. The lab&rsquo;s <code>netlab/env.py</code> has a small
<code>load_env()</code> that reads <code>.env</code> into <code>os.environ</code>, the same thing the
<a href="https://pypi.org/project/python-dotenv/">python-dotenv</a> package does; a variable already set in your shell
wins over the file.</p>
<p>So the code names the variable, and the value lives somewhere else:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">password</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;LAB_PASSWORD&#34;</span><span class="p">]</span>
</span></span></code></pre></div><p>The lab&rsquo;s check for task 6.1 tests this both ways. It fails if the lab&rsquo;s password appears anywhere in your
code, and it logs in with a deliberately wrong <code>LAB_PASSWORD</code> in the environment, expecting the login to fail.
If your code still gets in, the password didn&rsquo;t come from the environment.</p>
<hr>
<h2 id="what-happens-when-the-login-fails">What happens when the login fails?</h2>
<p>Netmiko raises an <strong>exception</strong>, the same mechanism as the <code>FileNotFoundError</code> in
<a href="/lessons/python-101/files-and-errors/">lesson 4</a>. A wrong password gives this, trimmed to its last part:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&gt;&gt;&gt; ConnectHandler(device_type=&#34;cisco_ios&#34;, host=&#34;r1&#34;, username=&#34;admin&#34;, password=&#34;wrong&#34;)
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">paramiko.ssh_exception.AuthenticationException: Authentication failed.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">During handling of the above exception, another exception occurred:
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">netmiko.exceptions.NetmikoAuthenticationException: Authentication to device failed.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Common causes of this problem are:
</span></span><span class="line"><span class="cl">1. Invalid username and password
</span></span><span class="line"><span class="cl">2. Incorrect SSH-key file
</span></span><span class="line"><span class="cl">3. Connecting to the wrong device
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Device settings: cisco_ios r1:22
</span></span></code></pre></div><p>Two tracebacks, read from the bottom up: paramiko failed to authenticate, and Netmiko caught that and raised
its own, clearer exception. A device that doesn&rsquo;t answer at all, here an address with nothing on it, gives a
different one after <code>conn_timeout</code> seconds (10 by default; the lab uses 5), with paramiko&rsquo;s
<em>Unable to connect to port 22 on 172.80.111.19</em> underneath:</p>
<p><code>netmiko.exceptions.NetmikoTimeoutException: TCP connection to device failed.</code></p>
<table>
  <thead>
      <tr>
          <th>Exception</th>
          <th>Means</th>
          <th>Usual cause</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>NetmikoAuthenticationException</code></td>
          <td>SSH connected, the login was refused</td>
          <td>wrong username or password, wrong <code>.env</code></td>
      </tr>
      <tr>
          <td><code>NetmikoTimeoutException</code></td>
          <td>nothing answered on port 22</td>
          <td>the device is down, wrong address, a firewall</td>
      </tr>
  </tbody>
</table>
<p>Both live in <code>netmiko.exceptions</code>, so you can catch each one with <code>try</code>/<code>except</code> and say what happened in
one line that names the device:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">netmiko.exceptions</span> <span class="kn">import</span> <span class="n">NetmikoAuthenticationException</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">conn</span> <span class="o">=</span> <span class="n">ConnectHandler</span><span class="p">(</span><span class="n">device_type</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="n">username</span><span class="o">=</span><span class="n">user</span><span class="p">,</span> <span class="n">password</span><span class="o">=</span><span class="n">password</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">except</span> <span class="n">NetmikoAuthenticationException</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="nb">print</span><span class="p">(</span><span class="s2">&#34;r1: login failed&#34;</span><span class="p">)</span>
</span></span></code></pre></div><p>The person running the script reads one line instead of two screens of traceback, and it tells them what to
fix.</p>
<hr>
<h2 id="what-is-an-object-and-what-is-a-class">What is an object, and what is a class?</h2>
<p>Look at what <code>ConnectHandler()</code> returned:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span>
</span></span><span class="line"><span class="cl"><span class="o">&lt;</span><span class="n">netmiko</span><span class="o">.</span><span class="n">cisco</span><span class="o">.</span><span class="n">cisco_ios</span><span class="o">.</span><span class="n">CiscoIosSSH</span> <span class="nb">object</span> <span class="n">at</span> <span class="mh">0xffff9c629cd0</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">conn</span><span class="o">.</span><span class="n">base_prompt</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;r1&#39;</span>
</span></span></code></pre></div><p><code>conn</code> is an <strong>object</strong>: a value that holds data and also has functions of its own. The data are its
<strong>attributes</strong>, read with a dot: <code>conn.base_prompt</code> is the prompt without the <code>#</code>. The functions are its
<strong>methods</strong>, called with a dot and parentheses: <code>conn.send_command(...)</code>, <code>conn.disconnect()</code>. You&rsquo;ve used
methods since lesson 2, because a string is an object too: <code>line.split()</code> is the <code>split</code> method of one
string.</p>
<p>A <strong>class</strong> is the kind of object, the blueprint every object of that kind is made from. <code>conn</code> is a
<code>CiscoIosSSH</code>; <code>r3</code> above is an <code>AristaSSH</code>. They&rsquo;re different classes with the same method names, so
<code>send_command()</code> works on both, and each does the right thing for its CLI. That&rsquo;s why the code doesn&rsquo;t
change between Bolt and Pulse: only <code>device_type</code> does, and <code>ConnectHandler()</code> picks the class from it.</p>
<p>You can make classes of your own. This course needs only one, and it&rsquo;s the simplest kind: a <strong>dataclass</strong>,
a class that only holds a few named values. The lab&rsquo;s inventory of devices uses one:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">dataclasses</span> <span class="kn">import</span> <span class="n">dataclass</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd">@dataclass</span>
</span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">Device</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
</span></span><span class="line"><span class="cl">    <span class="n">host</span><span class="p">:</span> <span class="nb">str</span>
</span></span><span class="line"><span class="cl">    <span class="n">device_type</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&#34;cisco_ios&#34;</span>
</span></span></code></pre></div><p>The lines inside list the <strong>fields</strong>, each with a type (<code>str</code>, a string) that Python records but doesn&rsquo;t
enforce. <code>= &quot;cisco_ios&quot;</code> is a default, like a function&rsquo;s default argument. The <code>@dataclass</code> line above the
class (a <strong>decorator</strong>) writes the rest for you: a way to create one, a readable printout, and <code>==</code> that
compares the fields:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">d</span> <span class="o">=</span> <span class="n">Device</span><span class="p">(</span><span class="s2">&#34;r3&#34;</span><span class="p">,</span> <span class="s2">&#34;172.80.111.13&#34;</span><span class="p">,</span> <span class="s2">&#34;arista_eos&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">d</span>
</span></span><span class="line"><span class="cl"><span class="n">Device</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;r3&#39;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s1">&#39;172.80.111.13&#39;</span><span class="p">,</span> <span class="n">device_type</span><span class="o">=</span><span class="s1">&#39;arista_eos&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">Device</span><span class="p">(</span><span class="s2">&#34;r1&#34;</span><span class="p">,</span> <span class="s2">&#34;172.80.111.11&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">Device</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;r1&#39;</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s1">&#39;172.80.111.11&#39;</span><span class="p">,</span> <span class="n">device_type</span><span class="o">=</span><span class="s1">&#39;cisco_ios&#39;</span><span class="p">)</span>
</span></span></code></pre></div><p>Why not a dict? <code>d.host</code> is shorter than <code>d[&quot;host&quot;]</code>, making one with a field missing fails at once (a
<code>TypeError</code>) where a dict would quietly lack the key, and the printout says what the thing is. The
<a href="https://docs.python.org/3/library/dataclasses.html">dataclasses documentation</a> has the rest. Classes that
build on other classes (<strong>inheritance</strong>) wait for ANTA, where every test you
<span class="link-soon">write yourself</span> is a class built on ANTA&rsquo;s <code>AntaTest</code>.</p>
<hr>
<h2 id="how-does-one-script-talk-to-every-device">How does one script talk to every device?</h2>
<p>With a list of <code>Device</code>s and a loop: connect, run the command, disconnect, keep the result in a dict under the
device&rsquo;s name. That&rsquo;s <code>collect()</code>, the function task 6.2 asks for, and the lab&rsquo;s <code>scripts/collect.py</code> prints a
line per device from what it returns:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/collect.py
</span></span><span class="line"><span class="cl">r1   6 IP interfaces, 5 up
</span></span><span class="line"><span class="cl">r2   6 IP interfaces, 5 up
</span></span><span class="line"><span class="cl">sw1  10 IP interfaces, 4 up
</span></span><span class="line"><span class="cl">r3   4 IP interfaces, 4 up
</span></span></code></pre></div><p>Three Bolt devices and a Pulse router, through one function, because each <code>Device</code> carries its own
<code>device_type</code>. The <code>disconnect()</code> goes in a <code>finally:</code> block, so the session is closed even when the command
fails. With r2&rsquo;s password changed (the lab&rsquo;s <code>make break T=6.2</code>), the same script stops at r2 with one line,
and exits 1 so a shell script or a CI job can tell it failed:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">$ python scripts/collect.py
</span></span><span class="line"><span class="cl">error: r2: login failed (check LAB_USER and LAB_PASSWORD)
</span></span></code></pre></div><p>The devices are done one at a time, and the first failure stops the run. Lesson 8 carries on past failures;
Nornir, in a later course, runs the same work on many devices at once.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>KeyError: 'LAB_PASSWORD'</code>.</strong> The variable isn&rsquo;t set: <code>.env</code> is missing (<code>make setup</code> makes it), or
<code>load_env()</code> wasn&rsquo;t called before <code>os.environ</code> was read.</p>
<p><strong>The password is in the code &ldquo;for now&rdquo;.</strong> It ends up in Git and in every copy of the script. Read it from the
environment from the first line.</p>
<p><strong><code>NetmikoAuthenticationException</code>, though <code>ssh</code> by hand works.</strong> Compare <code>.env</code> with what you typed, username
included.</p>
<p><strong><code>NetmikoTimeoutException</code>.</strong> Nothing answered on port 22. Check the address and that the device is up before
suspecting the code.</p>
<p><strong>The wrong <code>device_type</code>.</strong> The login may work, but the session preparation and the template are for another
CLI. Use the name your inventory gives that device.</p>
<p><strong><code>use_textfsm=True</code> returns a string.</strong> No template for that platform and command. Check <code>type()</code> before
looping.</p>
<p><strong>Sessions left open, and unread config output.</strong> Put <code>disconnect()</code> in a <code>finally:</code> block, and look for
<code>% Invalid</code> in what <code>send_config_set()</code> returns.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-6">Chapter 6 of the lab</a> has you write <code>connect()</code> and <code>interface_table()</code> with the
login read from <code>.env</code>, and run a script that prints r1&rsquo;s interfaces. Then you write the <code>Device</code> dataclass
and <code>collect()</code>, log in to r1, r2, sw1 and r3 (two device types through one function), and turn a refused
login into one clear line instead of a traceback.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>So far the list of devices is written in the script. <a href="/lessons/python-101/yaml-json-rest/">Lesson 7</a> moves it
out, into a YAML inventory file and then a REST API, and shows that both end up as the same list of dicts.</p>
]]></content:encoded></item><item><title>Parsing Show Output: Regular Expressions and TextFSM (Python 101, Lesson 5)</title><link>https://networkdoodles.com/lessons/python-101/regex-and-textfsm/</link><pubDate>Mon, 16 Nov 2026 09:00:00 -0500</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/lessons/python-101/regex-and-textfsm/</guid><category>python</category><category>network-automation</category><category>textfsm</category><category>howto</category><category>capeos</category><description>Turn show ip interface brief into data with a regular expression and with TextFSM and ntc-templates, see how a template is picked, and write your own.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<p><code>show</code> output is a table laid out for people, and the job of a parser is to turn it into data: a list of
dicts, one per row, with the same keys every time. A <strong>regular expression</strong> describes the shape of a line
and pulls out the parts you name. A <strong>TextFSM template</strong> does the same for a whole output, with a few rules
for which lines to skip, and <a href="https://github.com/networktocode/ntc-templates">ntc-templates</a> is a library of
979 of them, picked by platform and command. Use a regex for one value or a one-off; use ntc-templates when it
has a template for your command; write a template of your own when it doesn&rsquo;t.</p>
<p><a href="/lessons/python-101/strings-lists-dicts/">Lesson 2</a> did this by hand with <code>split()</code>, and you saw how much
care one two-word status needed. This lesson does it properly, twice, on the same <code>show ip interface brief</code>
from r1, and the <a href="/labs/python-101/">lab</a>&rsquo;s chapter 5 checks that both ways give exactly the same answer.</p>
<hr>
<h2 id="why-is-screen-scraping-fragile">Why is screen-scraping fragile?</h2>
<p><strong>Screen-scraping</strong> means reading data out of text that was laid out for a person to look at. Here&rsquo;s r1&rsquo;s
<code>show ip interface brief</code>, which you know from <a href="/lessons/networking-101/">Networking 101</a>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Interface              IP-Address      OK? Method Status                Protocol
</span></span><span class="line"><span class="cl">GigabitEthernet0       172.80.111.11   YES DHCP   up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet1       192.168.10.1    YES manual up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet2       10.0.12.1       YES manual up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet3       10.0.13.1       YES manual up                    up
</span></span><span class="line"><span class="cl">GigabitEthernet4       unassigned      YES unset  administratively down down
</span></span><span class="line"><span class="cl">Loopback0              10.255.0.1      YES manual up                    up
</span></span></code></pre></div><p>A person sees six columns. Python sees a string, and <code>split()</code> sees words:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">line</span> <span class="o">=</span> <span class="s2">&#34;GigabitEthernet4       unassigned      YES unset  administratively down down&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;unset&#39;</span><span class="p">,</span> <span class="s1">&#39;administratively&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">,</span> <span class="s1">&#39;down&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">())</span>
</span></span><span class="line"><span class="cl"><span class="mi">7</span>
</span></span></code></pre></div><p>Seven items where every other line has six, so &ldquo;the status is item 4&rdquo; is wrong on exactly the line you care
about. Every kind of change breaks a scraper a different way: a two-word value, a blank line, a header
repeated after a page break, a new column in the next software release, or the same command on another
platform with another layout.</p>
<p>What fixes it is <strong>structured data</strong>: values with names, like
<code>{&quot;interface&quot;: &quot;GigabitEthernet4&quot;, &quot;status&quot;: &quot;administratively down&quot;}</code>. Code that reads <code>row[&quot;status&quot;]</code>
doesn&rsquo;t care which column it came from. The best source of structured data is the device itself, over an
API, and lesson 7 does that. For the many commands that only print text, a parser is the place where the
fragile part lives, once, tested, so the rest of your code never sees it.</p>
<hr>
<h2 id="what-is-a-regular-expression">What is a regular expression?</h2>
<p>A <strong>regular expression</strong>, or <strong>regex</strong>, is a small pattern language for text. Python&rsquo;s is the
<a href="https://docs.python.org/3/library/re.html"><code>re</code> module</a>. Most characters match themselves, and a few have
meanings:</p>
<table>
  <thead>
      <tr>
          <th>Pattern</th>
          <th>Matches</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>\S</code></td>
          <td>one character that isn&rsquo;t whitespace</td>
      </tr>
      <tr>
          <td><code>\s</code></td>
          <td>one whitespace character: a space, a tab</td>
      </tr>
      <tr>
          <td><code>\d</code></td>
          <td>one digit</td>
      </tr>
      <tr>
          <td><code>+</code></td>
          <td>one or more of the thing before it: <code>\S+</code> is a word, <code>\s+</code> is a gap</td>
      </tr>
      <tr>
          <td><code>^</code> and <code>$</code></td>
          <td>the start and the end of the line</td>
      </tr>
      <tr>
          <td><code>up|down</code></td>
          <td>either one</td>
      </tr>
      <tr>
          <td><code>\.</code></td>
          <td>a real dot (a bare <code>.</code> means any character)</td>
      </tr>
      <tr>
          <td><code>( )</code></td>
          <td>a <strong>group</strong>: the part you want back</td>
      </tr>
  </tbody>
</table>
<p>Patterns are written as <strong>raw strings</strong>, <code>r&quot;...&quot;</code>, so that Python leaves the backslashes alone for <code>re</code> to
read. An address is <code>\d+\.\d+\.\d+\.\d+</code>; an interface name followed by a gap is <code>\S+\s+</code>.</p>
<h3 id="what-do-groups-give-you">What do groups give you?</h3>
<p>A group marks the part of the match you want to keep. <code>re.match()</code> returns a <strong>match object</strong>, and
<code>.group(1)</code> is the first group&rsquo;s text:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">re</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;(\S+)\s+(\S+)&#34;</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="n">m</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">)</span>
</span></span></code></pre></div><p>Numbers get confusing by the fourth group, so name them. <code>(?P&lt;name&gt;...)</code> is a <strong>named group</strong>, and
<code>groupdict()</code> hands the whole match back as a dict, which is already the shape you want:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">version</span> <span class="o">=</span> <span class="s2">&#34;CapeOS Software (CO-BOLT), Version 0.11.2, LAB USE ONLY&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;\((?P&lt;image&gt;[^)]+)\), Version (?P&lt;version&gt;[^,]+)&#34;</span><span class="p">,</span> <span class="n">version</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span><span class="p">[</span><span class="s2">&#34;image&#34;</span><span class="p">],</span> <span class="n">m</span><span class="p">[</span><span class="s2">&#34;version&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">(</span><span class="s1">&#39;CO-BOLT&#39;</span><span class="p">,</span> <span class="s1">&#39;0.11.2&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">m</span><span class="o">.</span><span class="n">groupdict</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;image&#39;</span><span class="p">:</span> <span class="s1">&#39;CO-BOLT&#39;</span><span class="p">,</span> <span class="s1">&#39;version&#39;</span><span class="p">:</span> <span class="s1">&#39;0.11.2&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p><code>[^)]+</code> means &ldquo;one or more characters that aren&rsquo;t <code>)</code>&rdquo;, a common way to say &ldquo;up to the next bracket&rdquo;.</p>
<h3 id="match-search-or-findall"><code>match</code>, <code>search</code> or <code>findall</code>?</h3>
<table>
  <thead>
      <tr>
          <th>Function</th>
          <th>Looks</th>
          <th>Returns</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>re.match(p, s)</code></td>
          <td>only at the start of <code>s</code></td>
          <td>a match object, or <code>None</code></td>
      </tr>
      <tr>
          <td><code>re.search(p, s)</code></td>
          <td>anywhere in <code>s</code>, first match</td>
          <td>a match object, or <code>None</code></td>
      </tr>
      <tr>
          <td><code>re.findall(p, s)</code></td>
          <td>everywhere in <code>s</code></td>
          <td>a list: strings for one group, tuples for several</td>
      </tr>
  </tbody>
</table>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;Version (\S+),&#34;</span><span class="p">,</span> <span class="n">version</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="kc">None</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;Version (\S+),&#34;</span><span class="p">,</span> <span class="n">version</span><span class="p">)</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="s1">&#39;0.11.2&#39;</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">text</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/brief-r1.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(\S+)\s+(\d+\.\d+\.\d+\.\d+)&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">,</span> <span class="n">re</span><span class="o">.</span><span class="n">M</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[(</span><span class="s1">&#39;GigabitEthernet0&#39;</span><span class="p">,</span> <span class="s1">&#39;172.80.111.11&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;192.168.10.1&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;GigabitEthernet2&#39;</span><span class="p">,</span> <span class="s1">&#39;10.0.12.1&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;GigabitEthernet3&#39;</span><span class="p">,</span> <span class="s1">&#39;10.0.13.1&#39;</span><span class="p">),</span> <span class="p">(</span><span class="s1">&#39;Loopback0&#39;</span><span class="p">,</span> <span class="s1">&#39;10.255.0.1&#39;</span><span class="p">)]</span>
</span></span></code></pre></div><p><code>re.match</code> says <code>None</code> because the line doesn&rsquo;t <em>start</em> with <code>Version</code>. <code>re.M</code> (multiline) makes <code>^</code> mean the
start of every line, not only the start of the text; without it, the same <code>findall</code> for unassigned
interfaces finds nothing:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(\S+)\s+unassigned&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(?P&lt;interface&gt;\S+)\s+unassigned&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">,</span> <span class="n">re</span><span class="o">.</span><span class="n">M</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">]</span>
</span></span></code></pre></div><p>A <code>None</code> you didn&rsquo;t expect is the classic regex failure: <code>.group(1)</code> on it raises <code>AttributeError: 'NoneType' object has no attribute 'group'</code>. Check the match before you use it.</p>
<hr>
<h2 id="how-does-a-regex-parse-a-whole-table">How does a regex parse a whole table?</h2>
<p>One pattern for one line, with a named group per column, applied to every line of the output. The status
column is the hard part, so it spells out the three values it can have:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">pattern</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;^(?P&lt;interface&gt;\S+)\s+(?P&lt;ip_address&gt;\S+)\s+(?P&lt;ok&gt;YES|NO)\s+(?P&lt;method&gt;\S+)\s+(?P&lt;status&gt;up|down|administratively down)\s+(?P&lt;protocol&gt;\S+)\s*$&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">pattern</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="n">line</span><span class="p">)</span><span class="o">.</span><span class="n">groupdict</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;ok&#39;</span><span class="p">:</span> <span class="s1">&#39;YES&#39;</span><span class="p">,</span> <span class="s1">&#39;method&#39;</span><span class="p">:</span> <span class="s1">&#39;unset&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;down&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">pattern</span><span class="o">.</span><span class="k">match</span><span class="p">(</span><span class="s2">&#34;Interface              IP-Address      OK? Method Status                Protocol&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl"><span class="kc">None</span>
</span></span></code></pre></div><p><code>re.compile()</code> turns the pattern into an object once, so a loop doesn&rsquo;t rebuild it for every line. Because
the pattern is strict (<code>YES|NO</code> in the third column, a known status), the header line doesn&rsquo;t match, and
neither does a blank line: the loop keeps the lines that match and skips the rest. That&rsquo;s the whole of a
regex parser, and in the lab it&rsquo;s about ten lines.</p>
<p>The strictness cuts both ways. On r3, which runs CapeOS Pulse, the same command prints a different table
(<code>Ethernet1  10.0.13.2/30  up  up  1500</code>, no <code>OK?</code> column), and the Bolt pattern matches none of it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">netlab.parsers</span> <span class="kn">import</span> <span class="n">BRIEF_LINE</span><span class="p">,</span> <span class="n">parse_brief_regex</span><span class="p">,</span> <span class="n">parse_brief_ntc</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">pulse</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;checks/data/pulse-brief-r3.txt&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_brief_regex</span><span class="p">(</span><span class="n">pulse</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span></code></pre></div><p>That&rsquo;s the right result for a pattern written for one layout. It&rsquo;s also the reason a hand-written regex
doesn&rsquo;t scale to every command on every platform you run.</p>
<hr>
<h2 id="what-is-textfsm">What is TextFSM?</h2>
<p><a href="https://github.com/google/textfsm/wiki/TextFSM">TextFSM</a> is a Python module, written at Google, that parses
text with a <strong>template</strong>: a small file that names the values you want and gives the rules for finding them. FSM
stands for <em>finite state machine</em>: the template has <strong>states</strong>, and the parser is in one state at a time,
which lets it treat the header, the body and the footer of an output differently. This is a complete template
for two columns of the table above:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Value INTERFACE (\S+)
</span></span><span class="line"><span class="cl">Value STATUS (up|down|administratively down)
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">Start
</span></span><span class="line"><span class="cl">  ^${INTERFACE}\s+\S+\s+(YES|NO)\s+\S+\s+${STATUS}\s -&gt; Record
</span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">textfsm</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&#34;scratch/brief_two.textfsm&#34;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>     <span class="n">fsm</span> <span class="o">=</span> <span class="n">textfsm</span><span class="o">.</span><span class="n">TextFSM</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="o">...</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">fsm</span><span class="o">.</span><span class="n">header</span>
</span></span><span class="line"><span class="cl"><span class="p">[</span><span class="s1">&#39;INTERFACE&#39;</span><span class="p">,</span> <span class="s1">&#39;STATUS&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">fsm</span><span class="o">.</span><span class="n">ParseText</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[[</span><span class="s1">&#39;GigabitEthernet0&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet2&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet3&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;Loopback0&#39;</span><span class="p">,</span> <span class="s1">&#39;up&#39;</span><span class="p">]]</span>
</span></span></code></pre></div><p>A template has two parts:</p>
<ul>
<li><strong>Values</strong>, at the top: <code>Value NAME (regex)</code>, one per column. The regex in brackets says what that value
looks like. A value can have options before its name: <code>Required</code> (a row without it is dropped), <code>Filldown</code>
(keep it for the next rows, for a value printed once above several lines), <code>List</code> (collect several matches
into a list) and <code>Key</code>.</li>
<li><strong>States</strong>, after a blank line: a name on its own line, then <strong>rules</strong>, indented, each a regex that starts
with <code>^</code>. <code>${INTERFACE}</code> inside a rule stands for that value&rsquo;s regex, and whatever it matches is saved
under that name. Every template has a <code>Start</code> state.</li>
</ul>
<p>TextFSM reads the output a line at a time and tries the current state&rsquo;s rules in order. The first rule that
matches wins; a line that matches no rule is ignored. After the <code>-&gt;</code> comes the <strong>action</strong>: <code>Record</code> saves the
values collected so far as one row and clears them; a state name, like <code>-&gt; Servers</code>, moves to that state; and
<code>-&gt; Error</code> stops the parse with an error, which is how a template says &ldquo;I&rsquo;ve seen a line I don&rsquo;t understand&rdquo;.
At the end of the text, TextFSM records whatever is left. A literal <code>$</code> at the end of a rule is written <code>$$</code>,
because a single <code>$</code> starts a value&rsquo;s name.</p>
<p>The result is a list of rows in <code>fsm.header</code>&rsquo;s order. Zip the two together, and lower-case the names, and
you have the same list of dicts the regex gave you.</p>
<hr>
<h2 id="how-does-ntc-templates-pick-a-template">How does ntc-templates pick a template?</h2>
<p>Network to Code&rsquo;s <strong>ntc-templates</strong> is a package of TextFSM templates for common commands, community written
and tested, 979 of them in version 9.3.0. You don&rsquo;t name a template. You say which <strong>platform</strong> the output
came from and which <strong>command</strong> made it, and <code>parse_output()</code> finds the template and runs it:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">ntc_templates.parse</span> <span class="kn">import</span> <span class="n">parse_output</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">text</span><span class="p">)[</span><span class="mi">4</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">,</span> <span class="s1">&#39;proto&#39;</span><span class="p">:</span> <span class="s1">&#39;down&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>Its field names are its own (<code>proto</code>, not <code>protocol</code>), and it doesn&rsquo;t turn <code>unassigned</code> into <code>None</code>. The
lab&rsquo;s <code>parse_brief_ntc()</code> maps those two so its rows equal the regex&rsquo;s exactly. The platform is <code>cisco_ios</code>
because <a href="/posts/capeos-lab-network-os/">CapeOS</a> Bolt prints this command the way that platform does; lesson 6 says more about why the lab
uses those names.</p>
<p>The choice is made by a file in the package, <code>templates/index</code>. Each line is a template file, a hostname
pattern, a platform and a command pattern:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">cisco_ios_show_ip_interface_brief.textfsm, .*, cisco_ios, sh[[ow]] ip int[[erface]] br[[ief]]
</span></span></code></pre></div><p><code>sh[[ow]]</code> means <code>sh</code>, <code>sho</code> or <code>show</code>, so the abbreviations you type on a device work too:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;sh ip int br&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">text</span><span class="p">)[</span><span class="mi">4</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;GigabitEthernet4&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;unassigned&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;administratively down&#39;</span><span class="p">,</span> <span class="s1">&#39;proto&#39;</span><span class="p">:</span> <span class="s1">&#39;down&#39;</span><span class="p">}</span>
</span></span></code></pre></div><p>The index has 139 entries for <code>cisco_ios</code> and 47 for <code>arista_eos</code>. Pulse output goes to the <code>arista_eos</code>
template, which knows its columns, MTU included:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;arista_eos&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">pulse</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">{</span><span class="s1">&#39;interface&#39;</span><span class="p">:</span> <span class="s1">&#39;Ethernet1&#39;</span><span class="p">,</span> <span class="s1">&#39;ip_address&#39;</span><span class="p">:</span> <span class="s1">&#39;10.0.13.2/30&#39;</span><span class="p">,</span> <span class="s1">&#39;status&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;protocol&#39;</span><span class="p">:</span> <span class="s1">&#39;up&#39;</span><span class="p">,</span> <span class="s1">&#39;mtu&#39;</span><span class="p">:</span> <span class="s1">&#39;1500&#39;</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="o">&gt;&gt;&gt;</span> <span class="n">parse_output</span><span class="p">(</span><span class="n">platform</span><span class="o">=</span><span class="s2">&#34;cisco_ios&#34;</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="s2">&#34;show ip interface brief&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">pulse</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="p">[]</span>
</span></span></code></pre></div><p>The second line is a trap worth remembering: the wrong platform gives an empty list, not an error. And when
there&rsquo;s no template at all, you get an exception. ntc-templates 9.3.0 has nothing for <code>show ntp associations</code>
on <code>cisco_ios</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&gt;&gt;&gt; parse_output(platform=&#34;cisco_ios&#34;, command=&#34;show ntp associations&#34;, data=open(&#34;checks/data/ntp-r1.txt&#34;).read())
</span></span><span class="line"><span class="cl">Traceback (most recent call last):
</span></span><span class="line"><span class="cl">  ...
</span></span><span class="line"><span class="cl">ntc_templates.parse.ParsingException: Unable to parse command &#34;show ntp associations&#34; on platform cisco_ios - No template found for attributes: &#34;{&#39;Command&#39;: &#39;show ntp associations&#39;, &#39;Platform&#39;: &#39;cisco_ios&#39;}&#34;
</span></span></code></pre></div><p>That&rsquo;s the lab&rsquo;s task 5.2: you write that template yourself.</p>
<hr>
<h2 id="regex-or-textfsm-which-one-when">Regex or TextFSM: which one, when?</h2>
<table>
  <thead>
      <tr>
          <th>You want</th>
          <th>Use</th>
          <th>Why</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>one value from one output: a version, an uptime, a serial</td>
          <td>a regex, <code>re.search</code></td>
          <td>one line of code; a template would be overkill</td>
      </tr>
      <tr>
          <td>a table from a command ntc-templates covers</td>
          <td><code>parse_output()</code></td>
          <td>someone has already written and tested it, across releases</td>
      </tr>
      <tr>
          <td>a table from a command it doesn&rsquo;t cover</td>
          <td>your own TextFSM template</td>
          <td>states, <code>Required</code> and <code>-&gt; Error</code> handle headers, footers and surprises</td>
      </tr>
      <tr>
          <td>data from a device that can give it to you structured</td>
          <td>its API or JSON output</td>
          <td>no parsing at all, and nothing to break</td>
      </tr>
  </tbody>
</table>
<p>Regex is the tool you&rsquo;ll use most often, for small things. TextFSM earns its place on tables, and on outputs
with sections, like <code>show interfaces</code>, where values printed once at the top apply to the lines below. In lesson
6, Netmiko&rsquo;s <code>use_textfsm=True</code> calls ntc-templates for you, straight after the command runs, so most of the
time <code>parse_output()</code> is one argument away.</p>
<hr>
<h2 id="what-goes-wrong">What goes wrong</h2>
<p><strong><code>AttributeError: 'NoneType' object has no attribute 'group'</code>.</strong> The regex didn&rsquo;t match, and <code>re.match</code> or
<code>re.search</code> returned <code>None</code>. Print the line you gave it; check whether it needed <code>search</code> rather than <code>match</code>.</p>
<p><strong><code>findall</code> returns <code>[]</code> on a whole output.</strong> <code>^</code> or <code>$</code> in the pattern without <code>re.M</code>.</p>
<p><strong>The header comes back as a row.</strong> The pattern is too loose: <code>\S+</code> matches <code>Interface</code> as happily as
<code>GigabitEthernet0</code>. Make one column strict, like <code>YES|NO</code>, or skip the header with a state.</p>
<p><strong>A two-word value split in half.</strong> <code>\S+</code> stops at the space in <code>administratively down</code>. Spell out the
values the column can have.</p>
<p><strong>An empty list from <code>parse_output()</code>, no error.</strong> The platform&rsquo;s wrong for this output, so a template ran and
nothing matched. Check what the device really is.</p>
<p><strong><code>ParsingException: ... No template found</code>.</strong> ntc-templates has nothing for that platform and command. Check
the index, or write the template.</p>
<p><strong><code>TextFSMError: State Error raised</code>.</strong> A line hit an <code>-&gt; Error</code> rule. The message gives the rule&rsquo;s line
number in the template and the input line it didn&rsquo;t expect.</p>
<p><strong>A different answer after an upgrade.</strong> A new ntc-templates release can rename a field. That&rsquo;s why the lab
pins 9.3.0.</p>
<hr>
<h2 id="try-it">Try it</h2>
<p><a href="/labs/python-101/#chapter-5">Chapter 5 of the lab</a> has you write <code>BRIEF_LINE</code> and <code>parse_brief_regex()</code>, then
<code>parse_brief_ntc()</code> with ntc-templates, and the check makes them agree on outputs they haven&rsquo;t seen. Then you
write <code>templates/ntp_associations.textfsm</code>, the template ntc-templates doesn&rsquo;t have, for r1&rsquo;s
<code>show ntp associations</code>.</p>
<h2 id="whats-next">What&rsquo;s next</h2>
<p>So far the show output has come from files. <a href="/lessons/python-101/netmiko/">Lesson 6</a> gets it from the devices
themselves, over SSH with Netmiko, with <code>use_textfsm=True</code> doing this lesson&rsquo;s parsing on the way back.</p>
]]></content:encoded></item></channel></rss>