<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>Observability · Network Doodles</title><link>https://networkdoodles.com/tags/observability/</link><description>Network automation, from first playbook to agentic operations. Ansible, NetBox, MCP, Python and the occasional doodle.</description><language>en-us</language><managingEditor>David Henderson</managingEditor><lastBuildDate>Sat, 21 Nov 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://networkdoodles.com/tags/observability/index.xml" rel="self" type="application/rss+xml"/><item><title>Your Network Automation ROI Number Is Probably Marketing (AutoCon 6, Part 5)</title><link>https://networkdoodles.com/posts/network-automation-roi/</link><pubDate>Sat, 21 Nov 2026 00:00:00 +0000</pubDate><dc:creator>David Henderson</dc:creator><guid>https://networkdoodles.com/posts/network-automation-roi/</guid><category>network-automation</category><category>automation</category><category>observability</category><category>devops</category><description>Jonathon Davis's six-step method for automation ROI: a human baseline, oversight, real costs, priced failures, and productivity kept apart from savings.</description><content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[<div class="details admonition abstract open">
        <div class="details-summary admonition-title "><i class="icon admonition-icon icon-abstract"></i>Short answer<i class="details-icon  admonition-icon admonition-icon-arrow-right"></i></div>
        <div class="details-content">
            <div class="admonition-content">Vendor time-saved dashboards guess the minutes, count only their own successful runs, treat automation as free
and annualize the best week, so finance discounts them. Measure it yourself: stopwatch a human, subtract the
oversight that stayed, charge build, recurring and per-run costs, price the failures, keep productivity apart
from hard savings, and compute payback from trailing actuals.</div>
        </div>
    </div>
<p>Jonathon Davis opened his <a href="https://networkautomation.forum/autocon6">AutoCon 6</a> talk with a script he wrote
years ago to dial 50 modems and open a ticket for any that didn&rsquo;t answer. Nobody sat down with a list anymore,
and the check went from monthly to weekly. Everyone agreed it was a success. When leadership asked what it was
worth, he had no answer. He could feel the savings, but he couldn&rsquo;t put a number on them.</p>
<p>That&rsquo;s the normal case. His argument: part of the network automation adoption ceiling is an accounting ceiling,
and accounting is something we can fix.</p>
<h2 id="why-dont-finance-teams-trust-automation-time-saved-numbers">Why don&rsquo;t finance teams trust automation time-saved numbers?</h2>
<p>Most automation value is never measured. What does get measured usually comes from the platform that did the
work, and its time-saved dashboard inflates in four ways:</p>
<ul>
<li><strong>Minutes</strong> are an estimate someone typed in, with no human oversight subtracted.</li>
<li><strong>Runs</strong> count only that platform, and only successes.</li>
<li><strong>Cost</strong> is zero. A failed run that burned compute and tokens is invisible, or counted as savings.</li>
<li><strong>Window</strong> is whatever looks biggest. A good week times 52.</li>
</ul>
<p>None of that is fraud, but it&rsquo;s a platform grading its own homework. Finance discounts it on sight, and they&rsquo;re
right to.</p>
<h2 id="how-do-you-measure-what-a-network-automation-returned">How do you measure what a network automation returned?</h2>
<ol>
<li><strong>Baseline from a human.</strong> Stopwatch the manual task. Only runs that replace a real manual run earn labor
value. Extra runs because automation made it cheap get counted but earn nothing.</li>
<li><strong>Subtract oversight.</strong> Babysitting time comes off the top.</li>
<li><strong>Charge real costs:</strong> build, recurring and per-run, including tokens. Engineering time is the biggest cost
and the one most often left out.</li>
<li><strong>Price failures.</strong> Every run pays its cost. Only successful runs earn value.</li>
<li><strong>Separate productivity from hard savings.</strong> Hours returned are productivity. Hard savings means a budget line
went down. Cost avoidance is planned spend that didn&rsquo;t happen. Calling productivity hard savings is the most
common mistake.</li>
<li><strong>Payback from trailing actuals.</strong> Choose the window first (90 days is a good start) and don&rsquo;t move it.</li>
</ol>
<p>His modem example: 190 net minutes per pass, 12 of 52 annual passes with a manual counterpart, about 38 hours a
year returned. At a $75 loaded rate that&rsquo;s about $2,850 of productivity, <em>not</em> hard savings, because no budget
line moved. Build cost about $400, payback about 1.7 months.</p>
<p>His slide for leadership was three lines: about 38 NOC hours a year, paid back in about 1.7 months, risk
reduction not included. A small number he can defend beats a big one he can&rsquo;t.</p>
<p>He open-sourced the method as the <a href="https://github.com/LumaTrack/automation-roi-starter">Automation ROI Starter Kit</a>:
MIT licensed, built on Prometheus, Grafana and one standard-library Python file, with no vendor in the data path.
The guardrails are built in. There&rsquo;s no default for manual minutes, oversight is a required field, runs without a
baseline get a count but no dollars, and productivity and hard savings are never summed. His instructions: fork
it, put your measured numbers in, report every run.</p>
<h2 id="my-take-say-what-it-isnt-and-the-rest-gets-believed">My take: say what it isn&rsquo;t, and the rest gets believed</h2>
<p>This was the most useful talk of the week for anyone who has to justify automation spend. In pre-sales I get
asked for payback on almost every engagement, and too often the only number available is a vendor&rsquo;s time-saved
estimate. The productivity versus hard savings split is the part I&rsquo;m taking home. Saying &ldquo;this returns 38 hours,
it doesn&rsquo;t cut a budget line&rdquo; up front makes every other number in the deck believable.</p>
<p>The other part I&rsquo;d underline: report failures next to successes. Reporting only successes is marketing. It also
explains the survey finding from part 1 of this series, <a href="/posts/automate-network-troubleshooting-from-alerts/">Where Do You Start Automating Network Troubleshooting?</a>:
about half of organizations don&rsquo;t measure automation at all. Nobody wants to publish a number they know won&rsquo;t
survive scrutiny.</p>
<p>Where I&rsquo;d add something is risk reduction. The formula rightly leaves it out of payback, but for read-only
automation like backups, validation and finding dead backup paths before you need them, risk reduction is most
of the value. Jonathon suggested pricing it as the cost of a past outage times the reduction in probability, and
discounting it hard. I&rsquo;d track it in its own field, never summed into payback, with the same discipline as
everything else. The outage-cost side of that is a <a href="/posts/observability-business-impact/">blast radius question</a>,
which is one more reason observability comes before automation.</p>
<h2 id="what-four-questions-should-you-ask-of-any-automation-roi-number">What four questions should you ask of any automation ROI number?</h2>
<ol>
<li>Whose minutes? Measured or guessed? Oversight subtracted?</li>
<li>Which runs? One platform or all? Successes only?</li>
<li>What did it cost? Build, recurring, per-run, failures?</li>
<li>What window? Trailing actuals, or the best week annualized?</li>
</ol>
<p>He ran the questions against a real platform&rsquo;s time-saved dashboard, and it failed all four: a typed-in estimate
per workflow with no oversight field, its own successes but not its failures, no rate or costs with failed runs
missing entirely, and a one-week total that&rsquo;s easy to annualize. Then he ran them against the kit: your measured
minutes with oversight subtracted, every run including failures, build, recurring, per-run and token costs with
failures priced, and only the runs Prometheus actually observed.</p>
<p>Run them over your own numbers before someone in finance does.</p>
<p>This is part 5 of my <a href="/series/autocon6/">AutoCon 6 series</a>.</p>
]]></content:encoded></item></channel></rss>