online posts 16
wireshark 101

task 8.2 · chapter 8 of 8: a broken path · 25 min

The mystery fault

objective

The lab breaks the path to web one of three ways, at random. Capture at three points along the path and find out where the packets stop and why, from the packets alone.

you edit, in the labanswers/8.2.yml

open a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter

given
The faultmake break X=mystery (don’t read what it prints: scroll it away)
Capturemake capture AT=laptop:eth1,r1:eth2,r2:eth2 NAME=8.2 X=webip
Afterwardsmake fix
Your answersanswers/8.2.yml: broken_router (r1 or r2) and cause (acl or route)
new to this? step by stepbeginner
  1. make break X=mystery, then make capture AT=laptop:eth1,r1:eth2,r2:eth2 NAME=8.2 X=webip.
  2. Filter tcp && frame.interface_name == "..." for each point in turn. Where does the SYN stop appearing? Does a SYN-ACK appear anywhere?
  3. Answer the router and the cause, then make fix and make check T=8.2.

more in meet the lab

1 explain

Divide the path

Silence is a clue, if you know where it starts

When a connection to a server just hangs, the client’s capture shows one thing: SYNs going out, again and again, and nothing coming back (task 6.2 told you why that’s different from a refusal). The client can’t tell you any more than that. The packets went somewhere, and stopped.

The method is the one you’d use for a dead string of Christmas lights: check halfway along. Capture at several points on the path at the same time, then follow one packet from point to point. The fault is between the last place you see it and the first place you don’t.

in one sentence Capture at several points at once and follow one packet: the fault is between the last point that saw it and the first that didn’t.

Two kinds of silence

On this path, two very different faults look the same from the laptop:

CauseWhat happensWhat the captures show
acl: an access list on a router drops the trafficthe SYN reaches the router and goes no furtherthe SYN on the points before the router, on none after it
route: a router has no route back to the laptopthe SYN gets all the way to web; web answers; the answer dies on the way backthe SYN everywhere, a SYN-ACK leaving web that never reaches the laptop, and an ICMP Network unreachable from that router back to web

An access list (ACL) is a router’s list of permit and deny rules for packets. A router that denies a packet can tell the sender, with an ICMP administratively prohibited message, or drop it silently. The lab’s fault is set up with no ip unreachables, so it drops silently: the most common setting on real networks, and the hardest to debug.

A warning about one file, three cables

When one packet is captured at three points, it’s in your file three times. Wireshark’s TCP analysis doesn’t know they’re the same packet seen in three places: it marks the second and third copies as Retransmission or Dup ACK. Filtering the display to one interface doesn’t help: the flags were worked out over the whole file, before the filter. So use frame.interface_name to see where a packet got to, and don’t trust the tcp.analysis flags in a file with several capture points. If you need them, save one interface’s frames to a file of their own (File › Export Specified Packets, with that interface’s filter applied) and open that.

2 show

Three points on the path

  laptop ──[laptop:eth1]── sw1 ── r1 ──[r1:eth2]── r2 ──[r2:eth2]── web

The capture points, from the laptop outwards:

PointWhat it tells you
laptop:eth1what the laptop sent and received
r1:eth2what got through r1, heading to r2 (and what came back from r2)
r2:eth2what got through r2 and reached web’s cable (and what web sent)

The traffic is X=webip: the laptop opens http://192.168.2.10/ by its address, so a DNS problem can’t get in the way. A good capture plan always takes away the things you aren’t testing.

Here’s how one of the three faults begins, all three points in one listing:

 2	laptop:eth1	39608 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 ...
 3	r1:eth2	[TCP Retransmission] 39608 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 ...
 4	r2:eth2	[TCP Retransmission] 39608 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 ...
 5	r2:eth2	80 → 39608 [SYN, ACK] Seq=0 Ack=1 Win=65160 Len=0 MSS=1460 ...
 7	laptop:eth1	[TCP Retransmission] 39608 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 ...

Frames 3 and 4 aren’t retransmissions: they’re frame 2, seen further along.

4 challenge

Find it

There’s no follow-along this time: this is the whole chapter in one task.

  1. Break it, and don’t read the commands it prints:

    run in the terminal
    make break X=mystery
    clear
    
  2. Capture at all three points while the laptop tries to open the page:

    run in the terminal
    make capture AT=laptop:eth1,r1:eth2,r2:eth2 NAME=8.2 X=webip
    
  3. Open /captures/8.2.pcapng, and for each point in turn filter:

    tcp && frame.interface_name == "laptop:eth1"
    

    (then "r1:eth2", then "r2:eth2"). Where does the SYN stop? Is there a SYN-ACK anywhere?

  4. Write your verdict, then undo the fault:

    editanswers/8.2.ymlCtrl+P → 8.2.yml
    broken_router: r?      # r1 or r2
    cause: ?               # acl or route
    
    run in the terminal
    make fix
    

    make check T=8.2 still knows which fault it was after the fix. Want another go? make break X=mystery again picks a new one at random (there are three).

hint

SYN on laptop:eth1 only: it didn’t get through r1. On laptop:eth1 and r1:eth2 but not r2:eth2: it didn’t get through r2.

hint

A SYN-ACK on r2:eth2 means web got the SYN and answered. Then the problem is on the way back. Filter icmp: did a router say anything about it?

hint

Prove it on the router: click r2 on the Topology tab, then in the r2 tab show ip route and show access-lists.

show the solutionanswers/8.2.yml

or pull it into your lab: make solve T=8.2, then git diff --staged shows what changed

It depends on the fault you got:

What you sawbroken_routercause
SYN on laptop:eth1 onlyr1acl
SYN on laptop:eth1 and r1:eth2, nowhere on r2:eth2r2acl
SYN on all three, SYN-ACK and an ICMP Network unreachable on r2:eth2 onlyr2route

In the last case, r2 has lost its route back to 192.168.1.0/24, so web’s answer never gets back to r1. r2 says so, but to web, not to you: the ICMP Network unreachable goes to the sender of the packet it couldn’t route, which is web. The laptop never hears about it. Notice what doesn’t help: from the laptop, all three look identical, and ping from web to r2 works fine. Only the captures at the right points tell them apart.

5 check: run it, then prove it

$ make check T=8.2

when it's right, you'll see

╭─ task 8.2 · the mystery fault
│  ✓ captures/8.2.pcapng: taken on laptop:eth1, r1:eth2, r2:eth2, ... frames
│  ✓ the router where the packets stop (challenge): ...
│  ✓ the cause: acl (a filter drops them) or route (no way back) (challenge): ...
╰─ 3/3 checks passed  verified