task 8.3 · chapter 8 of 8: a broken path · 20 min
The black hole
objective
Someone turned on jumbo frames on the laptop and on web, but not on the network between them. Ping works and HTTPS hangs. Find the frame that never gets through, and where it disappears.
answers/8.3.ymlopen a file in VS Code: Ctrl+P (Cmd+P on a Mac), type its name, Enter
| The fault | make break X=mtu: the laptop’s and web’s MTU set to 9000 |
|---|---|
| Capture | make capture AT=laptop:eth1,sw1:eth1 NAME=8.3 X=tls |
| Afterwards | make fix |
| Your answers | answers/8.3.yml |
new to this? step by stepbeginner
make break X=mtu; from the laptop,ping -c 2 web.labworks,curl -m 5 https://web.lab/doesn’t.make capture AT=laptop:eth1,sw1:eth1 NAME=8.3 X=tls.- Filter
frame.len > 1514: which interface has them, which doesn’t? Note the frame length. - Challenge: who dropped them, and why.
make fix, thenmake check T=8.3.
more in meet the lab
When small packets work and big ones don’t
MTU
Every link has an MTU (maximum transmission unit): the biggest packet it can carry in one frame. On ordinary Ethernet it’s 1,500 bytes of IP packet, which makes a 1,514-byte frame once the Ethernet header is added. Some networks use jumbo frames, often 9,000 bytes, to move data with fewer packets.
At the start of a TCP connection, each side announces its MSS (maximum segment size): the biggest chunk of data it can take, worked out from its own MTU (1,460 bytes for 1,500; 8,960 for 9,000). Each side then sends segments no bigger than the other’s MSS.
The black hole
Here’s the trap. If two hosts both have jumbo frames, they each announce 8,960, and each happily sends big segments. But if a link between them only takes 1,500, the big frames don’t fit. A router would at least be able to send back an ICMP message saying too big (that’s how Path MTU Discovery works). A switch can’t send anything: it works at layer 2, has no IP address on the path, and drops the oversized frame without a word.
So small packets (the handshake, ping, DNS) all work, and the first big one disappears. The connection opens and then hangs. That’s called a black hole: packets go in and nothing comes out, and nobody’s told.
in one sentence An MTU mismatch lets small packets through and silently drops big ones: the connection opens, then hangs.
Why TLS finds it first
Remember task 7.1: the laptop’s TLS 1.3 Client Hello is 1,558 bytes, because of its post-quantum key share. On a normal 1,500-byte link TCP splits it in two, but with the laptop thinking it has 9,000 bytes to play with, it goes as one segment. HTTPS breaks before anything else does, while ping, DNS and plain HTTP requests, all small, carry on working.
Sent, sent, sent
Both capture points, TCP only:
11 laptop:eth1 74 42898 → 443 [SYN] Seq=0 Win=62720 Len=0 MSS=8960 ...
12 sw1:eth1 74 [TCP Retransmission] 42898 → 443 [SYN] Seq=0 ... MSS=8960 ...
13 sw1:eth1 74 443 → 42898 [SYN, ACK] Seq=0 Ack=1 ... MSS=8960 ...
14 laptop:eth1 74 [TCP Retransmission] 443 → 42898 [SYN, ACK] Seq=0 Ack=1 ... MSS=8960 ...
15 laptop:eth1 66 42898 → 443 [ACK] Seq=1 Ack=1 ...
16 sw1:eth1 66 [TCP Dup ACK 15#1] 42898 → 443 [ACK] Seq=1 Ack=1 ...
17 laptop:eth1 1624 Client Hello (SNI=web.lab)
18 laptop:eth1 1624 [TCP Retransmission] 42898 → 443 [PSH, ACK] Seq=1 Ack=1 ... Len=1558
19 laptop:eth1 1624 [TCP Retransmission] 42898 → 443 [PSH, ACK] Seq=1 Ack=1 ... Len=1558
...
The third column is the frame length. The handshake crosses both points (the Retransmission and Dup ACK
labels on frames 12 to 16 are the same packets seen twice: task 8.2 explains why to ignore them here). Both sides announced MSS=8960. Then
the Client Hello, 1,624 bytes, goes out on the laptop’s cable again and again, and never once shows up on
sw1:eth1.
Reproduce it, then capture it
Break it, in the VS Code tab’s terminal:
run in the terminalmake break X=mtuThen see the symptom from the laptop: click laptop on the Topology tab, and in the laptop tab:
run in the laptop tabping -c 2 web.lab curl -m 5 https://web.lab/ ip link show eth1Ping answers. curl gives up after five seconds.
ip linkshowsmtu 9000.Capture the laptop’s cable and the switch’s port towards r1 together, while the laptop tries HTTPS:
run in the terminalmake capture AT=laptop:eth1,sw1:eth1 NAME=8.3 X=tlsOpen
/captures/8.3.pcapngand filter the frames too big for a normal cable:frame.len > 1514Note their length, and which interface they’re on. Then add
&& frame.interface_name == "sw1:eth1".Fill in the follow-along lines:
editanswers/8.3.ymlCtrl+P → 8.3.ymlstuck_frame_bytes: 1624 big_frames_past_switch: 0
Who, and why
dropped_by: the box that dropped the frames;laptop_mtu: the laptop’s MTU, the cause.
Then put it right with make fix, and try the curl again.
hint
The frames are on the laptop’s cable and not on the far side of the switch. Only one box is in between.
hint
Why didn’t anything tell the laptop? Which kinds of device can send ICMP too big?
show the solutionanswers/8.3.yml
or pull it into your lab: make solve T=8.3, then git diff --staged shows what changed
dropped_by: sw1
laptop_mtu: 9000
sw1’s ports take 1,500-byte packets. A 1,624-byte frame arrived from the laptop, didn’t fit, and was dropped. A switch has no way to say so, so the laptop kept retransmitting into the hole until curl gave up.
The fix is to make every device on a network agree on the MTU: either jumbo frames everywhere on the path, or
nowhere. And the habit to keep: when ping works but the application doesn’t, filter frame.len > 1514 and
look for frames that are sent over and over and never acknowledged.
5 check: run it, then prove it
$ make check T=8.3when it's right, you'll see
╭─ task 8.3 · the black hole
│ ✓ captures/8.3.pcapng: taken on laptop:eth1, sw1:eth1, ... frames
│ ✓ the capture has the laptop's oversized frames: the fault was on
│ ✓ the size of the frame that never got an answer (follow along): ...
│ ✓ how many of them reached r1's side of the switch (follow along): 0
│ ✓ the box that dropped them (challenge): sw1
│ ✓ the laptop's MTU, the cause (challenge): 9000
╰─ 6/6 checks passed verified