IPv6 not working before pinging the gateway

[Marc Peters]

Hi,

i have a server at the german hosting provider Hetzner. They provide
IPv6. You get a /64 assigned for your host. The problem is, that IPv6
doesn't work right after a reboot, but you have to ping the gateway
first and after that, everything works as expected. For that i have a
line in roots crontab:

@reboot sleep 10 && ping6 -c 10 fe80::1\%em0 > /dev/null

mpi@ suggested to stop working around this and fixing it. He asked for
the output of the routing table before pinging the gateway without IPv6
access and after pinging the gateway with working IPv6.

Before:
~ $ route -n show
Routing tables

Internet:
Destination Gateway Flags Refs Use Mtu Prio
Iface
default 136.243.67.65 UGS 107 402 - 8
em0
224/4 127.0.0.1 URS 0 0 32768 8
lo0
127/8 127.0.0.1 UGRS 0 0 32768 8
lo0
127.0.0.1 127.0.0.1 UHhl 12 409 32768 1
lo0
136.243.67.64/26 136.243.67.92 UCn 1 0 - 4
em0
136.243.67.65 cc:e1:7f:07:e0:88 UHLch 1 3 - 3
em0
136.243.67.92 30:85:a9:a4:ce:5e UHLl 0 121 - 1
em0
136.243.67.127 136.243.67.92 UHb 0 0 - 1
em0

Internet6:
Destination Gateway Flags
Refs Use Mtu Prio Iface
default fe80::1%em0 UGS
2 233 - 8 em0
::/96 ::1 UGRS
0 0 32768 8 lo0
::/104 ::1 UGRS
0 0 32768 8 lo0
::1 ::1 UHhl
15 74 32768 1 lo0
::127.0.0.0/104 ::1 UGRS
0 0 32768 8 lo0
::224.0.0.0/100 ::1 UGRS
0 0 32768 8 lo0
::255.0.0.0/104 ::1 UGRS
0 0 32768 8 lo0
::ffff:0.0.0.0/96 ::1 UGRS
0 0 32768 8 lo0
2002::/24 ::1 UGRS
0 0 32768 8 lo0
2002:7f00::/24 ::1 UGRS
0 0 32768 8 lo0
2002:e000::/20 ::1 UGRS
0 0 32768 8 lo0
2002:ff00::/24 ::1 UGRS
0 0 32768 8 lo0
2a01:4f8:212:216c::/64 2a01:4f8:212:216c::2 UCPn
0 2 - 4 em0
2a01:4f8:212:216c::/64 2a01:4f8:212:216c::25 UCPn
0 0 - 4 em0
2a01:4f8:212:216c::/64 2a01:4f8:212:216c::1:443 UCPn
0 0 - 4 em0
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e UHLl
0 23 - 1 em0
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e UHLl
0 4 - 1 em0
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e UHLl
0 67 - 1 em0
fe80::/10 ::1 UGRS
0 1 32768 8 lo0
fec0::/10 ::1 UGRS
0 0 32768 8 lo0
fe80::%em0/64 fe80::3285:a9ff:fea4:ce5e%em0 UCn
1 0 - 4 em0
fe80::1%em0 link#1 UHLch
1 2 - 3 em0
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e UHLl
0 0 - 1 em0
fe80::1%lo0 fe80::1%lo0 UHl
0 0 32768 1 lo0
ff01::/16 ::1 UGRS
0 1 32768 8 lo0
ff01::%em0/32 fe80::3285:a9ff:fea4:ce5e%em0 Um
0 3 - 4 em0
ff01::%lo0/32 ::1 Um
0 1 32768 4 lo0
ff02::/16 ::1 UGRS
0 1 32768 8 lo0
ff02::%em0/32 fe80::3285:a9ff:fea4:ce5e%em0 Um
0 3 - 4 em0
ff02::%lo0/32 ::1 Um
0 1 32768 4 lo0

Output from not working ping6:
~ $ ping6 www.google.de
PING www.google.de (2a00:1450:4001:821::2003): 56 data bytes
ping6: failed to get receiving hop limit
ping6: failed to get receiving hop limit
ping6: failed to get receiving hop limit
ping6: failed to get receiving hop limit
^C
--- www.google.de ping statistics ---
9 packets transmitted, 0 packets received, 100.0% packet loss


now i ping the GW:
~ $ ping6 -c 5 fe80::1%em0
PING fe80::1%em0 (fe80::1%em0): 56 data bytes
64 bytes from fe80::1%em0: icmp_seq=0 hlim=64 time=1.228 ms
64 bytes from fe80::1%em0: icmp_seq=1 hlim=64 time=0.603 ms
64 bytes from fe80::1%em0: icmp_seq=2 hlim=64 time=0.615 ms
64 bytes from fe80::1%em0: icmp_seq=3 hlim=64 time=0.621 ms
64 bytes from fe80::1%em0: icmp_seq=4 hlim=64 time=0.641 ms

--- fe80::1%em0 ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.603/0.742/1.228/0.244 ms

After:
~ $ route -n show
Routing tables

Internet:
Destination Gateway Flags Refs Use Mtu Prio
Iface
default 136.243.67.65 UGS 14 553 - 8
em0
224/4 127.0.0.1 URS 0 0 32768 8
lo0
116.31.116.39 136.243.67.65 UGHD 0 540 - L 8
em0
127/8 127.0.0.1 UGRS 0 0 32768 8
lo0
127.0.0.1 127.0.0.1 UHhl 12 1340 32768 1
lo0
136.243.67.64/26 136.243.67.92 UCn 1 0 - 4
em0
136.243.67.65 cc:e1:7f:07:e0:88 UHLch 2 4 - 3
em0
136.243.67.92 30:85:a9:a4:ce:5e UHLl 0 172 - 1
em0
136.243.67.127 136.243.67.92 UHb 0 0 - 1
em0

Internet6:
Destination Gateway Flags
Refs Use Mtu Prio Iface
default fe80::1%em0 UGS
0 456 - 8 em0
::/96 ::1 UGRS
0 0 32768 8 lo0
::/104 ::1 UGRS
0 0 32768 8 lo0
::1 ::1 UHhl
15 99 32768 1 lo0
::127.0.0.0/104 ::1 UGRS
0 0 32768 8 lo0
::224.0.0.0/100 ::1 UGRS
0 0 32768 8 lo0
::255.0.0.0/104 ::1 UGRS
0 0 32768 8 lo0
::ffff:0.0.0.0/96 ::1 UGRS
0 0 32768 8 lo0
2002::/24 ::1 UGRS
0 0 32768 8 lo0
2002:7f00::/24 ::1 UGRS
0 0 32768 8 lo0
2002:e000::/20 ::1 UGRS
0 0 32768 8 lo0
2002:ff00::/24 ::1 UGRS
0 0 32768 8 lo0
2a01:4f8:212:216c::/64 2a01:4f8:212:216c::2 UCPn
0 2 - 4 em0
2a01:4f8:212:216c::/64 2a01:4f8:212:216c::25 UCPn
0 0 - 4 em0
2a01:4f8:212:216c::/64 2a01:4f8:212:216c::1:443 UCPn
0 0 - 4 em0
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e UHLl
0 49 - 1 em0
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e UHLl
0 51 - 1 em0
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e UHLl
0 105 - 1 em0
fe80::/10 ::1 UGRS
0 1 32768 8 lo0
fec0::/10 ::1 UGRS
0 0 32768 8 lo0
fe80::%em0/64 fe80::3285:a9ff:fea4:ce5e%em0 UCn
1 0 - 4 em0
fe80::1%em0 cc:e1:7f:07:e0:88 UHLch
1 9 - 3 em0
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e UHLl
0 7 - 1 em0
fe80::1%lo0 fe80::1%lo0 UHl
0 0 32768 1 lo0
ff01::/16 ::1 UGRS
0 1 32768 8 lo0
ff01::%em0/32 fe80::3285:a9ff:fea4:ce5e%em0 Um
0 3 - 4 em0
ff01::%lo0/32 ::1 Um
0 1 32768 4 lo0
ff02::/16 ::1 UGRS
0 1 32768 8 lo0
ff02::%em0/32 fe80::3285:a9ff:fea4:ce5e%em0 Um
0 4 - 4 em0
ff02::%lo0/32 ::1 Um
0 1 32768 4 lo0


pinging google.de does work now:
~ $ ping6 www.google.de
PING www.google.de (2a00:1450:4001:821::2003): 56 data bytes
64 bytes from 2a00:1450:4001:821::2003: icmp_seq=0 hlim=56 time=5.085 ms
64 bytes from 2a00:1450:4001:821::2003: icmp_seq=1 hlim=56 time=5.048 ms
64 bytes from 2a00:1450:4001:821::2003: icmp_seq=2 hlim=56 time=5.052 ms
^C
--- www.google.de ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.048/5.062/5.085/0.016 ms


If any information is missing, i will be happy to provide it.

Maybe someone can shed some light on this and fix it.

Thanks,
Marc



dmesg:
OpenBSD 6.1 (GENERIC.MP) #7: Mon Jun 12 20:41:01 CEST 2017


***@syspatch-61-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

real mem = 17057746944 (16267MB)

avail mem = 16536100864 (15770MB)

mpath0 at root

scsibus0 at mpath0: 256 targets

mainbus0 at root

bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xeb300 (102 entries)

bios0: vendor American Megatrends Inc. version "2106" date 07/16/2012

bios0: ASUSTeK Computer INC. P8B WS

acpi0 at bios0: rev 2

acpi0: sleep states S0 S1 S3 S4 S5

acpi0: tables DSDT FACP APIC MCFG HPET SSDT SSDT SSDT DMAR

acpi0: wakeup devices P0P1(S4) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4)
PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4)
PXSX(S4) RP08(S4) PEGP(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits

acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat

cpu0 at mainbus0: apid 0 (boot processor)

cpu0: Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz, 3400.45 MHz

cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SS
E4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT


cpu4: 256KB 64b/line 8-way L2 cache

cpu4: smt 1, core 0, package 0

cpu5 at mainbus0: apid 3 (application processor)

cpu5: Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz, 3400.02 MHz

cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SS
E4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT


cpu5: 256KB 64b/line 8-way L2 cache

cpu5: smt 1, core 1, package 0

cpu6 at mainbus0: apid 5 (application processor)

cpu6: Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz, 3400.02 MHz

cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SS
E4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT


cpu6: 256KB 64b/line 8-way L2 cache

cpu6: smt 1, core 2, package 0

cpu7 at mainbus0: apid 7 (application processor)

cpu7: Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz, 3400.02 MHz

cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SS
E4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT


cpu7: 256KB 64b/line 8-way L2 cache

cpu7: smt 1, core 3, package 0

ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins

acpimcfg0 at acpi0 addr 0xf8000000, bus 0-63

acpihpet0 at acpi0: 14318179 Hz

acpiprt0 at acpi0: bus 0 (PCI0)

acpiprt1 at acpi0: bus 6 (P0P1)

acpiprt2 at acpi0: bus 3 (RP01)

acpiprt3 at acpi0: bus -1 (RP02)

acpiprt4 at acpi0:

drm0 at inteldrm0

usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb2 at pci0 dev 28 function 0 "Intel 6 Series PCIE" rev 0xb5: msi
pci3 at ppb2 bus 3
ppb3 at pci0 dev 28 function 5 "Intel 6 Series PCIE" rev 0xb5: msi
pci4 at ppb3 bus 4
em0 at pci4 dev 0 function 0 "Intel 82574L" rev 0x00: msi, address
30:85:a9:a4:ce:5e
ppb4 at pci0 dev 28 function 7 "Intel 6 Series PCIE" rev 0xb5: msi
pci5 at ppb4 bus 5
xhci0 at pci5 dev 0 function 0 "ASMedia ASM1042 xHCI" rev 0x00: msi
usb1 at xhci0: USB revision 3.0
uhub1 at usb1 configuration 1 interface 0 "ASMedia xHCI root hub" rev
3.00/1.00 addr 1
ehci1 at pci0 dev 29 function 0 "Intel 6 Series USB" rev 0x05: apic 2 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb5 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci6 at ppb5 bus 6
pcib0 at pci0 dev 31 function 0 "Intel C206 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x05: msi,
AHCI 1.3
ahci0: port 0: 6.0Gb/s
ahci0: port 1: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, ST33000650NS, 0004> SCSI3 0/direct
fixed naa.5000c5004dfb2697
sd0: 2861588MB, 512 bytes/sector, 5860533168 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, ST33000650NS, 0004> SCSI3 0/direct
fixed naa.5000c5004dfb2367
sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 6 Series SMBus" rev 0x05: apic
2 int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: stts2002
sdtemp1 at iic0 addr 0x19: stts2002
sdtemp2 at iic0 addr 0x1a: stts2002
sdtemp3 at iic0 addr 0x1b: stts2002
spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM ECC PC3-10600 with thermal sensor
spdmem1 at iic0 addr 0x51: 4GB DDR3 SDRAM ECC PC3-10600 with thermal sensor
spdmem2 at iic0 addr 0x52: 4GB DDR3 SDRAM ECC PC3-10600 with thermal sensor
spdmem3 at iic0 addr 0x53: 4GB DDR3 SDRAM ECC PC3-10600 with thermal sensor
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33
lm1 at wbsio0 port 0x290/8: NCT6776F
vmm0 at mainbus0: VMX/EPT
uhub3 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uhub4 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uhidev0 at uhub4 port 5 configuration 1 interface 0 "GASIA PS2toUSB
Adapter" rev 1.10/2.01 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub4 port 5 configuration 1 interface 1 "GASIA PS2toUSB
Adapter" rev 1.10/2.01 addr 3
uhidev1: iclass 3/1, 3 report ids
ums0 at uhidev1 reportid 1: 5 buttons, Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
uhid1 at uhidev1 reportid 3: input=2, output=0, feature=0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd2 at scsibus3 targ 1 lun 0: <OPENBSD, SR RAID 1, 006> SCSI2 0/direct fixed
sd2: 2861588MB, 512 bytes/sector, 5860532576 sectors
root on sd2a (7e4e73c2d1d85347.a) swap on sd2b dump on sd2b

[Stuart Henderson]

Post by Marc Peters
Hi,
i have a server at the german hosting provider Hetzner. They provide
IPv6. You get a /64 assigned for your host. The problem is, that IPv6
doesn't work right after a reboot, but you have to ping the gateway
first and after that, everything works as expected. For that i have a
@reboot sleep 10 && ping6 -c 10 fe80::1\%em0 > /dev/null

How are your PF rules? Do they allow NDP packets to pass? If you're
unsure, I would try "pass log inet6 proto icmp6" or similar.

(this might be a bit of a surprise if used to IPv4 where address
resolution is done by a separate protocol that PF doesn't block).

Post by Marc Peters
the output of the routing table before pinging the gateway without IPv6
access and after pinging the gateway with working IPv6.

This is the main difference between them:

-fe80::1%em0 link#1 UHLch 1 2 - 3 em0
+fe80::1%em0 cc:e1:7f:07:e0:88 UHLch 1 9 - 3 em0

[Marc Peters]

Post by Stuart Henderson
How are your PF rules? Do they allow NDP packets to pass? If you're
unsure, I would try "pass log inet6 proto icmp6" or similar.
(this might be a bit of a surprise if used to IPv4 where address
resolution is done by a separate protocol that PF doesn't block).

I don't block any icmp6:
pass inet6 proto icmp6 all

is already present in my /etc/pf.conf

Killing the ndp entry brings my connection down, too:

~ # ndp -na
Neighbor Linklayer Address Netif Expire
S Flags
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e em0 permanent R l
fe80::1%em0 cc:e1:7f:07:e0:88 em0 23h59m54s S R
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e em0 permanent R l

~ # ndp -d fe80::1%em0

fe80::1%em0 (fe80::1%em0) deleted

~ # ping6 www.google.de

PING www.google.de (2a00:1450:4001:821::2003): 56 data bytes

^C

--- www.google.de ping statistics ---

13 packets transmitted, 0 packets received, 100.0% packet loss

~ # ping6 fe80::1%em0

PING fe80::1%em0 (fe80::1%em0): 56 data bytes

64 bytes from fe80::1%em0: icmp_seq=0 hlim=64 time=9.001 ms

64 bytes from fe80::1%em0: icmp_seq=1 hlim=64 time=0.610 ms

^C

--- fe80::1%em0 ping statistics ---

2 packets transmitted, 2 packets received, 0.0% packet loss

round-trip min/avg/max/std-dev = 0.610/4.806/9.001/4.196 ms

~ # ping6 www.google.de

PING www.google.de (2a00:1450:4001:821::2003): 56 data bytes

64 bytes from 2a00:1450:4001:821::2003: icmp_seq=0 hlim=56 time=5.014 ms

64 bytes from 2a00:1450:4001:821::2003: icmp_seq=1 hlim=56 time=5.045 ms

^C

--- www.google.de ping statistics ---

2 packets transmitted, 2 packets received, 0.0% packet loss

round-trip min/avg/max/std-dev = 5.014/5.029/5.045/0.015 ms

Is there any way for us to fix it or is it just a misconfiguration at
Hetzner?

[Stuart Henderson]

Post by Marc Peters

Post by Stuart Henderson
How are your PF rules? Do they allow NDP packets to pass? If you're
unsure, I would try "pass log inet6 proto icmp6" or similar.
(this might be a bit of a surprise if used to IPv4 where address
resolution is done by a separate protocol that PF doesn't block).

pass inet6 proto icmp6 all
is already present in my /etc/pf.conf

Are there any other rules which might interfere with this one? This
issue feels very much like NDP not getting through in some circumstances.

For instance I had problems at an IXP where one peer was sourcing the
NDP from an fe80:: address which was getting blocked by a too-restrictive
"drop junk packets" type of rule. Everyone else was sending them with a
"real" source address which wasn't triggering that rule - it took a
while to track down!

I would want to be 100% sure of this before digging deeper (e.g. with
"match log(matches) proto icmp6" at the top of the ruleset and watching
pflog when flushing ndp).

I think the step after that would be seeing what you get from nd6 debug
messages, either you can build a kernel with the ND6_DEBUG option, or if
you can break into DDB, you don't actually need a new kernel, just
'w nd6_debug 1' and 'c' should do the trick - then see what shows up
in /var/log/messages.

[Marc Peters]

Post by Stuart Henderson

Post by Marc Peters

Post by Stuart Henderson
How are your PF rules? Do they allow NDP packets to pass? If you're
unsure, I would try "pass log inet6 proto icmp6" or similar.
(this might be a bit of a surprise if used to IPv4 where address
resolution is done by a separate protocol that PF doesn't block).

pass inet6 proto icmp6 all
is already present in my /etc/pf.conf

Are there any other rules which might interfere with this one? This
issue feels very much like NDP not getting through in some circumstances.

Here is the running set:
~ # pfctl -sr
block drop log all
block drop in log quick from <bad-ssh> to any
match in all scrub (no-df random-id)
match log (matches) proto ipv6-icmp all
pass out on egress proto tcp all flags S/SA
pass out on egress proto udp all
pass out on egress proto icmp all
pass in on em0 inet proto tcp from any to 136.243.67.92 port = 22 flags
S/SA keep state (source-track rule, max-src-conn 15
, max-src-conn-rate 2/60, overload <bad-ssh> flush global, src.track 60)
pass in on em0 inet6 proto tcp from any to fe80::3285:a9ff:fea4:ce5e
port = 22 flags S/SA keep state (source-track rule, ma
x-src-conn 15, max-src-conn-rate 2/60, overload <bad-ssh> flush global,
src.track 60)
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::2 port =
22 flags S/SA keep state (source-track rule, max-src
-conn 15, max-src-conn-rate 2/60, overload <bad-ssh> flush global,
src.track 60)
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::25 port =
22 flags S/SA keep state (source-track rule, max-sr
c-conn 15, max-src-conn-rate 2/60, overload <bad-ssh> flush global,
src.track 60)
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::1:443 port
= 22 flags S/SA keep state (source-track rule, max
-src-conn 15, max-src-conn-rate 2/60, overload <bad-ssh> flush global,
src.track 60)
pass in on em0 inet6 proto tcp from any to fe80::3285:a9ff:fea4:ce5e
port = 587 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::2 port =
587 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::25 port =
587 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::1:443 port
= 587 flags S/SA
pass in on em0 inet6 proto tcp from any to fe80::3285:a9ff:fea4:ce5e
port = 993 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::2 port =
993 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::25 port =
993 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::1:443 port
= 993 flags S/SA
pass in on em0 inet proto tcp from any to 136.243.67.92 port = 587 flags
S/SA
pass in on em0 inet proto tcp from any to 136.243.67.92 port = 993 flags
S/SA
pass in on em0 proto udp from any to any port 33433 >< 33626
pass inet proto icmp all icmp-type echoreq
pass inet6 proto ipv6-icmp all
pass in log on egress inet proto tcp from any to any port = 25 flags
S/SA rdr-to 127.0.0.1 port 8025
pass in log (to pflog1) on egress proto tcp from <nospamd> to any port =
25 flags S/SA
pass in log (to pflog1) on egress proto tcp from <spamd-white> to any
port = 25 flags S/SA
pass in log (to pflog1) on egress inet6 proto tcp from any to any port =
25 flags S/SA
pass in log (to pflog1) quick on egress proto tcp from
<bgp-spamd-bypass> to any port = 25 flags S/SA
pass out log (to pflog1) on egress proto tcp from any to any port = 25
flags S/SA
pass in on em0 inet6 proto tcp from any to fe80::3285:a9ff:fea4:ce5e
port = 80 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::2 port =
80 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::25 port =
80 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::1:443 port
= 80 flags S/SA
pass in on em0 inet6 proto tcp from any to fe80::3285:a9ff:fea4:ce5e
port = 443 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::2 port =
443 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::25 port =
443 flags S/SA
pass in on em0 inet6 proto tcp from any to 2a01:4f8:212:216c::1:443 port
= 443 flags S/SA
pass in on em0 inet proto tcp from any to 136.243.67.92 port = 80 flags S/SA
pass in on em0 inet proto tcp from any to 136.243.67.92 port = 443 flags
S/SA
block drop in on ! lo0 proto tcp from any to any port 6000:6010
block drop in on ! lo inet6 from ::1 to any
block drop in on ! lo inet from 127.0.0.0/8 to any
block drop in inet6 from ::1 to any
block drop in on lo0 inet6 from fe80::1 to any
block drop in on ! em0 inet6 from 2a01:4f8:212:216c::/64 to any
block drop in on em0 inet6 from fe80::3285:a9ff:fea4:ce5e to any
block drop in inet6 from 2a01:4f8:212:216c::2 to any
block drop in inet6 from 2a01:4f8:212:216c::25 to any
block drop in inet6 from 2a01:4f8:212:216c::1:443 to any
block drop in inet from 127.0.0.1 to any
block drop in on ! em0 inet from 136.243.67.64/26 to any
block drop in inet from 136.243.67.92 to any

Post by Stuart Henderson
For instance I had problems at an IXP where one peer was sourcing the
NDP from an fe80:: address which was getting blocked by a too-restrictive
"drop junk packets" type of rule. Everyone else was sending them with a
"real" source address which wasn't triggering that rule - it took a
while to track down!
I would want to be 100% sure of this before digging deeper (e.g. with
"match log(matches) proto icmp6" at the top of the ruleset and watching
pflog when flushing ndp).

flushed and try to ping google:
~ # tcpdump -eni pflog0

tcpdump: WARNING: snaplen raised from 116 to 160

tcpdump: listening on pflog0, link-type PFLOG

17:51:40.802154 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:51:40.802161 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:51:40.802164 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:51:44.315096 rule 0/(match) block in on em0: 14.102.54.71.44014 >
136.243.67.92.23: S 2297643868:2297643868(0) win 17694
17:51:54.352603 rule 0/(match) block in on em0: 183.83.2.130.17434 >
136.243.67.92.23: S 2297643868:2297643868(0) win 8562
17:52:10.602114 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:52:10.602120 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:52:10.602125 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:52:12.006267 rule 0/(match) block in on em0: 91.223.133.13.58044 >
136.243.67.92.4447: S 3013916555:3013916555(0) win 10
24

17:52:16.856321 rule 0/(match) block in on em0: 61.231.101.145.59990 >
136.243.67.92.445: S 4146551513:4146551513(0) win 81
92 <mss 1440,nop,wscale 2,nop,nop,sackOK> (DF)













17:52:40.782015 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:52:40.782021 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:52:40.782026 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: [|icmp6]
17:52:55.907212 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:821::2003: icmp6: echo request
17:52:55.907217 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:821::2003: icmp6: echo request
17:52:55.907221 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:821::2003: icmp6: echo request
17:52:55.907233 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
17:52:55.907237 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
17:52:55.907240 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
17:53:35.791950 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
17:53:35.791956 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
17:53:35.791960 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
^C
22 packets received by filter

Post by Stuart Henderson
I think the step after that would be seeing what you get from nd6 debug
messages, either you can build a kernel with the ND6_DEBUG option, or if
you can break into DDB, you don't actually need a new kernel, just
'w nd6_debug 1' and 'c' should do the trick - then see what shows up
in /var/log/messages.

I would prefer the new kernel for this, as this is an offsite machine.

[Stefan Sperling]

Post by Marc Peters
Is there any way for us to fix it or is it just a misconfiguration at
Hetzner?

It might help to look at what is actually going over the wire
while pings are stuck: tcpdump -n -i em0 ip6

[Marc Peters]

Post by Stefan Sperling

Post by Marc Peters
Is there any way for us to fix it or is it just a misconfiguration at
Hetzner?

It might help to look at what is actually going over the wire
while pings are stuck: tcpdump -n -i em0 ip6

right after flushing the ndp and trying to ping google:

~ # tcpdump -n -i em0 ip6

tcpdump: listening on em0, link-type EN10MB

17:52:55.907249 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:52:55.908742 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:52:56.901975 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:52:56.902528 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:52:57.901975 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:52:57.902535 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:52:58.911999 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:52:58.914579 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:52:59.911971 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:52:59.914208 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:00.911968 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:00.913858 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:01.912130 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:01.914075 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:02.911972 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:02.913241 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:03.911961 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:03.913586 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:04.912121 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:04.914017 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:05.827118 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 3374201201:3374201220(19) ack
2980031881 wi
n 267 <nop,nop,timestamp 2273063251 659659698>: BGP (KEEPALIVE)
[flowlabel 0x569b1]
17:53:05.911961 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:05.914806 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:06.826922 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 0:19(19) ack 1 win 267
<nop,nop,timestamp 2
273063253 659659698>: BGP (KEEPALIVE) [flowlabel 0x569b1]

17:53:06.911954 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:06.913726 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:07.912113 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:07.913632 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:08.826940 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 0:19(19) ack 1 win 267
<nop,nop,timestamp 2
273063257 659659698>: BGP (KEEPALIVE) [flowlabel 0x569b1]

17:53:08.911949 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:08.913862 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:09.911951 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:09.913609 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:10.912108 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:10.914284 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:11.911942 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:11.913738 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:12.826657 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 0:19(19) ack 1 win 267
<nop,nop,timestamp 2
273063265 659659698>: BGP (KEEPALIVE) [flowlabel 0x569b1]
17:53:12.911941 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:12.913520 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:13.912098 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:13.913934 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:14.911935 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:14.914060 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:15.911935 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:15.913487 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:16.912092 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:16.913561 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:17.911928 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:17.913791 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:18.911925 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:18.914068 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:20.791941 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:20.792476 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:20.826523 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 0:19(19) ack 1 win 267
<nop,nop,timestamp 2
273063281 659659698>: BGP (KEEPALIVE) [flowlabel 0x569b1]
17:53:21.791925 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:21.792468 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:22.791917 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:22.792472 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:35.791970 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:35.792466 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:35.806953 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 19:38(19) ack 1 win 267 <nop,nop,timestamp
2273063311 659659698>: BGP (KEEPALIVE) [flowlabel 0x569b1]
17:53:36.791905 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:36.792441 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
17:53:36.825804 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.26066: P 0:38(38) ack 1 win 267
<nop,nop,timestamp 2
273063313 659659698>: BGP (KEEPALIVE) [flowlabel 0x569b1]
17:53:37.791886 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
17:53:37.792437 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
^C
380 packets received by filter
0 packets dropped by kernel

[Martin Pieuchot]

Post by Stefan Sperling

Post by Marc Peters
Is there any way for us to fix it or is it just a misconfiguration at
Hetzner?

It might help to look at what is actually going over the wire
while pings are stuck: tcpdump -n -i em0 ip6

Could you set net.inet6.icmp6.nd6_debug to 1 and redo this?

Do you see anything in the log?

[Marc Peters]

Post by Martin Pieuchot

Post by Stefan Sperling

Post by Marc Peters
Is there any way for us to fix it or is it just a misconfiguration at
Hetzner?

It might help to look at what is actually going over the wire
while pings are stuck: tcpdump -n -i em0 ip6

Could you set net.inet6.icmp6.nd6_debug to 1 and redo this?
Do you see anything in the log?

Haven't got time to rebuild the kernel with debug options yet, or is
this not needed?

[Stefan Sperling]

Post by Marc Peters
Haven't got time to rebuild the kernel with debug options yet, or is
this not needed?

Not needed. Just run sysctl net.inet6.icmp6.nd6_debug=1

[Marc Peters]

Post by Martin Pieuchot

Post by Stefan Sperling

Post by Marc Peters
Is there any way for us to fix it or is it just a misconfiguration at
Hetzner?

It might help to look at what is actually going over the wire
while pings are stuck: tcpdump -n -i em0 ip6

Could you set net.inet6.icmp6.nd6_debug to 1 and redo this?
Do you see anything in the log?

Rebooting the box with the sysctl active show following /var/log/messages:

Jun 26 12:25:35 arafel /bsd: nd6_na_input: ND packet from non-neighbor
Jun 26 12:25:35 arafel apmd: battery status: absent. external power
status: not known. estimated battery life 0%
Jun 26 12:25:36 arafel /bsd: nd6_na_input: ND packet from non-neighbor
Jun 26 12:26:07 arafel last message repeated 15 times
Jun 26 12:28:08 arafel last message repeated 61 times


This is, what i did, including the tcpdumps:

~ # sysctl net.inet6.icmp6.nd6_debug

net.inet6.icmp6.nd6_debug=1
***@arafel
~ # ndp -na
Neighbor Linklayer Address Netif Expire
S Flags
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e em0 permanent R l
fe80::1%em0 (incomplete) em0 expired
I 1
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e em0 permanent R l
***@arafel
~ #
I-search:
~ # ndp -na
~ # ndp -d fe80::1%em0
fe80::1%em0 (fe80::1%em0) deleted
***@arafel
~ # ping6 www.google.de
^C
***@arafel
~ # ndp -na
Neighbor Linklayer Address Netif Expire
S Flags
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e em0 permanent R l
fe80::1%em0 (incomplete) em0 1s
I 2
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e em0 permanent R l
***@arafel
~ # ping6 fe80::1%em0
PING fe80::1%em0 (fe80::1%em0): 56 data bytes
64 bytes from fe80::1%em0: icmp_seq=4 hlim=64 time=821.274 ms
64 bytes from fe80::1%em0: icmp_seq=5 hlim=64 time=1.836 ms
64 bytes from fe80::1%em0: icmp_seq=6 hlim=64 time=0.636 ms
64 bytes from fe80::1%em0: icmp_seq=7 hlim=64 time=0.595 ms
64 bytes from fe80::1%em0: icmp_seq=8 hlim=64 time=0.633 ms
64 bytes from fe80::1%em0: icmp_seq=9 hlim=64 time=1.617 ms
^C
--- fe80::1%em0 ping statistics ---
10 packets transmitted, 6 packets received, 40.0% packet loss
round-trip min/avg/max/std-dev = 0.595/137.765/821.274/305.675 ms
***@arafel
~ # ping6 www.google.de
PING www.google.de (2a00:1450:4001:81e::2003): 56 data bytes
64 bytes from 2a00:1450:4001:81e::2003: icmp_seq=0 hlim=56 time=5.073 ms
64 bytes from 2a00:1450:4001:81e::2003: icmp_seq=1 hlim=56 time=5.019 ms
64 bytes from 2a00:1450:4001:81e::2003: icmp_seq=2 hlim=56 time=5.077 ms
^C
--- www.google.de ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.019/5.056/5.077/0.027 ms
***@arafel
~ # ndp -na
Neighbor Linklayer Address Netif Expire
S Flags
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e em0 permanent R l
fe80::1%em0 cc:e1:7f:07:e0:88 em0 13s R R
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e em0 permanent R l
***@arafel
~ # ndp -d fe80::1%em0
fe80::1%em0 (fe80::1%em0) deleted
***@arafel
~ # ping6 www.google.de
PING www.google.de (2a00:1450:4001:81e::2003): 56 data bytes
^C
--- www.google.de ping statistics ---
5 packets transmitted, 0 packets received, 100.0% packet loss
***@arafel
~ # ndp -na
Neighbor Linklayer Address Netif Expire
S Flags
2a01:4f8:212:216c::2 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::25 30:85:a9:a4:ce:5e em0 permanent R l
2a01:4f8:212:216c::1:443 30:85:a9:a4:ce:5e em0 permanent R l
fe80::1%em0 cc:e1:7f:07:e0:88 em0 expired I R
fe80::3285:a9ff:fea4:ce5e%em0 30:85:a9:a4:ce:5e em0 permanent R l
***@arafel
~ # ping6 www.google.de
PING www.google.de (2a00:1450:4001:81e::2003): 56 data bytes
^C
--- www.google.de ping statistics ---
9 packets transmitted, 0 packets received, 100.0% packet loss
***@arafel
~ # ping fe80::1%em0
ping: no address associated with name
***@arafel
~ # ping6 fe80::1%em0
PING fe80::1%em0 (fe80::1%em0): 56 data bytes
64 bytes from fe80::1%em0: icmp_seq=0 hlim=64 time=10.294 ms
64 bytes from fe80::1%em0: icmp_seq=1 hlim=64 time=0.599 ms
64 bytes from fe80::1%em0: icmp_seq=2 hlim=64 time=0.640 ms
^C
--- fe80::1%em0 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.599/3.844/10.294/4.561 ms
***@arafel
~ # ping6 www.google.de
PING www.google.de (2a00:1450:4001:81e::2003): 56 data bytes
64 bytes from 2a00:1450:4001:81e::2003: icmp_seq=0 hlim=56 time=5.067 ms
64 bytes from 2a00:1450:4001:81e::2003: icmp_seq=1 hlim=56 time=5.007 ms
^C
--- www.google.de ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.007/5.037/5.067/0.030 ms

~ # tcpdump -eni pflog0
tcpdump: WARNING: snaplen raised from 116 to 160
tcpdump: listening on pflog0, link-type PFLOG
12:29:41.398517 rule 0/(match) block in on em0: 93.174.93.136.54993 >
136.243.67.92.1111: S 1582464766:1582464766(0) win 102
4
12:29:44.678646 rule 0/(match) block in on em0: 88.255.141.18.51498 >
136.243.67.92.445: S 1424326710:1424326710(0) win 8192
<mss 1460,nop,wscale 8,nop,nop,sackOK> (DF)
12:30:08.321078 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
12:30:08.321084 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
12:30:08.321087 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
12:30:15.713275 rule 0/(match) block out on em0: 136.243.67.92.22 >
91.197.232.107.58392: F 2633281877:2633281877(0) ack 425
8121315 win 271 <nop,nop,timestamp 1328506330 864465987> (DF)
12:30:27.534638 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:30:27.534640 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:30:27.534643 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:30:29.204646 rule 0/(match) block in on em0: 83.7.100.86.19892 >
136.243.67.92.23: S 2297643868:2297643868(0) win 31620 <
mss 1452>
12:30:32.363192 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: [|icmp6]
12:30:32.363199 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: [|icmp6]
12:30:32.363203 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: [|icmp6]
12:30:42.311185 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:30:42.311191 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:30:42.311196 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:30:56.742347 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:30:56.742353 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:30:56.742356 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:30:56.742369 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
12:30:56.742374 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
12:30:56.742377 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: [|icmp6]
12:31:00.218490 rule 0/(match) block in on em0: 5.153.178.1.60341 >
136.243.67.92.445: S 3678376091:3678376091(0) win 8192 <
mss 1460,nop,wscale 2,nop,nop,sackOK> (DF)
12:31:07.122538 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:31:07.122541 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:31:07.122544 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:31:19.713063 rule 0/(match) block out on em0: 136.243.67.92.22 >
91.197.232.107.58392: F 0:0(0) ack 1 win 271 <nop,nop,ti
mestamp 1328506458 864465987> (DF)
12:31:26.110829 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:31:26.110834 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:31:26.110838 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:31:26.110853 rule 3/(match) match out on em0:
fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: [|icmp6]
12:31:26.110856 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: [|icmp6]
12:31:26.110860 rule 24/(match) pass out on em0:
fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: [|icmp6]
12:31:28.976105 rule 0/(match) block in on em0: 202.134.2.14.56116 >
136.243.67.92.445: S 4044183613:4044183613(0) win 8192
<mss 1460,nop,wscale 2,nop,nop,sackOK> (DF)
12:31:31.415639 rule 3/(match) match out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:31:31.415644 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:31:31.415647 rule 24/(match) pass out on em0:
2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003: icmp6: echo request
12:31:46.942162 rule 0/(match) block in on em0: 180.66.99.62.44499 >
136.243.67.92.1900: udp 94 (DF)
12:31:48.326376 rule 0/(match) block in on em0: 114.42.192.116.63733 >
136.243.67.92.445: S 1383329208:1383329208(0) win 819
2 <mss 1440,nop,wscale 2,nop,nop,sackOK> (DF)
^C
39 packets received by filter
0 packets dropped by kernel

~ # tcpdump -n -i em0 ip6

tcpdump: listening on em0, link-type EN10MB

12:30:08.321097 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:08.321669 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:09.313286 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:09.313841 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:10.313292 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:10.313766 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:11.326948 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:11.327500 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:12.323296 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:12.323844 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:13.323279 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:13.323837 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:14.323687 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:14.324219 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:15.323269 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:15.323742 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:16.323265 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:16.323803 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:18.795505 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:18.796040 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:19.793247 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:19.794197 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:20.793245 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:20.793840 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:21.809654 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:21.810181 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:22.803237 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:22.803796 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:23.803229 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:23.803773 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:24.803593 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:24.804136 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:25.803318 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:25.803750 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:26.803221 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:26.803696 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:28.105383 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:28.105964 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:29.103205 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:29.103845 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:30.103201 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:30.103699 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:31.363366 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:31.363955 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:32.363219 fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:32.363746 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: neighbor
adv: tgt is fe80::1 [class 0xc0]
12:30:32.363786 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request

12:30:32.364358 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:30:32.543200 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request

12:30:32.544968 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:30:33.543196 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request

12:30:33.543762 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:30:34.543223 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request

12:30:34.543746 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:30:34.589536 2a01:4f8:212:216c::1:443.54168 >
2001:67c:2218:2::4:1.53: 34657 [1au] A? ns2.nic.fr. (39) [flowlabel 0x49d8
9]

12:30:34.600365 2a01:4f8:212:216c::1:443.53631 >
2001:660:5301:1e::101.53: 43024 [1au] A? imag.imag.fr. (41) [flowlabel 0x4
107a]

12:30:34.604574 2001:67c:2218:2::4:1.53 >
2a01:4f8:212:216c::1:443.54168: 34657*- 2/0/1 A 192.93.0.4, (349)
12:30:34.627114 2001:660:5301:1e::101.53 >
2a01:4f8:212:216c::1:443.53631: 43024*- 1/0/1 A 129.88.30.1 (57)
12:30:35.543214 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request

12:30:35.543743 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:30:36.543228 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request

12:30:36.544751 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:30:42.311209 2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003:
icmp6: echo request
12:30:42.316167 2a00:1450:4001:81e::2003 > 2a01:4f8:212:216c::1:443:
icmp6: echo reply
12:30:43.313185 2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003:
icmp6: echo request
12:30:43.318108 2a00:1450:4001:81e::2003 > 2a01:4f8:212:216c::1:443:
icmp6: echo reply
12:30:44.313194 2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003:
icmp6: echo request
12:30:44.318163 2a00:1450:4001:81e::2003 > 2a01:4f8:212:216c::1:443:
icmp6: echo reply
12:30:56.742386 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:56.742964 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:57.733161 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:57.733701 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:58.733153 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:58.733671 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:30:59.743221 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:59.743749 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:00.743167 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:00.743734 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:01.743160 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:01.743746 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:07.122552 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:07.123095 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:08.113108 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1 12:31:08.113670
2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6: neighbor adv: tgt is
fe80::1 [class 0xc0]
12:31:09.113102 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:09.113664 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:10.123113 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:10.123660 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:11.123096 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1 12:31:11.123652
2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6: neighbor adv: tgt is
fe80::1 [class 0xc0]
12:31:12.123089 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1 12:31:12.123658
2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6: neighbor adv: tgt is
fe80::1 [class 0xc0]
12:31:13.123246 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:13.123774 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:14.123081 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:14.123633 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:15.123075 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:15.123596 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]
12:31:26.110869 fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:31:26.117905 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: neighbor
adv: tgt is fe80::1 [class 0xc0] 12:31:26.117972
fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:31:26.121017 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply

12:31:27.113085 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:31:27.113590 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply
12:31:28.103057 fe80::3285:a9ff:fea4:ce5e > fe80::1: icmp6: echo request
12:31:28.103604 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: echo reply
12:31:31.415658 2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003:
icmp6: echo request
12:31:31.420614 2a00:1450:4001:81e::2003 > 2a01:4f8:212:216c::1:443:
icmp6: echo reply
12:31:32.423044 2a01:4f8:212:216c::1:443 > 2a00:1450:4001:81e::2003:
icmp6: echo request
12:31:32.427963 2a00:1450:4001:81e::2003 > 2a01:4f8:212:216c::1:443:
icmp6: echo reply
12:31:43.783017 2a01:4f8:212:216c::1:443.18059 >
2a00:15a8:0:100:0:d91f:50aa:1.179: S 558055419:558055419(0) win 16384 <mss
1440,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 1323955660 0>
[flowlabel 0x80fe5]
12:31:43.788441 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.18059: S 1159803442:1159803442(0) ack 558055420
win 16384 <mss 1440,nop,wscale 6,nop,nop,timestamp 3320032842 1323955660>
12:31:43.788514 2a01:4f8:212:216c::1:443.18059 >
2a00:15a8:0:100:0:d91f:50aa:1.179: . ack 1 win 256 <nop,nop,timestamp 1323
955660 3320032842> [flowlabel 0x80fe5]
12:31:43.788622 2a01:4f8:212:216c::1:443.18059 >
2a00:15a8:0:100:0:d91f:50aa:1.179: P 1:54(53) ack 1 win 256 <nop,nop,times
tamp 1323955660 3320032842>: BGP [|BGP OPEN] [flowlabel 0x80fe5]
12:31:43.794670 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.18059: P 1:48(47) ack 54 win 266
<nop,nop,timestamp 3320032842 1323955660>: BGP [|BGP OPEN] [flowlabel
0x6d122]
12:31:43.794818 2a01:4f8:212:216c::1:443.18059 >
2a00:15a8:0:100:0:d91f:50aa:1.179: P 54:73(19) ack 48 win 256
<nop,nop,timestamp 1323955660 3320032842>: BGP (KEEPALIVE) [flowlabel
0x80fe5]
12:31:43.795049 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.18059: P 48:67(19) ack 54 win 267 <nop,nop,tim
estamp 3320032842 1323955660>: BGP (KEEPALIVE) [flowlabel 0x6d122]
12:31:43.796767 2a01:4f8:212:216c::1:443.18059 >
2a00:15a8:0:100:0:d91f:50aa:1.179: P 73:102(29) ack 67 win 256 <nop,nop,ti
mestamp 1323955660 3320032842>: BGP (UPDATE: (Path attributes:
(MP_UNREACH_NLRI[O] IPv6 Unicast, Withdraw))) [flowlabel 0x8
0fe5]
12:31:43.802216 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.18059: . ack 102 win 267 <nop,nop,timestamp 33
20032842 1323955660> [flowlabel 0x6d122]
12:31:43.803313 2a00:15a8:0:100:0:d91f:50aa:1.179 >
2a01:4f8:212:216c::1:443.18059: P 67:295(228) ack 102 win 267
<nop,nop,timestamp 3320032842 1323955660>: BGP [|BGP UPDATE] [flowlabel
0x6d122]
12:31:44.002939 2a01:4f8:212:216c::1:443.18059 >
2a00:15a8:0:100:0:d91f:50aa:1.179: . ack 295 win 256 <nop,nop,timestamp 13
23955661 3320032842> [flowlabel 0x80fe5]
^C
3448 packets received by filter
0 packets dropped by kernel

[Martin Pieuchot]

Post by Marc Peters

Post by Martin Pieuchot
[...]
Could you set net.inet6.icmp6.nd6_debug to 1 and redo this?
Do you see anything in the log?

Jun 26 12:25:35 arafel /bsd: nd6_na_input: ND packet from non-neighbor

Indeed.

Your machine is asking with the following address:

2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6: neighbor sol: who has fe80::1

Your provider is answering with an address that doesn't match your
subnet:

2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6: neighbor adv: tgt is fe80::1 [class 0xc0]

It'd be nice if somebody could tell us what the RFCs say about this
case. Florian do you have an idea? Should we fix something or should
Marc tell his provider to fix his setup?

[R0me0 ***]

Did you open a trouble ticket on Hetzner?

Post by Martin Pieuchot

Post by Marc Peters

Post by Martin Pieuchot
[...]
Could you set net.inet6.icmp6.nd6_debug to 1 and redo this?
Do you see anything in the log?

Rebooting the box with the sysctl active show following
Jun 26 12:25:35 arafel /bsd: nd6_na_input: ND packet from non-neighbor

Indeed.
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6: neighbor sol: who has fe80::1
Your provider is answering with an address that doesn't match your
2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6: neighbor adv: tgt is
fe80::1 [class 0xc0]
It'd be nice if somebody could tell us what the RFCs say about this
case. Florian do you have an idea? Should we fix something or should
Marc tell his provider to fix his setup?

[Florian Obser]

Post by Martin Pieuchot

Post by Marc Peters

Post by Martin Pieuchot
[...]
Could you set net.inet6.icmp6.nd6_debug to 1 and redo this?
Do you see anything in the log?

Jun 26 12:25:35 arafel /bsd: nd6_na_input: ND packet from non-neighbor

Indeed.
2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6: neighbor sol: who has fe80::1
Your provider is answering with an address that doesn't match your
2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6: neighbor adv: tgt is fe80::1 [class 0xc0]
It'd be nice if somebody could tell us what the RFCs say about this
case. Florian do you have an idea? Should we fix something or should
Marc tell his provider to fix his setup?

this was introduced by claudio@ in rev. 1.53 of nd6_nbr.c:

If a neighbor solictation isn't from the unspecified address, make sure
that the source address matches one of the interfaces address prefixes.
From NetBSD, tested by todd@ and naddy@


netbsd added this in their rev 1.89 and 1.90:

If a neighbor solictation isn't from the unspecified address, make sure
that the source address matches one of the interfaces address prefixes.

and:
Generalize previous fix so that both NS and NA packets are checked.


However, I don't get why. Other than being extra paranoia defending
against a misbehaving router maybe?. We already check a hop limit of
255, so the packet had to be generated on-link and not forwarded by a
router.

RFC4861 defines a "valid advertisement":

7.1.2. Validation of Neighbor Advertisements

A node MUST silently discard any received Neighbor Advertisement
messages that do not satisfy all of the following validity checks:

- The IP Hop Limit field has a value of 255, i.e., the packet
could not possibly have been forwarded by a router.

- ICMP Checksum is valid.

- ICMP Code is 0.

- ICMP length (derived from the IP length) is 24 or more octets.

- Target Address is not a multicast address.

- If the IP Destination Address is a multicast address the
Solicited flag is zero.

- All included options have a length that is greater than zero.

The contents of the Reserved field, and of any unrecognized options,
MUST be ignored. Future, backward-compatible changes to the protocol
may specify the contents of the Reserved field or add new options;
backward-incompatible changes may use different Code values.

The contents of any defined options that are not specified to be used
with Neighbor Advertisement messages MUST be ignored and the packet
processed as normal. The only defined option that may appear is the
Target Link-Layer Address option.

A Neighbor Advertisements that passes the validity checks is called a
"valid advertisement".

( https://tools.ietf.org/html/rfc4861#section-7.1.2 )

So I think we should remove the check by reverting claudio's commit.
I can cook a diff if we we agree to move in that direction.

The router might however still be misbehaving. The RFC does not
clearly state from where the respons must be send, it only has this:

7.2.4. Sending Solicited Neighbor Advertisements
[...]
The Target Address of the advertisement is copied from the Target Address
of the solicitation.

I think we would accept the adv if the router responded from fe80::1

[Patrik Lundin]

Post by Florian Obser

Post by Martin Pieuchot
It'd be nice if somebody could tell us what the RFCs say about this
case. Florian do you have an idea? Should we fix something or should
Marc tell his provider to fix his setup?

If a neighbor solictation isn't from the unspecified address, make sure
that the source address matches one of the interfaces address prefixes.
If a neighbor solictation isn't from the unspecified address, make sure
that the source address matches one of the interfaces address prefixes.
Generalize previous fix so that both NS and NA packets are checked.
However, I don't get why. Other than being extra paranoia defending
against a misbehaving router maybe?. We already check a hop limit of
255, so the packet had to be generated on-link and not forwarded by a
router.

This discussion reminded me of a similar thread a few years back:
http://marc.info/?l=openbsd-misc&m=136057739111931&w=2

It among other things brings up CVE-2008-2476. Maby some of it is
relevant now as well?

[Florian Obser]

Post by Patrik Lundin

Post by Florian Obser

Post by Martin Pieuchot
It'd be nice if somebody could tell us what the RFCs say about this
case. Florian do you have an idea? Should we fix something or should
Marc tell his provider to fix his setup?

If a neighbor solictation isn't from the unspecified address, make sure
that the source address matches one of the interfaces address prefixes.
If a neighbor solictation isn't from the unspecified address, make sure
that the source address matches one of the interfaces address prefixes.
Generalize previous fix so that both NS and NA packets are checked.
However, I don't get why. Other than being extra paranoia defending
against a misbehaving router maybe?. We already check a hop limit of
255, so the packet had to be generated on-link and not forwarded by a
router.

http://marc.info/?l=openbsd-misc&m=136057739111931&w=2
It among other things brings up CVE-2008-2476. Maby some of it is
relevant now as well?

Aha, now I get it. It defends against overriding the neighbour cache
for something attached to interface0 from interface1.

We could try to make this smarter, because it actually does more.
Which is the problem here, it defends against overriding (or setting)
the neighbour cache for interface0 from interface0.

However, let's look at the tcpdump output again:

12:30:08.321097 2a01:4f8:212:216c::1:443 > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:08.321669 2a01:4f8::a:21:b > 2a01:4f8:212:216c::1:443: icmp6:
neighbor adv: tgt is fe80::1 [class 0xc0]

This is not working and I have some suspicions why we get that answer:

1) the router probably does not have an address out of 2a01:4f8:212:216c::/64
configured
2) it sees a neihgbour sol from a global address for a link local address
3) uh oh, what should I do, it's probably bad to answer with a link local
address to a global address (so can't use fe80::1, it would probably
work though) let's use another global address we have lying around
4) openbsd goes: no no, you are not a neighbor, who are you?

then later:
12:30:32.363219 fe80::3285:a9ff:fea4:ce5e > ff02::1:ff00:1: icmp6:
neighbor sol: who has fe80::1
12:30:32.363746 fe80::1 > fe80::3285:a9ff:fea4:ce5e: icmp6: neighbor
adv: tgt is fe80::1 [class 0xc0]

Note that this time around we ask the same neighbour sol question, but
from a link local address, and everything works.

Basically the upstream router is in source address selection hell.

Let's use a link local address as source instead. Not sure if this is
the best place to do it though.

Marc, does this fix your problem?

Comments, OKs?

diff --git nd6_nbr.c nd6_nbr.c
index fa8d3ed1472..086eeef87ba 100644
--- nd6_nbr.c
+++ nd6_nbr.c
@@ -445,7 +445,8 @@ nd6_ns_output(struct ifnet *ifp, struct in6_addr *daddr6,
* We use the source address for the prompting packet
* (saddr6), if:
* - saddr6 is given from the caller (by giving "ln"), and
- * - saddr6 belongs to the outgoing interface.
+ * - saddr6 belongs to the outgoing interface and
+ * - if taddr is link local saddr6 musst be link local as well
* Otherwise, we perform the source address selection as usual.
*/
struct ip6_hdr *hip6; /* hold ip6 */
@@ -453,9 +454,12 @@ nd6_ns_output(struct ifnet *ifp, struct in6_addr *daddr6,

if (ln && ln->ln_hold) {
hip6 = mtod(ln->ln_hold, struct ip6_hdr *);
- if (sizeof(*hip6) <= ln->ln_hold->m_len)
+ if (sizeof(*hip6) <= ln->ln_hold->m_len) {
saddr6 = &hip6->ip6_src;
- else
+ if (saddr6 && IN6_IS_ADDR_LINKLOCAL(taddr6) &&
+ !IN6_IS_ADDR_LINKLOCAL(saddr6))
+ saddr6 = NULL;
+ } else
saddr6 = NULL;
} else
saddr6 = NULL;

[Marc Peters]

Post by Florian Obser
Marc, does this fix your problem?
Comments, OKs?
diff --git nd6_nbr.c nd6_nbr.c
index fa8d3ed1472..086eeef87ba 100644
--- nd6_nbr.c
+++ nd6_nbr.c
@@ -445,7 +445,8 @@ nd6_ns_output(struct ifnet *ifp, struct in6_addr *daddr6,
* We use the source address for the prompting packet
* - saddr6 is given from the caller (by giving "ln"), and
- * - saddr6 belongs to the outgoing interface.
+ * - saddr6 belongs to the outgoing interface and
+ * - if taddr is link local saddr6 musst be link local as well
* Otherwise, we perform the source address selection as usual.
*/
struct ip6_hdr *hip6; /* hold ip6 */
@@ -453,9 +454,12 @@ nd6_ns_output(struct ifnet *ifp, struct in6_addr *daddr6,
if (ln && ln->ln_hold) {
hip6 = mtod(ln->ln_hold, struct ip6_hdr *);
- if (sizeof(*hip6) <= ln->ln_hold->m_len)
+ if (sizeof(*hip6) <= ln->ln_hold->m_len) {
saddr6 = &hip6->ip6_src;
- else
+ if (saddr6 && IN6_IS_ADDR_LINKLOCAL(taddr6) &&
+ !IN6_IS_ADDR_LINKLOCAL(saddr6))
+ saddr6 = NULL;
+ } else
saddr6 = NULL;
} else
saddr6 = NULL;

Hi Florian,

applied the patch to 6.1-STABLE and it fixes my issues.

Thanks a lot.

Cheers,
Marc

[Stefan Sperling]

Post by Florian Obser
Comments, OKs?

s/saddr6 musst/saddr6 must/

then ok

Post by Florian Obser
diff --git nd6_nbr.c nd6_nbr.c
index fa8d3ed1472..086eeef87ba 100644
--- nd6_nbr.c
+++ nd6_nbr.c
@@ -445,7 +445,8 @@ nd6_ns_output(struct ifnet *ifp, struct in6_addr *daddr6,
* We use the source address for the prompting packet
* - saddr6 is given from the caller (by giving "ln"), and
- * - saddr6 belongs to the outgoing interface.
+ * - saddr6 belongs to the outgoing interface and
+ * - if taddr is link local saddr6 musst be link local as well
* Otherwise, we perform the source address selection as usual.
*/
struct ip6_hdr *hip6; /* hold ip6 */
@@ -453,9 +454,12 @@ nd6_ns_output(struct ifnet *ifp, struct in6_addr *daddr6,
if (ln && ln->ln_hold) {
hip6 = mtod(ln->ln_hold, struct ip6_hdr *);
- if (sizeof(*hip6) <= ln->ln_hold->m_len)
+ if (sizeof(*hip6) <= ln->ln_hold->m_len) {
saddr6 = &hip6->ip6_src;
- else
+ if (saddr6 && IN6_IS_ADDR_LINKLOCAL(taddr6) &&
+ !IN6_IS_ADDR_LINKLOCAL(saddr6))
+ saddr6 = NULL;
+ } else
saddr6 = NULL;
} else
saddr6 = NULL;
--
I'm not entirely sure you are real.