Discussion:
MP_LOCKDEBUG - changing mtu - ddb console
Hrvoje Popovski
2017-06-15 16:43:54 UTC
Permalink
Hi all,

test box is configured with kern.pool_debug=1 and option MP_LOCKDEBUG.

while sending traffic (9000 byte) over ix interfaces and at the same
time changing mtu on sink interface (ix1) from 9000 to 1500 drops me to
ddb console.

it seems that i can't trigger dropping to ddb console with
kern.pool_debug=0.

without MP_LOCKDEBUG, it seems that after changing mtu on ix1 interface
while forwarding traffic, i can't see any traffic on ix1 interface. but
this i need to investigate a little more.

this is plain ip forwarding without pseudo interfaces or pf.

ddb traces are in attachment.



OpenBSD 6.1-current (GENERIC.MP) #3: Thu Jun 15 16:03:36 CEST 2017
***@x3550m4.srce.hr:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34314846208 (32725MB)
avail mem = 33269080064 (31727MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67b000 (84 entries)
bios0: vendor IBM version "-[D7E158DUS-2.40]-" date 04/10/2017
bios0: IBM IBM System x3550 M4 -[791425Z]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC S
SDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX
7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4) MRPG(S4)
MRPH(S4) M
RPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.27 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2100271740 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 8 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 4, package 0
cpu5 at mainbus0: apid 10 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 5, package 0
cpu6 at mainbus0: apid 32 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 0, package 1
cpu7 at mainbus0: apid 34 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 0, core 1, package 1
cpu8 at mainbus0: apid 36 (application processor)
cpu8: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu8:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu8: 256KB 64b/line 8-way L2 cache
cpu8: smt 0, core 2, package 1
cpu9 at mainbus0: apid 38 (application processor)
cpu9: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu9:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu9: 256KB 64b/line 8-way L2 cache
cpu9: smt 0, core 3, package 1
cpu10 at mainbus0: apid 40 (application processor)
cpu10: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu10:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,
VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADL
INE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,S
MEP,ERMS,SENSOR,ARAT
cpu10: 256KB 64b/line 8-way L2 cache
cpu10: smt 0, core 4, package 1
cpu11 at mainbus0: apid 42 (application processor)
cpu11: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.01 MHz
cpu11:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,
VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADL
INE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,S
MEP,ERMS,SENSOR,ARAT
cpu11: 256KB 64b/line 8-way L2 cache
cpu11: smt 0, core 5, package 1
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 24 pins
ioapic2 at mainbus0: apid 10 pa 0xfec40000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0x80000000, bus 0-255
acpiprt0 at acpi0: bus 0 (IOH0)
acpiprt1 at acpi0: bus 12 (MRP1)
acpiprt2 at acpi0: bus 17 (DCC0)
acpiprt3 at acpi0: bus 22 (MRP3)
acpiprt4 at acpi0: bus 27 (MRP5)
acpiprt5 at acpi0: bus 6 (PEX0)
acpiprt6 at acpi0: bus 128 (IOH1)
acpiprt7 at acpi0: bus 134 (MRPD)
acpicpu0 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu1 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu2 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu3 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu4 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu5 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu6 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu7 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu8 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu9 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu10 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
acpicpu11 at acpi0: C2(***@59 ***@0x10), C1(***@3 mwait.1)
"PNP0003" at acpi0 not configured
"IPI0001" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0501" at acpi0 not configured
tpm0 at acpi0: TPM_ addr 0xfed40000/0x5000: WEC WPCT200 rev 0x47
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E5 v2 Host" rev 0x04
ppb0 at pci0 dev 1 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci1 at ppb0 bus 12
12:0:0: mem address conflict 0xfff80000/0x80000
12:0:1: mem address conflict 0xfff80000/0x80000
bge0 at pci1 dev 0 function 0 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x572000
0), APE firmware NCSI 1.0.60.0: msi, address 00:10:18:c1:e4:e2
brgphy0 at bge0 phy 1: BCM5720C 10/100/1000baseT PHY, rev. 0
bge1 at pci1 dev 0 function 1 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x572000
0), APE firmware NCSI 1.0.60.0: msi, address 00:10:18:c1:e4:e3
brgphy1 at bge1 phy 2: BCM5720C 10/100/1000baseT PHY, rev. 0
ppb1 at pci0 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci2 at ppb1 bus 17
ppb2 at pci2 dev 0 function 0 "IDT 89HPES24T3G2" rev 0x02
pci3 at ppb2 bus 18
ppb3 at pci3 dev 2 function 0 "IDT 89HPES24T3G2" rev 0x02
pci4 at ppb3 bus 19
19:0:0: mem address conflict 0xfff80000/0x80000
myx0 at pci4 dev 0 function 0 "Myricom Z8E" rev 0x01: msi, model
10G-PCIE2-8BL2
-2S, address 00:60:dd:45:ba:f8
ppb4 at pci3 dev 4 function 0 "IDT 89HPES24T3G2" rev 0x02
pci5 at ppb4 bus 20
20:0:0: mem address conflict 0xfff80000/0x80000
myx1 at pci5 dev 0 function 0 "Myricom Z8E" rev 0x01: msi, model
10G-PCIE2-8BL2
-2S, address 00:60:dd:45:ba:f9
ppb5 at pci0 dev 3 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci6 at ppb5 bus 22
22:0:0: mem address conflict 0xfff80000/0x80000
22:0:1: mem address conflict 0xfff80000/0x80000
ix0 at pci6 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:9
6:a0
ix1 at pci6 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:9
6:a1
ppb6 at pci0 dev 3 function 2 "Intel E5 v2 PCIE" rev 0x04: msi
pci7 at ppb6 bus 27
mfii0 at pci7 dev 0 function 0 "Symbios Logic MegaRAID SAS2208" rev
0x05: msi
mfii0: "ServeRAID M5110", firmware 23.34.0-0023, 512MB cache
scsibus1 at mfii0: 64 targets
sd0 at scsibus1 targ 0 lun 0: <IBM, ServeRAID M5110, 3.46> SCSI3
0/direct fixed
naa.600605b006c3a0b01a582bd010e4c053
sd0: 952720MB, 512 bytes/sector, 1951170560 sectors
scsibus2 at mfii0: 256 targets
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci0 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci0 dev 5 function 2 not configured
ppb7 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x06
pci8 at ppb7 bus 32
ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x06: apic 8 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 a
ddr 1
ppb8 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb6: msi
pci9 at ppb8 bus 6
em0 at pci9 dev 0 function 0 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:12
em1 at pci9 dev 0 function 1 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:13
em2 at pci9 dev 0 function 2 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:14
em3 at pci9 dev 0 function 3 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:15
ppb9 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb6: msi
pci10 at ppb9 bus 1
ppb10 at pci10 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci11 at ppb10 bus 2
ppb11 at pci11 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci12 at ppb11 bus 3
ppb12 at pci12 dev 0 function 0 "Renesas SH7757 PCIE-PCI" rev 0x00
pci13 at ppb12 bus 4
vga1 at pci13 dev 0 function 0 "Matrox MGA G200eR" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb13 at pci11 dev 1 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci14 at ppb13 bus 5
ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x06: apic 8 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 a
ddr 1
ppb14 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa6
pci15 at ppb14 bus 33
pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x06
ichiic0 at pci0 dev 31 function 3 "Intel C600 SMBus" rev 0x06: apic 8 int 11
iic0 at ichiic0
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
pci16 at mainbus0 bus 128
128:4:0: mem address conflict 0x90000000/0x4000
128:4:1: mem address conflict 0x90004000/0x4000
128:4:2: mem address conflict 0x90008000/0x4000
128:4:3: mem address conflict 0x9000c000/0x4000
128:4:4: mem address conflict 0x90010000/0x4000
128:4:5: mem address conflict 0x90014000/0x4000
128:4:6: mem address conflict 0x90018000/0x4000
128:4:7: mem address conflict 0x9001c000/0x4000
ppb15 at pci16 dev 0 function 0 "Intel E5 v2 PCIE" rev 0x04


[85/18108]
pci17 at ppb15 bus 129
ppb16 at pci16 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci18 at ppb16 bus 134
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci16 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci16 dev 5 function 2 not configured
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev
2.00/0.00 addr 2
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev
2.00/0.00 addr 2
uhub4 at uhub3 port 1 configuration 1 interface 0 "no manufacturer
Gadget USB H
UB" rev 2.00/0.00 addr 3
uhub4: device problem, disabling port 1
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (2aee1933a7f750e8.a) swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
uhidev0 at uhub4 port 1 configuration 1 interface 0 "Avocent
Keyboard/Mouse Fun
ction" rev 2.00/0.00 addr 4
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub4 port 1 configuration 1 interface 1 "Avocent
Keyboard/Mouse Fun
ction" rev 2.00/0.00 addr 4
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub4 port 1 configuration 1 interface 2 "Avocent
Keyboard/Mouse Fun
ction" rev 2.00/0.00 addr 4
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
Hrvoje Popovski
2017-06-15 23:05:40 UTC
Permalink
Post by Hrvoje Popovski
Hi all,
test box is configured with kern.pool_debug=1 and option MP_LOCKDEBUG.
it wasn't easy but i managed to get ddb once more :)


***@x3550m4:~
# ____mmpp__lloocckk((00xxffffffffffffffff881199dd33110000)):: l o c k
sp u n o u
t Stop
ped at db_enter+0x9: leave
ddb{3}>

ddb{3}> trace
db_enter(ffffffff81759640,8,10,ffff800023c19c60,286,8) at db_enter+0x9
___mp_lock(ffffffff819d3100,ffffffff817cb590,1,1,ffff800023bfcba0,ffff800023bfa
8e0) at ___mp_lock+0x7c
___mp_acquire_count(ffffffff819d3100,1,ffff800023bfcba0,ffff800023bfa8e0,ffffff
ff819ee4c0,ffffffff812fd602) at ___mp_acquire_count+0x33
mi_switch(0,0,20,ffff800023c19dc0,ffff800023bfa8e0,ffff800023c19db0) at
mi_swit
ch+0x22b
sleep_finish(ffff800023c19dc0,1,ffffffff8196b118,ffff800023c19dc0,0,0)
at sleep
_finish+0xc2
msleep(ffffffff8196b100,ffffffff8196b118,20,ffffffff818bab30,0,ffffffff815059c4
) at msleep+0x15a
taskq_next_work(ffffffff8196b100,ffff800023c19eb0,ffffffff8163ef40,ffffffff8163
ef40,0,ffff800023c19eb0) at taskq_next_work+0x5c
taskq_thread(ffffffff8196b100,2a2,ffffffff8196b100,ffffffff8153f3e0,0,ffff80002
3c19f10) at taskq_thread+0x6e
end trace frame: 0x0, count: -8
ddb{3}>


ddb{3}> ps
PID TID PPID UID S FLAGS WAIT COMMAND
32205 60024 25238 0 3 0x100083 ttyin ksh
25238 60344 23584 1000 3 0x10008b pause ksh
23584 432865 99984 1000 3 0x90 select sshd
99984 464293 71663 0 3 0x82 poll sshd
96076 278513 1 0 7 0x100083 ksh
11752 53701 1 0 3 0x100083 ttyin getty
69171 27429 1 0 3 0x100083 ttyin getty
69200 91192 1 0 3 0x100083 ttyin getty
92776 285925 1 0 3 0x100083 ttyin getty
26373 188878 1 0 3 0x100083 ttyin getty
45353 519788 1 0 3 0x100098 poll cron
91381 65045 53364 95 3 0x100092 kqread smtpd
10267 328542 53364 103 3 0x100092 kqread smtpd
98406 368838 53364 95 3 0x100092 kqread smtpd
8295 211658 53364 95 3 0x100092 kqread smtpd
85768 337946 53364 95 3 0x100092 kqread smtpd
74979 385483 53364 95 3 0x100092 kqread smtpd
53364 36145 1 0 3 0x100080 kqread smtpd
71663 298878 1 0 3 0x80 select sshd
9902 31803 54613 83 3 0x100092 poll ntpd
54613 32611 73441 83 7 0x100492 ntpd
73441 351399 1 0 3 0x100080 poll ntpd
66109 87818 58238 73 3 0x100090 kqread syslogd
58238 467469 1 0 3 0x100082 netio syslogd
13544 209242 0 0 3 0x14200 pgzero zerothread
39157 12877 0 0 3 0x14200 aiodoned aiodoned
19554 144818 0 0 3 0x14200 syncer update
68070 269755 0 0 3 0x14200 cleaner cleaner
23375 84783 0 0 3 0x14200 reaper reaper
79244 245489 0 0 3 0x14200 pgdaemon pagedaemon
83411 507629 0 0 3 0x14200 bored crynlk
25000 100498 0 0 3 0x14200 bored crypto
84490 176412 0 0 7 0x14600 pfpurge
30681 168786 0 0 3 0x14200 usbtsk usbtask
97245 300783 0 0 3 0x14200 usbatsk usbatsk
8325 394728 0 0 3 0x40014200 acpi0 acpi0
5351 134226 0 0 7 0x40014200 idle11
96374 271675 0 0 7 0x40014200 idle10
29831 518602 0 0 7 0x40014200 idle9
62097 210555 0 0 7 0x40014200 idle8
57996 501532 0 0 7 0x40014200 idle7
89701 520237 0 0 3 0x40014200 idle6
21707 248620 0 0 3 0x40014200 idle5
43773 255753 0 0 7 0x40014200 idle4
32145 465833 0 0 3 0x40014200 idle3
98125 83253 0 0 3 0x40014200 idle2
72875 19314 0 0 3 0x40014200 idle1
45781 276621 0 0 3 0x14200 bored sensors
61039 420 0 0 7 0x14200 softnet
3358 465476 0 0 3 0x14200 bored systqmp
*61325 139908 0 0 7 0x14200 systq
77839 507536 0 0 7 0x40014200 softclock
18382 422172 0 0 3 0x40014200 idle0
44455 317177 0 0 3 0x14200 bored sbar
1 283311 0 0 3 0x82 wait init
0 0 -1 0 3 0x10200 scheduler swapper
ddb{3}>
Martin Pieuchot
2017-06-19 08:25:50 UTC
Permalink
Post by Hrvoje Popovski
Hi all,
test box is configured with kern.pool_debug=1 and option MP_LOCKDEBUG.
while sending traffic (9000 byte) over ix interfaces and at the same
time changing mtu on sink interface (ix1) from 9000 to 1500 drops me to
ddb console.
it seems that i can't trigger dropping to ddb console with
kern.pool_debug=0.
But the machine do not freeze right?

Do you still enter DDB with a bigger value of __mp_lock_spinout?
Post by Hrvoje Popovski
without MP_LOCKDEBUG, it seems that after changing mtu on ix1 interface
while forwarding traffic, i can't see any traffic on ix1 interface. but
this i need to investigate a little more.
this is plain ip forwarding without pseudo interfaces or pf.
ddb traces are in attachment.
OpenBSD 6.1-current (GENERIC.MP) #3: Thu Jun 15 16:03:36 CEST 2017
real mem = 34314846208 (32725MB)
avail mem = 33269080064 (31727MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0: vendor IBM version "-[D7E158DUS-2.40]-" date 04/10/2017
bios0: IBM IBM System x3550 M4 -[791425Z]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC S
SDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX
7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4) MRPG(S4)
MRPH(S4) M
RPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2100271740 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 8 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 4, package 0
cpu5 at mainbus0: apid 10 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 5, package 0
cpu6 at mainbus0: apid 32 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 0, package 1
cpu7 at mainbus0: apid 34 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 0, core 1, package 1
cpu8 at mainbus0: apid 36 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu8: 256KB 64b/line 8-way L2 cache
cpu8: smt 0, core 2, package 1
cpu9 at mainbus0: apid 38 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,SENSOR,ARAT
cpu9: 256KB 64b/line 8-way L2 cache
cpu9: smt 0, core 3, package 1
cpu10 at mainbus0: apid 40 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,
VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADL
INE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,S
MEP,ERMS,SENSOR,ARAT
cpu10: 256KB 64b/line 8-way L2 cache
cpu10: smt 0, core 4, package 1
cpu11 at mainbus0: apid 42 (application processor)
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,
VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADL
INE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,S
MEP,ERMS,SENSOR,ARAT
cpu11: 256KB 64b/line 8-way L2 cache
cpu11: smt 0, core 5, package 1
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 24 pins
ioapic2 at mainbus0: apid 10 pa 0xfec40000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0x80000000, bus 0-255
acpiprt0 at acpi0: bus 0 (IOH0)
acpiprt1 at acpi0: bus 12 (MRP1)
acpiprt2 at acpi0: bus 17 (DCC0)
acpiprt3 at acpi0: bus 22 (MRP3)
acpiprt4 at acpi0: bus 27 (MRP5)
acpiprt5 at acpi0: bus 6 (PEX0)
acpiprt6 at acpi0: bus 128 (IOH1)
acpiprt7 at acpi0: bus 134 (MRPD)
"PNP0003" at acpi0 not configured
"IPI0001" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0501" at acpi0 not configured
tpm0 at acpi0: TPM_ addr 0xfed40000/0x5000: WEC WPCT200 rev 0x47
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E5 v2 Host" rev 0x04
ppb0 at pci0 dev 1 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci1 at ppb0 bus 12
12:0:0: mem address conflict 0xfff80000/0x80000
12:0:1: mem address conflict 0xfff80000/0x80000
bge0 at pci1 dev 0 function 0 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x572000
0), APE firmware NCSI 1.0.60.0: msi, address 00:10:18:c1:e4:e2
brgphy0 at bge0 phy 1: BCM5720C 10/100/1000baseT PHY, rev. 0
bge1 at pci1 dev 0 function 1 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x572000
0), APE firmware NCSI 1.0.60.0: msi, address 00:10:18:c1:e4:e3
brgphy1 at bge1 phy 2: BCM5720C 10/100/1000baseT PHY, rev. 0
ppb1 at pci0 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci2 at ppb1 bus 17
ppb2 at pci2 dev 0 function 0 "IDT 89HPES24T3G2" rev 0x02
pci3 at ppb2 bus 18
ppb3 at pci3 dev 2 function 0 "IDT 89HPES24T3G2" rev 0x02
pci4 at ppb3 bus 19
19:0:0: mem address conflict 0xfff80000/0x80000
myx0 at pci4 dev 0 function 0 "Myricom Z8E" rev 0x01: msi, model
10G-PCIE2-8BL2
-2S, address 00:60:dd:45:ba:f8
ppb4 at pci3 dev 4 function 0 "IDT 89HPES24T3G2" rev 0x02
pci5 at ppb4 bus 20
20:0:0: mem address conflict 0xfff80000/0x80000
myx1 at pci5 dev 0 function 0 "Myricom Z8E" rev 0x01: msi, model
10G-PCIE2-8BL2
-2S, address 00:60:dd:45:ba:f9
ppb5 at pci0 dev 3 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci6 at ppb5 bus 22
22:0:0: mem address conflict 0xfff80000/0x80000
22:0:1: mem address conflict 0xfff80000/0x80000
ix0 at pci6 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:9
6:a0
ix1 at pci6 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:9
6:a1
ppb6 at pci0 dev 3 function 2 "Intel E5 v2 PCIE" rev 0x04: msi
pci7 at ppb6 bus 27
mfii0 at pci7 dev 0 function 0 "Symbios Logic MegaRAID SAS2208" rev
0x05: msi
mfii0: "ServeRAID M5110", firmware 23.34.0-0023, 512MB cache
scsibus1 at mfii0: 64 targets
sd0 at scsibus1 targ 0 lun 0: <IBM, ServeRAID M5110, 3.46> SCSI3
0/direct fixed
naa.600605b006c3a0b01a582bd010e4c053
sd0: 952720MB, 512 bytes/sector, 1951170560 sectors
scsibus2 at mfii0: 256 targets
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci0 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci0 dev 5 function 2 not configured
ppb7 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x06
pci8 at ppb7 bus 32
ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x06: apic 8 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 a
ddr 1
ppb8 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb6: msi
pci9 at ppb8 bus 6
em0 at pci9 dev 0 function 0 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:12
em1 at pci9 dev 0 function 1 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:13
em2 at pci9 dev 0 function 2 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:14
em3 at pci9 dev 0 function 3 "Intel I350" rev 0x01: msi, address
40:f2:e9:ec:b4
:15
ppb9 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb6: msi
pci10 at ppb9 bus 1
ppb10 at pci10 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci11 at ppb10 bus 2
ppb11 at pci11 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci12 at ppb11 bus 3
ppb12 at pci12 dev 0 function 0 "Renesas SH7757 PCIE-PCI" rev 0x00
pci13 at ppb12 bus 4
vga1 at pci13 dev 0 function 0 "Matrox MGA G200eR" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb13 at pci11 dev 1 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci14 at ppb13 bus 5
ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x06: apic 8 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 a
ddr 1
ppb14 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa6
pci15 at ppb14 bus 33
pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x06
ichiic0 at pci0 dev 31 function 3 "Intel C600 SMBus" rev 0x06: apic 8 int 11
iic0 at ichiic0
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
pci16 at mainbus0 bus 128
128:4:0: mem address conflict 0x90000000/0x4000
128:4:1: mem address conflict 0x90004000/0x4000
128:4:2: mem address conflict 0x90008000/0x4000
128:4:3: mem address conflict 0x9000c000/0x4000
128:4:4: mem address conflict 0x90010000/0x4000
128:4:5: mem address conflict 0x90014000/0x4000
128:4:6: mem address conflict 0x90018000/0x4000
128:4:7: mem address conflict 0x9001c000/0x4000
ppb15 at pci16 dev 0 function 0 "Intel E5 v2 PCIE" rev 0x04
[85/18108]
pci17 at ppb15 bus 129
ppb16 at pci16 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci18 at ppb16 bus 134
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci16 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci16 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci16 dev 5 function 2 not configured
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev
2.00/0.00 addr 2
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev
2.00/0.00 addr 2
uhub4 at uhub3 port 1 configuration 1 interface 0 "no manufacturer
Gadget USB H
UB" rev 2.00/0.00 addr 3
uhub4: device problem, disabling port 1
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (2aee1933a7f750e8.a) swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
uhidev0 at uhub4 port 1 configuration 1 interface 0 "Avocent
Keyboard/Mouse Fun
ction" rev 2.00/0.00 addr 4
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub4 port 1 configuration 1 interface 1 "Avocent
Keyboard/Mouse Fun
ction" rev 2.00/0.00 addr 4
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub4 port 1 configuration 1 interface 2 "Avocent
Keyboard/Mouse Fun
ction" rev 2.00/0.00 addr 4
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
# ifconfig ix
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9000
lladdr a0:36:9f:2e:96:a0
index 3 priority 0 llprio 3
media: Ethernet autoselect (10GSFP+Cu full-duplex,rxpause,txpause)
status: active
inet 192.168.11.1 netmask 0xfffffffc broadcast 192.168.11.3
ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9000
lladdr a0:36:9f:2e:96:a1
index 4 priority 0 llprio 3
media: Ethernet autoselect (10GSFP+Cu full-duplex,rxpause,txpause)
status: active
inet 192.168.12.1 netmask 0xfffffffc broadcast 192.168.12.3
# ifconfig ix1 mtu 1500
# __mp_lock(0xffffffff819e5a20): lock spun outStopped at db_enter+0x9: leave
ddb{1}> trace
db_enter(ffffffff81884fe8,8,10,ffff800023c97ce0,282,8) at db_enter+0x9
___mp_lock(ffffffff819e5a20,ffffffff81939ab0,1,1,ffff800023c79170,ffff800023c98
010) at ___mp_lock+0x7c
___mp_acquire_count(ffffffff819e5a20,1,ffff800023c79170,ffff800023c98010,ffffff
ff819e7200,ffffffff81667882) at ___mp_acquire_count+0x33
mi_switch(ffffffff81a00170,ffffffff8192d3d6,0,ffff800023c97e40,ffff800023c98010
,ffff800023c97e30) at mi_switch+0x22b
sleep_finish(ffff800023c97e40,1,28,ffff800023c97e40,ffffffff81a00170,ffffffff81
92d3d6) at sleep_finish+0xc2
tsleep(ffffffff81a00170,28,ffffffff8192d3d6,0,ffff800023c98010,0) at tsleep+0x1
64
sched_sync(ffff800023c98010,2a2,0,0,ffff800023c98010,ffffffff811e89d0) at sched
_sync+0x1d6
end trace frame: 0x0, count: -7
ddb{1}>
ddb{1}> ps
PID TID PPID UID S FLAGS WAIT COMMAND
90211 173972 1 0 7 0x100083 ksh
49416 350139 1 0 3 0x100083 ttyin getty
76263 320744 1 0 3 0x100083 ttyin getty
67418 467704 1 0 3 0x100083 ttyin getty
2507 209424 1 0 3 0x100083 ttyin getty
39830 298933 1 0 3 0x100083 ttyin getty
16844 121935 1 0 3 0x100098 poll cron
1996 388057 15917 95 3 0x100092 kqread smtpd
50983 455639 15917 103 3 0x100092 kqread smtpd
32750 261433 15917 95 3 0x100092 kqread smtpd
98003 155465 15917 95 3 0x100092 kqread smtpd
66529 293417 15917 95 3 0x100092 kqread smtpd
67220 458973 15917 95 3 0x100092 kqread smtpd
15917 239051 1 0 3 0x100080 kqread smtpd
60680 469781 1 0 3 0x80 select sshd
64899 147364 39896 83 3 0x100092 poll ntpd
39896 523282 77847 83 3 0x100092 poll ntpd
77847 465982 1 0 3 0x100080 poll ntpd
67535 92132 39254 73 3 0x100090 kqread syslogd
39254 320074 1 0 3 0x100082 netio syslogd
26613 57176 0 0 3 0x14200 pgzero zerothread
88504 340949 0 0 3 0x14200 aiodoned aiodoned
*77211 223387 0 0 7 0x14200 update
99991 196282 0 0 3 0x14200 cleaner cleaner
72079 23890 0 0 3 0x14200 reaper reaper
20741 275460 0 0 3 0x14200 pgdaemon pagedaemon
48988 72107 0 0 3 0x14200 bored crynlk
76396 445498 0 0 3 0x14200 bored crypto
46485 14982 0 0 7 0x14600 pfpurge
35334 373436 0 0 3 0x14200 usbtsk usbtask
99136 145473 0 0 3 0x14200 usbatsk usbatsk
94006 465245 0 0 3 0x40014200 acpi0 acpi0
90754 125248 0 0 7 0x40014200 idle11
62737 125323 0 0 7 0x40014200 idle10
73466 12196 0 0 7 0x40014200 idle9
1119 126734 0 0 7 0x40014200 idle8
63337 421118 0 0 7 0x40014200 idle7
60651 337134 0 0 3 0x40014200 idle6
1151 518800 0 0 3 0x40014200 idle5
26371 210902 0 0 3 0x40014200 idle4
49267 93946 0 0 7 0x40014200 idle3
99422 273194 0 0 3 0x40014200 idle2
52099 215802 0 0 3 0x40014200 idle1
15954 394329 0 0 3 0x14200 bored sensors
87018 123322 0 0 7 0x14200 softnet
31914 320684 0 0 3 0x14200 bored systqmp
51334 100638 0 0 7 0x14200 systq
20437 386613 0 0 2 0x40014200 softclock
26318 231633 0 0 7 0x40014200 idle0
9501 248801 0 0 3 0x14200 bored sbar
1 126966 0 0 3 0x82 wait init
0 0 -1 0 3 0x10200 scheduler swapper
ddb{1}> mach ddbcpu 0
Stopped at db_enter+0x9: leave
ddb{0}> trace
db_enter(ffffffff8196ca20,ffff800023c0f878,0,10,ffff800023c0f718,286) at db_ent
er+0x9
x86_ipi_handler(ffff80000120f000,ffff800023c0f914,ffff8000000dbc00,7,10,e0) at x
86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
ixgbe_tx_ctx_setup(ffff8000000dbc00,ffffff007e16af00,ffff800023c0f910,ffff80002
3c0f914,0,ffff80000120f000) at ixgbe_tx_ctx_setup+0x53
ixgbe_encap(ffff8000000dbc00,ffffff007e16af00,ffff8000000e0000,ffff8000000e0278
,ffff8000000e0048,ffff8000000dbc00) at ixgbe_encap+0xbd
ixgbe_start(ffff8000000e0278,7,ffff8000000dbc00,ffff8000000e02f0,ffff8000000e02
78,ffffffff8132ea30) at ixgbe_start+0xaf
ifq_serialize(ffff8000000e0278,ffff8000000e0340,42,ffffff006a225420,ffff8000011
aa630,42) at ifq_serialize+0xf8
ixgbe_txeof(ffff8000000dbc00,800,ffff800023c0fb48,ffff8000000dbc00,ffff8000000d
f040,ffff8000000e0000) at ixgbe_txeof+0x234
ixgbe_intr(ffff8000000e0000,0,0,ffffffff8196ca20,ffff8000000c3880,4) at ixgbe_i
ntr+0x1fc
intr_handler(ffff800023c0fb48,ffff8000000c3880,0,ffff8000000c3900,4,7) at intr_
handler+0x6f
Xintr_ioapic_edge22() at Xintr_ioapic_edge22+0xc9
--- interrupt ---
Xspllower(4,1,ffffffff81a04160,ffffffff8196ca20,ffffffff81956ff8,ffffffff81956f
e0) at Xspllower+0xc
task_add(ffffffff81956fe0,ffffffff81a04160,ffffffff819e7d80,ffffffff814fcce0,0,
ffffffff81420060) at task_add+0x83
timeout_run(ffffffff81a03f20,ffffffff819e7d80,ffffffff819e7d98,0,3,8) at timeou
t_run+0x4c
softclock(0,4,ffff800000504940,ffff800000504940,ffffffff811a91fc,ffff800023c0fd
38) at softclock+0x146
softintr_dispatch(0,ffff800023c0fda0,0,ffff8000000c3900,ffffffff815e1080,7) at s
oftintr_dispatch+0x9f
Xsoftclock() at Xsoftclock+0x1f
--- interrupt ---
end trace frame: 0x0, count: -17
ddb{0}> mach ddbcpu 2
Stopped at db_enter+0x9: leave
ddb{2}> trace
db_enter(ffff8000000a0000,ffff800000530000,0,10,ffff800023d616b8,286) at db_ent
er+0x9
x86_ipi_handler(ffff80000009e010,ffffffff819e5a20,7c07b,c,ffff800000530000,0) a
t x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
___mp_lock(ffffffff819e5a20,ffffffff81939ab0,1,1,ffff800023d4d178,ffff800023d38
020) at ___mp_lock+0x4d
___mp_acquire_count(ffffffff819e5a20,1,ffff800023d4d178,ffff800023d38020,ffffff
078f7c4720,ffffffff81667882) at ___mp_acquire_count+0x33
mi_switch(ffff8000005adc10,ffffffff8196a868,0,ffff800023d61930,ffff800023d38020
,ffff800023d61920) at mi_switch+0x22b
sleep_finish(ffff800023d61930,1,119,ffff800023d61930,ffff8000005adc10,ffffffff8
196a868) at sleep_finish+0xc2
tsleep(ffff8000005adc10,119,ffffffff8196a868,0,0,443) at tsleep+0x164
ttysleep(ffff8000005adc00,ffff8000005adc10,119,ffffffff8196a868,0,ffff800023d38
020) at ttysleep+0x42
ttread(ffff8000005adc00,ffff800023d61d20,0,800,ffff800023d61d20,ffffff078705abd
8) at ttread+0x1c5
spec_read(ffff800023d61b80,2001,ffffff078408b2e0,ffffff078705abd8,1,0) at spec_
read+0x301
VOP_READ(ffffff078705abd8,ffff800023d61d20,0,ffffff087f7d3000,3d8b07b445621e71,
1) at VOP_READ+0x4f
vn_read(ffffff078408b2b0,ffffff078408b2e0,ffff800023d61d20,ffffff087f7d3000,fff
fff078408b2b0,ffff800023d61eb0) at vn_read+0xaf
dofilereadv(ffff800023d38020,0,ffffff078408b2b0,ffff800023d61de0,1,0) at dofile
readv+0x247
sys_read(ffff800023d38020,ffff800023d61e60,ffff800023d61eb0,ffff800023d38020,1,
ffff800023d61f20) at sys_read+0x95
syscall() at syscall+0x29f
--- syscall (number 3) ---
end of kernel
end trace frame: 0x7f7ffffd4aff, count: -16
ddb{2}> mach ddbcpu 3
Stopped at db_enter+0x9: leave
ddb{3}> trace
db_enter(ffff8000000a2000,0,ffff8000000a2000,10,ffff800023c37d28,286) at db_ent
er+0x9
x86_ipi_handler(ffff8000000a2000,ffff8000000b8d00,ffff80000005bee0,0,0,ffff8000
23c37da0) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
acpicpu_idle(ffff800023bfb630,ffff8000000a2000,ffff8000000a2010,0,3d8b07b445621
e71,286) at acpicpu_idle+0x242
cpu_idle_cycle(ffff8000000a2000,2a2,0,ffff8000000a2000,ffffffff8170c670,0) at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -5
ddb{3}> mach ddbcpu 4
Stopped at db_enter+0x9: leave
ddb{4}> trace
db_enter(ffff8000000a4000,ffff8000217e10f0,ffff8000217e1000,10,ffff800023c77be8
,286) at db_enter+0x9
x86_ipi_handler(ffff8000000a0010,ffffffff819e5a20,7c07c,c,ffffffff815e102c,ffff
8000000a2000) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
___mp_lock(ffffffff819e5a20,ffffffff81939ab0,1,1,ffff800023c78000,ffff800023c4f
1c8) at ___mp_lock+0x4a
___mp_acquire_count(ffffffff819e5a20,1,ffff800023c78000,ffff800023c4f1c8,ffffff
ff819e7200,ffffffff81667882) at ___mp_acquire_count+0x33
mi_switch(ffffffff814b28a0,ffffffff81874657,64,ffff800023c77e60,ffff800023c4f1c
8,ffff800023c77e50) at mi_switch+0x22b
sleep_finish(ffff800023c77e60,1,20,ffff800023c77e60,ffffffff814b28a0,ffffffff81
874657) at sleep_finish+0xc2
tsleep(ffffffff814b28a0,20,ffffffff81874657,64,0,0) at tsleep+0x164
pf_purge_thread(ffff800023c4f1c8,2a2,ffffffff814b28a0,0,ffffffff811a91fc,ffff80
0023c77f10) at pf_purge_thread+0x55
end trace frame: 0x0, count: -9
ddb{4}>
ddb{4}> mach ddbcpu 5
Stopped at db_enter+0x9: leave
ddb{5}> trace
db_enter(ffff8000000a7000,ffffffff819efaf0,0,10,ffff800023c19b48,286) at db_ent
er+0x9
x86_ipi_handler(ffff8000000a4010,ffffffff819e5a20,7c07d,c,ffff800023c19bb0,ffff
ffff819fff60) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
___mp_lock(ffffffff819e5a20,ffffffff81939ab0,1,1,ffff800023bfcba0,ffff800023bfa
8e0) at ___mp_lock+0x4a
___mp_acquire_count(ffffffff819e5a20,1,ffff800023bfcba0,ffff800023bfa8e0,ffffff
ff819e7200,ffffffff81667882) at ___mp_acquire_count+0x33
mi_switch(0,0,20,ffff800023c19dc0,ffff800023bfa8e0,ffff800023c19db0) at mi_swit
ch+0x22b
sleep_finish(ffff800023c19dc0,1,ffffffff81956ff8,ffff800023c19dc0,0,0) at sleep
_finish+0xc2
msleep(ffffffff81956fe0,ffffffff81956ff8,20,ffffffff81806781,0,ffffffff8160f474
) at msleep+0x15a
taskq_next_work(ffffffff81956fe0,ffff800023c19eb0,ffffffff813a3a20,ffffffff813a
3a20,0,ffff800023c19eb0) at taskq_next_work+0x5c
taskq_thread(ffffffff81956fe0,2a2,ffffffff81956fe0,ffffffff8121d910,0,ffff80002
3c19f10) at taskq_thread+0x6e
end trace frame: 0x0, count: -10
ddb{5}>
ddb{5}> mach ddbcpu 6
Stopped at db_enter+0x9: leave
ddb{6}> trace
db_enter(ffff8000000a9000,ffffff006979e500,ffff8000000e0278,10,ffff800023c23958
,282) at db_enter+0x9
x86_ipi_handler(ffff80000001ded0,ffff800023c23cc0,ffffff007e13eb50,7,ffff800023
c239c0,ffffffff8132f3ad) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
ip_fragment(ffffff0066f59500,0,ffff800023c23cc0,1,0,0) at ip_fragment
ip_forward(ffffff0066f59500,ffff8000000dd048,ffffff0786a4e708,0,1600000c,160000
0c) at ip_forward+0x1e7
ipv4_input(ffff8000000dd048,ffffff0066f59500,800,800,ffffff007e13eb42,ffff80000
00dd048) at ipv4_input+0x4f7
ether_input(ffff8000000dd048,ffffff0066f59500,0,ffffffff813a3a20,ffff8000000dd2
40,ffff8000000dd048) at ether_input+0xbd
if_input_process(3,ffff800023c23eb0,0,0,ffff800023c23eb0,ffff800000019080) at i
f_input_process+0xfa
taskq_thread(ffff800000019080,2a2,ffff800000019080,ffffffff8121d910,0,ffff80002
3c23f10) at taskq_thread+0x79
end trace frame: 0x0, count: -9
ddb{6}>
ddb{6}> mach ddbcpu 7
Stopped at db_enter+0x9: leave
ddb{7}> trace
db_enter(ffff8000000ab000,ffff800023c4e008,ffff8000000ab000,10,ffff800023c4dd28
,286) at db_enter+0x9
x86_ipi_handler(ffff8000000ab000,ffff800000014e00,ffff80000005bf60,0,3b48a538,f
fffffff81776ec0) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
acpicpu_idle(ffff800023c4e008,ffff8000000ab000,ffff8000000ab010,0,3d8b07b445621
e71,286) at acpicpu_idle+0x242
cpu_idle_cycle(ffff8000000ab000,2a2,0,ffff8000000ab000,ffffffff8170c670,0) at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -5
ddb{7}>
ddb{7}> mach ddbcpu 8
Stopped at db_enter+0x9: leave
ddb{8}> trace
db_enter(ffff8000000ad000,0,ffff8000000ad000,10,ffff800023c54d28,286) at db_ent
er+0x9
x86_ipi_handler(ffff8000000ad000,ffff8000000b8000,ffff80000005b220,0,3b48a541,f
fffffff81776ec0) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
acpicpu_idle(ffff800023c4e240,ffff8000000ad000,ffff8000000ad010,0,3d8b07b445621
e71,286) at acpicpu_idle+0x242
cpu_idle_cycle(ffff8000000ad000,2a2,0,ffff8000000ad000,ffffffff8170c670,0) at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -5
ddb{8}> mach ddbcpu 9
Stopped at db_enter+0x9: leave
ddb{9}> trace
db_enter(ffff8000000b0000,9f,ffff8000000b0000,10,ffff800023c59d28,286) at db_en
ter+0x9
x86_ipi_handler(ffff8000000b0000,ffff8000000b8100,ffff8000000c7040,0,3b48a52e,f
fffffff81776ec0) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
acpicpu_idle(ffff800023c4e478,ffff8000000b0000,ffff8000000b0010,0,3d8b07b445621
e71,286) at acpicpu_idle+0x242
cpu_idle_cycle(ffff8000000b0000,2a2,0,ffff8000000b0000,ffffffff8170c670,0) at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -5
ddb{9}>
ddb{9}> mach ddbcpu 0xa
Stopped at db_enter+0x9: leave
ddb{10}> trace
db_enter(ffff8000000b2000,0,ffff8000000b2000,10,ffff800023c5ed28,286) at db_ent
er+0x9
x86_ipi_handler(ffff8000000b2000,ffff800000014400,ffff8000000c7160,0,3b48a54b,f
fffffff81776ec0) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
acpicpu_idle(ffff800023c4e6b0,ffff8000000b2000,ffff8000000b2010,0,3d8b07b445621
e71,286) at acpicpu_idle+0x242
cpu_idle_cycle(ffff8000000b2000,2a2,0,ffff8000000b2000,ffffffff8170c670,0) at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -5
ddb{10}>
ddb{10}> mach ddbcpu 0xb
Stopped at db_enter+0x9: leave
ddb{11}> trace
db_enter(ffff8000000b4000,0,ffff8000000b4000,10,ffff800023c63d28,286) at db_ent
er+0x9
x86_ipi_handler(ffff8000000b4000,ffff8000000b8200,ffff8000000c7280,0,3b48a500,f
fffffff81776ec0) at x86_ipi_handler+0x85
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
acpicpu_idle(ffff800023c4e8e8,ffff8000000b4000,ffff8000000b4010,0,3d8b07b445621
e71,286) at acpicpu_idle+0x242
cpu_idle_cycle(ffff8000000b4000,2a2,0,ffff8000000b4000,ffffffff8170c670,0) at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -5
ddb{11}>
Hrvoje Popovski
2017-06-19 09:15:30 UTC
Permalink
Post by Martin Pieuchot
Post by Hrvoje Popovski
Hi all,
test box is configured with kern.pool_debug=1 and option MP_LOCKDEBUG.
while sending traffic (9000 byte) over ix interfaces and at the same
time changing mtu on sink interface (ix1) from 9000 to 1500 drops me to
ddb console.
it seems that i can't trigger dropping to ddb console with
kern.pool_debug=0.
But the machine do not freeze right?
Right.
Post by Martin Pieuchot
Do you still enter DDB with a bigger value of __mp_lock_spinout?
Don't know what that means ...
Martin Pieuchot
2017-06-19 09:23:33 UTC
Permalink
Post by Hrvoje Popovski
Post by Martin Pieuchot
Post by Hrvoje Popovski
Hi all,
test box is configured with kern.pool_debug=1 and option MP_LOCKDEBUG.
while sending traffic (9000 byte) over ix interfaces and at the same
time changing mtu on sink interface (ix1) from 9000 to 1500 drops me to
ddb console.
it seems that i can't trigger dropping to ddb console with
kern.pool_debug=0.
But the machine do not freeze right?
Right.
Post by Martin Pieuchot
Do you still enter DDB with a bigger value of __mp_lock_spinout?
Don't know what that means ...
That means that MP_LOCKDEBUG works the following way:

- Each time you spin on the lock without taking it a counter is increased
- If this counter reaches the value of __mp_lock_spinout the machine enter DDB

So if this value is too small you will enter DDB everytime there's
contention. If it's too big you'll have to wait minutes before entering
DDB.

I just suggested you increase the default value:

Index: kern/kern_lock.c
===================================================================
RCS file: /cvs/src/sys/kern/kern_lock.c,v
retrieving revision 1.49
diff -u -p -r1.49 kern_lock.c
--- kern/kern_lock.c 20 Apr 2017 15:06:47 -0000 1.49
+++ kern/kern_lock.c 19 Jun 2017 09:22:39 -0000
@@ -43,7 +43,7 @@

#ifdef MP_LOCKDEBUG
/* CPU-dependent timing, this needs to be settable from ddb. */
-int __mp_lock_spinout = 200000000;
+int __mp_lock_spinout = 2000000000;
#endif

#if defined(MULTIPROCESSOR)
Hrvoje Popovski
2017-06-19 14:25:27 UTC
Permalink
Post by Martin Pieuchot
- Each time you spin on the lock without taking it a counter is increased
- If this counter reaches the value of __mp_lock_spinout the machine enter DDB
So if this value is too small you will enter DDB everytime there's
contention. If it's too big you'll have to wait minutes before entering
DDB.
Thank you for explanation.
It seems that i can't trigger dropping to ddb with increased
__mp_lock_spinout.

Loading...