Stuart Henderson
2017-07-17 09:53:53 UTC
I ran into "vmxnet3_getbuf: buffer has mbuf" on an esxi VM (5.5.0 -
VM version 8) running librenms and unifi (java/mongodb). Lots of snmp
fetches, lots of disk io from rrdtool, fairly high memory utilisation.
It's running June 20 snapshot (amd64 MP). Probably a couple of weeks
uptime. I can provide dmesg after a reboot, but wanted to check if
anyone has requests for more information from DDB first (in any event
I'll need to reboot it in a couple of hours).
OCR'd and quickly proofread below, screenshots at
Loading Image...
Loading Image...
Loading Image...
Iogin: panic: vmxnet3_getbuf: buffer has mbuf
Stopped at db enter+0x9: leave
TID PID UID PRFLAGS PFLAGS CPU COMMAND
161561 79703 755 0x2 0 2 snmpget
4987 44005 755 0x2 0 3 snmpwalk
*243557 4788 755 0x2 0 0 snmpget
399589 81644 0 0x10 0x4000000 1 bacula-fd
db_enter(ffff8000236b2aa0,10202,8,10,ffff8000236b2a88,286) at db_enter+0x9
panic(ffffffff8177d708,ffff8000236b2c08,1d0,1d0,ffff8000001ae560,ffff8000001ac0
48) at panic+0x102
vmxnet3_getbuf(ffff8000001ac000,ffff8000001ae560,ffffff00bff18000,ffff8000001ae
560,ffffff008c331c00,ffffff00bff1fca0) at vmxnet3_getbuf+0x12b
vmxnet3_rxintr(ffff8000001ac000,ffff8000001ae560,ffff8000236b2d48,0,ffffffff819
6c920,ffff8000001fe580) at vmxnet3_rxintr+0x256
vmxnet3_intr(ffff8000001ac000,ffff8000237e7878,0,ffffff01ab0f42e0,ffffff0036900
b98,ffffff01ab0f6270) at vmxnet3_intr+0x67
intr_handler(ffff8000236b2d48,ffff8000001fe580,5,ffff8000001fe500,0,7) at intr
handler+0x6f
_intr_ioapic_level10() at _intr_ioapic_level10+0xcd
--- interrupt ---
___mp_lock(ffffffff81a1f400,45,ffff8000237e7878,30000,10,ffff8000236b2e20) at _
__mp_lock+0x42
syscall() at syscall+0x328
--- syscall (number 5) ---
end trace frame: 0x0, count: 6
0x86733ea9c6a:
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports. Insufficient info makes it difficult to find and fix bugs.
ddb{0}> sh reg
rdi 0x1
rsi 0x286 mptramp_gdt32_desc+0x264
rbp 0xffff8000236b2a98
rbx 0xffffffff8177d708 __func__.6599+0x16
rdx 0
rcx 0xffffffff8196c920 cpu_info_primary
rax 0x1
r8 0xffff8000236b29a8
r9 0x1
r10 0xffff8000236b2948
r11 0x8
r12 0x100 mptramp_gdt32_desc+0xde
r13 0xffff8000236b2aa8
r14 0xffff8000001ac000
r15 0xffffff00bff18000
rip 0xffffffff81024129 db_enter+0x9
cs 0x8
rfIags 0x286 mptramp_gdt32_desc+0x264
rsp 0xffff8000236b2a88
ss 0x10
db_enter+0x9: Ieave
VM version 8) running librenms and unifi (java/mongodb). Lots of snmp
fetches, lots of disk io from rrdtool, fairly high memory utilisation.
It's running June 20 snapshot (amd64 MP). Probably a couple of weeks
uptime. I can provide dmesg after a reboot, but wanted to check if
anyone has requests for more information from DDB first (in any event
I'll need to reboot it in a couple of hours).
OCR'd and quickly proofread below, screenshots at
Loading Image...
Loading Image...
Loading Image...
Iogin: panic: vmxnet3_getbuf: buffer has mbuf
Stopped at db enter+0x9: leave
TID PID UID PRFLAGS PFLAGS CPU COMMAND
161561 79703 755 0x2 0 2 snmpget
4987 44005 755 0x2 0 3 snmpwalk
*243557 4788 755 0x2 0 0 snmpget
399589 81644 0 0x10 0x4000000 1 bacula-fd
db_enter(ffff8000236b2aa0,10202,8,10,ffff8000236b2a88,286) at db_enter+0x9
panic(ffffffff8177d708,ffff8000236b2c08,1d0,1d0,ffff8000001ae560,ffff8000001ac0
48) at panic+0x102
vmxnet3_getbuf(ffff8000001ac000,ffff8000001ae560,ffffff00bff18000,ffff8000001ae
560,ffffff008c331c00,ffffff00bff1fca0) at vmxnet3_getbuf+0x12b
vmxnet3_rxintr(ffff8000001ac000,ffff8000001ae560,ffff8000236b2d48,0,ffffffff819
6c920,ffff8000001fe580) at vmxnet3_rxintr+0x256
vmxnet3_intr(ffff8000001ac000,ffff8000237e7878,0,ffffff01ab0f42e0,ffffff0036900
b98,ffffff01ab0f6270) at vmxnet3_intr+0x67
intr_handler(ffff8000236b2d48,ffff8000001fe580,5,ffff8000001fe500,0,7) at intr
handler+0x6f
_intr_ioapic_level10() at _intr_ioapic_level10+0xcd
--- interrupt ---
___mp_lock(ffffffff81a1f400,45,ffff8000237e7878,30000,10,ffff8000236b2e20) at _
__mp_lock+0x42
syscall() at syscall+0x328
--- syscall (number 5) ---
end trace frame: 0x0, count: 6
0x86733ea9c6a:
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports. Insufficient info makes it difficult to find and fix bugs.
ddb{0}> sh reg
rdi 0x1
rsi 0x286 mptramp_gdt32_desc+0x264
rbp 0xffff8000236b2a98
rbx 0xffffffff8177d708 __func__.6599+0x16
rdx 0
rcx 0xffffffff8196c920 cpu_info_primary
rax 0x1
r8 0xffff8000236b29a8
r9 0x1
r10 0xffff8000236b2948
r11 0x8
r12 0x100 mptramp_gdt32_desc+0xde
r13 0xffff8000236b2aa8
r14 0xffff8000001ac000
r15 0xffffff00bff18000
rip 0xffffffff81024129 db_enter+0x9
cs 0x8
rfIags 0x286 mptramp_gdt32_desc+0x264
rsp 0xffff8000236b2a88
ss 0x10
db_enter+0x9: Ieave