zhuravlev Posted May 13, 2010 Posted May 13, 2010 Доброе время суток уважаемые . Появилась проблема на серверах доступа под управлением FreeBSD 7.0 и 7.1 происходит постепенная или рывками утечка оперативной памяти после чего у него отказывает сетевая подсистема, т.е. полностью перестает работать сеть. Бывает это на разных сегментных серверах, т.е. может глючить по 2 раза в сутки целую неделю а потом простоять а аптайме не менее 3х месяцев и так может случится с любым из сегментных маршрутизаторов. Down Up интерфейса не лечит, лечит только ребут системы. Подставил костыль, скрипт пингует раз в минуту стабильный сервер если пинга нет, то скрипт перезагружает сервер ибо в противном случае приходилось дергать вручную. Данные сервера выполняют следующий функционал: 1. доступ в сеть абонентов при помощи VLAN; 2. доступ абонентов в сеть при помощи РРРоЕ на базе mpd5.1 до 200 сессий, авторизация происходит статически из файла mpd.secret т.к. Radius-сервер находится далеко; 3. автоматическая блокировка абонентов-должников при помощи ipfw, т.е. на каждого абонента для его статического IP-адреса на вот таком примере: deny ip from 10.10.57.185 to any deny ip from any to 10.10.57.185 данные правила динамически выгружает биллинг на сервер. 4. статическая маршрутизация между серверами доступа около 120 маршрутов и дефолт роут смотрящий на пограничный маршрутизатор, трафик не шейпируем. 5. прокачка Интернет и локального межсегментного трафика. В сегментную сеть смотрит 2 гигабитных интерфейса а в межсегментную сеть один гигабитный интерфейс. Коммутируемое оборудование используем D-Link DGS-3426. 5 коммутаторов DGS-3426 образуют кольцо от них уже идут ответвления. На кольцевых коммутаторах настроен RSTP который отлично работает. Ниже иллюстрирую лог загрузки ядра системы: May 12 16:31:21 Lnk-GW6 kernel: Copyright © 1992-2009 The FreeBSD Project. May 12 16:31:21 Lnk-GW6 kernel: Copyright © 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 May 12 16:31:21 Lnk-GW6 kernel: The Regents of the University of California. All rights reserved. May 12 16:31:21 Lnk-GW6 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. May 12 16:31:21 Lnk-GW6 kernel: FreeBSD 7.1-RELEASE #1: Tue Mar 9 22:49:24 KRAT 2010 May 12 16:31:21 Lnk-GW6 kernel: alex@Lnk-GW6 :/usr/obj/usr/src/sys/GW6LNK May 12 16:31:21 Lnk-GW6 kernel: Timecounter "i8254" frequency 1193182 Hz quality 0 May 12 16:31:21 Lnk-GW6 kernel: CPU: Intel® Xeon® CPU E5450 @ 3.00GHz (3009.97-MHz 686-class CPU) May 12 16:31:21 Lnk-GW6 kernel: Origin = "GenuineIntel" Id = 0x1067a Stepping = 10 May 12 16:31:21 Lnk-GW6 kernel: Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLU SH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> May 12 16:31:21 Lnk-GW6 kernel: Features2=0x40ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,<b26>> May 12 16:31:22 Lnk-GW6 kernel: AMD Features=0x20100000<NX,LM> May 12 16:31:22 Lnk-GW6 kernel: AMD Features2=0x1<LAHF> May 12 16:31:22 Lnk-GW6 kernel: Cores per package: 4 May 12 16:31:22 Lnk-GW6 kernel: real memory = 2680160256 (2556 MB) May 12 16:31:22 Lnk-GW6 kernel: avail memory = 2613112832 (2492 MB) May 12 16:31:22 Lnk-GW6 kernel: ACPI APIC Table: <INTEL S5000VSA> May 12 16:31:22 Lnk-GW6 kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs May 12 16:31:22 Lnk-GW6 kernel: cpu0 (BSP): APIC ID: 0 May 12 16:31:22 Lnk-GW6 kernel: cpu1 (AP): APIC ID: 1 May 12 16:31:22 Lnk-GW6 kernel: cpu2 (AP): APIC ID: 2 May 12 16:31:22 Lnk-GW6 kernel: cpu3 (AP): APIC ID: 3 May 12 16:31:22 Lnk-GW6 kernel: cpu4 (AP): APIC ID: 4 May 12 16:31:22 Lnk-GW6 kernel: cpu5 (AP): APIC ID: 5 May 12 16:31:22 Lnk-GW6 kernel: cpu6 (AP): APIC ID: 6 May 12 16:31:22 Lnk-GW6 kernel: cpu7 (AP): APIC ID: 7 May 12 16:31:22 Lnk-GW6 kernel: ioapic0 <Version 2.0> irqs 0-23 on motherboard May 12 16:31:22 Lnk-GW6 kernel: ioapic1 <Version 2.0> irqs 24-47 on motherboard May 12 16:31:22 Lnk-GW6 kernel: lapic0: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: kbd1 at kbdmux0 May 12 16:31:22 Lnk-GW6 kernel: ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) May 12 16:31:22 Lnk-GW6 kernel: acpi0: <INTEL S5000VSA> on motherboard May 12 16:31:22 Lnk-GW6 kernel: acpi0: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: acpi0: Power Button (fixed) May 12 16:31:22 Lnk-GW6 kernel: acpi0: reservation of 0, a0000 (3) failed May 12 16:31:22 Lnk-GW6 kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 May 12 16:31:22 Lnk-GW6 kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 May 12 16:31:22 Lnk-GW6 kernel: acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 May 12 16:31:22 Lnk-GW6 kernel: Timecounter "HPET" frequency 14318180 Hz quality 900 May 12 16:31:22 Lnk-GW6 kernel: acpi_button0: <Sleep Button> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: pcib0: <ACPI Host-PCI bridge> port 0xca2,0xca3,0xcf8-0xcff on acpi0 May 12 16:31:22 Lnk-GW6 kernel: pci0: <ACPI PCI bus> on pcib0 May 12 16:31:22 Lnk-GW6 kernel: pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0 May 12 16:31:22 Lnk-GW6 kernel: pci1: <ACPI PCI bus> on pcib1 May 12 16:31:22 Lnk-GW6 kernel: pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1 May 12 16:31:22 Lnk-GW6 kernel: pci2: <ACPI PCI bus> on pcib2 May 12 16:31:22 Lnk-GW6 kernel: pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2 May 12 16:31:22 Lnk-GW6 kernel: pci3: <ACPI PCI bus> on pcib3 May 12 16:31:22 Lnk-GW6 kernel: em0: <Intel® PRO/1000 Network Connection 6.9.6> port 0x3020-0x303f mem 0xb8c60000-0xb8c7ffff,0xb8c40000-0xb8c5ffff irq 16 at device 0.0 on pci3 May 12 16:31:22 Lnk-GW6 kernel: em0: Using MSI interrupt May 12 16:31:22 Lnk-GW6 kernel: em0: [FILTER] May 12 16:31:22 Lnk-GW6 kernel: em0: Ethernet address: 00:15:17:95:12:c0 May 12 16:31:22 Lnk-GW6 kernel: em1: <Intel® PRO/1000 Network Connection 6.9.6> port 0x3000-0x301f mem 0xb8c20000-0xb8c3ffff,0xb8c00000-0xb8c1ffff irq 17 at device 0.1 on pci3 May 12 16:31:22 Lnk-GW6 kernel: em1: Using MSI interrupt May 12 16:31:22 Lnk-GW6 kernel: em1: [FILTER] May 12 16:31:22 Lnk-GW6 kernel: em1: Ethernet address: 00:15:17:95:12:c1 May 12 16:31:22 Lnk-GW6 kernel: pcib4: <ACPI PCI-PCI bridge> irq 17 at device 1.0 on pci2 May 12 16:31:22 Lnk-GW6 kernel: pci4: <ACPI PCI bus> on pcib4 May 12 16:31:22 Lnk-GW6 kernel: pcib5: <PCI-PCI bridge> at device 0.0 on pci4 May 12 16:31:22 Lnk-GW6 kernel: pci5: <PCI bus> on pcib5 May 12 16:31:22 Lnk-GW6 kernel: aac0: <Adaptec RAID 3405> mem 0xb8a00000-0xb8bfffff irq 19 at device 14.0 on pci5 May 12 16:31:22 Lnk-GW6 kernel: aac0: Enable Raw I/O May 12 16:31:22 Lnk-GW6 kernel: aac0: Enable 64-bit array May 12 16:31:22 Lnk-GW6 kernel: aac0: New comm. interface enabled May 12 16:31:22 Lnk-GW6 kernel: aac0: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: aac0: Adaptec 3405, aac driver 2.0.0-1 May 12 16:31:22 Lnk-GW6 kernel: aacp0: <SCSI Passthrough Bus> on aac0 May 12 16:31:22 Lnk-GW6 kernel: aacp1: <SCSI Passthrough Bus> on aac0 May 12 16:31:22 Lnk-GW6 kernel: aacp2: <SCSI Passthrough Bus> on aac0 May 12 16:31:22 Lnk-GW6 kernel: pcib6: <PCI-PCI bridge> at device 0.2 on pci4 May 12 16:31:22 Lnk-GW6 kernel: pci6: <PCI bus> on pcib6 May 12 16:31:22 Lnk-GW6 kernel: pcib7: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2 May 12 16:31:22 Lnk-GW6 kernel: pci7: <ACPI PCI bus> on pcib7 May 12 16:31:22 Lnk-GW6 kernel: em2: <Intel® PRO/1000 Network Connection 6.9.6> port 0x2020-0x203f mem 0xb8820000-0xb883ffff,0xb8400000-0xb87fffff irq 18 at device 0.0 on pci7 May 12 16:31:22 Lnk-GW6 kernel: em2: Using MSI interrupt May 12 16:31:22 Lnk-GW6 kernel: em2: [FILTER] May 12 16:31:22 Lnk-GW6 kernel: em2: Ethernet address: 00:15:17:61:4a:dc May 12 16:31:22 Lnk-GW6 kernel: em3: <Intel® PRO/1000 Network Connection 6.9.6> port 0x2000-0x201f mem 0xb8800000-0xb881ffff,0xb8000000-0xb83fffff irq 19 at device 0.1 on pci7 May 12 16:31:22 Lnk-GW6 kernel: em3: Using MSI interrupt May 12 16:31:22 Lnk-GW6 kernel: em3: [FILTER] May 12 16:31:22 Lnk-GW6 kernel: em3: Ethernet address: 00:15:17:61:4a:dd May 12 16:31:22 Lnk-GW6 kernel: pcib8: <ACPI PCI-PCI bridge> at device 0.3 on pci1 May 12 16:31:22 Lnk-GW6 kernel: pci8: <ACPI PCI bus> on pcib8 May 12 16:31:22 Lnk-GW6 kernel: pcib9: <PCI-PCI bridge> at device 3.0 on pci0 May 12 16:31:22 Lnk-GW6 kernel: pci9: <PCI bus> on pcib9 May 12 16:31:22 Lnk-GW6 kernel: pci0: <base peripheral> at device 8.0 (no driver attached) May 12 16:31:22 Lnk-GW6 kernel: pcib10: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0 May 12 16:31:22 Lnk-GW6 kernel: pci10: <ACPI PCI bus> on pcib10 May 12 16:31:22 Lnk-GW6 kernel: uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x4080-0x409f irq 23 at device 29.0 on pci0 May 12 16:31:22 Lnk-GW6 kernel: uhci0: [GIANT-LOCKED] May 12 16:31:22 Lnk-GW6 kernel: uhci0: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: usb0: <Intel 631XESB/632XESB/3100 USB controller USB-1> on uhci0 May 12 16:31:22 Lnk-GW6 kernel: usb0: USB revision 1.0 May 12 16:31:22 Lnk-GW6 kernel: uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0 May 12 16:31:22 Lnk-GW6 kernel: uhub0: 2 ports with 2 removable, self powered May 12 16:31:22 Lnk-GW6 kernel: uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x4060-0x407f irq 22 at device 29.1 on pci0 May 12 16:31:22 Lnk-GW6 kernel: uhci1: [GIANT-LOCKED] May 12 16:31:22 Lnk-GW6 kernel: uhci1: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: usb1: <Intel 631XESB/632XESB/3100 USB controller USB-2> on uhci1 May 12 16:31:22 Lnk-GW6 kernel: usb1: USB revision 1.0 May 12 16:31:22 Lnk-GW6 kernel: uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1 May 12 16:31:22 Lnk-GW6 kernel: uhub1: 2 ports with 2 removable, self powered May 12 16:31:22 Lnk-GW6 kernel: uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x4040-0x405f irq 23 at device 29.2 on pci0 May 12 16:31:22 Lnk-GW6 kernel: uhci2: [GIANT-LOCKED] May 12 16:31:22 Lnk-GW6 kernel: uhci2: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: usb2: <Intel 631XESB/632XESB/3100 USB controller USB-3> on uhci2 May 12 16:31:22 Lnk-GW6 kernel: usb2: USB revision 1.0 May 12 16:31:22 Lnk-GW6 kernel: uhub2: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2 May 12 16:31:22 Lnk-GW6 kernel: uhub2: 2 ports with 2 removable, self powered May 12 16:31:22 Lnk-GW6 kernel: uhci3: <Intel 631XESB/632XESB/3100 USB controller USB-4> port 0x4020-0x403f irq 22 at device 29.3 on pci0 May 12 16:31:22 Lnk-GW6 kernel: uhci3: [GIANT-LOCKED] May 12 16:31:22 Lnk-GW6 kernel: uhci3: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: usb3: <Intel 631XESB/632XESB/3100 USB controller USB-4> on uhci3 May 12 16:31:22 Lnk-GW6 kernel: usb3: USB revision 1.0 May 12 16:31:22 Lnk-GW6 kernel: uhub3: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3 May 12 16:31:22 Lnk-GW6 kernel: uhub3: 2 ports with 2 removable, self powered May 12 16:31:22 Lnk-GW6 kernel: ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xb8f00400-0xb8f007ff irq 23 at device 29.7 on pci0 May 12 16:31:22 Lnk-GW6 kernel: ehci0: [GIANT-LOCKED] May 12 16:31:22 Lnk-GW6 kernel: ehci0: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: usb4: EHCI version 1.0 May 12 16:31:22 Lnk-GW6 kernel: usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 May 12 16:31:22 Lnk-GW6 kernel: usb4: <Intel 63XXESB USB 2.0 controller> on ehci0 May 12 16:31:22 Lnk-GW6 kernel: usb4: USB revision 2.0 May 12 16:31:22 Lnk-GW6 kernel: uhub4: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4 May 12 16:31:22 Lnk-GW6 kernel: uhub4: 8 ports with 8 removable, self powered May 12 16:31:22 Lnk-GW6 kernel: pcib11: <ACPI PCI-PCI bridge> at device 30.0 on pci0 May 12 16:31:22 Lnk-GW6 kernel: pci11: <ACPI PCI bus> on pcib11 May 12 16:31:22 Lnk-GW6 kernel: vgapci0: <VGA-compatible display> port 0x1000-0x10ff mem 0xb0000000-0xb7ffffff,0xb8e00000-0xb8e0ffff irq 17 at device 12.0 on pci11 May 12 16:31:22 Lnk-GW6 kernel: isab0: <PCI-ISA bridge> at device 31.0 on pci0 May 12 16:31:22 Lnk-GW6 kernel: isa0: <ISA bus> on isab0 May 12 16:31:22 Lnk-GW6 kernel: atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x40b0-0x40bf irq 20 at device 31.1 on pci0 May 12 16:31:22 Lnk-GW6 kernel: ata0: <ATA channel 0> on atapci0 May 12 16:31:22 Lnk-GW6 kernel: ata0: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: ata1: <ATA channel 1> on atapci0 May 12 16:31:22 Lnk-GW6 kernel: ata1: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: atapci1: <Intel 63XXESB2 SATA300 controller> port 0x40c8-0x40cf,0x40e4-0x40e7,0x40c0-0x40c7,0x40e0-0x40e3,0x40a0-0x40af mem 0xb8f00000-0xb8f003ff irq 20 at device 31.2 on pci0 May 12 16:31:22 Lnk-GW6 kernel: atapci1: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: ata2: <ATA channel 0> on atapci1 May 12 16:31:22 Lnk-GW6 kernel: ata2: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: ata3: <ATA channel 1> on atapci1 May 12 16:31:22 Lnk-GW6 kernel: ata3: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: pci0: <serial bus, SMBus> at device 31.3 (no driver attached) May 12 16:31:22 Lnk-GW6 kernel: atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 May 12 16:31:22 Lnk-GW6 kernel: atkbd0: <AT Keyboard> irq 1 on atkbdc0 May 12 16:31:22 Lnk-GW6 kernel: kbd0 at atkbd0 May 12 16:31:22 Lnk-GW6 kernel: atkbd0: [GIANT-LOCKED] May 12 16:31:22 Lnk-GW6 kernel: atkbd0: [iTHREAD] May 12 16:31:22 Lnk-GW6 kernel: sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 May 12 16:31:22 Lnk-GW6 kernel: sio0: type 16550A May 12 16:31:22 Lnk-GW6 kernel: sio0: [FILTER] May 12 16:31:22 Lnk-GW6 kernel: sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 May 12 16:31:22 Lnk-GW6 kernel: sio1: type 16550A May 12 16:31:22 Lnk-GW6 kernel: sio1: [FILTER] May 12 16:31:22 Lnk-GW6 kernel: cpu0: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est0: <Enhanced SpeedStep Frequency Control> on cpu0 May 12 16:31:22 Lnk-GW6 kernel: p4tcc0: <CPU Frequency Thermal Control> on cpu0 May 12 16:31:22 Lnk-GW6 kernel: cpu1: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est1: <Enhanced SpeedStep Frequency Control> on cpu1 May 12 16:31:22 Lnk-GW6 kernel: p4tcc1: <CPU Frequency Thermal Control> on cpu1 May 12 16:31:22 Lnk-GW6 kernel: cpu2: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est2: <Enhanced SpeedStep Frequency Control> on cpu2 May 12 16:31:22 Lnk-GW6 kernel: p4tcc2: <CPU Frequency Thermal Control> on cpu2 May 12 16:31:22 Lnk-GW6 kernel: cpu3: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est3: <Enhanced SpeedStep Frequency Control> on cpu3 May 12 16:31:22 Lnk-GW6 kernel: p4tcc3: <CPU Frequency Thermal Control> on cpu3 May 12 16:31:22 Lnk-GW6 kernel: cpu4: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est4: <Enhanced SpeedStep Frequency Control> on cpu4 May 12 16:31:22 Lnk-GW6 kernel: p4tcc4: <CPU Frequency Thermal Control> on cpu4 May 12 16:31:22 Lnk-GW6 kernel: cpu5: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est5: <Enhanced SpeedStep Frequency Control> on cpu5 May 12 16:31:22 Lnk-GW6 kernel: p4tcc5: <CPU Frequency Thermal Control> on cpu5 May 12 16:31:22 Lnk-GW6 kernel: cpu6: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est6: <Enhanced SpeedStep Frequency Control> on cpu6 May 12 16:31:22 Lnk-GW6 kernel: p4tcc6: <CPU Frequency Thermal Control> on cpu6 May 12 16:31:22 Lnk-GW6 kernel: cpu7: <ACPI CPU> on acpi0 May 12 16:31:22 Lnk-GW6 kernel: est7: <Enhanced SpeedStep Frequency Control> on cpu7 May 12 16:31:22 Lnk-GW6 kernel: p4tcc7: <CPU Frequency Thermal Control> on cpu7 May 12 16:31:22 Lnk-GW6 kernel: pmtimer0 on isa0 May 12 16:31:22 Lnk-GW6 kernel: orm0: <ISA Option ROMs> at iomem 0xc0000-0xc8fff,0xc9000-0xcefff,0xcf000-0xcffff,0xd0000-0xd0fff pnpid ORM0000 on isa0 May 12 16:31:22 Lnk-GW6 kernel: ppc0: parallel port not found. May 12 16:31:22 Lnk-GW6 kernel: sc0: <System console> at flags 0x100 on isa0 May 12 16:31:22 Lnk-GW6 kernel: sc0: VGA <16 virtual consoles, flags=0x300> May 12 16:31:22 Lnk-GW6 kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 May 12 16:31:22 Lnk-GW6 kernel: Timecounters tick every 0.833 msec May 12 16:31:22 Lnk-GW6 kernel: ipfw2 (+ipv6) initialized, divert enabled, nat loadable, rule-based forwarding enabled, default to accept, logging disabled May 12 16:31:22 Lnk-GW6 kernel: acd0: DVDROM <TSSTcorpDVD-ROM SH-D162D/SB01> at ata0-master UDMA33 May 12 16:31:22 Lnk-GW6 kernel: aacd0: <RAID 0/1> on aac0 May 12 16:31:22 Lnk-GW6 kernel: aacd0: 478600MB (980172800 sectors) May 12 16:31:22 Lnk-GW6 kernel: lapic1: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #1 Launched! May 12 16:31:22 Lnk-GW6 kernel: lapic6: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #6 Launched! May 12 16:31:22 Lnk-GW6 kernel: lapic3: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #3 Launched! May 12 16:31:22 Lnk-GW6 kernel: lapic2: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #2 Launched! May 12 16:31:22 Lnk-GW6 kernel: lapic7: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #7 Launched! May 12 16:31:22 Lnk-GW6 kernel: lapic5: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #5 Launched! May 12 16:31:22 Lnk-GW6 kernel: lapic4: Forcing LINT1 to edge trigger May 12 16:31:22 Lnk-GW6 kernel: SMP: AP CPU #4 Launched! May 12 16:31:22 Lnk-GW6 kernel: Trying to mount root from ufs:/dev/aacd0s1a May 12 16:31:22 Lnk-GW6 savecore: no dumps found May 12 16:31:22 Lnk-GW6 kernel: em3: link state changed to DOWN May 12 16:31:24 Lnk-GW6 kernel: em2: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan612: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan613: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan614: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan615: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan611: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan616: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan617: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: vlan600: link state changed to UP May 12 16:31:24 Lnk-GW6 kernel: em0: link state changed to UP May 12 16:31:25 Lnk-GW6 kernel: em3: link state changed to UP May 12 16:34:19 Lnk-GW6 kernel: arp: 172.2.60.42 is on vlan600 but got reply from 00:21:91:13:25:f6 on em2 May 12 16:34:19 Lnk-GW6 kernel: arp: 172.2.60.63 moved from 00:21:91:13:25:c9 to 00:21:91:13:26:23 on vlan600 May 12 16:34:19 Lnk-GW6 kernel: arp: 172.2.60.63 moved from 00:21:91:13:26:23 to 00:21:91:13:25:c9 on vlan600 May 12 16:34:19 Lnk-GW6 kernel: arp: 172.2.60.37 is on vlan600 but got reply from 00:21:91:f4:a0:f8 on em2 Далее представлю Вам вывод команды netstat –m # netstat -m 1058/3307/4365 mbufs in use (current/cache/total) 867/1849/2716/33792 mbuf clusters in use (current/cache/total/max) 867/1693 mbuf+clusters out of packet secondary zone in use (current/cache) 0/135/135/16896 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/8448 9k jumbo clusters in use (current/cache/total/max) 0/0/0/4224 16k jumbo clusters in use (current/cache/total/max) 1998K/5064K/7063K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/6/8704 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Вывод команды top –S last pid: 16962; load averages: 0.44, 0.40, 0.32 up 0+21:26:11 14:05:49 96 processes: 9 running, 70 sleeping, 1 stopped, 16 waiting CPU: 0.0% user, 0.0% nice, 3.7% system, 0.2% interrupt, 96.1% idle Mem: 20M Active, 346M Inact, 171M Wired, 136K Cache, 112M Buf, 1959M Free Swap: 16G Total, 16G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 12 root 1 171 ki31 0K 8K CPU6 6 21.4H 100.00% idle: cpu6 15 root 1 171 ki31 0K 8K CPU3 3 21.3H 100.00% idle: cpu3 17 root 1 171 ki31 0K 8K RUN 1 21.2H 100.00% idle: cpu1 11 root 1 171 ki31 0K 8K CPU7 7 21.2H 100.00% idle: cpu7 18 root 1 171 ki31 0K 8K CPU0 0 21.2H 100.00% idle: cpu0 14 root 1 171 ki31 0K 8K CPU4 4 20.8H 100.00% idle: cpu4 13 root 1 171 ki31 0K 8K CPU5 5 20.0H 94.24% idle: cpu5 16 root 1 171 ki31 0K 8K CPU2 2 19.6H 89.31% idle: cpu2 29 root 1 -68 - 0K 8K - 2 103:59 17.04% em0 taskq 34 root 1 -68 - 0K 8K - 5 85:30 11.62% em3 taskq 33 root 1 -68 - 0K 8K - 4 33:20 1.66% em2 taskq 20 root 1 -32 - 0K 8K WAIT 7 11:31 0.88% swi4: clock sio 50 root 1 -68 - 0K 8K - 3 5:07 0.00% dummynet 767 root 1 44 0 8808K 5096K select 1 4:05 0.00% snmpd 22 root 1 -16 - 0K 8K - 1 2:47 0.00% yarrow 56 root 1 20 - 0K 8K syncer 3 1:56 0.00% syncer Очень сильно беспокоит 346M Inact потому что не зависимо от нагрузок на сервер этот параметр постоянно растет за сутки до этого данный параметр был 96М а параметр 1959M Free постоянно пропорционально убывает. Опции ядра: options IPFIREWALL options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_FORWARD options IPDIVERT options DUMMYNET device vlan options DEVICE_POLLING options HZ=1200 options NETGRAPH options NETGRAPH_PPPOE options NETGRAPH_SOCKET options NETGRAPH_CISCO options NETGRAPH_ECHO options NETGRAPH_FRAME_RELAY options NETGRAPH_HOLE options NETGRAPH_KSOCKET options NETGRAPH_LMI options NETGRAPH_TTY options NETGRAPH_ASYNC options NETGRAPH_BPF options NETGRAPH_ETHER options NETGRAPH_IFACE options NETGRAPH_KSOCKET options NETGRAPH_L2TP options NETGRAPH_PPP options NETGRAPH_PPTPGRE options NETGRAPH_TEE options NETGRAPH_UI options NETGRAPH_VJC Также немного оттюнил систему при этом выключил Polling ибо очень тормозило саму передачу трафик и пинг был огромный. Polling был отключен задолго до подобных глюков. sysctl -w kern.polling.enable=0 sysctl -w kern.ipc.somaxconn=2048 sysctl -w net.inet.tcp.blackhole=1 sysctl -w net.inet.udp.blackhole=1 sysctl -w net.inet.icmp.drop_redirect=1 sysctl -w net.inet.ip.redirect=0 sysctl -w net.inet.icmp.log_redirect=1 sysctl -w net.inet6.ip6.redirect=0 #Netgraph PPPoE sysctl -w net.graph.maxdgram=128000 sysctl -w net.graph.recvspace=128000 Подскажите куда копать, куда смотреть ? Куда девается память ? P.S. Перед отказом сетевой подсистемы оперативной памяти судя по графикам израсходовано по разному, не более 6-й части. Вставить ник Quote
jab Posted May 13, 2010 Posted May 13, 2010 Inactive не будет чиститься пока есть Free. Абсолютно нормальное поведение. На кой вставлен POLLING на восьмигоршковом XEON'е при том, что он не юзается - непонятно. Где полный sysctl ? Где логи в момент гипотетического "зависания сетевой подсистемы" ? Для начала - убрать поллинг из ядра, проапгрейдиться до 7-STABLE. Вставить ник Quote
zhuravlev Posted May 13, 2010 Author Posted May 13, 2010 Inactive не будет чиститься пока есть Free. Абсолютно нормальное поведение. На кой вставлен POLLING на восьмигоршковом XEON'е при том, что он не юзается - непонятно. Где полный sysctl ? Где логи в момент гипотетического "зависания сетевой подсистемы" ? Для начала - убрать поллинг из ядра, проапгрейдиться до 7-STABLE. как получить полный sysctl? sysctl -a? Там очень много всего)) в логах messages и dmesg.today нечего нет при этом. Или где-нибудь еще можно посмотреть и как? options HZ=1200 тоже убрать из ядра? а версия системы собственно таква: Lnk-GW6# uname -a FreeBSD Lnk-GW6 7.1-RELEASE FreeBSD 7.1-RELEASE #1: Tue Mar 9 22:49:24 KRAT 2010 Вставить ник Quote
.png.5d2afa2996cc6a85d0f2c09b92dd0a28.png)
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.