Jump to content
Калькуляторы

Freebsd 9.1 mpd pppoe высокие interrupt

Reply to this topic

roysbike   

roysbike
Posted (edited)

Коллеги , помогите разобраться . Я уже писал в другой ветке. По поводу высокой загрузки на прерывания. Суть проблемы. Когда сервер в uptime 1 день , то он хоть 1500 мбит и 3000 сессий прокачает через себя. А уже прошло 2 нед , производительность. серверла упала в 5 раз. Тоесть прерывания на igb2 и igb3 загруженны больше чем 2 нед назад при той же нагрузке. Используем Freebsd 9.1

-mpd5.1 PPPoE.

-2 карты. igb0 и igb1 объеденены lagg0 . igb1 и igb2 в lagg1 (+ 100 вланов) В каждом из которых слушает PPPoE.

-шейпер NG_car. NAT не используем

 

top -SHPI

 

last pid: 13537;  load averages:  3.03,  3.11,  3.17                                                       up 16+10:09:45  21:02:39
168 processes: 10 running, 123 sleeping, 35 waiting
CPU 0:  2.0% user,  0.0% nice,  2.0% system, 22.0% interrupt, 74.0% idle
CPU 1:  1.0% user,  0.0% nice,  9.0% system, 18.0% interrupt, 72.0% idle
CPU 2:  0.0% user,  0.0% nice,  0.0% system, 95.0% interrupt,  5.0% idle
CPU 3:  0.0% user,  0.0% nice,  0.0% system, 97.0% interrupt,  3.0% idle
Mem: 371M Active, 756M Inact, 803M Wired, 622M Buf, 3998M Free
Swap: 4096M Total, 4096M Free

 PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
  12 root     -92    -     0K   640K CPU2    2 165.0H 98.19% intr{irq266: igb2:que}
  12 root     -92    -     0K   640K CPU3    3 142.2H 96.88% intr{irq271: igb3:que}
  11 root     155 ki31     0K    64K RUN     1 349.8H 89.36% idle{idle: cpu1}
  11 root     155 ki31     0K    64K RUN     0 346.7H 82.96% idle{idle: cpu0}
  11 root     155 ki31     0K    64K RUN     3 245.8H  7.28% idle{idle: cpu3}
  12 root     -92    -     0K   640K WAIT    0 727:03  7.08% intr{irq256: igb0:que}
  11 root     155 ki31     0K    64K RUN     2 225.4H  6.59% idle{idle: cpu2}
  12 root     -92    -     0K   640K CPU1    1 706:33  5.47% intr{irq261: igb1:que}
10497 root      22    0 37596K  9992K select  0 609:50  2.88% snmpd
  12 root     -92    -     0K   640K WAIT    0 486:58  2.10% intr{irq258: igb0:que}
  12 root     -92    -     0K   640K RUN     1 463:05  1.95% intr{irq263: igb1:que}
  12 root     -92    -     0K   640K WAIT    0 474:21  1.76% intr{irq257: igb0:que}
  12 root     -92    -     0K   640K RUN     1 472:57  1.46% intr{irq264: igb1:que}
  12 root     -92    -     0K   640K WAIT    1 458:29  1.46% intr{irq262: igb1:que}
  12 root     -92    -     0K   640K WAIT    0 478:28  1.07% intr{irq259: igb0:que}
11960 root      21    0   513M   416M select  0 423:24  0.88% mpd5{mpd5}
   0 root     -92    0     0K   400K -       1 223:52  0.49% kernel{igb2 que}
  12 root     -92    -     0K   640K WAIT    0 110:10  0.10% intr{irq272: igb3:que}

 

netstat -w1 -h

pppoe2# netstat -w1 -h
           input        (Total)           output
  packets  errs idrops      bytes    packets  errs      bytes colls
     148k     0     0       114M       153k     0       135M     0
     144k     0     0       110M       148k     0       130M     0
     146k     0     0       112M       151k     0       133M     0
     149k     0     0       114M       154k     0       134M     0

 

   2 users    Load  4.14  3.65  3.37                  Dec 17 21:07

Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP PAGER
       Tot   Share      Tot    Share    Free           in   out     in   out
Act  458884    7900  1203916     9500 4095116  count
All  595452    9476 1075234k    24308          pages
Proc:                                                            Interrupts
 r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        cow   45084 total
            32       79k   14 1373  40k  116    1        zfod        atkbd0 1
                                                         ozfod       atapci0+
1.6%Sys  51.6%Intr  0.6%User  0.0%Nice 46.3%Idle        %ozfod       uhci2 ehci
|    |    |    |    |    |    |    |    |    |    |       daefr  1126 cpu0:timer
=++++++++++++++++++++++++++                               prcfr 11391 igb0:que 0
                                       15 dtbuf       16 totfr  3173 igb0:que 1
Namei     Name-cache   Dir-cache    174740 desvn          react  3117 igb0:que 2
  Calls    hits   %    hits   %    109743 numvn          pdwak  2598 igb0:que 3
      7       7 100                 43684 frevn          pdpgs       igb0:link
                                                         intrn  3037 igb1:que 0
Disks  ada0   cd0 pass0 pass1                      822056 wire   2629 igb1:que 1
KB/t   0.00  0.00  0.00  0.00                      379148 act    2757 igb1:que 2
tps       0     0     0     0                      773292 inact  2718 igb1:que 3
MB/s   0.00  0.00  0.00  0.00                             cache       igb1:link
%busy     0     0     0     0                     4095116 free    384 igb2:que 0
                                                  636608 buf    1723 igb2:que 1
                                                                   6 igb2:que 2
                                                                   6 igb2:que 3
                                                                     igb2:link
                                                                1951 igb3:que 0
                                                                5105 igb3:que 1
                                                                   4 igb3:que 2
                                                                   5 igb3:que 3
                                                                     igb3:link
                                                                   8 bge0 276
                                                                1109 cpu2:timer

 

vmstat -i

 

pppoe2# vmstat -i
interrupt                          total       rate
irq1: atkbd0                           6          0
irq17: atapci0+                  2136946          1
irq22: uhci2 ehci0*                   24          0
cpu0:timer                    1581496883       1114
irq256: igb0:que 0           13595613578       9579
irq257: igb0:que 1            2876400233       2026
irq258: igb0:que 2            2936825540       2069
irq259: igb0:que 3            2911018664       2051
irq260: igb0:link                      2          0
irq261: igb1:que 0            2906488111       2047
irq262: igb1:que 1            2812819112       1981
irq263: igb1:que 2            2858171296       2013
irq264: igb1:que 3            2909989331       2050
irq265: igb1:link                      3          0
irq266: igb2:que 0            6629949321       4671
irq267: igb2:que 1            4866698493       3428
irq268: igb2:que 2              15741476         11
irq269: igb2:que 3              17885510         12
irq270: igb2:link                      2          0
irq271: igb3:que 0            6550547197       4615
irq272: igb3:que 1            5356778722       3774
irq273: igb3:que 2              15320193         10
irq274: igb3:que 3              14152079          9
irq275: igb3:link                      2          0
irq276: bge0                    20594459         14
cpu2:timer                    1521906386       1072
cpu1:timer                    1433646684       1010
cpu3:timer                    1535140223       1081
Total                        63369320476      44647

 

814 сессий

 

loader.conf

pppoe2# cat /boot/loader.conf | grep -v "#"
kern.maxfiles="25000"
net.graph.maxdata=65536
net.graph.maxalloc=65536
hw.igb.rxd=4096
hw.igb.txd=4096
hw.igb.max_interrupt_rate=16000
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.inet.tcp.tcbhashsize=4096
kern.ipc.nsfbufs=10240

 

sysctl.conf

pppoe2# cat /etc/sysctl.conf | grep -v "#"
net.graph.maxdgram=8388608
net.graph.recvspace=8388608
kern.ipc.maxsockbuf=83886080
net.inet.ip.fastforwarding=1
dev.igb.2.rx_processing_limit=4096
dev.igb.3.rx_processing_limit=4096
kern.random.sys.harvest.ethernet=0
kern.random.sys.harvest.point_to_point=0
kern.random.sys.harvest.interrupt=0
net.inet.icmp.icmplim=2000

Edited by roysbike

Share this post

Link to post
Share on other sites

roysbike   

roysbike
Posted

net.isr

уже обсуждалось не раз на форуме.

да проывал , еще хуже становилось. Но попробую еще раз .

Share this post

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Followers 0
Go to topic listing Программное обеспечение, биллинг и *unix системы