[megahertz0]

Есть один вот такой шеститонник:

Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
 1   48  CEF720 48 port 1000mb SFP              WS-X6748-SFP       
 2   48  48 port 10/100/1000mb EtherModule      WS-X6148V-GE-TX    
 3   16  16 port 1000mb GBIC ethernet           WS-X6416-GBIC      
 4   16  16 port 1000mb GBIC ethernet           WS-X6416-GBIC      
 5   16  16 port 1000mb GBIC ethernet           WS-X6416-GBIC      
 6    2  Supervisor Engine 720 (Active)         WS-SUP720-3BXL     

Вот что показывает в процессах

cgw130#sh proc cpu sorted 1m | ex 0.00
CPU utilization for five seconds: 37%/35%; one minute: 40%; five minutes: 41%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
310       19836     18244       1087  0.39%  1.00%  1.12%   1 SSH Process
 12      625688   1101676        567  0.95%  0.45%  0.41%   0 ARP Input
277      459824   2296209        200  0.07%  0.33%  0.38%   0 IP Input
301        5960  22688267          0  0.15%  0.16%  0.15%   0 Ethernet Msec Ti
358       96548    263503        366  0.15%  0.10%  0.10%   0 CEF: IPv4 proces
 52       14688    183003         80  0.07%  0.07%  0.07%   0 Per-Second Jobs
271       38820    143480        270  0.07%  0.07%  0.07%   0 CDP Protocol
379       79004     91312        865  0.07%  0.04%  0.05%   0 HIDDEN VLAN Proc
521       40728    748540         54  0.07%  0.03%  0.02%   0 EIGRP-IPv4 Hello

Как видно, процессы грузят процессор максимум на 2-3 процента. Остальное прерывания...

 

Свитч работает как L3-ядро, раздает и принимает маршруты через EIGRP и дефолт от бордера. Роутит гига 2-3.

Route-map проверил, везде есть set ip next-hop или set ip default next-hop, больше ничего лишнего. Есть немного мультикаста для опытов в незначительных количествах. На L3-интерфесах прописано no ip redirects, no ip unreachables.

 

Вот что показывает show platform hardware capacity:

cgw130#sh platform hardware capacity
System Resources
 PFC operating mode: PFC3BXL
 Supervisor redundancy mode: administratively sso, operationally sso
 Switching resources: Module   Part number               Series      CEF mode
                      1        WS-X6748-SFP              CEF720           CEF
                      2        WS-X6148V-GE-TX          classic           CEF
                      3        WS-X6416-GBIC            classic           CEF
                      4        WS-X6416-GBIC            classic           CEF
                      5        WS-X6416-GBIC            classic           CEF
                      6        WS-SUP720-3BXL        supervisor           CEF

Power Resources
 Power supply redundancy mode: administratively redundant
                               operationally redundant
 System power: 3830W, 0W (0%) inline, 1454W (38%) total allocated
 Powered devices: 0 total, 0 Class3, 0 Class2, 0 Class1, 0 Class0, 0 Cisco
                       Inline-Pwr  Inline-Pwr
                       Limit       Allocated   %Limit
Slot Card-Type          Watts       Watts
---- ------------------ ----------- ----------- -------
2    WS-F6K-VPWR-GE      399.84       34.44     9%

Flash/NVRAM Resources
 Usage: Module Device               Bytes:      Total          Used     %Used
        1      dfc#1-bootflash:              15990784             0        0%
        6  RP  bootflash:                    65536000      58889268       90%
        6  SP  disk0:                       512073728     380059648       74%
        6  SP  sup-bootflash:                65536000      57776228       88%
        6  SP  const_nvram:                    129004         10704        8%
        6  SP  nvram:                         1964024         84669        4%

CPU Resources
 CPU utilization: Module             5 seconds       1 minute       5 minutes
                  1                   0% /  0%             1%              1%
                  6  RP              39% / 36%            38%             38%
                  6  SP              10% /  0%            13%             14%
 Processor memory: Module   Bytes:       Total           Used           %Used
                   1                 198955040       38597396             19%
                   6  RP             890769392      126667448             14%
                   6  SP             825301860      137668352             17%
 I/O memory: Module         Bytes:       Total           Used           %Used
             6  RP                    67108864       21605604             32%
             6  SP                    67108864       20884952             31%

EOBC Resources
 Module                     Packets/sec     Total packets     Dropped packets
 1          Rx:                      18          36911501                   2
            Tx:                      12           4232857                   4
 6  RP      Rx:                      93          16866182                  28
            Tx:                      92          16824992                   0
 6  SP      Rx:                      45           8994856                 758
            Tx:                      52          10134990                   0

VLAN Resources
 VLANs: 4094 total, 174 VTP, 3 extended, 42 internal, 3875 free

L2 Forwarding Resources
          MAC Table usage:   Module  Collisions  Total       Used       %Used
                             6                0  65536       1309          2%

            VPN CAM usage:                       Total       Used       %Used
                                                   512          0          0%
L3 Forwarding Resources
            FIB TCAM usage:                     Total        Used       %Used
                 72 bits (IPv4, MPLS, EoM)     524288         692          1%
                144 bits (IP mcast, IPv6)      262144          22          1%

                    detail:      Protocol                    Used       %Used
                                 IPv4                         690          1%
                                 MPLS                           1          1%
                                 EoM                            1          1%

                                 IPv6                           1          1%
                                 IPv4 mcast                    18          1%
                                 IPv6 mcast                     3          1%

           Adjacency usage:                     Total        Used       %Used
                                              1048576         540          1%

    Forwarding engine load:
                    Module       pps   peak-pps                     peak-time
                    6         505933     505933  15:10:01 MSK Tue Dec 10 2013

Netflow Resources
         TCAM utilization:       Module       Created      Failed       %Used
                                 6                  3           0          0%
         ICAM utilization:       Module       Created      Failed       %Used
                                 6                  0           0          0%

                Flowmasks:   Mask#   Type        Features
                     IPv4:       0   reserved    none
                     IPv4:       1   unused      none
                     IPv4:       2   unused      none
                     IPv4:       3   reserved    none

                     IPv6:       0   reserved    none
                     IPv6:       1   unused      none
                     IPv6:       2   unused      none
                     IPv6:       3   reserved    none

CPU Rate Limiters Resources
            Rate limiters:       Total         Used      Reserved       %Used
                   Layer 3           9            4             1         44%
                   Layer 2           5            3             3         60%

ACL/QoS TCAM Resources
 Key: ACLent - ACL TCAM entries, ACLmsk - ACL TCAM masks, AND - ANDOR,
      QoSent - QoS TCAM entries, QOSmsk - QoS TCAM masks, OR - ORAND,
      Lbl-in - ingress label, Lbl-eg - egress label, LOUsrc - LOU source,
      LOUdst - LOU destination, ADJ - ACL adjacency

 Module ACLent ACLmsk QoSent QoSmsk Lbl-in Lbl-eg LOUsrc LOUdst  AND  OR  ADJ
 6          1%     2%     1%     1%     1%     1%     0%     0%   0%  0%   1%

L3 Multicast Resources
 IPv4 replication mode: ingress
 IPv6 replication mode: ingress
 Bi-directional PIM Designated Forwarder Table usage: 4 total, 0 (0%) used
 Replication capability: Module                              IPv4        IPv6
                         1                                 egress      egress
                         2                                ingress     ingress
                         3                                ingress     ingress
                         4                                ingress     ingress
                         5                                ingress     ingress
                         6                                 egress      egress
 MET table Entries: Module                             Total    Used    %Used
                    6                                  65516       8       1%

QoS Policer Resources
 Aggregate policers: Module                      Total         Used     %Used
                     6                            1024           16        1%
 Microflow policer configurations: Module        Total         Used     %Used
                                   6                64            1        1%

Switch Fabric Resources
 Bus utilization: current: 21%, peak was 29% at 22:52:40 MSK Mon Dec 9 2013
 Fabric utilization:     Ingress                    Egress
   Module  Chanl  Speed  rate  peak                 rate  peak
   1       0        20G    0%    1% @15:10 10Dec13    0%    2% @13:42 10Dec13
   1       1        20G    0%    1% @15:10 10Dec13    1%    2% @15:09 10Dec13
   6       0        20G    1%    2% @15:09 10Dec13    1%    1% @15:10 10Dec13
 Switching mode: Module                                        Switching mode
                 1                                                       acef
                 6                                                        bus

Interface Resources
 Interface drops:
   Module    Total drops:    Tx            Rx      Highest drop port:  Tx  Rx
   2                 8485918619             0                          41   0
   4                          0          2562                           0   1
   5                         93             0                          13   0

 Interface buffer sizes:
   Module                            Bytes:     Tx buffer           Rx buffer
   1     (asic-1)                                 1221120              173504
   2     (asic-1)                                 1081344              147456
   3     (asic-1)                                  442368               81920
   4     (asic-1)                                  442368               81920
   5     (asic-1)                                  442368               81920
IBC Resources
 Module                     Packets/sec     Total packets     Dropped packets
 6  RP      Rx:                   12022        1685304825                   0
            Tx:                   11984        1680951314                   0
 6  SP      Rx:                      17           3203186                   1
            Tx:                      26           5492406                   0

SPAN Resources
 Source sessions: 16 maximum, 1 used
   Type                             Max      Used
   Local                              2(*)      1
   Local-tx                          14         0
   RSPAN source                       2(*)      0
   ERSPAN source                      2(*)      0
   Capture                            1(*)      0
   Service module                     1(*)      0
   OAM loopback                       1(*)      0
     * - shared source sessions and the total can not exceed 2
 Destination sessions: 64 maximum, 0 used
   Type                             Max      Used
   RSPAN destination                 64(*)      0
   ERSPAN destination                23(*)      0
     * - shared destination sessions and the total can not exceed 64

Multicast LTL Resources
 Usage:   30656 Total, 1282 Used

 

В обшем, какая-то магия. Уже все источники нагрузки на проц ликвидировал. Непонятно куда смотреть. Вот это читал.

[Hoax]

попробуйте show cef not-cef-switched и show ip traffic посмотреть

Изменено 10 декабря, 2013 пользователем Hoax

[megahertz0]

Попробовал уобрать PBR, вернее в ACL поставил deny. Вот что получилось.

cgw130#sh proc cpu sorted 1m | ex 0.00
CPU utilization for five seconds: 6%/3%; one minute: 21%; five minutes: 27%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 12      657152   1157196        567  0.23%  0.52%  0.43%   0 ARP Input
506     1291868   5786425        223  0.87%  0.46%  0.46%   0 Port manager per
277      476016   2360408        201  0.15%  0.34%  0.44%   0 IP Input
301        6148  23256240          0  0.15%  0.15%  0.15%   0 Ethernet Msec Ti
310       28968     26279       1102  0.31%  0.12%  0.04%   1 SSH Process
358       98976    270092        366  0.07%  0.11%  0.10%   0 CEF: IPv4 proces
 52       15060    187538         80  0.07%  0.07%  0.07%   0 Per-Second Jobs
271       39768    147017        270  0.07%  0.07%  0.07%   0 CDP Protocol
563      116476    273888        425  0.07%  0.04%  0.05%   0 SNMP ENGINE
317      125396     55637       2253  0.07%  0.04%  0.05%   0 QOS Stats Gather
409        2276   5735617          0  0.07%  0.03%  0.02%   0 RADIUS
521       41776    766092         54  0.07%  0.03%  0.02%   0 EIGRP-IPv4 Hello
246        2508   5737611          0  0.07%  0.03%  0.02%   0 ACE Tunnel Task
567        6160    226306         27  0.07%  0.02%  0.02%   0 LLDP Protocol

[SergeiK]

Попробуйте почитать про Control Plane Policing (CoPP).

Даже просто включение данной фичи позволит посмотреть, куда попадают пакеты, и помочь в диагностике.

Если что-то создает проблемы, там же можно ограничить.

[megahertz0]

Вопрос в том, почему PBR нагружает так сильно проц? PBRом трафик по акцес-листу из определенных подсетей заворачивается в BRAS, другим акцес-листом заворачивается в линуксовый тазик для НАТа. Оба next-hop директли-коннектед для шеститонника. Выглядит вот так:

 

route-map map-default permit 10
match ip address Subnets-isg
set ip default next-hop 10.20.0.2
!
route-map map-default permit 20
match ip address xx
set ip next-hop 10.252.0.1
!
route-map map-default permit 30
match ip address yy
set ip next-hop 10.252.0.1
!
route-map map-default permit 40
match ip address xx
set ip next-hop 10.252.0.1



ip access-list extended Subnets-isg
permit ip 10.21.0.0 0.0.255.255 any
deny   ip any any
ip access-list extended xx
permit ip 10.201.38.0 0.0.1.255 any
deny   ip any any
ip access-list extended yy
permit ip 10.201.34.0 0.0.1.255 any
deny   ip any any
ip access-list extended zz
permit ip 10.201.10.0 0.0.1.255 any
deny   ip any any

Изменено 10 декабря, 2013 пользователем megahertz0

[DVM-Avgoor]

Вопрос в том, почему PBR нагружает так сильно проц?

 

 

Потому что вы что-то сделали не так. Или перебрали через лимит, или используете PBR не совместимый с железным L3.

Честно говоря, это вообще редкость чтобы на 65ой что-то уползало в CPU. Такое бывает или преднамеренно (но лечится CoPP), либо кто-то не читал доки по железке.

[DVM-Avgoor]

Покажите настройки интерфейсов где это применяется. Что там с cef опциями...

[DVM-Avgoor]

И вам вдогонку, раз уж сегодня вечер Cisco PBR на форуме:

 

Using a PBR route-map without a set statement - Any traffic that matches a PBR route-map with no set statement will be punted. This is due to the fact that we need to program the next-hop in hardware and if the next-hop is not known, this traffic must be punted to determine the next hop. Configure a set statement OR remove the policy route from the interface.

 

When configuring IPv4 routing and addresses, follow these guidelines and restrictions:

•See the command reference for information about the maximum-paths command.

 

•The Policy Feature Card (PFC) and any Distributed Feature Cards (DFCs) provide hardware support for policy-based routing (PBR) for route-map sequences that use the match ip address, set ip next-hop, and ip default next-hop PBR keywords.

 

When configuring PBR, follow these guidelines and restrictions:

–The PFC provides hardware support for PBR configured on a tunnel interface.

 

–The PFC does not provide hardware support for PBR configured with the set ip next-hop keywords if the next hop is a tunnel interface.

 

–To avoid high CPU utilization, do not configure an address in the same subnet as the next hop.

 

–If the RP address falls within the range of a PBR ACL, traffic addressed to the RP is policy routed in hardware instead of being forwarded to the RP. To prevent policy routing of traffic addressed to the RP, configure PBR ACLs to deny traffic addressed to the RP.

 

–Any options in Cisco IOS ACLs that provide filtering in a PBR route-map that would cause flows to be sent to the RP to be switched in software are ignored. For example, logging is not supported in ACEs in Cisco IOS ACLs that provide filtering in PBR route-maps.

 

–PBR traffic through switching module ports where PBR is configured is routed in software if the switching module resets. (CSCee92191)

 

–Any permit route-map sequence with no set statement will cause matching traffic to be processed by the RP.

 

–In Cisco IOS Release 12.2(33)SXH4 and later releases, for efficient use of hardware resources, enter the platform ipv4 pbr optimize tcam command in global configuration mode when configuring multiple PBR sequences (or a single PBR sequence with multiple ACLs) in which more than one PBR ACL contains DENY entries. In earlier releases, we recommend avoiding this type of configuration. (CSCsr45495)

 

–In Cisco IOS Release 12.2(33)SXH4 and later releases, the BOOTP/DHCP traffic will be dropped unless explicitly permitted. In Cisco IOS Release 12.2(18)SXF, BOOTP/DHCP packets are not subjected to a PBR configured in the ingress interfaces and the BOOTP/DHCP packets are forwarded to the BOOTP/DHCP server, although they are not explicitly permitted.

 

[megahertz0]

Грамотно заданный вопрос - половина ответа. В общем, все оказалось просто. Некст-хоп находился в той же подсети, что и SVI на шеститоннике. А циска это прямо не рекомендует вот здесь. Сколько не читал эту доку - на пункт про ту же подсеть внимания не обращал.

В итоге пересадил БРАС в отдельный влан с подсеткой, поднял на шеститоннике SVI и уже туда свернул трафик. Все, теперь загрузка в норме

CPU utilization for five seconds: 3%/0%; one minute: 3%; five minutes: 3%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 12      759892   1416017        536  0.31%  0.50%  0.43%   0 ARP Input
 52       15524    207120         74  0.07%  0.07%  0.07%   0 Per-Second Jobs
271       42312    162394        260  0.07%  0.07%  0.07%   0 CDP Protocol
277      516184   2611040        197  0.07%  0.08%  0.07%   0 IP Input
301        6372  25723224          0  0.07%  0.12%  0.13%   0 Ethernet Msec Ti
310         216       188       1148  0.71%  0.21%  0.06%   1 SSH Process
317      137200     61482       2231  0.07%  0.03%  0.04%   0 QOS Stats Gather
358      107312    298650        359  0.07%  0.11%  0.11%   0 CEF: IPv4 proces
379       84304    103320        815  0.07%  0.03%  0.04%   0 HIDDEN VLAN Proc
506     1388388   6393824        217  0.79%  0.42%  0.41%   0 Port manager per
521       44472    842143         52  0.07%  0.03%  0.02%   0 EIGRP-IPv4 Hello
569      141948    721859        196  0.07%  0.05%  0.06%   0 NTP

[DVM-Avgoor]

Ну, в списке выше оно есть. Иногда бывает полезно прочитать мануал. :)

Хотя признаю что у циски такие мануалы, что и читать не хочется. Какие-то вечно дезинтегрированные, что-ли...

[megahertz0]

Ну, в списке выше оно есть. Иногда бывает полезно прочитать мануал. :)

Догнал я в чем проблема я раньше чем вы выложили, но сути дела это не меняет :)

[dmvy]

Вопрос в том, почему PBR нагружает так сильно проц? PBRом трафик по акцес-листу из определенных подсетей заворачивается в BRAS, другим акцес-листом заворачивается в линуксовый тазик для НАТа. Оба next-hop директли-коннектед для шеститонника. Выглядит вот так:

 

route-map map-default permit 10
match ip address Subnets-isg
set ip default next-hop 10.20.0.2
!

ip access-list extended Subnets-isg
permit ip 10.21.0.0 0.0.255.255 any
deny   ip any any

нахожу лишним deny ip any any

по логике на next-hop будут уходить пакеты только те, которые попали под permit и никакие другие. deny не нужен. Вероятно он и нагрузил CPU.

[megahertz0]

Deny не убирал, просто некст-хоп поменял. Это помогло.

[DVM-Avgoor]

по логике на next-hop будут уходить пакеты только те, которые попали под permit и никакие другие. deny не нужен. Вероятно он и нагрузил CPU.

 

 

А там в любом же случае deny в конце акцес-листа. Даже если его не вписывать явно :)

[joesm]

Использовать deny в ACL для PBR крайне не рекомендуется Cisco.

В части железа такие штуки переводят обработку трафика в софтовый режим.

[Mackiavelly]

Друзья аналогичная ситуация  железка почти какая же все способы выше не помогли

#sh proc cpu s | e 0.00
CPU utilization for five seconds: 34%/22%; one minute: 39%; five minutes: 40%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  70   1192706321281279185         93  3.67%  3.73%  3.74%   0 Net Input
 331   770944392 161987754       4759  2.07%  2.31%  2.64%   0 IP Input
 364   565020840 162517702       3476  1.83%  1.74%  1.74%   0 CEF: IPv4 proces
  23   5098194761011377796        504  1.35%  1.17%  1.18%   0 ARP Input
 336        1828      1938        943  0.55%  0.23%  0.06%   2 Virtual Exec
 307    80085992 173847082        460  0.39%  0.18%  0.17%   0 QOS Stats Gather
 642    82004072 423305036        193  0.23%  0.21%  0.23%   0 OSPF-1 Router
 290     47881841284350644          3  0.23%  0.21%  0.21%   0 Ethernet Msec Ti
 301   105999336 524496695        202  0.15%  0.09%  0.08%   0 DHCPD Receive
 244    38868076  16653854       2333  0.15%  0.15%  0.15%   0 Compute load avg
 644    50721844 499074765        101  0.07%  0.11%  0.14%   0 OSPF-1 Hello
 277    20115388  55665571        361  0.07%  0.07%  0.07%   0 esw_vlan_stat_pr
 401     8648944  22355026        386  0.07%  0.03%  0.02%   0 HIDDEN VLAN Proc
 424    38624064 280693199        137  0.07%  0.09%  0.07%   0 ADJ resolve proc
 391     9230884  66937486        137  0.07%  0.05%  0.06%   0 L3 Manager
 354      814884  16406482         49  0.07%  0.02%  0.01%   0 XDR mcast
 329     2804392 695403877          4  0.07%  0.09%  0.07%   0 VRRS Main thread
  57    10226608  52043623        196  0.07%  0.07%  0.07%   0 Per-Second Jobs
#

#sh platform hardware capacity
System Resources
  PFC operating mode: PFC3B
  Supervisor redundancy mode: administratively sso, operationally sso
  Switching resources: Module   Part number               Series      CEF mode
                       1        WS-X6704-10GE             CEF720          dCEF
                       3        WS-X6148-GE-TX           classic           CEF
                       4        WS-X6724-SFP              CEF720           CEF
                       5        WS-SUP720-3B          supervisor           CEF
                       7        WS-X6748-GE-TX            CEF720           CEF

Power Resources
  Power supply redundancy mode: administratively redundant
                                operationally redundant
  System power: 2331W, 0W (0%) inline, 1496W (64%) total allocated
  Powered devices: 0 total, 0 Class4, 0 Class3, 0 Class2, 0 Class1, 0 Class0, 0 Cisco

Flash/NVRAM Resources
  Usage: Module Device               Bytes:      Total          Used     %Used
         1      dfc#1-bootflash:              15990784        649960        4%
         4      cfc#4-bootflash:              15990784             0        0%
         5  RP  bootflash:                    65536000        762772        1%
         5  SP  sup-bootdisk:                512106496     434470912       85%
         5  SP  const_nvram:                    129004           676        1%
         5  SP  nvram:                         1964024          4186        1%
         7      cfc#7-bootflash:              15990784             0        0%

CPU Resources
  CPU utilization: Module             5 seconds       1 minute       5 minutes
                   1                   6% /  0%             6%              6%
                   4                   1% /  0%             2%              2%
                   5  RP              37% / 24%            39%             40%
                   5  SP              22% /  5%            21%             21%
                   7                   2% /  1%             2%              2%
  Processor memory: Module   Bytes:       Total           Used           %Used
                    1                 192743808      117181188             61%
                    4                 192743808       49932660             26%
                    5  RP             344375296      205893688             60%
                    5  SP             336606744      169022700             50%
                    7                 192743808       50474948             26%
  I/O memory: Module         Bytes:       Total           Used           %Used
              5  RP                    67108864       16487652             25%
              5  SP                    67108864       15767000             23%

EOBC Resources
  Module                     Packets/sec     Total packets     Dropped packets
  1          Rx:                      34       17786673953                   0
             Tx:                      29        1969346147                   3
  4          Rx:                      29       17786672808                   0
             Tx:                      23        1472955257                   7
  5  RP      Rx:                     158        7808831938                   0
             Tx:                     158        7690291506                   0
  5  SP      Rx:                      55        3215564127                   0
             Tx:                      62        3530334423                   0
  7          Rx:                      59       17786673582                   0
             Tx:                      53        1965180215                   3

VLAN Resources
  VLANs: 4094 total, 7 VTP, 249 extended, 13 internal, 3825 free

L2 Forwarding Resources
           MAC Table usage:   Module  Collisions  Total       Used       %Used
                              1                0  65536       3750          6%
                              5                0  65536      10508         16%

             VPN CAM usage:                       Total       Used       %Used
                                                    512          0          0%
L3 Forwarding Resources
             FIB TCAM usage:                     Total        Used       %Used
                  72 bits (IPv4, MPLS, EoM)     196608       39642         20%
                 144 bits (IP mcast, IPv6)       32768          18          1%

                     detail:      Protocol                    Used       %Used
                                  IPv4                       39640         20%
                                  MPLS                           1          1%
                                  EoM                            1          1%

                                  IPv6                          11          1%
                                  IPv4 mcast                     4          1%
                                  IPv6 mcast                     3          1%

            Adjacency usage:                     Total        Used       %Used
                                               1048576        9637          1%

     Forwarding engine load:
                     Module       pps   peak-pps                     peak-time
                     1         666459    3842773  11:51:11 EET Tue Dec 5 2017
                     5         791744   13408903  21:01:52 EET Sat Jul 22 2017

Netflow Resources
          TCAM utilization:       Module       Created      Failed       %Used
                                  1                  4           0          0%
                                  5                  4           0          0%
          ICAM utilization:       Module       Created      Failed       %Used
                                  1                  0           0          0%
                                  5                  0           0          0%

                 Flowmasks:   Mask#   Type        Features
                      IPv4:       0   reserved    none
                      IPv4:       1   Intf Des    Intf NDE L3 Feature
                      IPv4:       2   unused      none
                      IPv4:       3   reserved    none

                      IPv6:       0   reserved    none
                      IPv6:       1   unused      none
                      IPv6:       2   unused      none
                      IPv6:       3   reserved    none

CPU Rate Limiters Resources
             Rate limiters:       Total         Used      Reserved       %Used
                    Layer 3           9            4             1         44%
                    Layer 2           5            3             3         60%

ACL/QoS TCAM Resources
  Key: ACLent - ACL TCAM entries, ACLmsk - ACL TCAM masks, AND - ANDOR,
       QoSent - QoS TCAM entries, QOSmsk - QoS TCAM masks, OR - ORAND,
       Lbl-in - ingress label, Lbl-eg - egress label, LOUsrc - LOU source,
       LOUdst - LOU destination, ADJ - ACL adjacency

  Module ACLent ACLmsk QoSent QoSmsk Lbl-in Lbl-eg LOUsrc LOUdst  AND  OR  ADJ
  1          1%     3%     1%     1%     1%     1%     0%     3%   0%  0%   1%
  5          1%     3%     1%     1%     1%     1%     0%     3%   0%  0%   1%

L3 Multicast Resources
  IPv4 replication mode: ingress
  IPv6 replication mode: ingress
  Bi-directional PIM Designated Forwarder Table usage: 4 total, 0 (0%) used
  Replication capability: Module                              IPv4        IPv6
                          1                                 egress      egress
                          3                                ingress     ingress
                          4                                 egress      egress
                          5                                 egress      egress
                          7                                 egress      egress
  MET table Entries: Module                             Total    Used    %Used
                     1                                  65512       6       1%
                     5                                  65512       6       1%

QoS Policer Resources
  Aggregate policers: Module                      Total         Used     %Used
                      1                            1024            1        1%
                      5                            1024            1        1%
  Microflow policer configurations: Module        Total         Used     %Used
                                    1                64            1        1%
                                    5                64            1        1%

Switch Fabric Resources
  Bus utilization: current: 13%, peak was 30% at 17:29:09 EET Wed Nov 8 2017
  Fabric utilization:     Ingress                    Egress
    Module  Chanl  Speed  rate  peak                 rate  peak
    1       0        20G    6%   30% @23:56 31Dec17    8%   24% @20:58 23Nov17
    1       1        20G   24%   51% @20:31 25Jan18   10%   32% @19:56 05Jan18
    4       0        20G    3%   12% @19:52 05Nov17   14%   47% @21:49 08Nov17
    5       0        20G    1%    6% @12:40 24Jan17    1%    9% @09:22 28Dec17
    7       0        20G    4%   13% @16:07 05Jan18   12%   42% @20:45 12Nov17
    7       1        20G   17%   52% @21:02 12Nov17    9%   41% @00:29 18Jun17
  Switching mode: Module                                        Switching mode
                  1                                                       dcef
                  4                                                       acef
                  5                                                        bus
                  7                                                       acef

Interface Resources
  Interface drops:
    Module    Total drops:    Tx            Rx      Highest drop port:  Tx  Rx
    1                    1819226             0                           4   0
    3                  217017048         16911                          41  10
    4                 6969248528             0                           9   0
    7                 1506436275           456                          37  23

  Interface buffer sizes:
    Mod/Port                      Bytes:  Tx buffer       Rx buffer
    1/1                                    14622592         2068416
    1/2                                    14622592         2068416
    1/3                                    14622592         2068416
    1/4                                    14622592         2068416
    3/1                                     1081344          147456
    3/2                                     1081344          147456
    3/3                                     1081344          147456
    3/4                                     1081344          147456
    3/5                                     1081344          147456
    3/6                                     1081344          147456
    3/7                                     1081344          147456
    3/8                                     1081344          147456
    3/9                                     1081344          147456
    3/10                                    1081344          147456
    3/11                                    1081344          147456
    3/12                                    1081344          147456
    3/13                                    1081344          147456
    3/14                                    1081344          147456
    3/15                                    1081344          147456
    3/16                                    1081344          147456
    3/17                                    1081344          147456
    3/18                                    1081344          147456
    3/19                                    1081344          147456
    3/20                                    1081344          147456
    3/21                                    1081344          147456
    3/22                                    1081344          147456
    3/23                                    1081344          147456
    3/24                                    1081344          147456
    3/25                                    1081344          147456
    3/26                                    1081344          147456
    3/27                                    1081344          147456
    3/28                                    1081344          147456
    3/29                                    1081344          147456
    3/30                                    1081344          147456
    3/31                                    1081344          147456
    3/32                                    1081344          147456
    3/33                                    1081344          147456
    3/34                                    1081344          147456
    3/35                                    1081344          147456
    3/36                                    1081344          147456
    3/37                                    1081344          147456
    3/38                                    1081344          147456
    3/39                                    1081344          147456
    3/40                                    1081344          147456
    3/41                                    1081344          147456
    3/42                                    1081344          147456
    3/43                                    1081344          147456
    3/44                                    1081344          147456
    3/45                                    1081344          147456
    3/46                                    1081344          147456
    3/47                                    1081344          147456
    3/48                                    1081344          147456
    4/1                                     1221120          173504
    4/2                                     1221120          173504
    4/3                                     1221120          173504
    4/4                                     1221120          173504
    4/5                                     1221120          173504
    4/6                                     1221120          173504
    4/7                                     1221120          173504
    4/8                                     1221120          173504
    4/9                                     1221120          173504
    4/10                                    1221120          173504
    4/11                                    1221120          173504
    4/12                                    1221120          173504
    4/13                                    1221120          173504
    4/14                                    1221120          173504
    4/15                                    1221120          173504
    4/16                                    1221120          173504
    4/17                                    1221120          173504
    4/18                                    1221120          173504
    4/19                                    1221120          173504
    4/20                                    1221120          173504
    4/21                                    1221120          173504
    4/22                                    1221120          173504
    4/23                                    1221120          173504
    4/24                                    1221120          173504
    7/1                                     1221120          173504
    7/2                                     1221120          173504
    7/3                                     1221120          173504
    7/4                                     1221120          173504
    7/5                                     1221120          173504
    7/6                                     1221120          173504
    7/7                                     1221120          173504
    7/8                                     1221120          173504
    7/9                                     1221120          173504
    7/10                                    1221120          173504
    7/11                                    1221120          173504
    7/12                                    1221120          173504
    7/13                                    1221120          173504
    7/14                                    1221120          173504
    7/15                                    1221120          173504
    7/16                                    1221120          173504
    7/17                                    1221120          173504
    7/18                                    1221120          173504
    7/19                                    1221120          173504
    7/20                                    1221120          173504
    7/21                                    1221120          173504
    7/22                                    1221120          173504
    7/23                                    1221120          173504
    7/24                                    1221120          173504
    7/25                                    1221120          173504
    7/26                                    1221120          173504
    7/27                                    1221120          173504
    7/28                                    1221120          173504
    7/29                                    1221120          173504
    7/30                                    1221120          173504
    7/31                                    1221120          173504
    7/32                                    1221120          173504
    7/33                                    1221120          173504
    7/34                                    1221120          173504
    7/35                                    1221120          173504
    7/36                                    1221120          173504
    7/37                                    1221120          173504
    7/38                                    1221120          173504
    7/39                                    1221120          173504
    7/40                                    1221120          173504
    7/41                                    1221120          173504
    7/42                                    1221120          173504
    7/43                                    1221120          173504
    7/44                                    1221120          173504
    7/45                                    1221120          173504
    7/46                                    1221120          173504
    7/47                                    1221120          173504
    7/48                                    1221120          173504
IBC Resources
  Module                     Packets/sec     Total packets     Dropped packets
  5  RP      Rx:                    7314       28500044717            36729368
             Tx:                     195        5401242255                   0
  5  SP      Rx:                     116        3499801711               97353
             Tx:                    2962       87527142668                   0

SPAN Resources
  Source sessions: 16 maximum, 0 used
    Type                             Max      Used
    Local                              2(*)      0
    Local-tx                          14         0
    RSPAN source                       2(*)      0
    ERSPAN source                      2(*)      0
    Capture                            1(*)      0
    Service module                     1(*)      0
    OAM loopback                       1(*)      0
    Reflector                          1(*)      0
      * - shared source sessions and the total can not exceed 2
  Destination sessions: 64 maximum, 0 used
    Type                             Max      Used
    RSPAN destination                 64(*)      0
    ERSPAN destination                23(*)      0
      * - shared destination sessions and the total can not exceed 64

Multicast LTL Resources
  Usage:   55232 Total, 6963 Used
#

#sh ver
Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9-M), Version 15.1(2)SY7, RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Sun 13-Mar-16 07:31 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)SX7, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9-M), Version 15.1(2)SY7, RELEASE SOFTWARE (fc4)

Router uptime is 1 year, 21 weeks, 5 days, 12 hours, 10 minutes
Uptime for this control processor is 1 year, 21 weeks, 5 days, 11 hours, 24 minutes
System returned to ROM by reload at 00:59:06 EET Tue Aug 30 2016 (SP by reload)
System restarted at 01:01:54 EET Tue Aug 30 2016
System image file is "sup-bootdisk:s72033-adventerprisek9-mz.151-2.SY7.bin"
Last reload reason: Reload Command

cisco WS-C6509 (R7000) processor (revision 2.0) with 458720K/65536K bytes of memory.
Processor board ID SCA0417052N
SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache
Last reset from s/w reset
245 Virtual Ethernet interfaces
122 Gigabit Ethernet interfaces
4 Ten Gigabit Ethernet interfaces
1917K bytes of non-volatile configuration memory.

65536K bytes of Flash internal SIMM (Sector size 512K).
Configuration register is 0x2102

#

Суть все работало как часики в один момент прерывания поднялись с 1-3% до 25-30%, конфиг не менялся, можно сказать само по себе, но в шеститонниках такое не бывает...

Куда копать понять не могу...

Изменено 29 января, 2018 пользователем Mackiavelly

[vurd]

show ibc brief

[Mackiavelly]

6 minutes ago, vurd said:

show ibc brief

#show ibc brief
=========================== Inband counters and statistics ===========================
Interface information:
        Interface IBC0/0(idb 0x48B7F638)
        5 minute  rx rate 9124000 bits/sec  7260 packets/sec
        5 minute  tx rate 116000 bits/sec  188 packets/sec
        28527416707 Packets input, 3398136801082 bytes
        0 broadcasts received
        5402096384 Packets outputs, 630114855984 bytes
        541867908 broadcasts sent
        36729368 Inband input packet drops
        0 Inband output packet drops
        2 IBC resets

        *** Inband Generic Counters***
        0 Packets Input, 0 Bytes
        0 Broadcasts Input
        0 Packets Output
        0 Total Drops
        0 software bridged paks
        0 Packets Fast switching paks
        Potential/Actual paks CEF switched 0/0
        Potential/Actual paks tag CEF switched 0/0
        0 packets xconnect CEF switched
        0 Packets xconnect L2 switched
        0 packets xconnect dropped
        0 Packets immediately punted
        Potential/Actual paks copied to process level 0/0
        Potential/Actual paks copied to L2 process level 0/0
        0 Packets L2 processed
        0 packets L2 switched
        0 Encapsulation corrections
        Internally intercepted packets:
         0 exc, 0 dindex (0 sp, 0 old index), 0 cap1, 0 cap2
        Earl8 shimmed paks rcvd:
         0 pseudoipv6 0 svchdr 0 shim

        *** INBAND PACKET DROPS & FAILURES ***:
          0 runt 0 length error
          0 shim_processing 0 shim_error
          0 no idb 0 get_l2_idb failures 0 if_input changes
          0 intercept 0 spd
          0 Number of times intercept function(s) returning incorrect values
          0 bridge loopback 0 diag
          0 Rx packets with unicast IP and multicast MAC
          0 fastsend 0 soutput 0 no mac descriptors 0 NULL if_outputs
          0 throttle 0 throttle sneaks
          0 encapsulation 0 get_lif
          0 if_input corrections 0 uncorrectable if_inputs
          0 corr ecc errors
          0 null parse packet
          0 shim insertions attempted at tx, 0 failed
          0 times les cef vector not set
          0 times les cef vector disabled
        packet drop trace:
Intercept Vectors:
Inband Intercepts:


#

 

[vurd]

9 мегабит сыпется на цпу, делайте миррор с цпу и смотрите что там летит. Вангую мультикаст.

[Mackiavelly]

23 минуты назад, vurd сказал:

9 мегабит сыпется на цпу, делайте миррор с цпу и смотрите что там летит. Вангую мультикаст.

Тоесть скорее всего, хомячек шлет на шелезку мультикаст?

[vurd]

22 минуты назад, Mackiavelly сказал:

Тоесть скорее всего, хомячек шлет на шелезку мультикаст?

Я без понятия, что там и куда у вас шлет. Нужно снять дамп. Снимите - всё узнаете.

 

monitor session 3 type local
 source cpu rp both
 destination interface gigabitEthernet 4/48
 no shutdown
!

 

[Mackiavelly]

1 час назад, vurd сказал:

Я без понятия, что там и куда у вас шлет. Нужно снять дамп. Снимите - всё узнаете.

 

monitor session 3 type local
 source cpu rp both
 destination interface gigabitEthernet 4/48
 no shutdown
!

 

Спасибо, нашел и пристрелил, трафик генерил dlink роутер который сошел с ума и слал кучу запросов dhcpv6

Frame 278476: 161 bytes on wire (1288 bits), 161 bytes captured (1288 bits) on interface 0
    Interface id: 0 (\Device\NPF_{943AC067-18F4-4A96-A094-1F786715D536})
        Interface name: \Device\NPF_{943AC067-18F4-4A96-A094-1F786715D536}
    Encapsulation type: Ethernet (1)
    Arrival Time: Jan 29, 2018 15:43:40.009917000 Финляндия (зима)
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1517233420.009917000 seconds
    [Time delta from previous captured frame: 0.000002000 seconds]
    [Time delta from previous displayed frame: 0.000002000 seconds]
    [Time since reference or first frame: 68.972064000 seconds]
    Frame Number: 278476
    Frame Length: 161 bytes (1288 bits)
    Capture Length: 161 bytes (1288 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ipv6:udp:dhcpv6]
    [Coloring Rule Name: UDP]
    [Coloring Rule String: udp]
Ethernet II, Src: D-LinkIn_c8:4e:69 (90:8d:78:c8:4e:69), Dst: IPv6mcast_01:00:02 (33:33:00:01:00:02)
    Destination: IPv6mcast_01:00:02 (33:33:00:01:00:02)
        Address: IPv6mcast_01:00:02 (33:33:00:01:00:02)
        .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: D-LinkIn_c8:4e:69 (90:8d:78:c8:4e:69)
        Address: D-LinkIn_c8:4e:69 (90:8d:78:c8:4e:69)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv6 (0x86dd)
Internet Protocol Version 6, Src: fe80::928d:78ff:fec8:4e69, Dst: ff02::1:2
    0110 .... = Version: 6
    .... 0000 0000 .... .... .... .... .... = Traffic Class: 0x00 (DSCP: CS0, ECN: Not-ECT)
        .... 0000 00.. .... .... .... .... .... = Differentiated Services Codepoint: Default (0)
        .... .... ..00 .... .... .... .... .... = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    .... .... .... 0000 0000 0000 0000 0000 = Flow Label: 0x00000
    Payload Length: 107
    Next Header: UDP (17)
    Hop Limit: 64
    Source: fe80::928d:78ff:fec8:4e69
    Destination: ff02::1:2
    [Source SA MAC: D-LinkIn_c8:4e:69 (90:8d:78:c8:4e:69)]
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 546, Dst Port: 547
    Source Port: 546
    Destination Port: 547
    Length: 107
    Checksum: 0xbd69 [unverified]
    [Checksum Status: Unverified]
    [Stream index: 1164]
DHCPv6
    Message type: Solicit (1)
    Transaction ID: 0x870fcf
    Elapsed time
        Option: Elapsed time (8)
        Length: 2
        Value: ffff
        Elapsed time: 655350ms
    Option Request
        Option: Option Request (6)
        Length: 16
        Value: 00150016001700180038001600400043
        Requested Option code: SIP Server Domain Name List (21)
        Requested Option code: SIP Servers IPv6 Address List (22)
        Requested Option code: DNS recursive name server (23)
        Requested Option code: Domain Search List (24)
        Requested Option code: NTP Server (56)
        Requested Option code: SIP Servers IPv6 Address List (22)
        Requested Option code: Dual-Stack Lite AFTR Name (64)
        Requested Option code: Prefix Exclude (67)
    Client Identifier
        Option: Client Identifier (1)
        Length: 10
        Value: 00030001908d78c84e69
        DUID: 00030001908d78c84e69
        DUID Type: link-layer address (3)
        Hardware type: Ethernet (1)
        Link-layer address: 90:8d:78:c8:4e:69
    Reconfigure Accept
        Option: Reconfigure Accept (20)
        Length: 0
    Fully Qualified Domain Name
        Option: Fully Qualified Domain Name (39)
        Length: 15
        Value: 000c446c696e6b2d526f7574657200
        0000 0... = Reserved: 0x00
        .... .0.. = N bit: Server should perform DNS updates
        .... ..0. = O bit: Server has not overridden client's S bit preference
        .... ...0 = S bit: Server should not perform forward DNS updates
        Client FQDN: Dlink-Router
    Identity Association for Non-temporary Address
        Option: Identity Association for Non-temporary Address (3)
        Length: 12
        Value: 000000010000000000000000
        IAID: 00000001
        T1: 0
        T2: 0
    Identity Association for Prefix Delegation
        Option: Identity Association for Prefix Delegation (25)
        Length: 12
        Value: 000000010000000000000000
        IAID: 00000001
        T1: 0
        T2: 0

я его мак покаместь просто дропаю с интерфейса, щас буду думать как в дальнейшем отсреливать таких товарищей

[zhenya`]

netdr) 

[myst]

20 часов назад, Mackiavelly сказал:

я его мак покаместь просто дропаю с интерфейса, щас буду думать как в дальнейшем отсреливать таких товарищей

ну воркараунд выше дали, почти полностью универсальный для подобных случаев - CoPP

[Mackiavelly]

А как вообще заблокировать весь ipv6 c определенного влан интерфейса, пробывал через ACL ipv6 не помогло

IPv6 access list ipv6-deny
    deny ipv6 any any sequence 10
    deny tcp any any sequence 20
    deny udp any any sequence 30
    deny icmp any any sequence 40

вешал на IN(да да, я в курсе что в конце листа есть общий deny), все равно данный длинк роутер уваливался ко мне на проц