AlexandraL0505 Posted September 20 · Report post Все привет! уже приличное время копаюсь и не могу найти проблему, что так жрет CPU на 6506. show processes cpu detailed sorted 5s | exclude 0.0 CPU utilization for five seconds: 88%/73%; one minute: 79%; five minutes: 76% PID/TID 5Sec 1Min 5Min Process Prio STATE CPU 16406 82.8% 73.7% 71.0% ios-base 91d12h 6 61.3% 54.2% 51.6% 21 Intr 54d20h 1 7.7% 2.9% 2.1% 10 Receive 96.117 7 4.0% 3.2% 3.2% 22 Intr 6d03h 18 2.6% 1.4% 1.7% 10 Receive 64.390 14 2.0% 2.3% 2.3% 10 Receive 40.148 13 1.7% 1.8% 1.9% 10 Reply 71.391 21 1.1% 2.5% 2.0% 10 Receive 89.288 8 0.7% 0.8% 0.8% 23 Intr 1d21h 2 0.6% 0.9% 1.0% 5 Ready 3d09h 10 0.6% 0.8% 0.6% 10 Reply 44.931 22 0.4% 1.7% 1.7% 10 Receive 21m20s 11 0.1% 0.1% 0.1% 10 Receive 7h33m Process sbin/ios-base, type IOS, PID = 16406 CPU utilization for five seconds: 9%/73%; one minute: 8%; five minutes: 8% Task Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Prio Task Name 568 200851218 312947570 641 1.25% 1.01% 1.30% 0 M IP SNMP 384 120386427 31852908 3779 0.71% 0.71% 0.71% 0 L CEF: IPv4 proces 4 72323354 397624140 181 0.53% 0.49% 0.46% 0 M Service Task 323 562457721175116890 47 0.50% 0.71% 0.55% 0 M IP Input 139 32658493 18558323 1759 0.31% 0.23% 0.22% 0 M IPC LC Message H 565 13479722 133440646 101 0.23% 0.13% 0.13% 0 M IPv6 Input 319 33647505 391760446 85 0.21% 0.26% 0.24% 0 M network_out Svc 45 32943232 32320097 1019 0.19% 0.19% 0.19% 0 M GraphIt 148 19596945 3684109 5319 0.11% 0.11% 0.11% 0 M Compute load avg PID/TID 5Sec 1Min 5Min Process Prio STATE CPU 16425 2.5% 2.6% 2.5% tcp.proc 2d21h 4 2.1% 1.6% 1.4% 10 Receive 10h58m 7 0.3% 0.9% 1.1% 10 Receive 16h01m PID/TID 5Sec 1Min 5Min Process Prio STATE CPU 16431 0.6% 0.6% 0.5% iprouting.iosproc 1d08h 1 0.3% 0.2% 0.1% 10 Receive 4h58m 9 0.2% 0.2% 0.2% 10 Receive 5m56s 3 0.1% 0.2% 0.1% 10 Receive 5h34m show platform hardware capacity System Resources PFC operating mode: PFC3C Supervisor redundancy mode: administratively sso, operationally sso Switching resources: Sw/Mod Part number Series CEF mode 1/1 WS-X6708-10GE CEF720 dCEF 1/2 WS-X6724-SFP CEF720 CEF 1/3 WS-X6724-SFP CEF720 CEF 1/5 VS-S720-10G supervisor CEF 2/1 WS-X6708-10GE CEF720 dCEF 2/2 WS-X6724-SFP CEF720 CEF 2/5 VS-S720-10G supervisor CEF Power Resources switch 1 Power supply redundancy mode: administratively redundant operationally redundant System power: 2771W, 0W (0%) inline, 1512W (55%) total allocated Powered devices: 0 total, 0 Class3, 0 Class2, 0 Class1, 0 Class0, 0 Cisco Power Resources switch 2 Power supply redundancy mode: administratively redundant operationally redundant System power: 2771W, 0W (0%) inline, 1386W (50%) total allocated Powered devices: 0 total, 0 Class3, 0 Class2, 0 Class1, 0 Class0, 0 Cisco Flash/NVRAM Resources Use: Sw/Mod Device Bytes: Total Used %Used 1/5 RP bootflash: 65536000 458556 1% 1/5 SP disk0: 1024573440 12763136 1% 1/5 SP sup-bootdisk: 1024557056 242909184 24% 1/5 SP const_nvram: 129004 11096 9% 1/5 SP nvram: 1964024 572308 29% 2/5 slavenvram: 1964024 571284 29% 2/5 slaveconst_nvram: 129004 11096 9% 2/5 slavedisk0: 1024573440 0 0% 2/5 slavesup-bootdisk: 1024557056 242909184 24% 2/5 slavebootflash: 65536000 507084 1% CPU Resources CPU utilization: Sw/Mod 5 seconds 1 minute 5 minutes 1/1 5% / 4% 7% 7% 1/2 5% / 6% 6% 6% 1/3 0% / 2% 1% 0% 1/5 RP 4% / 56% 7% 8% 1/5 SP 7% / 8% 8% 9% 2/1 10% / 4% 7% 7% 2/2 9% / 7% 10% 9% 2/5 RP 2% / 2% 1% 1% 2/5 SP 5% / 7% 7% 7% Processor memory: Sw/Mod Bytes: Total Used %Used 1/1 1073741824 247468032 23% 1/2 268435456 203055104 76% 1/3 268435456 203063296 76% 1/5 RP 1073741824 418324480 39% 1/5 SP 1073741824 473317376 44% 2/1 1073741824 246771712 23% 2/2 268435456 202440704 75% 2/5 RP 1073741824 404062208 38% 2/5 SP 1073741824 472711168 44% I/O memory: Sw/Mod Bytes: Total Used %Used 1/1 11796240 11442472 97% 1/2 8126344 7886544 97% 1/3 8126344 7886544 97% 1/5 RP 12058384 10715832 89% 1/5 SP 8388608 8104784 97% 2/1 11796240 11442472 97% 2/2 8126344 7886544 97% 2/5 RP 12058384 10715832 89% 2/5 SP 8388608 8104304 97% EOBC Resources Sw/Mod Packets/sec Total packets Dropped packets 1/1 Rx: 59 6466085028 0 Tx: 55 1118230862 4 1/2 Rx: 14 6466090476 0 Tx: 8 397731764 4 1/3 Rx: 8 6466090477 0 Tx: 3 324700703 4 1/5 RP Rx: 173 2914355455 6 Tx: 116 2056019752 0 1/5 SP Rx: 94 1676899658 1 Tx: 142 2569525402 0 2/1 Rx: 54 3192422344 0 Tx: 57 1071247309 4 2/2 Rx: 12 3192427007 0 Tx: 12 479493963 4 2/5 RP Rx: 0 108782189 0 Tx: 0 43355575 0 2/5 SP Rx: 75 1594148592 1 Tx: 76 1598435373 0 VLAN Resources VLANs: 4094 total, 180 VTP, 33 extended, 20 internal, 3861 free L2 Forwarding Resources MAC Table usage: Sw/Mod Collisions Total Used %Used 1/1 0 98304 2695 3% 1/5 0 98304 2691 3% 2/1 0 98304 2699 3% 2/5 0 98304 2697 3% VPN CAM usage: Total Used %Used 512 0 0% L3 Forwarding Resources FIB TCAM usage: Total Used %Used 72 bits (IPv4, MPLS, EoM) 196608 3330 2% 144 bits (IP mcast, IPv6) 32768 22 1% detail: Protocol Used %Used IPv4 3330 2% MPLS 0 0% EoM 0 0% IPv6 15 1% IPv4 mcast 4 1% IPv6 mcast 3 1% Adjacency usage: Total Used %Used 1048576 3532 1% Forwarding engine load: Sw/Mod pps peak-pps peak-time 1/1 479000 1934937 18:28:48 MSK Fri Aug 16 2024 1/5 43818 356800 02:22:26 MSK Sun Aug 25 2024 2/1 636174 1872379 11:50:02 MSK Thu Aug 15 2024 2/5 15656 138111 09:39:25 MSK Thu Jul 4 2024 Netflow Resources TCAM utilization: Sw/Mod Created Failed %Used 1/1 13914 0 10% 1/5 41 0 0% 2/1 13653 0 10% 2/5 23 0 0% ICAM utilization: Sw/Mod Created Failed %Used 1/1 0 0 0% 1/5 0 0 0% 2/1 0 0 0% 2/5 0 0 0% Flowmasks: Mask# Type Features IPv4: 0 reserved none IPv4: 1 Intf Ful NAT_INGRESS NAT_EGRESS FM_GUARDIAN IPv4: 2 unused none IPv4: 3 reserved none IPv6: 0 reserved none IPv6: 1 unused none IPv6: 2 unused none IPv6: 3 reserved none CPU Rate Limiters Resources Rate limiters: Total Used Reserved %Used Layer 3 9 5 1 56% Layer 2 5 3 3 60% ACL/QoS TCAM Resources Key: ACLent - ACL TCAM entries, ACLmsk - ACL TCAM masks, AND - ANDOR, QoSent - QoS TCAM entries, QOSmsk - QoS TCAM masks, OR - ORAND, Lbl-in - ingress label, Lbl-eg - egress label, LOUsrc - LOU source, LOUdst - LOU destination, ADJ - ACL adjacency Sw/Mod ACLent ACLmsk QoSent QoSmsk Lbl-in Lbl-eg LOUsrc LOUdst AND OR ADJ 1/1 40% 94% 1% 1% 2% 2% 50% 47% 6% 0% 1% 1/5 40% 94% 1% 1% 2% 2% 50% 47% 6% 0% 1% 2/1 40% 93% 1% 1% 2% 2% 50% 47% 6% 0% 1% 2/5 40% 93% 1% 1% 2% 2% 50% 47% 6% 0% 1% L3 Multicast Resources IPv4 replication mode: egress IPv6 replication mode: egress Bi-directional PIM Designated Forwarder Table usage: 4 total, 0 (0%) used Replication capability: Sw/Mod IPv4 IPv6 1/1 egress egress 1/2 egress egress 1/3 egress egress 1/5 egress egress 2/1 egress egress 2/2 egress egress 2/5 egress egress MET table Entries: Sw/Mod Total Used %Used 1/1 65516 6 1% 1/5 65516 8 1% 2/1 65516 6 1% 2/5 65516 6 1% QoS Policer Resources Aggregate policers: Sw/Mod Total Used %Used 1/1 1024 1 1% 1/5 1024 1 1% 2/1 1024 1 1% 2/5 1024 1 1% Microflow policer configurations: Sw/Mod Total Used %Used 1/1 64 1 1% 1/5 64 1 1% 2/1 64 1 1% 2/5 64 1 1% Switch Fabric Resources: Switch 1 Bus utilization: current: 0%, peak was 3% at 13:42:37 MSK Mon Aug 19 2024 Fabric utilization: Ingress Egress Module Chanl Speed rate peak rate peak 1 0 20G 6% 60% @10:30 03Sep24 0% 63% @01:25 06Sep24 1 1 20G 0% 99% @18:38 16Aug24 5% 58% @10:43 06Sep24 2 0 20G 0% 10% @19:19 15Sep24 2% 68% @18:38 16Aug24 3 0 20G 0% 0% 0% 0% 5 0 20G 0% 49% @13:25 15Aug24 0% 46% @14:14 23Apr24 5 1 20G 0% 9% @02:29 07Sep24 0% 19% @15:01 17Sep24 Switching mode: Module Switching mode 1 dcef 2 acef 3 acef 5 dcef Switch Fabric Resources: Switch 2 Bus utilization: current: 0%, peak was 3% at 17:11:52 MSK Mon Aug 19 2024 Fabric utilization: Ingress Egress Module Chanl Speed rate peak rate peak 1 0 20G 35% 99% @10:08 15Jul24 0% 49% @05:17 06Sep24 1 1 20G 2% 99% @17:11 19Aug24 25% 57% @23:15 15Sep24 2 0 20G 0% 10% @18:38 15Sep24 2% 70% @12:59 13Aug24 5 0 20G 0% 46% @14:27 23Apr24 10% 99% @13:26 15Aug24 5 1 20G 0% 0% 0% 18% @08:52 09Sep24 Switching mode: Module Switching mode 1 dcef 2 acef 5 dcef Interface Resources Interface drops: Sw/Mod Total drops: Tx Rx Highest drop port: Tx Rx 1/1 18381445 0 1 0 1/2 348690507 0 7 0 2/1 167962783 0 1 0 2/2 2541104421 0 7 0 2/5 238417575 0 5 0 Interface buffer sizes: Sw/Mod Bytes: Tx buffer Rx buffer 1/1 (asic-1) 91889216 109296640 1/2 (asic-1) 1221120 173504 1/3 (asic-1) 1221120 173504 1/5 (asic-1) 8640448 10127232 1/5 (asic-2) 91887168 109293568 2/1 (asic-1) 91889216 109296640 2/2 (asic-1) 1221120 173504 2/5 (asic-1) 8640448 10127232 2/5 (asic-2) 91887168 109293568 IBC Resources Sw/Mod Packets/sec Total packets Dropped packets 1/5 RP Rx: 36657 261437506508 454884 Tx: 36448 259097495636 0 1/5 SP Rx: 126 2258634933 0 Tx: 123 1896726725 0 2/5 RP Rx: 0 6713952 0 Tx: 0 51334451 0 2/5 SP Rx: 119 2134308737 0 Tx: 124 2236271525 0 SPAN Resources Source sessions: 16 maximum, 1 used Type Max Used Local 2(*) 0 Local-tx 14 0 RSPAN source 2(*) 1 ERSPAN source 2(*) 0 Capture 1(*) 0 Service module 1(*) 0 OAM loopback 1(*) 0 * - shared source sessions and the total can not exceed 2 Destination sessions: 64 maximum, 0 used Type Max Used RSPAN destination 64(*) 0 ERSPAN destination 23(*) 0 * - shared destination sessions and the total can not exceed 64 Multicast LTL Resources Usage: 20416 Total, 11866 Used show ibc brief Interface information: Interface IBC0/0(idb 0x60011A60) Hardware is Mistral IBC (revision 5) 5 minute rx rate 145408000 bits/sec, 37309 packets/sec 5 minute tx rate 145110000 bits/sec, 37105 packets/sec 261423941108 packets input, 166677484970411 bytes 0 broadcasts received 259084007824 packets output, 166158660155233 bytes 121749865 broadcasts sent 454872 Inband input packet drops 0 Bridge Packet loopback drops 11113 Rx packets dropped with Multicast MAC and Unicast IP 1680443003 Packets out of 2213189326 CEF Switched, 0 Packets out of 0 Tag CEF Switched 28131387 Packets Fast Switched 0 Giant Tx packets dropped 0 Packets SLB Switched, 0 Packets CWAN Switched Potential/Actual paks copied to process level 1812134565/1811901186 (233379 dropped, 144037 spd drops) 918540091 paks copied at vlan soutput 1862 paks copied at ibc fast soutput 0 paks copied at ibc fastsend 3392059567 inband interrupts 55330098 transmit ring cleanups 586891549 ibl inputs 55330114 total tx interrupts set 55330114 tx ints due to packets outstanding 0 tx ints due low free buffers in pool 0 tx ints due to application setting tx dma done batch size=32 buffers free minimum before tx int=4 mistral ran out of tx descriptors 0 times mistral tx interrupt inconsisteny occured 0 times Label switched pkts dropped: 0 Xconnect pkts processed: 0, dropped: 0 Xconnect pkt reflection drops: 0 IBC resets = 4; last at 16:05:16.372 MSK Thu Mar 14 2024 MISTRAL ERROR COUNTERS System address timeouts = 0 BUS errors = 0 IBC Address timeouts = 0 (addr 0x0) Page CRC errors = 0 IBL CRC errors = 0 ECC Correctable errors = 0 Packets with padding removed (297/6/9) = 312 Packets expanded (977940376/0) = 977940376 Packets attempted tail end expansion > 1 page and were dropped = 0 IP packets dropped with frag offset of 1 = 0 89342 total packets dropped on throttled interfaces (89089 low, 253 medium, 0 high) 0 medium, 0 high prio packets allowed in due to selective throttling 0 packets corrected for if_input 0 packets were uncorrectable for if_input 0 exception packets dropped with service acceleration enabled Hazard Illegal packet length = 0 Illegal Offset = 0 Hazard Packet underflow = 0 Packet Overflow = 0 IBL fill hang count = 0 Unencapsed packets = 0 LBIC RXQ Drop pkt count = 23615 LBIC drop pkt count = 0 LBIC Drop pkt stick = 0 Думала в SPAN трафик отзеркалить, но на этой циске нету monitor session ... source cpu rp и sp: monitor session 2 source ? interface SPAN source interface remote SPAN source Remote switch Switch number of IDS module vlan SPAN source VLAN Трафик конечно не маленький, но раньше CPU максимум был в 40%. А потом в один прекрасный момент стал упираться в полочку 90-100%. что-то с этим надо делать ... Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
jffulcrum Posted September 20 · Report post Не видно версии IOS в вашей простыне, напишите, некоторые уцелевшие "мангаловоды" умеют кодировать от запоя по фотографии подсказать сразу по версии. 1 час назад, AlexandraL0505 сказал: но на этой циске нету monitor session ... source cpu rp и sp: Сталкивался с таким, там есть геморный метод, по show red (вроде бы...может show mod)определить, на каком слоте суп висит и хватать трафик с него через monitor на пустом (не подключенном) интерфейсе. Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
AlexandraL0505 Posted September 20 · Report post Цитата Не видно версии IOS в вашей простыне show ver Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-VM), Version 12.2(33)SXI5, RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2010 by Cisco Systems, Inc. Compiled Sat 23-Oct-10 17:57 by prod_rel_team ROM: System Bootstrap, Version 12.2(17r)SX7, RELEASE SOFTWARE (fc1) ******** uptime is 3 years, 29 weeks, 4 days, 6 hours, 2 minutes Uptime for this control processor is 27 weeks, 20 hours, 25 minutes Time since ******** switched to active is 27 weeks, 19 hours, 53 minutes System returned to ROM by Stateful Switchover at 15:28:50 MSK Thu Mar 14 2024 (SP by reload) System restarted at 15:35:22 MSK Thu Mar 14 2024 System image file is "sup-bootdisk:s72033-ipservicesk9_wan-vz.122-33.SXI5.bin" Last reload reason: Reason unspecified cisco WS-C6506-E (R7000) processor (revision 1.2) with 1040384K/8192K bytes of memory. Processor board ID SAL15119EGU SR71000 CPU at 600Mhz, Implementation 1284, Rev 1.2, 512KB L2 Cache Last reset from s/w reset 162 Virtual Ethernet interfaces 78 Gigabit Ethernet interfaces 20 Ten Gigabit Ethernet interfaces 1917K bytes of non-volatile configuration memory. 65536K bytes of Flash internal SIMM (Sector size 512K). Configuration register is 0x2102 Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
jffulcrum Posted September 20 · Report post 14 минут назад, AlexandraL0505 сказал: 12.2(33)SXI5 Странно, SXI ветка, и нет source cpu в monitor. По версии - у вас NetFlow собирается? Я помню, что на таких старых версиях сидели из-за багов с NetFlow в последующих. Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
AlexandraL0505 Posted September 20 · Report post нет не настроен сбор netflow но есть кэш, если это может нести какую-то инфу show ip cache flow ------------------------------------------------------------------------------- Displaying software-switched flow entries on the MSFC in Module 1/5: IP packet size distribution (14303673 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .004 .987 .001 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .000 .000 .000 .001 .004 .000 .000 .000 .000 .000 .000 IP Flow Switching Cache, 278544 bytes 1 active, 4095 inactive, 293927 added 14826614 ager polls, 0 flow alloc failures Active flows timeout in 1 minutes Inactive flows timeout in 15 seconds IP Sub Flow Cache, 33992 bytes 0 active, 1024 inactive, 0 added, 0 added to flow 0 alloc failures, 0 force free 1 chunk, 0 chunks added last clearing of statistics 23w1d Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec) -------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow TCP-other 15310 0.0 11 736 0.0 0.1 1.5 UDP-other 330 0.0 2 253 0.0 5.7 15.5 ICMP 278251 0.0 50 84 3.2 49.7 3.0 IGMP 36 0.0 2 32 0.0 0.6 15.4 Total: 293927 0.0 48 91 3.3 47.1 2.9 Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
nixx Posted September 20 · Report post ios достаточно древний, я с ~14 года на 15й версии сидел. стандартное начало поисков - сдампить трафик процессора и разбираться в пакетах, что лишнее, а что не очень (ну и резать лишнее соответственно, причем лучше ДО циски) debug netdr capture rx show netdr captured-packets ну и откуда у вас 99% занятость каналов кроссбара вылетала? я понимаю, что это был июль-август, но тем не менее, может там и щас колбасится в районе 98%. Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
AlexandraL0505 Posted September 23 · Report post Цитата debug netdr capture rx show netdr captured-packets за пару секунд дебага насыпало ~4000 пакетов. Т.е. они все летят на цп? лишнее резать будет занятно)) Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
jffulcrum Posted September 23 · Report post Удивительно, как можно было дожить на мангале, не имея настроенного COPP. Можно прямо с примера https://forum.nag.ru/index.php?/topic/97814-cat-6500-trablshuting/ начать Вставить ник Quote Ответить с цитированием Share this post Link to post Share on other sites More sharing options...
