_J_ Posted September 3, 2018 Posted September 3, 2018 Всем привет. Есть коммутатор WS-C6509-E с одним движком VS-S720-10G. Mod Ports Card Type Model Serial No. --- ----- -------------------------------------- ------------------ ----------- 3 48 CEF720 48 port 1000mb SFP WS-X6748-SFP SAL1048936F 4 8 CEF720 8 port 10GE with DFC WS-X6708-10GE SAL17257D8J 5 5 Supervisor Engine 720 10GE (Active) VS-S720-10G SAL13442UP3 7 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL1029VG8K 8 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL12458ZYL Mod MAC addresses Hw Fw Sw Status --- ---------------------------------- ------ ------------ ------------ ------- 3 0019.5500.bda4 to 0019.5500.bdd3 1.8 12.2(14r)S5 12.2(33)SXI6 Ok 4 e4d3.f1a3.e818 to e4d3.f1a3.e81f 3.6 12.2(18r)S1 12.2(33)SXI6 Ok 5 0021.a094.b538 to 0021.a094.b53f 3.1 8.5(3) 12.2(33)SXI6 Ok 7 0018.18c6.c940 to 0018.18c6.c96f 2.4 12.2(14r)S5 12.2(33)SXI6 Ok 8 0021.a096.d3a0 to 0021.a096.d3cf 3.0 12.2(18r)S1 12.2(33)SXI6 Ok Mod Sub-Module Model Serial Hw Status ---- --------------------------- ------------------ ----------- ------- ------- 3 Centralized Forwarding Card WS-F6700-CFC SAD104402S8 3.0 Ok 4 Distributed Forwarding Card WS-F6700-DFC3CXL SAL17257R8W 1.8 Ok 5 Policy Feature Card 3 VS-F6K-PFC3CXL SAL13431WZG 1.2 Ok 5 MSFC3 Daughterboard VS-F6K-MSFC3 SAL13442JNA 2.0 Ok 7 Centralized Forwarding Card WS-F6700-CFC SAL1223T0R5 4.0 Ok 8 Distributed Forwarding Card WS-F6700-DFC3BXL SAL1124R668 5.3 Ok В него установлена линейная плата WS-X6748-GE-TX (48 медных гиговых порта) с DFC Mod Ports Card Type Model Serial No. --- ----- -------------------------------------- ------------------ ----------- 8 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL12458ZYL Mod MAC addresses Hw Fw Sw Status --- ---------------------------------- ------ ------------ ------------ ------- 8 0021.a096.d3a0 to 0021.a096.d3cf 3.0 12.2(18r)S1 12.2(33)SXI6 Ok Mod Sub-Module Model Serial Hw Status ---- --------------------------- ------------------ ----------- ------- ------- 8 Distributed Forwarding Card WS-F6700-DFC3BXL SAL1124R668 5.3 Ok Mod Online Diag Status ---- ------------------- 8 Pass И в этом восьмом модуле наблюдаются потери до других модулей при суммарном трафике через модуль более 20Мбит/с. При переносе модуля в девятый слот картина не меняется, но вот в седьмом всё отлично. Получается во фрейме глючные слоты, в данном случае 8 и 9? Или чтото-гдето можно посмотреть/крутануть? Вставить ник Quote
VolanD666 Posted September 4, 2018 Posted September 4, 2018 А другого модуля нет проверить в этих слотах? Вставить ник Quote
_J_ Posted September 5, 2018 Author Posted September 5, 2018 Есть, проверял. Устанавливал плату как с cfc, так и с dfc. Ситуация повторяется. Дело не в модуле явно. Также заметил, что наблюдаются потери даже в самом модуле, при пропуске трафика с первого на второй порт. Работала железка давно и вот просто так, без каких-либо механических воздействий, начала дропать трафик в слотах. Вставить ник Quote
zhenya` Posted September 5, 2018 Posted September 5, 2018 может попробовать diagnostic bootup level complete? и hw-module module 8 reset ну и show counters interface GigabitEthernet для какого-нить проблемного порта. Вставить ник Quote
_J_ Posted September 5, 2018 Author Posted September 5, 2018 9 часов назад, zhenya` сказал: может попробовать diagnostic bootup level complete? и hw-module module 8 reset ну и show counters interface GigabitEthernet для какого-нить проблемного порта. Там немного другая команда у меня - diagnostic start module 8 test complete Скрытый текст Sep 5 14:22:33.718 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestFibDevices{ID=13} has completed successfully Sep 5 14:22:33.718 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestIPv4FibShortcut{ID=14} ... Sep 5 14:22:34.026 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:22:41.527 MSK: %SYS-3-CPUHOG: Task is running for (2045)msecs, more than (2000)msecs (0/0),process = L3 Manager.(s72033_rp-adventerprisek9_wan-8-dso-b.so+0x277FE0) ([30:-9]6+0x3CF7BC) ([30:-9]11+0x4D4B04) ([31:-8]-+0x4D475C) ([40:0]+0x4D71D0) ([30:-9]3+0x544DF4) ([40:0]+0x544C50) ([30:-9]14+0x2700E8) ([41:0]+0x2700C0) ([41:0]+0x2700E8) ([41:0]+0x2700C0) Sep 5 14:22:43.879 MSK: %SYS-3-CPUHOG: Task is running for (4020)msecs, more than (2000)msecs (0/0),process = L3 Manager.(s72033_rp-adventerprisek9_wan-1-dso-b.so+0x4D717C) ([30:-9]3+0x544DF4) ([40:0]+0x544C50) ([30:-9]14+0x2700E8) ([41:0]+0x2700C0) ([30:-9]3+0x544DF4) ([40:0]+0x544C50) ([30:-9]14+0x2700E8) ([41:0]+0x2700C0) ([41:0]+0x2700E8) ([41:0]+0x2700C0) Sep 5 14:22:45.879 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:22:45.922 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:22:46.798 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestIPv4FibShortcut{ID=14} has completed successfully Sep 5 14:22:46.798 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestIPv6FibShortcut{ID=15} ... Sep 5 14:22:47.055 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:22:47.021 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:22:50.675 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:22:50.662 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:22:54.725 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestIPv6FibShortcut{ID=15} has completed successfully Sep 5 14:22:54.725 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestNATFibShortcut{ID=17} ... Sep 5 14:22:54.795 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:22:54.757 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:22:59.039 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:22:59.022 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:00.123 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:00.067 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestNATFibShortcut{ID=17} has completed successfully Sep 5 14:23:00.067 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestMPLSFibShortcut{ID=18} ... Sep 5 14:23:00.115 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:03.987 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:03.956 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:05.019 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:04.891 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestMPLSFibShortcut{ID=18} has completed successfully Sep 5 14:23:04.891 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestL3Capture{ID=19} ... Sep 5 14:23:04.999 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:09.675 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:09.643 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:10.419 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestL3Capture{ID=19} has completed successfully Sep 5 14:23:10.419 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestL3VlanMet{ID=20} ... Sep 5 14:23:11.071 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:11.020 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:14.135 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:14.104 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:15.099 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/25, changed state to administratively down Sep 5 14:23:15.052 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/25, changed state to administratively down Sep 5 14:23:17.739 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/25, changed state to down Sep 5 14:23:17.733 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/25, changed state to down Sep 5 14:23:18.389 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestL3VlanMet{ID=20} has completed successfully Sep 5 14:23:18.389 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestIngressSpan{ID=21} ... Sep 5 14:23:18.495 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:18.437 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:22.011 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:22.012 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:23.059 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/25, changed state to administratively down Sep 5 14:23:23.004 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/25, changed state to administratively down Sep 5 14:23:25.999 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/25, changed state to down Sep 5 14:23:26.006 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/25, changed state to down Sep 5 14:23:27.371 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:26.698 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestIngressSpan{ID=21} has completed successfully Sep 5 14:23:26.698 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestEgressSpan{ID=22} ... Sep 5 14:23:27.349 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestEgressSpan{ID=22} has completed successfully Sep 5 14:23:27.349 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestAclPermit{ID=23} ... Sep 5 14:23:27.377 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:29.951 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:29.959 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:30.623 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:30.603 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestAclPermit{ID=23} has completed successfully Sep 5 14:23:30.603 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestAclDeny{ID=24} ... Sep 5 14:23:30.631 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:42.628 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:42.854 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:44.668 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:44.647 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestAclDeny{ID=24} has completed successfully Sep 5 14:23:44.647 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestQos{ID=25} ... Sep 5 14:23:44.683 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:49.404 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:49.407 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:50.052 MSK: %LINK-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:50.031 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestQos{ID=25} has completed successfully Sep 5 14:23:50.031 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestNetflowShortcut{ID=26} ... Sep 5 14:23:50.063 MSK: %LINK-SP-5-CHANGED: Interface GigabitEthernet8/1, changed state to administratively down Sep 5 14:23:52.584 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:52.576 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/1, changed state to down Sep 5 14:23:53.732 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestNetflowShortcut{ID=26} has completed successfully Sep 5 14:23:53.732 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestFirmwareDiagStatus{ID=30} ... Sep 5 14:23:53.744 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestFirmwareDiagStatus{ID=30} has completed successfully Sep 5 14:23:53.744 MSK: %DIAG-SP-6-TEST_RUNNING: Module 8: Running TestOBFL{ID=33} ... Sep 5 14:23:53.748 MSK: %DIAG-SP-6-TEST_OK: Module 8: TestOBFL{ID=33} has completed successfully Сброс Скрытый текст hw-module module 8 reset Proceed with reload of module?[confirm] % reset issued for module 8 CoreSwitch# Sep 5 14:33:58.697 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/47, changed state to down Sep 5 14:33:58.717 MSK: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/47, changed state to down Sep 5 14:33:58.745 MSK: %LINK-3-UPDOWN: Interface GigabitEthernet8/48, changed state to down Sep 5 14:33:58.773 MSK: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/48, changed state to down Sep 5 14:33:58.831 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/47, changed state to down Sep 5 14:33:58.831 MSK: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/48, changed state to down Sep 5 14:33:58.847 MSK: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/47, changed state to down Sep 5 14:33:58.847 MSK: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/48, changed state to down Sep 5 14:33:59.562 MSK: %C6KPWR-SP-4-DISABLED: power to module in slot 8 set off (User Request) Sep 5 11:33:59.566: SP: scp_dnld_oir_status(8): status = 0 Sep 5 11:33:59.566: SP: scp_dnld_module 8 : 0 : 0: during state online, got event 2(disabled) Sep 5 11:33:59.566: SP: @@@ scp_dnld_module 8 : 0 : 0: online -> disabled Sep 5 11:33:59.566: SP: Slot 8 changed status to OFFLINE Sep 5 11:33:59.566: SP: disable_action(8/0): Sep 5 11:33:59.566: SP: Stop timer Sep 5 11:33:59.566: SP: stop_download_process() slot/proc 8/0, download_pid: -1 my_pid: 404 image_fd: -1 Sep 5 11:33:59.566: SP: disable_action: invoking oir_ios_offline for 8/0 Sep 5 11:33:59.598: SP: scp_dnld_module 8 : 1 : 0: during state online, got event 2(disabled) Sep 5 11:33:59.598: SP: @@@ scp_dnld_module 8 : 1 : 0: online -> disabled Sep 5 11:33:59.598: SP: Slot 8 changed status to OFFLINE Sep 5 11:33:59.598: SP: disable_action(8/1): Sep 5 11:33:59.598: SP: Stop timer Sep 5 11:33:59.598: SP: stop_download_process() slot/proc 8/1, download_pid: -1 my_pid: 404 image_fd: -1 Sep 5 11:34:14.610: SP: scp_dnld_oir_status(8): status = 1 Sep 5 11:34:14.610: SP: sm_debug_init : debug already initialized Sep 5 11:34:14.610: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_enabled, got event 1(enabled) Sep 5 11:34:14.610: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_enabled -> enabled Sep 5 11:34:14.610: SP: Start REGN timer for 8 with 240 sec Sep 5 11:34:30.138: SP: make_ios_dnld_instance: 8/1 Sep 5 11:34:30.138: SP: sm_debug_init : debug already initialized Sep 5 11:34:30.138: SP: scp_dnld_module 8 : 1 : 0: during state wait_til_enabled, got event 1(enabled) Sep 5 11:34:30.138: SP: @@@ scp_dnld_module 8 : 1 : 0: wait_til_enabled -> enabled Sep 5 11:34:30.138: SP: Start REGN timer for 8 with 240 sec Sep 5 11:34:30.138: SP: Card 8 sent image download request Sep 5 11:34:30.138: SP: (scp_start_download) 8/1 Sep 5 11:34:30.138: SP: (scp_start_download) 8/1: Started D/L Process, pid 149 Sep 5 11:34:30.798: SP: scp_adjust_timeouts_for_disk_file: slot/proc 8/1 Sep 5 11:34:30.798: SP: get_card_image: slot/proc 8/1: UBIN patch image on flash opened (microcode:/C2LC) Sep 5 11:34:30.798: SP: Downloading image to card at slot 8 Sep 5 11:34:30.798: SP: Image type for slot 8: IOS Sep 5 11:34:40.101: SP: Download to slot 8 Done! Sep 5 11:34:40.101: SP: scp_dnld_module 8 : 1 : 0: during state enabled, got event 10(online) Sep 5 11:34:40.101: SP: @@@ scp_dnld_module 8 : 1 : 0: enabled -> online Sep 5 11:34:40.101: SP: Stop timer Sep 5 11:34:40.101: SP: Slot 8 changed status to ONLINE Sep 5 11:34:40.101: SP: scp_download_process_teardown() mypid 149, slot/proc 8/1, image_fd -1 Sep 5 11:35:38.655: SP: scp_dnld_ipc_rcv_findmaster: setting scp_rcv_ipc_findmaster[8] == TRUE Sep 5 11:35:39.551: SP: scp_icc_slave_up: Stop timer for 8/0 Sep 5 11:35:39.551: SP: scp_icc_slave_up: IOS-RUNNING for 8/0 Sep 5 11:36:02.732: SP: scp_dnld_module 8 : 0 : 0: during state enabled, got event 5(registered) Sep 5 11:36:02.732: SP: @@@ scp_dnld_module 8 : 0 : 0: enabled -> wait_til_boot_ready Sep 5 11:36:02.732: SP: Stop timer Sep 5 11:36:02.732: SP: Start BOOT_RDY timer for 8 with 30000 msec Sep 5 11:36:02.848: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_boot_ready, got event 6(boot_ready) Sep 5 11:36:02.848: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_boot_ready -> wait_til_downloaded Sep 5 11:36:02.848: SP: Stop timer Sep 5 11:36:02.848: SP: Start DNLD timer for 8 with 120 sec Sep 5 11:36:02.848: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_downloaded, got event 4(dnld_completed) Sep 5 11:36:02.848: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_downloaded -> wait_til_ready Sep 5 11:36:02.848: SP: Stop timer Sep 5 11:36:02.848: SP: Start EXEC_CODE timer for 8 with 90 sec Sep 5 11:36:02.852: SP: Received Run-ready from slot 8 Sep 5 11:36:02.852: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_ready, got event 8(ready) Sep 5 11:36:02.852: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_ready -> wait_til_running Sep 5 11:36:02.852: SP: Stop timer Sep 5 11:36:02.852: SP: Start RUN_RDY timer for 8 with 90 sec Sep 5 11:36:02.852: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_running, got event 8(ready) Sep 5 11:36:02.852: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_running -> ready_missed Sep 5 11:36:02.852: SP: Stop timer Sep 5 11:36:02.852: SP: scp_dnld_module 8 : 0 : 0: idle during state ready_missed Sep 5 11:36:02.852: SP: @@@ scp_dnld_module 8 : 0 : 0: ready_missed -> wait_til_running Sep 5 11:36:02.852: SP: Start RUN_RDY timer for 8 with 90 sec Sep 5 11:36:02.856: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_running, got event 9(running) Sep 5 11:36:02.856: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_running -> wait_til_online Sep 5 11:36:02.856: SP: Stop timer Sep 5 11:36:02.856: SP: Start ONLINE timer for 8 Sep 5 11:36:02.856: SP: start_timer_online_action: timer controlled by lcc_configure() and online-timer Sep 5 11:36:02.856: SP: start_timer_online_action():SCP-ONLINE set online_timer to: 660 sec. Sep 5 11:36:02.856: SP: is_module_under_rapid_boot: slot: 8, status: 0 Sep 5 11:36:03.765: SP: scp_dnld_ios_online 8 : 0: *Jan 1 00:00:12.540: DaughterBoard (Distributed Forwarding Card 3) Firmware compiled 24-Jan-11 15:48 by integ Build [100] Sep 5 11:36:03.747: %SYS-DFC8-5-RESTART: System restarted -- Cisco IOS Software, c6lc2 Software (c6lc2-SP-VM), Version 12.2(33)SXI6, RELEASE SOFTWARE (fc4) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2011 by Cisco Systems, Inc. Compiled Mon 28-Mar-11 18:09 by prod_rel_team Sep 5 11:36:03.755: DFC8: Currently running ROMMON from S (Gold) region Sep 5 11:36:22.488: SP: is_module_under_rapid_boot: slot: 8, status: 0 Sep 5 14:36:22.488 MSK: %DIAG-SP-6-RUN_MINIMUM: Module 8: Running Minimal Diagnostics... Sep 5 11:36:22.488: SP: Stopping scp_online_timer for slot: 8 Sep 5 11:36:23.181: SP: Restarting scp_online_timer for slot: 8. (360sec) Sep 5 14:38:14.443 MSK: %DIAG-SP-6-DIAG_OK: Module 8: Passed Online Diagnostics Sep 5 11:38:14.523: SP: is_module_under_rapid_boot: slot: 8, status: 0 Sep 5 11:38:14.523: SP: is_module_under_rapid_boot: slot: 8, status: 0 Sep 5 11:38:14.523: SP: scp_online_active_process(8): diag_action 0, result 1 Sep 5 11:38:14.523: SP: scp_dnld_module 8 : 0 : 0: during state wait_til_online, got event 10(online) Sep 5 11:38:14.523: SP: @@@ scp_dnld_module 8 : 0 : 0: wait_til_online -> online Sep 5 11:38:14.523: SP: Stop timer Sep 5 11:38:14.523: SP: Slot 8 changed status to ONLINE Sep 5 11:38:14.535: SP: stop_scp_online_timer():No timer running for slot:8, proc: 0 На интерфейсе всё хорошо по счетчикам Скрытый текст GigabitEthernet8/47 is up, line protocol is up (connected) Hardware is C6k 1000Mb 802.3, address is 0021.a096.d3ce (bia 0021.a096.d3ce) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT input flow-control is off, output flow-control is off Clock mode is auto ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:25, output 00:00:09, output hang never Last clearing of "show interface" counters never Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 844000 bits/sec, 69 packets/sec 421658 packets input, 291328018 bytes, 0 no buffer Received 74829 broadcasts (64485 multicasts) 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 0 multicast, 0 pause input 0 input packets with dribble condition detected 10485271 packets output, 3620349940 bytes, 0 underruns 0 output errors, 0 collisions, 12 interface resets 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier, 0 PAUSE output 0 output buffer failures, 0 output buffers swapped out Только одно странно - по нулям в статистике 5 minute input rate 0 bits/sec, 0 packets/sec хотя счетчики пакетов бегут Вставить ник Quote
zhenya` Posted September 5, 2018 Posted September 5, 2018 Чем вы дропы измеряете то? первая команда в conf t Вставить ник Quote
_J_ Posted September 5, 2018 Author Posted September 5, 2018 30 минут назад, zhenya` сказал: Чем вы дропы измеряете то? В порт Gi8/47 подключен микротик с запущенным bandwidth test server. Как только начинаем генерировать трафик на/из него, связь рвется ну и пинги пропадают На графиках порядка 20Мбит/с загрузка теста. Тот же микротик перевтыкаем в плату на седьмом слоте - сразу разгоняется до гигабита и ни единого потерянного пинга. 36 минут назад, zhenya` сказал: первая команда в conf t А это как посмотреть? Вставить ник Quote
zhenya` Posted September 5, 2018 Posted September 5, 2018 А в 48ом порту такое же поведение? Вставить ник Quote
_J_ Posted September 5, 2018 Author Posted September 5, 2018 Да, в 48-м всё также Скрытый текст Hardware is C6k 1000Mb 802.3, address is 0021.a096.d3cf (bia 0021.a096.d3cf) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT input flow-control is off, output flow-control is off Clock mode is auto ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:24, output 00:00:26, output hang never Last clearing of "show interface" counters never Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 5000 bits/sec, 3 packets/sec 23971 packets input, 2858583 bytes, 0 no buffer Received 23315 broadcasts (15120 multicasts) 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 0 multicast, 0 pause input 0 input packets with dribble condition detected Причем если гонять трафик с 47 на 48 (в одном влане, т.е. трафик не покидает слота) получаем всё те-же 20Мбит/с и далее потери. Вывод show counters Скрытый текст sh counters interface gigabitEthernet 8/47 64 bit counters: 0. rxHCTotalPkts = 28242 1. txHCTotalPkts = 160221 2. rxHCUnicastPkts = 25187 3. txHCUnicastPkts = 68685 4. rxHCMulticastPkts = 2712 5. txHCMulticastPkts = 16458 6. rxHCBroadcastPkts = 334 7. txHCBroadcastPkts = 75087 8. rxHCOctets = 6090788 9. txHCOctets = 75867517 10. rxTxHCPkts64Octets = 24716 11. rxTxHCPkts65to127Octets = 52246 12. rxTxHCPkts128to255Octets = 9543 13. rxTxHCPkts256to511Octets = 69927 14. rxTxHCpkts512to1023Octets = 277 15. rxTxHCpkts1024to1518Octets = 31763 16. txHCTrunkFrames = 9 17. rxHCTrunkFrames = 9 18. rxHCDropEvents = 0 32 bit counters: 0. rxCRCAlignErrors = 0 1. rxUndersizedPkts = 0 2. rxOversizedPkts = 0 3. rxFragmentPkts = 0 4. rxJabbers = 0 5. txCollisions = 0 6. ifInErrors = 0 7. ifOutErrors = 0 8. ifInDiscards = 0 9. ifInUnknownProtos = 0 10. ifOutDiscards = 0 11. txDelayExceededDiscards = 0 12. txCRC = 0 13. linkChange = 1 14. wrongEncapFrames = 0 All Port Counters 1. InPackets = 28242 2. InOctets = 6090788 3. InUcastPkts = 25187 4. InMcastPkts = 2712 5. InBcastPkts = 334 6. OutPackets = 160200 7. OutOctets = 75862515 8. OutUcastPkts = 68685 9. OutMcastPkts = 16453 10. OutBcastPkts = 75071 11. AlignErr = 0 12. FCSErr = 0 13. XmitErr = 0 14. RcvErr = 0 15. UnderSize = 0 16. SingleCol = 0 17. MultiCol = 0 18. LateCol = 0 19. ExcessiveCol = 0 20. CarrierSense = 0 21. Runts = 0 22. Giants = 0 23. InDiscards = 0 24. OutDiscards = 0 25. InErrors = 0 26. OutErrors = 0 27. InUnknownProtos = 0 28. txCRC = 0 29. TrunkFramesTx = 9 30. TrunkFramesRx = 9 31. WrongEncap = 0 32. Broadcast_suppression_discards = 0 33. Multicast_suppression_discards = 0 34. Unicast_suppression_discards = 0 35. rxTxHCPkts64Octets = 24711 36. rxTxHCPkts65to127Octets = 52243 37. rxTxHCPkts128to255Octets = 9542 38. rxTxHCPkts256to511Octets = 69915 39. rxTxHCpkts512to1023Octets = 277 40. rxTxHCpkts1024to1518Octets = 31763 41. DropEvents = 0 42. CRCAlignErrors = 0 43. UndersizedPkts = 0 44. OversizedPkts = 0 45. FragmentPkts = 0 46. Jabbers = 0 47. Collisions = 0 48. DelayExceededDiscards = 0 49. bpduOutlost = 0 50. qos0Outlost = 0 51. qos1Outlost = 0 52. qos2Outlost = 0 53. qos3Outlost = 0 54. qos4Outlost = 0 55. qos5Outlost = 0 56. qos6Outlost = 0 57. qos7Outlost = 0 58. qos8Outlost = 0 59. qos9Outlost = 0 60. qos10Outlost = 0 61. qos11Outlost = 0 62. qos12Outlost = 0 63. qos13Outlost = 0 64. qos14Outlost = 0 65. qos15Outlost = 0 66. qos16Outlost = 0 67. qos17Outlost = 0 68. qos18Outlost = 0 69. qos19Outlost = 0 70. qos20Outlost = 0 71. qos21Outlost = 0 72. qos22Outlost = 0 73. qos23Outlost = 0 74. qos24Outlost = 0 75. qos25Outlost = 0 76. qos26Outlost = 0 77. qos27Outlost = 0 78. qos28Outlost = 0 79. qos29Outlost = 0 80. qos30Outlost = 0 81. qos31Outlost = 0 82. bpduCbicOutlost = 0 83. qos0CbicOutlost = 0 84. qos1CbicOutlost = 0 85. qos2CbicOutlost = 0 86. qos3CbicOutlost = 0 87. bpduInlost = 0 88. qos0Inlost = 0 89. qos1Inlost = 0 90. qos2Inlost = 0 91. qos3Inlost = 0 92. qos4Inlost = 0 93. qos5Inlost = 0 94. qos6Inlost = 0 95. qos7Inlost = 0 96. qos8Inlost = 0 97. qos9Inlost = 0 98. qos10Inlost = 0 99. qos11Inlost = 0 100. qos12Inlost = 0 101. qos13Inlost = 0 102. qos14Inlost = 0 103. qos15Inlost = 0 104. qos16Inlost = 0 105. qos17Inlost = 0 106. qos18Inlost = 0 107. qos19Inlost = 0 108. qos20Inlost = 0 109. qos21Inlost = 0 110. qos22Inlost = 0 111. qos23Inlost = 0 112. qos24Inlost = 0 113. qos25Inlost = 0 114. qos26Inlost = 0 115. qos27Inlost = 0 116. qos28Inlost = 0 117. qos29Inlost = 0 118. qos30Inlost = 0 119. qos31Inlost = 0 120. pqueInlost = 0 121. Overruns = 0 122. maxIndex = 0 CoreSwitch>sh counters interface gigabitEthernet 8/48 64 bit counters: 0. rxHCTotalPkts = 357 1. txHCTotalPkts = 4836 2. rxHCUnicastPkts = 0 3. txHCUnicastPkts = 11 4. rxHCMulticastPkts = 36 5. txHCMulticastPkts = 899 6. rxHCBroadcastPkts = 303 7. txHCBroadcastPkts = 3944 8. rxHCOctets = 31284 9. txHCOctets = 1217945 10. rxTxHCPkts64Octets = 625 11. rxTxHCPkts65to127Octets = 1470 12. rxTxHCPkts128to255Octets = 81 13. rxTxHCPkts256to511Octets = 3034 14. rxTxHCpkts512to1023Octets = 1 15. rxTxHCpkts1024to1518Octets = 0 16. txHCTrunkFrames = 18 17. rxHCTrunkFrames = 18 18. rxHCDropEvents = 0 32 bit counters: 0. rxCRCAlignErrors = 0 1. rxUndersizedPkts = 0 2. rxOversizedPkts = 0 3. rxFragmentPkts = 0 4. rxJabbers = 0 5. txCollisions = 0 6. ifInErrors = 0 7. ifOutErrors = 0 8. ifInDiscards = 0 9. ifInUnknownProtos = 0 10. ifOutDiscards = 0 11. txDelayExceededDiscards = 0 12. txCRC = 0 13. linkChange = 573 14. wrongEncapFrames = 0 All Port Counters 1. InPackets = 357 2. InOctets = 31284 3. InUcastPkts = 0 4. InMcastPkts = 36 5. InBcastPkts = 303 6. OutPackets = 4825 7. OutOctets = 1214867 8. OutUcastPkts = 11 9. OutMcastPkts = 897 10. OutBcastPkts = 3935 11. AlignErr = 0 12. FCSErr = 0 13. XmitErr = 0 14. RcvErr = 0 15. UnderSize = 0 16. SingleCol = 0 17. MultiCol = 0 18. LateCol = 0 19. ExcessiveCol = 0 20. CarrierSense = 0 21. Runts = 0 22. Giants = 0 23. InDiscards = 0 24. OutDiscards = 0 25. InErrors = 0 26. OutErrors = 0 27. InUnknownProtos = 0 28. txCRC = 0 29. TrunkFramesTx = 18 30. TrunkFramesRx = 18 31. WrongEncap = 0 32. Broadcast_suppression_discards = 0 33. Multicast_suppression_discards = 0 34. Unicast_suppression_discards = 0 35. rxTxHCPkts64Octets = 624 36. rxTxHCPkts65to127Octets = 1468 37. rxTxHCPkts128to255Octets = 81 38. rxTxHCPkts256to511Octets = 3026 39. rxTxHCpkts512to1023Octets = 1 40. rxTxHCpkts1024to1518Octets = 0 41. DropEvents = 0 42. CRCAlignErrors = 0 43. UndersizedPkts = 0 44. OversizedPkts = 0 45. FragmentPkts = 0 46. Jabbers = 0 47. Collisions = 0 48. DelayExceededDiscards = 0 49. bpduOutlost = 0 50. qos0Outlost = 0 51. qos1Outlost = 0 52. qos2Outlost = 0 53. qos3Outlost = 0 54. qos4Outlost = 0 55. qos5Outlost = 0 56. qos6Outlost = 0 57. qos7Outlost = 0 58. qos8Outlost = 0 59. qos9Outlost = 0 60. qos10Outlost = 0 61. qos11Outlost = 0 62. qos12Outlost = 0 63. qos13Outlost = 0 64. qos14Outlost = 0 65. qos15Outlost = 0 66. qos16Outlost = 0 67. qos17Outlost = 0 68. qos18Outlost = 0 69. qos19Outlost = 0 70. qos20Outlost = 0 71. qos21Outlost = 0 72. qos22Outlost = 0 73. qos23Outlost = 0 74. qos24Outlost = 0 75. qos25Outlost = 0 76. qos26Outlost = 0 77. qos27Outlost = 0 78. qos28Outlost = 0 79. qos29Outlost = 0 80. qos30Outlost = 0 81. qos31Outlost = 0 82. bpduCbicOutlost = 0 83. qos0CbicOutlost = 0 84. qos1CbicOutlost = 0 85. qos2CbicOutlost = 0 86. qos3CbicOutlost = 0 87. bpduInlost = 0 88. qos0Inlost = 0 89. qos1Inlost = 0 90. qos2Inlost = 0 91. qos3Inlost = 0 92. qos4Inlost = 0 93. qos5Inlost = 0 94. qos6Inlost = 0 95. qos7Inlost = 0 96. qos8Inlost = 0 97. qos9Inlost = 0 98. qos10Inlost = 0 99. qos11Inlost = 0 100. qos12Inlost = 0 101. qos13Inlost = 0 102. qos14Inlost = 0 103. qos15Inlost = 0 104. qos16Inlost = 0 105. qos17Inlost = 0 106. qos18Inlost = 0 107. qos19Inlost = 0 108. qos20Inlost = 0 109. qos21Inlost = 0 110. qos22Inlost = 0 111. qos23Inlost = 0 112. qos24Inlost = 0 113. qos25Inlost = 0 114. qos26Inlost = 0 115. qos27Inlost = 0 116. qos28Inlost = 0 117. qos29Inlost = 0 118. qos30Inlost = 0 119. qos31Inlost = 0 120. pqueInlost = 0 121. Overruns = 0 122. maxIndex = 0 Вставить ник Quote
vurd Posted September 5, 2018 Posted September 5, 2018 Уберите ваши простынки под спойлеры плиз, невозможно тред читать. Вставить ник Quote
_J_ Posted September 5, 2018 Author Posted September 5, 2018 Пардоньте. Подправил. Вставить ник Quote
_J_ Posted September 22, 2018 Author Posted September 22, 2018 Тут вот ещё что происходит... Начинаю гонять трафик с порта на порт в одном влане и одном модуле по L2 между двумя микротиками через bandwidth test. Трафик не поднимается более 20-ти Мбит/с и через полминуты ловим подвисание консоли на секунд на 5, как по прямомму подключению, так и ssh. В логи сыпет Sep 21 19:18:53.438 MSK: %SYS-3-CPUHOG: Task is running for (2000)msecs, more than (2000)msecs (0/0),process = L3 Manager.(libc.so+0x40AD4) ([0:-3]s72033_rp-adventerprisek9_wan-1-dso-b+0x155920) ([30:-9]3+0x1751D4) ([30:-9]11+0x4D4B1C) ([31:-8]-+0x4D475C) ([40:0]+0x4D71D0) ([30:-9]3+0x544DF4) ([40:0]+0x544C50) ([30:-9]14+0x2700E8) ([41:0]+0x2700C0) ([41:0]+0x2700C0) Sep 21 19:18:55.802 MSK: %SYS-3-CPUHOG: Task is running for (4025)msecs, more than (2000)msecs (0/0),process = L3 Manager.(s72033_rp-adventerprisek9_wan-11-dso-b.so+0x4D7284) ([31:-8]-+0x4D40D8) ([40:0]+0x4D71D0) ([30:-9]3+0x544DF4) ([40:0]+0x544C50) ([30:-9]14+0x2700E8) ([41:0]+0x2700C0) ([30:-9]3+0x544C50) ([30:-9]14+0x2700E8) ([41:0]+0x2700C0) ([41:0]+0x2700C0) И далее отвал соседей по ospf Sep 21 19:19:22.275 MSK: %OSPF-5-ADJCHG: Process 1, Nbr 1.2.3.4 on Vlan1207 from FULL to DOWN, Neighbor Down: Dead timer expired Sep 21 19:19:22.527 MSK: %OSPF-5-ADJCHG: Process 1, Nbr 1.2.3.189 on Vlan1207 from FULL to DOWN, Neighbor Down: Dead timer expired Sep 21 19:19:24.083 MSK: %OSPF-5-ADJCHG: Process 1, Nbr 1.2.3.107 on Vlan1208 from FULL to DOWN, Neighbor Down: Dead timer expired Sep 21 19:19:27.139 MSK: %OSPF-5-ADJCHG: Process 1, Nbr 1.2.3.177 on Vlan1208 from FULL to DOWN, Neighbor Down: Dead timer expired Sep 21 19:19:26.831 MSK: %OSPF-5-ADJCHG: Process 1, Nbr 1.2.3.181 on Vlan1208 from FULL to DOWN, Neighbor Down: Dead timer expired Видно что колбасит процесс L3 Manager, но я же прогружаю l2. Не подскажете где можно посмотреть? Вставить ник Quote
zhenya` Posted September 22, 2018 Posted September 22, 2018 DFC имхо плохо. смените модули 7 и 8 и посмотрите на реакцию на модуль с CFC Вставить ник Quote
_J_ Posted September 22, 2018 Author Posted September 22, 2018 9 минут назад, zhenya` сказал: DFC имхо плохо. смените модули 7 и 8 и посмотрите на реакцию на модуль с CFC С CFC тоже самое. Может стоит обновить/поменять ios? Сейчас установлен Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-VM), Version 12.2(33)SXI6, RELEASE SOFTWARE (fc4) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2011 by Cisco Systems, Inc. Compiled Mon 28-Mar-11 17:27 by prod_rel_team ROM: System Bootstrap, Version 12.2(17r)SX5, RELEASE SOFTWARE (fc1) Вставить ник Quote
zhenya` Posted September 22, 2018 Posted September 22, 2018 Видимо проблема в шасси или супе. Вставить ник Quote
tros10 Posted September 23, 2018 Posted September 23, 2018 Если там DFC и потери в рамках одного модуля, то смотреть надо не в сторону софта ИМХО. Трафик не должен выходить за рамки асика если используются соседние порты и свитчится локально, без вовлечения фабрики. Поищите картинку с архитектурой 6748, что бы понять какие порты каким асикам принадлежат. Насколько я помню, там 4 асика, по 12 портов на каждый. Есть еще вариант, что карта с DFC убитая, а карты с CFC утыкаются в глючный PFC. Вставить ник Quote
_J_ Posted September 24, 2018 Author Posted September 24, 2018 20 часов назад, tros10 сказал: Поищите картинку с архитектурой 6748, что бы понять какие порты каким асикам принадлежат. Насколько я помню, там 4 асика, по 12 портов на каждый. Вот оно ---------- ---------- Видно что порты 46 и 48 находятся на одном asic и получается должны ходить напрямую, даже не задействуя CFC/DFC? Вставить ник Quote
UglyAdmin Posted September 24, 2018 Posted September 24, 2018 51 минуту назад, _J_ сказал: Видно что порты 46 и 48 находятся на одном asic и получается должны ходить напрямую, даже не задействуя CFC/DFC? ASIC'и там глупые и свитчевать внутри себя не умеют. Это делает DFC или PFC (в случае CFC). Вставить ник Quote
.png.5d2afa2996cc6a85d0f2c09b92dd0a28.png)
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.