Jump to content
Калькуляторы

Не балансится трафик на сервере Linux через Bonding интерфейс с LACP

Добрый день

Последний Дебиан 9.3,

сервер HP Proliant DL360

сетевые 4 шт Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20) объединены в bond интерфейс.
Проблема в том, что трафик не балансится на все интерфейсы и выше 1Гбит/с не поднимается.

root@nat:/home/tuxper# ethtool bond0
Settings for bond0:
        Supported ports: [ ]
        Supported link modes:   Not reported
        Supported pause frame use: No
        Supports auto-negotiation: No
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 4000Mb/s
        Duplex: Full
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Link detected: yes

Конфиг интерфейсов

/etc/network/interfaces

auto bond0
iface bond0 inet manual
        bond-mode 4
        bond-miimon 100
        bond-downdelay 200
        bond-updelay 200
        slaves enp2s0f0 enp2s0f1 enp3s0f0 enp3s0f1


auto vlanXXX
iface vlanXXX inet static
        address xx.xx.xx.2/24
        gateway xx.xx.xx.1
        dns-nameservers xx.xx.xx.249
        vlan-raw-device bond0

В /proc

Скрытый текст

root@nat:/home/tuxper# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 00:25:b3:de:9c:02
Active Aggregator Info:
        Aggregator ID: 2
        Number of ports: 4
        Actor Key: 9
        Partner Key: 5
        Partner Mac Address: 44:e4:d9:9a:cc:00

Slave Interface: enp2s0f0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:b3:de:9c:02
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 00:25:b3:de:9c:02
    port key: 9
    port priority: 255
    port number: 1
    port state: 61
details partner lacp pdu:
    system priority: 32768
    system mac address: 44:e4:d9:9a:cc:00
    oper key: 5
    port priority: 32768
    port number: 267
    port state: 61

Slave Interface: enp2s0f1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:b3:de:9c:04
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 00:25:b3:de:9c:02
    port key: 9
    port priority: 255
    port number: 2
    port state: 61
details partner lacp pdu:
    system priority: 32768
    system mac address: 44:e4:d9:9a:cc:00
    oper key: 5
    port priority: 32768
    port number: 265
    port state: 61

Slave Interface: enp3s0f0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:b3:de:9c:06
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 00:25:b3:de:9c:02
    port key: 9
    port priority: 255
    port number: 3
    port state: 61
details partner lacp pdu:
    system priority: 32768
    system mac address: 44:e4:d9:9a:cc:00
    oper key: 5
    port priority: 32768
    port number: 268
    port state: 61

Slave Interface: enp3s0f1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:b3:de:9c:08
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 00:25:b3:de:9c:02
    port key: 9
    port priority: 255
    port number: 4
    port state: 61
details partner lacp pdu:
    system priority: 32768
    system mac address: 44:e4:d9:9a:cc:00
    oper key: 5
    port priority: 32768
    port number: 269
    port state: 61
 


На свиче порт-ченнел поднялся, трафик ходит.

Меряю iperf3 с 3-х внешних серверов с разными белыми IP трафик выше 1Гбит/с с 1 интерфейса не поднимается.

Edited by Macro

Share this post


Link to post
Share on other sites

Debian 8.4

Настройка такая

iface bond1 inet static
    address 0.0.0.0
    bond_mode 4
    bond_miimon 100
    bond-slaves eth2 eth3
    bond-xmit_hash_policy layer2+3
auto bond1

На свиче LACP вроде балансируется.

Какой свич?

Share this post


Link to post
Share on other sites
20 минут назад, zhenya` сказал:

bond-xmit_hash_policy layer2+3

вообще-то у ТСа вообще

Transmit Hash Policy: layer2 (0)

 

Share this post


Link to post
Share on other sites

Всем спасибо!

Запилил bond-xmit-hash-policy layer2+3

 

Теперь трафик балансится, но почему то только между 3 интерфейсами - при 4-х в апе,

1 интерфейс всегда простаивает. 

На сервере алиасами 253 адреса, подключаюсь iperf'ом c 4 серверов на разные IP сервера с bond интерфейсом.

        lo               enp2s0f0            enp2s0f1            enp3s0f0            enp3s0f1             bond0              vlanXXX             vlanXXX
 KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out
    0.00      0.00   1531.25      0.41    135.27  113738.2      0.21  118715.7   1282.13  119864.7   2948.86  352319.0   2110.32  336947.7      0.00      0.00
    0.00      0.00   1527.67      0.98    158.84  114270.7      0.06  118605.5   1330.87  120255.2   3017.44  353132.4   2162.00  337747.0      0.00      0.00
    0.00      0.00   1540.88      0.41    164.47  114686.2      0.00  119001.4   1282.66  120380.9   2988.00  354068.9   2118.49  332546.5      0.00      0.00
    0.00      0.00   1543.07      0.34    171.20  109944.7      0.25  118620.5   1308.14  120087.4   3022.65  348652.9   2138.58  337935.2      0.00      0.00
    0.00      0.00   1530.38      0.92    146.98  115148.3      0.13  118874.4   1315.96  119794.9   2993.45  353818.5   2144.58  336750.9      0.00      0.00
    0.00      0.00   1532.87      0.40    169.75  114055.5      0.00  118279.2   1310.15  120255.1   3012.78  352590.3   2143.01  336750.8      0.00      0.00
    0.00      0.00   1539.45      0.34    156.05  113748.0      0.30  118565.8   1298.05  120204.4   2993.86  352518.5   2141.35  337252.5      0.00      0.00
    0.00      0.00   1547.60      0.34    147.12  114134.3      0.06  119425.3   1319.14  120495.5   3013.92  354055.4   2118.26  335688.0      0.00      0.00
    0.00      0.00   1514.03      0.34    169.75  113941.9      0.12  117176.5   1309.13  120214.5   2993.03  351333.2   2153.89  337442.1      0.00      0.00
    0.00      0.00   1536.37      0.34    167.91  114879.1      0.37  118476.6   1293.97  120176.5   2998.63  353532.5   2144.30  336362.3      0.00      0.00
    0.00      0.00   1551.12      0.28    166.79  113682.8      0.25  118797.2   1325.33  120182.8   3043.48  352663.0   2188.86  336419.8      0.00      0.00
    0.00      0.00   1561.69      0.34    197.21  113148.0      0.12  118743.4   1304.46  120217.8   3063.48  352109.5   2158.40  337653.8      0.00      0.00
    0.00      0.00   3082.64      0.34    161.30  114557.3      0.00  119294.6   1326.56  120515.2   4570.50  354367.5   2105.23  333260.7      0.00      0.00

 

Edited by Macro

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this