Перейти к содержимому
Калькуляторы

passilka

Новичок
 Просмотреть профиль  Активность

  • Публикации

    3

  • Зарегистрирован

  • Посещение

 Тип публикации 

Сообщения, опубликованные пользователем passilka

  1. Опубликовано · Жалоба на ответ

    При запуске множество ошибок:

    EAL: Error reading from file descriptor 18: Input/output error
EAL: Error reading from file descriptor 18: Input/output error
EAL: Error reading from file descriptor 18: Input/output error
EAL: Error reading from file descriptor 18: Input/output error
EAL: Error reading from file descriptor 18: Input/output error
EAL: Error reading from file descriptor 18: Input/output error
EAL: Error reading from file descriptor 18: Input/output error

     

    Лог extFilter:

     

     

    2017-03-21 08:34:55.727 [2403] Information Application - Setting mbuf size to 8191

    2017-03-21 08:34:56.773 [2403] Information Application - Master core is 0

    2017-03-21 08:34:57.047 [2403] Information Application - Port 0 MAC: 00:0c:29:26:ee:03

    2017-03-21 08:34:57.050 [2403] Warning Application - Pattern 'xn--80aeshrfifdjb.xn--p1ai' already present in the database from file /root/domains

    2017-03-21 08:34:57.071 [2403] Warning Application - Pattern 'new.xn--1---7cdbcb4aca7cbbcxxqb5ahnnpk8b2t.xn--p1acf' already present in the database from file /root/domains

    2017-03-21 08:34:57.077 [2403] Warning Application - Pattern 'igrovyeavtomatyvulkan.org' already present in the database from file /root/domains

    2017-03-21 08:34:57.094 [2403] Warning Application - Pattern 'flibusta.is' already present in the database from file /root/domains

    2017-03-21 08:34:57.096 [2403] Warning Application - Pattern 'bkrbet.top' already present in the database from file /root/domains

    2017-03-21 08:34:57.099 [2403] Warning Application - Pattern 'xn--e1awew.xn--p1ai' already present in the database from file /root/domains

    2017-03-21 08:34:57.102 [2403] Warning Application - Pattern 'kontorafonbet.ru' already present in the database from file /root/domains

    2017-03-21 08:34:57.102 [2403] Warning Application - Pattern 'xn--80aafc2bcqz.xn--p1ai' already present in the database from file /root/domains

    2017-03-21 08:34:57.118 [2403] Warning Application - Pattern 'xn---888-93d8c.xn--p1acf' already present in the database from file /root/domains

    2017-03-21 08:34:57.118 [2403] Warning Application - Pattern 'fbmetrix.club' already present in the database from file /root/domains

    2017-03-21 08:34:57.121 [2403] Warning Application - Pattern 'bk-leon2.ru' already present in the database from file /root/domains

    2017-03-21 08:34:57.124 [2403] Warning Application - Pattern 'bk3.xn--1--blcqqatr5b1a2j.xn--p1acf' already present in the database from file /root/domains

    2017-03-21 08:34:57.126 [2403] Warning Application - Pattern 'tennisi.kz' already present in the database from file /root/domains

    2017-03-21 08:34:57.127 [2403] Warning Application - Pattern 'proxy.flibusta.is' already present in the database from file /root/domains

    2017-03-21 08:34:57.133 [2403] Warning Application - Pattern 'betcity.rs' already present in the database from file /root/domains

    2017-03-21 08:34:57.133 [2403] Warning Application - Pattern 'bkrbet.ru' already present in the database from file /root/domains

    2017-03-21 08:34:57.134 [2403] Warning Application - Pattern 'www2.xn--90ahbfg3a0bc.com' already present in the database from file /root/domains

    2017-03-21 08:34:57.134 [2403] Warning Application - Pattern 'xn---1-6kcgqeizk4as6azb.xn--p1ai' already present in the database from file /root/domains

    2017-03-21 08:34:57.135 [2403] Warning Application - Pattern 'xn--4-7sbagbdpv5bpes5ad1ki.xn--80asehdb' already present in the database from file /root/domains

    2017-03-21 08:34:57.135 [2403] Warning Application - Pattern 'xn--1---7cdbcc3adabx4bp1biwi1a1d9b.xn--p1acf' already present in the database from file /root/domains

    2017-03-21 08:34:57.136 [2403] Warning Application - Pattern 'xn-----8kcledd0aci4aj1cb3o.xn--p1ai' already present in the database from file /root/domains

    2017-03-21 08:34:57.138 [2403] Warning Application - Pattern 'fbwebdn.net' already present in the database from file /root/domains

    2017-03-21 08:34:57.211 [2403] Warning Application - Pattern 'mixslots.com' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.270 [2403] Warning Application - Pattern 'ru-nur.com/▒▒▒▒▒-▒▒▒▒.html' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.273 [2403] Warning Application - Pattern 'nurr.ru/content/▒▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒-▒▒-▒▒▒▒▒▒-▒▒-▒▒▒▒▒▒▒-▒▒▒▒-▒▒-▒▒▒▒▒-▒▒-▒▒-▒▒▒▒▒▒-▒-▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.284 [2403] Warning Application - Pattern 'high-stone-forum.info/index.php/topic/1809-▒▒▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒-▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.289 [2403] Warning Application - Pattern 'ru.videosection.com/▒▒▒▒▒▒▒▒▒.html' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.292 [2403] Warning Application - Pattern 'shadow-biz.com/topic/408-▒▒▒▒▒▒-▒▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒▒-▒▒-▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.296 [2403] Warning Application - Pattern 'song5.ru/text/▒▒▒▒▒-▒▒▒▒▒▒▒▒-▒▒-▒▒▒▒▒-▒▒▒▒▒▒-▒▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.323 [2403] Warning Application - Pattern 'song5.ru/text/warriors-of-zion-▒-▒▒-▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.325 [2403] Warning Application - Pattern 'intpharm.ru/▒▒▒▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.326 [2403] Warning Application - Pattern 'gslot-home.com' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.343 [2403] Warning Application - Pattern 'mp3-muzyka.com/skachat-music-besplatno/▒▒▒▒▒▒▒▒+▒▒▒▒▒+▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.346 [2403] Warning Application - Pattern 'antanartia.tumblr.com/post/85745447369/▒▒▒▒▒▒▒▒-▒▒▒-▒▒▒▒▒▒▒▒▒▒-▒▒▒▒▒-▒▒-▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.351 [2403] Warning Application - Pattern 'high-stone-forum.info/index.php/topic/1609-▒▒▒-▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.366 [2403] Warning Application - Pattern '7pik.com/top/4290-▒▒▒-10-▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒▒-▒-▒▒▒▒▒.html' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.377 [2403] Warning Application - Pattern 'diplom-kazan-tut.com' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.381 [2403] Warning Application - Pattern 'song5.ru/text/▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒-▒▒▒▒▒-▒▒▒▒▒-▒▒▒-▒▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.385 [2403] Warning Application - Pattern 'melody24.net/search/▒▒▒▒▒▒▒▒▒+▒+▒▒▒▒▒▒▒▒/▒▒▒▒▒▒+▒▒+▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.387 [2403] Warning Application - Pattern 'muzogig.net/gig/▒▒▒%20▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.448 [2403] Warning Application - Pattern 'mp3-centr.ru/music/▒▒▒▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.451 [2403] Warning Application - Pattern 'vkmp3.su/mp3/▒▒▒▒▒%20▒▒▒▒▒%20▒▒▒▒▒%20▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.455 [2403] Warning Application - Pattern 'gigmuziki.com/search/▒▒▒%20▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.463 [2403] Warning Application - Pattern 'kriminall.biz/index.php?/topic/90-▒▒▒▒▒▒-▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒-▒▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒/' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.469 [2403] Warning Application - Pattern 'lurklurk.com/Encyclopedia_Dramatica/▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.497 [2403] Warning Application - Pattern 'song5.ru/text/▒▒▒▒▒▒-▒▒-▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.546 [2403] Warning Application - Pattern 'freemuzichka.com/tunes/▒▒▒▒▒▒▒▒%20-%20▒▒▒▒%20▒%20▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.574 [2403] Warning Application - Pattern 'freemuzichka.com/tunes/▒▒▒▒▒▒▒▒▒%20▒▒▒▒%20▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.589 [2403] Warning Application - Pattern 'freemuzichka.com/tunes/▒▒▒▒▒▒▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.614 [2403] Warning Application - Pattern 'sneg.audio/show/▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.615 [2403] Warning Application - Pattern 'song5.ru/text/▒▒▒▒▒▒-▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒-▒▒▒-▒▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.625 [2403] Warning Application - Pattern 'song5.ru/text/▒▒▒▒▒▒▒▒▒-▒▒▒▒▒▒▒▒-▒▒▒▒▒-▒▒▒▒▒-▒▒▒-▒▒▒▒▒▒▒▒▒▒▒▒-▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.630 [2403] Warning Application - Pattern 'en.osmannuritopbas.com/portfolio/▒▒-▒▒▒▒-▒▒▒▒▒▒▒▒▒▒▒' already present in the URL database from file /root/urls

    2017-03-21 08:34:57.631 [2403] Warning Application - Pattern '76▒▒▒▒▒▒▒.▒▒/' already present in the URL database from file /root/urls

     

     

     

    Конфиг extFilter:

     

    ; Переводить имя хоста в прописные буквы. Если url_normalization установлен в true, то не имеет значения.

    lower_host = true

     

    domainlist = /root/domains

    urllist = /root/urls

    ssllist = /root/ssl_host

    hostlist = /root/hosts

     

    ; Файл с портами для nDPI.

    ;protocols = /root/protos

     

    ; Список ip адресов/сетей для блокировки ssl если нет server_name в ssl hello пакете. Загружается если block_undetected_ssl = true.

    sslips = /root/ssl_ips

     

    ; если false, то будет послан rst пакет вместо редиректа. Default: false

    http_redirect = true

     

    redirect_url = http://gsglobalcom.ru/block.html

     

    ; HTTP код ответа. default: 302 Moved Temporarily

    http_code = 302 Found

     

    ; Что добавлять в redirect_url, line - строка из файла url, url - запрещенный url, none - ничего

    url_additional_info=none

     

     

    ; посылать tcp rst в сторону сервера от имени клиента. Default: false

    rst_to_server = true

     

    ; Default: 0 - disable

    statistic_interval = 300

     

    ; Default: false

    match_url_exactly = false

     

    ; Default: false

    block_undetected_ssl = false

     

    ; dpdk порт, где анализировать трафик

    dpdk_port = 0

     

    ; dpdk порт(ы), где анализировать трафик

    ;dpdk_ports = 0

     

    ; размер пула mbuf. Default: 8191

    ;mbuf_pool_size = 8191

     

    ; количество потоков по анализу трафика

    num_of_workers=1

     

    ; Какие ядра использовать. Default: все ядра, кроме management.

    core_mask = 7

     

    ; файл статистики (для extfilter-cacti)

    statisticsfile = /var/run/extFilter_stat

     

    ; mtu на интерфейсе для отправки пакетов в сторону абонентов. Default: 1500

    out_mtu = 1500

     

    ; Количество flow, обрабатываемых программой. Должно быть кратно 2.

    ;flowhash_size = 1048576

     

    ; количество тредов для отсылки уведомлений о блокировке

    ;num_of_senders = 1

     

    ; делать ли нормализацию url

    url_normalization = true

     

    ; удалять ли точку в конце имени хоста

    remove_dot = true

     

    [logging]

    loggers.root.level = information

    ;loggers.root.level = debug

    loggers.root.channel = fileChannel

    channels.fileChannel.class = FileChannel

    channels.fileChannel.path = /var/log/extFilter.log

    channels.fileChannel.rotation = 1 M

    channels.fileChannel.purgeCount = 4

    channels.fileChannel.archive = timestamp

    channels.fileChannel.formatter.class = PatternFormatter

    channels.fileChannel.formatter.pattern = %Y-%m-%d %H:%M:%S.%i [%P] %p %s - %t

    channels.fileChannel.formatter.times = local

     

     

     

    Проц такой

     

    ============================================================

    Core and Socket Information (as reported by '/proc/cpuinfo')

    ============================================================

     

    cores = [0, 1, 2, 3]

    sockets = [0]

     

    Socket 0

    --------

    Core 0 [0]

    Core 1 [1]

    Core 2 [2]

    Core 3 [3]

     

     

     

    Есть вопросы:

    Как высчитать flowhash_size для трафа в 200Мбит/с?

    Поддерждивает ли extFilter num_of_workers отличный от 1?

    В моём случае процессора core_mask используется по умолчанию (закоментировано в конфиге), а 7(10)=0111(2), и поэтому заранее в tuned.conf isolcpus=1,2,3, т.е. зарезервировано три ядра для dpdk. Так?

  2. Опубликовано · Жалоба на ответ

    ОК. Сделал так:

    [root@extFilter ~]# /usr/src/dpdk-stable-16.11.1/tools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:02:00.0 '82545EM Gigabit Ethernet Controller (Copper)' drv=igb_uio unused=e1000

Network devices using kernel driver
===================================
0000:03:00.0 'VMXNET3 Ethernet Controller' if=ens160 drv=vmxnet3 unused=igb_uio *Active*
0000:0b:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=ens192 drv=tg3 unused=igb_uio

Other network devices
=====================
<none>

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other crypto devices
====================
<none>
[root@extFilter ~]#

     

    При попытке запуска ошибка

    /usr/src/extfilter/src/extFilter  --config-file=/usr/src/extfilter/etc/extfilter.ini
...
EAL: Error reading from file descriptor 14: Input/output error
EAL: Error reading from file descriptor 14: Input/output error
EAL: Error reading from file descriptor 14: Input/output error
...

     

     

    Конфиг

    [root@extFilter ~]# cat /usr/src/extfilter/etc/extfilter.ini
; Переводить имя хоста в прописные буквы. Если url_normalization установлен в true, то не имеет значения.
lower_host = true

domainlist = /root/domains
urllist = /root/urls
ssllist = /root/ssl_host
hostlist = /root/hosts

; Файл с портами для nDPI.
;protocols = /root/protos

; Список ip адресов/сетей для блокировки ssl если нет server_name в ssl hello пакете. Загружается если block_undetected_ssl = true.
sslips = /root/ssl_ips

; если false, то будет послан rst пакет вместо редиректа. Default: false
http_redirect = true

redirect_url = http://gsglobalcom.ru/block.html

; HTTP код ответа. default: 302 Moved Temporarily
http_code = 302 Found

; Что добавлять в redirect_url, line - строка из файла url, url - запрещенный url, none - ничего
url_additional_info=none


; посылать tcp rst в сторону сервера от имени клиента. Default: false
rst_to_server = true

; Default: 0 - disable
statistic_interval = 300

; Default: false
match_url_exactly = false

; Default: false
block_undetected_ssl = false

; dpdk порт, где анализировать трафик
dpdk_port = 0

; dpdk порт(ы), где анализировать трафик
;dpdk_ports = 0,1

; размер пула mbuf. Default: 8191
;mbuf_pool_size = 8191

; количество потоков по анализу трафика
;num_of_workers=1

; Какие ядра использовать. Default: все ядра, кроме management.
core_mask = 3

; файл статистики (для extfilter-cacti)
statisticsfile = /var/run/extFilter_stat

; mtu на интерфейсе для отправки пакетов в сторону абонентов. Default: 1500
; out_mtu = 1500

; Количество flow, обрабатываемых программой. Должно быть кратно 2.
; flowhash_size = 1048576

; количество тредов для отсылки уведомлений о блокировке
; num_of_senders = 1

; делать ли нормализацию url
url_normalization = true

; удалять ли точку в конце имени хоста
remove_dot = true

[logging]
loggers.root.level = information
;loggers.root.level = debug
loggers.root.channel = fileChannel
channels.fileChannel.class = FileChannel
channels.fileChannel.path = /var/log/extFilter.log
channels.fileChannel.rotation = 1 M
channels.fileChannel.purgeCount = 4
channels.fileChannel.archive = timestamp
channels.fileChannel.formatter.class = PatternFormatter
channels.fileChannel.formatter.pattern = %Y-%m-%d %H:%M:%S.%i [%P] %p %s - %t
channels.fileChannel.formatter.times = local[root@extFilter ~]#

     

    В логе:

    [root@extFilter ~]# cat /var/log/extFilter.log
2017-03-20 16:04:46.384 [1126] Information Application - Setting mbuf size to 8191
2017-03-20 16:04:46.544 [1126] Information Application - Master core is 0
2017-03-20 16:04:46.544 [1126] Fatal Application - Minimum number of required cores is 3
2017-03-20 16:27:38.812 [1181] Information Application - Setting mbuf size to 8191
2017-03-20 16:27:39.942 [1181] Information Application - Master core is 0
2017-03-20 16:27:39.942 [1181] Fatal Application - Minimum number of required cores is 3
2017-03-20 16:40:45.857 [2296] Information Application - Setting mbuf size to 8191
2017-03-20 16:41:35.596 [2297] Information Application - Setting mbuf size to 8191
2017-03-20 16:41:36.403 [2297] Information Application - Master core is 0
2017-03-20 16:41:36.403 [2297] Fatal Application - Minimum number of required cores is 3
2017-03-20 16:59:45.074 [2360] Information Application - Setting mbuf size to 8191
2017-03-20 16:59:46.338 [2360] Information Application - Master core is 0
2017-03-20 16:59:46.338 [2360] Fatal Application - Minimum number of required cores is 3
[root@extFilter ~]#

     

    В моей ВМ 2 vCPU. Это занчит надо:

    1). добавить ядра в ВМ?

    2). переделать маску core_mask?

  3. Опубликовано · Жалоба на ответ

    Подключил к ВМ сетевую карту Intel e1000 (виртуализированна ESXi).

    И пробросил Broadcom BCM5719 напрямую в ВМ через DirectPath I/O.

    Как узнать, поддерживает ли DPDK BCM5719?

     

    [root@extFilter ~]# /usr/src/dpdk-stable-16.11.1/tools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:02:00.0 '82545EM Gigabit Ethernet Controller (Copper)' drv=igb_uio unused=e1000
0000:0b:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe' drv=igb_uio unused=tg3

Network devices using kernel driver
===================================
0000:03:00.0 'VMXNET3 Ethernet Controller' if=ens160 drv=vmxnet3 unused=igb_uio *Active*

Other network devices
=====================
<none>

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other crypto devices
====================
<none>

     

    Собрал extFilter.

    При проверке доступных портов показывает что их нет

    /usr/src/extfilter/src/extFilter  -l
Setting mbuf size to 8191
DPDK ports not specified!
Exception: DPDK ports not specified!
[root@extFilter ~]#