Jump to content

virtlib xen pv gentoo

tokra
 Share

Recommended Posts

tokra

tokra

Posted
Posted

Всем доброго времени суток, господа.

 

Что мы имеем.:

1. Сервер PowerEdge R910 - Dell 

processor	: 79
vendor_id	: GenuineIntel
cpu family	: 6
model		: 47
model name	: Intel(R) Xeon(R) CPU E7- 4870  @ 2.40GHz
stepping	: 2
microcode	: 0x36
cpu MHz		: 2394.060
cache size	: 30720 KB
physical id	: 3
siblings	: 20
core id		: 25
cpu cores	: 10
apicid		: 243
initial apicid	: 243
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu de tsc msr pae cx8 apic sep cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc pni pclmulqdq monitor est ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm ida arat dtherm
bogomips	: 4788.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual

 

2. Система gentoo adm64 с ядром 3.5.7-gentoo

 

3. Установлен xen и xen-tools

app-emulation/xen-4.1.1-r2 was built with the following:
USE="(multilib) -custom-cflags -debug -flask -pae -xsm"
CFLAGS=""

app-emulation/xen-tools-4.1.1-r6 was built with the following:
USE="hvm (multilib) qemu screen xend -api -custom-cflags -debug -doc -flask -pygrub"
CFLAGS="-fno-strict-overflow"
CXXFLAGS="-O2 -pipe -fno-strict-overflow"
LDFLAGS=""

 

xen используется для виртуализации в режиме Paravirtualization. В качестве гостевых ОС тот же самый gentoo.

Конфиг гостевой ОС выглядит так:

kernel = "/boot/DomU"
vcpus = 2
memory = 4096
name = "test"
vif = [ 'bridge=peth4' ]
disk = [ 'phy:/dev/vlxen/root,xvda,w', 'phy:/dev/vlxen/swap,xvdb,w' ]
root = "/dev/xvda ro" 
extra = "console=hvc0 xencons=xvc0"
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash	= 'restart'

 

В таком варианте все супер, все работает, делаешь xm create test.cfg и радуемся)

 

Дернул меня черт попробовать рулить всем этим добром через virtlib, т.к. есть успешный опыт работы с ним в KVM и как бы поддержка XEN есть.

Поставил себе libvirt

app-emulation/libvirt-0.10.2-r3 was built with the following:
USE="caps libvirtd lvm lxc macvtap (multilib) nls python udev vepa virt-network xen -audit -avahi -debug -iscsi -nfs -numa -openvz -parted -pcap -phyp -policykit -qemu -rbd -sasl (-selinux) -uml -virtualbox"

 

Поправил xend-config.sxp на предмет (xend-unix-server yes), сделал /etc/init.d/xend restart и /etc/init.d/libvirtd start. Вроде все норм.

Проверяем 

 virsh -c xen:///
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
   'quit' to quit

virsh #

вроде счастье есть.

 

Теперь решил преобразовать стандартный конфиг виртуалки в xml понятный libvirt

virsh -c xen:/// domxml-from-native xen-xm test.cfg 
<domain type='xen'>
 <name>test</name>
 <uuid>5e8e64d2-fa11-bf72-b2e1-4e8642fdf8a6</uuid>
 <memory unit='KiB'>4194304</memory>
 <currentMemory unit='KiB'>4194304</currentMemory>
 <vcpu placement='static'>2</vcpu>
 <os>
<type arch='x86_64' machine='xenpv'>linux</type>
<kernel>/boot/DomU</kernel>
<cmdline>console=hvc0 xencons=xvc0</cmdline>
 </os>
 <clock offset='utc' adjustment='reset'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
<disk type='block' device='disk'>
  <driver name='phy'/>
  <source dev='/dev/vlxen/root'/>
  <target dev='xvda' bus='xen'/>
</disk>
<disk type='block' device='disk'>
  <driver name='phy'/>
  <source dev='/dev/vlxen/swap'/>
  <target dev='xvdb' bus='xen'/>
</disk>
<interface type='bridge'>
  <mac address='00:00:00:00:00:00'/>
  <source bridge='peth4'/>
</interface>
<console type='pty'>
  <target type='xen' port='0'/>
</console>
 </devices>
</domain>

 

Это добро сохраняем в test.xml и пробуем запустить

virsh -c xen:///
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
   'quit' to quit

virsh # create test.xml 
Domain test created from test.xml

Вроде все номр, но делаем

virsh # list
Id	Name						   State
----------------------------------------------------
0	 Domain-0					   running

 

И как видим нифига.

 

Проверяю через xm create родной конфиг - все пучком стартует и работает.

 

Лезем в логи

tail -f libvirtd.log 
') unexpected exit status 1
2012-11-05 07:49:51.960+0000: 9375: error : ebiptablesDriverTestCLITools:4241 : Testing of ebtables command failed: Failure to execute command '$EBT -t nat -L' : 'FATAL: Module ebtables not found.
The kernel doesn't support the ebtables 'nat' table.'.

2012-11-05 07:50:38.515+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.516+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.517+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.518+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.519+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:54:23.824+0000: 9371: error : virConnectListAllDomains:8250 : this function is not supported by the connection driver: virConnectListAllDomains

 

Есть только такие сообщения на момент создания домена.

Поиск в лоб в гугле ничего вменяемого не дал.

 

Вот и вопрос к аудитории: использует ли кто-то в продакшене libvirt для управления гипервизором xen и если да, то сталкивался ли кто-то с такой проблемой).

tokra

tokra

Posted
  • Author
Posted (edited)

При старте средствами virsh kernel punic

[    0.393605] device-mapper: multipath queue-length: version 0.1.0 loaded
[    0.393609] device-mapper: multipath service-time: version 0.2.0 loaded
[    0.393942] dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
[    0.393952] No iBFT detected.
[    0.394132] usbcore: registered new interface driver usbhid
[    0.394138] usbhid: USB HID core driver
[    0.394157] Netfilter messages via NETLINK v0.30.
[    0.394169] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.394404] ctnetlink v0.93: registering with nfnetlink.
[    0.394566] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.394608] TCP: cubic registered
[    0.394612] Initializing XFRM netlink socket
[    0.394898] NET: Registered protocol family 10
[    0.395186] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.395219] sit: IPv6 over IPv4 tunneling driver
[    0.395476] NET: Registered protocol family 17
[    0.395506] 8021q: 802.1Q VLAN Support v1.8
[    0.395713] registered taskstats version 1
[    0.395774] console [netcon0] enabled
[    0.395780] netconsole: network logging started
[    0.395853] md: Waiting for all devices to be available before autodetect
[    0.395859] md: If you don't use raid, use raid=noautodetect
[    0.396064] md: Autodetecting RAID arrays.
[    0.396071] md: Scanned 0 and added 0 devices.
[    0.396074] md: autorun ...
[    0.396077] md: ... autorun DONE.
[    0.396106] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[    0.396112] Please append a correct "root=" boot option; here are the available partitions:
[    0.396120] ca00        20971520 xvda  driver: vbd
[    0.396125] ca10         6291456 xvdb  driver: vbd
[    0.396130] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    0.396137] Pid: 1, comm: swapper/0 Not tainted 3.5.7-gentoo #5
[    0.396141] Call Trace:
[    0.396150]  [<ffffffff815bce7e>] panic+0xb8/0x1ca
[    0.396154]  [<ffffffff815bcfd8>] ? printk+0x48/0x4a
[    0.396161]  [<ffffffff81a89f86>] mount_block_root+0x1d2/0x1f0
[    0.396166]  [<ffffffff81a8a194>] mount_root+0xe1/0xec
[    0.396171]  [<ffffffff81a8a304>] prepare_namespace+0x165/0x192
[    0.396176]  [<ffffffff81a89c52>] kernel_init+0x1b8/0x1ca
[    0.396182]  [<ffffffff81a894de>] ? loglevel+0x31/0x31
[    0.396187]  [<ffffffff815c4f64>] kernel_thread_helper+0x4/0x10
[    0.396193]  [<ffffffff815bee78>] ? retint_restore_args+0x5/0x6
[    0.396198]  [<ffffffff815c4f60>] ? gs_change+0x13/0x13

 

А теперь эта же виртуалка, но стартуем через xm create

*     127.0.0.0/8 via 127.0.0.1 ...
[ ok ]
* Mounting misc binary format filesystem ...
[ ok ]
* Loading custom binary format handlers ...
[ ok ]
* Activating swap devices ...
[ ok ]
* Initializing random number generator ...
[ ok ]
INIT: Entering runlevel: 3
* Bringing up interface eth0
*   10.30.10.149/24 ...
[ ok ]
*   Adding routes
*     default via 10.30.10.254 ...
[ ok ]
*   Waiting for IPv6 addresses ...
[ ok ]
* Mounting network filesystems ...
[ ok ]
* Starting syslog-ng ...
[ ok ]
* Setting clock via the NTP client 'ntpdate' ...

This is BigXEN_test.unknown_domain (Linux x86_64 3.5.7-gentoo) 03:27:43

BigXEN_test login:

 

Интересно девки пляшут)

 

Есть подозрения, что связанно с этим

Please append a correct "root=" boot option; here are the available partitions:
[    0.396120] ca00        20971520 xvda  driver: vbd
[    0.396125] ca10         6291456 xvdb  driver: vbd

 

Т.к. в родном конфиге присутствует

root = "/dev/xvda ro"

 

а вот уже в конфиге, который сгенерил libvirt только

<disk type='block' device='disk'>

<driver name='phy'/>

<source dev='/dev/vlxen/root'/>

<target dev='xvda' bus='xen'/>

</disk>

<disk type='block' device='disk'>

<driver name='phy'/>

<source dev='/dev/vlxen/swap'/>

<target dev='xvdb' bus='xen'/>

</disk>

Edited by tokra
tokra

tokra

Posted
  • Author
Posted

И так, подведем итоги)

 

Вся загвоздка была в <root>/dev/xvda</root>

 

Итоговый рабочий конфиг

<domain type='xen'>
 <name>test</name>
 <uuid>ffecd709-4db2-e410-37fc-03fa7d65f81e</uuid>
 <memory unit='KiB'>4194304</memory>
 <currentMemory unit='KiB'>4194304</currentMemory>
 <vcpu placement='static'>2</vcpu>
 <os>
   <type arch='x86_64' machine='xenpv'>linux</type>
   <kernel>/boot/DomU</kernel>
   <root>/dev/xvda</root>
   <cmdline>console=hvc0 xencons=xvc0</cmdline>
 </os>
 <clock offset='utc' adjustment='reset'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
   <disk type='block' device='disk'>
     <driver name='phy'/>
     <source dev='/dev/vlxen/root'/>
     <target dev='xvda' bus='xen'/>
   </disk>
   <disk type='block' device='disk'>
     <driver name='phy'/>
     <source dev='/dev/vlxen/swap'/>
     <target dev='xvdb' bus='xen'/>
   </disk>
   <interface type='bridge'>
     <mac address='00:00:00:00:00:00'/>
     <source bridge='peth4'/>
   </interface>
   <console type='pty'>
     <target type='xen' port='0'/>
   </console>
 </devices>
</domain>

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...
На сайте используются файлы cookie и сервисы аналитики для корректной работы форума и улучшения качества обслуживания. Продолжая использовать сайт, вы соглашаетесь с использованием файлов cookie и с Политикой конфиденциальности.