Jump to content
Калькуляторы

virtlib xen pv gentoo

Reply to this topic

tokra   

tokra
Posted

Всем доброго времени суток, господа.

 

Что мы имеем.:

1. Сервер PowerEdge R910 - Dell 

processor	: 79
vendor_id	: GenuineIntel
cpu family	: 6
model		: 47
model name	: Intel(R) Xeon(R) CPU E7- 4870  @ 2.40GHz
stepping	: 2
microcode	: 0x36
cpu MHz		: 2394.060
cache size	: 30720 KB
physical id	: 3
siblings	: 20
core id		: 25
cpu cores	: 10
apicid		: 243
initial apicid	: 243
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu de tsc msr pae cx8 apic sep cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc pni pclmulqdq monitor est ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm ida arat dtherm
bogomips	: 4788.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual

 

2. Система gentoo adm64 с ядром 3.5.7-gentoo

 

3. Установлен xen и xen-tools

app-emulation/xen-4.1.1-r2 was built with the following:
USE="(multilib) -custom-cflags -debug -flask -pae -xsm"
CFLAGS=""

app-emulation/xen-tools-4.1.1-r6 was built with the following:
USE="hvm (multilib) qemu screen xend -api -custom-cflags -debug -doc -flask -pygrub"
CFLAGS="-fno-strict-overflow"
CXXFLAGS="-O2 -pipe -fno-strict-overflow"
LDFLAGS=""

 

xen используется для виртуализации в режиме Paravirtualization. В качестве гостевых ОС тот же самый gentoo.

Конфиг гостевой ОС выглядит так:

kernel = "/boot/DomU"
vcpus = 2
memory = 4096
name = "test"
vif = [ 'bridge=peth4' ]
disk = [ 'phy:/dev/vlxen/root,xvda,w', 'phy:/dev/vlxen/swap,xvdb,w' ]
root = "/dev/xvda ro" 
extra = "console=hvc0 xencons=xvc0"
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash	= 'restart'

 

В таком варианте все супер, все работает, делаешь xm create test.cfg и радуемся)

 

Дернул меня черт попробовать рулить всем этим добром через virtlib, т.к. есть успешный опыт работы с ним в KVM и как бы поддержка XEN есть.

Поставил себе libvirt

app-emulation/libvirt-0.10.2-r3 was built with the following:
USE="caps libvirtd lvm lxc macvtap (multilib) nls python udev vepa virt-network xen -audit -avahi -debug -iscsi -nfs -numa -openvz -parted -pcap -phyp -policykit -qemu -rbd -sasl (-selinux) -uml -virtualbox"

 

Поправил xend-config.sxp на предмет (xend-unix-server yes), сделал /etc/init.d/xend restart и /etc/init.d/libvirtd start. Вроде все норм.

Проверяем 

 virsh -c xen:///
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
   'quit' to quit

virsh #

вроде счастье есть.

 

Теперь решил преобразовать стандартный конфиг виртуалки в xml понятный libvirt

virsh -c xen:/// domxml-from-native xen-xm test.cfg 
<domain type='xen'>
 <name>test</name>
 <uuid>5e8e64d2-fa11-bf72-b2e1-4e8642fdf8a6</uuid>
 <memory unit='KiB'>4194304</memory>
 <currentMemory unit='KiB'>4194304</currentMemory>
 <vcpu placement='static'>2</vcpu>
 <os>
<type arch='x86_64' machine='xenpv'>linux</type>
<kernel>/boot/DomU</kernel>
<cmdline>console=hvc0 xencons=xvc0</cmdline>
 </os>
 <clock offset='utc' adjustment='reset'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
<disk type='block' device='disk'>
  <driver name='phy'/>
  <source dev='/dev/vlxen/root'/>
  <target dev='xvda' bus='xen'/>
</disk>
<disk type='block' device='disk'>
  <driver name='phy'/>
  <source dev='/dev/vlxen/swap'/>
  <target dev='xvdb' bus='xen'/>
</disk>
<interface type='bridge'>
  <mac address='00:00:00:00:00:00'/>
  <source bridge='peth4'/>
</interface>
<console type='pty'>
  <target type='xen' port='0'/>
</console>
 </devices>
</domain>

 

Это добро сохраняем в test.xml и пробуем запустить

virsh -c xen:///
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
   'quit' to quit

virsh # create test.xml 
Domain test created from test.xml

Вроде все номр, но делаем

virsh # list
Id	Name						   State
----------------------------------------------------
0	 Domain-0					   running

 

И как видим нифига.

 

Проверяю через xm create родной конфиг - все пучком стартует и работает.

 

Лезем в логи

tail -f libvirtd.log 
') unexpected exit status 1
2012-11-05 07:49:51.960+0000: 9375: error : ebiptablesDriverTestCLITools:4241 : Testing of ebtables command failed: Failure to execute command '$EBT -t nat -L' : 'FATAL: Module ebtables not found.
The kernel doesn't support the ebtables 'nat' table.'.

2012-11-05 07:50:38.515+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.516+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.517+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.518+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:50:38.519+0000: 9367: error : xenHypervisorDoV2Sys:1007 : Unable to issue hypervisor ioctl 3166208: Permission denied
2012-11-05 07:54:23.824+0000: 9371: error : virConnectListAllDomains:8250 : this function is not supported by the connection driver: virConnectListAllDomains

 

Есть только такие сообщения на момент создания домена.

Поиск в лоб в гугле ничего вменяемого не дал.

 

Вот и вопрос к аудитории: использует ли кто-то в продакшене libvirt для управления гипервизором xen и если да, то сталкивался ли кто-то с такой проблемой).

Share this post

Link to post
Share on other sites

tokra   

tokra
Posted (edited)

При старте средствами virsh kernel punic

[    0.393605] device-mapper: multipath queue-length: version 0.1.0 loaded
[    0.393609] device-mapper: multipath service-time: version 0.2.0 loaded
[    0.393942] dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
[    0.393952] No iBFT detected.
[    0.394132] usbcore: registered new interface driver usbhid
[    0.394138] usbhid: USB HID core driver
[    0.394157] Netfilter messages via NETLINK v0.30.
[    0.394169] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.394404] ctnetlink v0.93: registering with nfnetlink.
[    0.394566] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.394608] TCP: cubic registered
[    0.394612] Initializing XFRM netlink socket
[    0.394898] NET: Registered protocol family 10
[    0.395186] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.395219] sit: IPv6 over IPv4 tunneling driver
[    0.395476] NET: Registered protocol family 17
[    0.395506] 8021q: 802.1Q VLAN Support v1.8
[    0.395713] registered taskstats version 1
[    0.395774] console [netcon0] enabled
[    0.395780] netconsole: network logging started
[    0.395853] md: Waiting for all devices to be available before autodetect
[    0.395859] md: If you don't use raid, use raid=noautodetect
[    0.396064] md: Autodetecting RAID arrays.
[    0.396071] md: Scanned 0 and added 0 devices.
[    0.396074] md: autorun ...
[    0.396077] md: ... autorun DONE.
[    0.396106] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[    0.396112] Please append a correct "root=" boot option; here are the available partitions:
[    0.396120] ca00        20971520 xvda  driver: vbd
[    0.396125] ca10         6291456 xvdb  driver: vbd
[    0.396130] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    0.396137] Pid: 1, comm: swapper/0 Not tainted 3.5.7-gentoo #5
[    0.396141] Call Trace:
[    0.396150]  [<ffffffff815bce7e>] panic+0xb8/0x1ca
[    0.396154]  [<ffffffff815bcfd8>] ? printk+0x48/0x4a
[    0.396161]  [<ffffffff81a89f86>] mount_block_root+0x1d2/0x1f0
[    0.396166]  [<ffffffff81a8a194>] mount_root+0xe1/0xec
[    0.396171]  [<ffffffff81a8a304>] prepare_namespace+0x165/0x192
[    0.396176]  [<ffffffff81a89c52>] kernel_init+0x1b8/0x1ca
[    0.396182]  [<ffffffff81a894de>] ? loglevel+0x31/0x31
[    0.396187]  [<ffffffff815c4f64>] kernel_thread_helper+0x4/0x10
[    0.396193]  [<ffffffff815bee78>] ? retint_restore_args+0x5/0x6
[    0.396198]  [<ffffffff815c4f60>] ? gs_change+0x13/0x13

 

А теперь эта же виртуалка, но стартуем через xm create

*     127.0.0.0/8 via 127.0.0.1 ...
[ ok ]
* Mounting misc binary format filesystem ...
[ ok ]
* Loading custom binary format handlers ...
[ ok ]
* Activating swap devices ...
[ ok ]
* Initializing random number generator ...
[ ok ]
INIT: Entering runlevel: 3
* Bringing up interface eth0
*   10.30.10.149/24 ...
[ ok ]
*   Adding routes
*     default via 10.30.10.254 ...
[ ok ]
*   Waiting for IPv6 addresses ...
[ ok ]
* Mounting network filesystems ...
[ ok ]
* Starting syslog-ng ...
[ ok ]
* Setting clock via the NTP client 'ntpdate' ...

This is BigXEN_test.unknown_domain (Linux x86_64 3.5.7-gentoo) 03:27:43

BigXEN_test login:

 

Интересно девки пляшут)

 

Есть подозрения, что связанно с этим

Please append a correct "root=" boot option; here are the available partitions:
[    0.396120] ca00        20971520 xvda  driver: vbd
[    0.396125] ca10         6291456 xvdb  driver: vbd

 

Т.к. в родном конфиге присутствует

root = "/dev/xvda ro"

 

а вот уже в конфиге, который сгенерил libvirt только

<disk type='block' device='disk'>

<driver name='phy'/>

<source dev='/dev/vlxen/root'/>

<target dev='xvda' bus='xen'/>

</disk>

<disk type='block' device='disk'>

<driver name='phy'/>

<source dev='/dev/vlxen/swap'/>

<target dev='xvdb' bus='xen'/>

</disk>

Edited by tokra

Share this post

Link to post
Share on other sites

tokra   

tokra
Posted

И так, подведем итоги)

 

Вся загвоздка была в <root>/dev/xvda</root>

 

Итоговый рабочий конфиг

<domain type='xen'>
 <name>test</name>
 <uuid>ffecd709-4db2-e410-37fc-03fa7d65f81e</uuid>
 <memory unit='KiB'>4194304</memory>
 <currentMemory unit='KiB'>4194304</currentMemory>
 <vcpu placement='static'>2</vcpu>
 <os>
   <type arch='x86_64' machine='xenpv'>linux</type>
   <kernel>/boot/DomU</kernel>
   <root>/dev/xvda</root>
   <cmdline>console=hvc0 xencons=xvc0</cmdline>
 </os>
 <clock offset='utc' adjustment='reset'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
   <disk type='block' device='disk'>
     <driver name='phy'/>
     <source dev='/dev/vlxen/root'/>
     <target dev='xvda' bus='xen'/>
   </disk>
   <disk type='block' device='disk'>
     <driver name='phy'/>
     <source dev='/dev/vlxen/swap'/>
     <target dev='xvdb' bus='xen'/>
   </disk>
   <interface type='bridge'>
     <mac address='00:00:00:00:00:00'/>
     <source bridge='peth4'/>
   </interface>
   <console type='pty'>
     <target type='xen' port='0'/>
   </console>
 </devices>
</domain>

Share this post

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Followers 0
Go to topic listing Программное обеспечение, биллинг и *unix системы