圖片來源：openvswitch 官網
換了新環境，首要任務就是重新建立開發與測試環境，但是發現好一陣子沒當黑手，東西又忘光光了，只好回去翻一下筆記，卻發現之前寫的安裝Xen 4.x 和 OpenvSwitch for centos 6.1~6.2，已經不夠用了，主要的原因如下：
- 這次懶得裝Xen了，反正開發環境用KVM擋著應該夠了
- CentOS 也已經升級到6.3 了
- 然後之前openvswitch 是直接從source code build ，這次應該可以直接產出rpm檔
安裝
#
# 一開始先準備 for rpm build 的套件
#
sudo yum -y install rpm-build gcc openssl-devel make kernel-devel module-init-tools autoconf redhat-rpm-config
接下來就是簡化的安裝步驟
#
# 利用rpmbuild 去產生rpm檔
#
rpmbuild -bb rhel/openvswitch.spec
cd ~/rpmbuild/RPMS/x86_64/
sudo yum install kmod-openvswitch-1.4.3-1.el6.x86_64.rpm
sudo yum install openvswitch-1.4.3-1.x86_64.rpm
用rpm的好處就是許多設定檔它都幫你設定好，程式和kernel module也都已經幫你裝好了，剩下來的你只需要再確認一下就ok，不過這邊另外要注意的是，centos內建的libvirt 是比較舊的版本，libvirt 從0.9.11開始才有支援openvswitch，如果沒有自己去更新，就會發現在virt-manger 怎麼去新增vm都沒有辦法加入網卡，所以下面就繼續記錄如何更新libvirt，我是拿fedora 17的版本的SRPM來rebuild
＃
＃ 安裝 Extra Packages for Enterprise Linux repository configuration
＃
wget http://mirror.chpc.utah.edu/pub/epel/6/x86_64/epel-release-6-7.noarch.rpm
sudo yum install epel-release-6-7.noarch.rpm
#
# 抓fedora 17 的 source rpm 回來重build
#
wget http://libvirt.org/sources/libvirt-0.10.0-1.fc17.src.rpm
sudo rpm -ivh libvirt-0.10.0-1.fc17.src.rpm
#
# 在重build之前，要把這套件相依的套件都要一起裝起來
#
sudo yum install avahi libxml2-devel gnutls-devel device-mapper-devel \
python-devel libnl-devel dejavu-lgc-sans-fonts openssl-devel yajl-devel \
avahi-devel libssh2-devel libcurl-devel xhtml1-dtds readline-devel \
ncurses-devel libtasn1-devel augeas libpciaccess-devel sanlock-devel \
libpcap-devel cyrus-sasl-devel parted-devel numactl-devel libcap-ng-devel \
netcf-devel audit-libs-devel systemtap-sdt-devel scrub numad libblkid-devel\
ibuuid-devel.x86_64 sanlock-lib.x86_64
sudo yum install ftp://rpmfind.net/linux/centos/6.3/os/x86_64/Packages/numad-0.5-4.20120522git.el6.x86_64.rpm
#
#開始重build
#
rpmbuild -bb ~/rpmbuild/SPECS/libvirt.spec
#
# build 好之後先把舊的套件移除
#
sudo yum remove libvirt-client
#
# 然後再安裝新的套件
#
cd ~/rpmbuild/RPMS/x86_64/
sudo yum install libvirt-*
sudo yum install virt-manager
#
# 最後再把libvirt重新啓動
#
sudo service libvirtd start
sudo chkconfig libvirtd on
設定
整個在openvswitch 在使用的經驗中，最讓我不解的就是到底br0 需不需要設ip，如果以傳統硬體機器來說，bridge本身應該不需要設定ip啊? (除非是我想錯了...)，以架構來說應該長得像下圖一樣。
+--------------------------------------------+
| host |
|--------------------------------------------|
| +----------+ +----------+ |
| | VM1 | ....... | VMn | |
| +----+-----+ +-----+----+ |
+------|-----------------------------|-------+
| +---+----+ +-----+----+ |
| | tap0 | ....... | tapn | |
| ++---+----+------------------+-----+----++ |
| | | ovs | | |
| | +-------------++--------------+ | |
| | +----+-----+ | |
| | | br0 | | |
| | +----+-----+ | |
| +------------------|---------------------+ |
| | |
| +-----------+ |
| | |
| +-----+-----+ +----------------+ |
| | | | | |
| | eth0 | | eth1 | |
+--+-----------+----------+----------------+-+
但是在openvswitch 上面的FAQ 開宗明義就說要把ip設定在br0 上...
Q: I created a bridge and added my Ethernet port to it, using commands
like these:
ovs-vsctl add-br br0
ovs-vsctl add-port br0 eth0
and as soon as I ran the "add-port" command I lost all connectivity
through eth0. Help!
A: A physical Ethernet device that is part of an Open vSwitch bridge
should not have an IP address. If one does, then that IP address
will not be fully functional.
You can restore functionality by moving the IP address to an Open
vSwitch "internal" device, such as the network device named after
the bridge itself. For example, assuming that eth0's IP address is
192.168.128.5, you could run the commands below to fix up the
situation:
ifconfig eth0 0.0.0.0
ifconfig br0 192.168.128.5
(If your only connection to the machine running OVS is through the
IP address in question, then you would want to run all of these
commands on a single command line, or put them into a script.) If
there were any additional routes assigned to eth0, then you would
also want to use commands to adjust these routes to go through br0.
If you use DHCP to obtain an IP address, then you should kill the
DHCP client that was listening on the physical Ethernet interface
(e.g. eth0) and start one listening on the internal interface
(e.g. br0). You might still need to manually clear the IP address
from the physical interface (e.g. with "ifconfig eth0 0.0.0.0").
There is no compelling reason why Open vSwitch must work this way.
However, this is the way that the Linux kernel bridge module has
always worked, so it's a model that those accustomed to Linux
bridging are already used to. Also, the model that most people
expect is not implementable without kernel changes on all the
versions of Linux that Open vSwitch supports.
By the way, this issue is not specific to physical Ethernet
devices. It applies to all network devices except Open vswitch
"internal" devices.
但是我設定好之後，從virt-manager 去啓動vm 又遇到這幾個問題：
- Unable to add bridge br0 port vnet0: No such device
- Unable to add bridge br0 port vnet0: Operation not supported
後來我又遇到一個笨問題，就是因為參考最上面的圖的架構，所以我一直沒有把eth1加入br0，造成我的vm一直無法連到外面，後來參考這篇文章"why vm can't ping host eth1"，才讓我茅塞頓開解決這個問題。
目前看起來openvswitch 是可以正常運作了....
沒有留言 :
張貼留言