Lab 1: Virtual Technelogy: UML and KVM


本實驗示範如何運用 Paravirtualization (UML) 與 Full virtualization (KVM) 建立虛擬叢集環境.


理學大樓 821 機房
CPU 4 AMD Opteron(tm) Processor 6128 processors
(16 cpu cores)
Memory 8G
Disk space 500G*2
O.S. Debian squeeze
CPU 4 AMD Opteron(tm) Processor 6128 processors
(16 cpu cores)
Memory 8G
Disk space 500G*2
O.S. Debian squeeze


隱藏網址: 因為這是公眾網頁, 為了安全, 除了 av-09, 所有網址都該殘缺.

入口機器(跳板), av-09:

  1. 建立 User Mode Linux, UML, 虛擬機器.
    1. UML Compilation
    2. UML Template Creation

    1. Login CSIE-Cloud
        $ ssh -X cloud@
        $ ssh -X ctsp${Tn}@CSIE-Cloud01 # Replace CSIE-Cloud01 by IP
    2. Check whether you got linux.uml and its related module
      註: 下列幾步已經由助教完成.
    3.   # The lines leading by "#" sign need not to be done.
        # $ sudo apt-get install uml-utilities vde2
        # $ ls -l /usr/local/lib/uml 
        # If you get No such file or directory 
        # $ sudo mkdir /usr/local/lib/uml 
        # $ sudo chown cloud:cloud /usr/local/lib/uml 
        # $ cd /usr/local/lib/uml 
        # $ scp hsu@as:/backup/tmp/uml-2011-08-01.tgz .
        # $ tar -zxvf uml-2011-08-01.tgz 
        # $ ls -l 
        # $ rm uml-2011-08-01.tgz 
        # Later on, these files will be re-generated.
        # $ sudo cp linux.uml /usr/local/bin
    4. Now, get template and related tools from our server
        $ Tn={YOUR_TEAM_NUMBER}
        $ cd /src3
        $ sudo mkdir /src3/UML-ctsp${Tn}
        $ sudo chown ctsp${Tn}:ctsp${Tn} /src3/UML-ctsp${Tn}
        $ cd /src3/UML-ctsp${Tn} 
        $ rm -rf /src3/UML-ctsp${Tn}/*
        $ cp -r /src3/UML/* . 
        $ ln Config-UML-Rfs Config-UML-Rfs-UmlSwitch 
        $ gunzip DebSqz-UltraLight.ext3.gz 
        $ sudo mount -o loop DebSqz-UltraLight.ext3 /mnt/ctsp${Tn} 
        # In another xterm, 避免頻繁切換路徑
        Login CSIE-Cloud again
        $ ssh -X cloud@
        $ ssh -X cloud@CSIE-Cloud01 # Replace CSIE-Cloud01 by IP
        $ sudo rm -rf /mnt/ctsp${Tn}/lib/modules/* 
        $ cd /usr/local/lib/uml/lib/modules/ 
       # 請以 man 指令查詢說明
        $ man find
        $ man cpio
        $ find . -print | sudo cpio -pdm /mnt/ctsp${Tn}/lib/modules/ 
        $ sync; sync 
        $ ls -l /mnt/ctsp${Tn}/lib/modules 
        # Back to the original xterm 
       $ sudo umount /mnt/ctsp${Tn} 
    5. Configure and Test VM
        $ Config-UML-Rfs
      ./Config-UML-Rfs root-fs-pathname uml-hostname UML-IP Ether-Card [TAP-No]
          TAP-No is optional, if need tap to be different from tap0.
        $ Config-UML-Rfs ./DebSqz-UltraLight.ext3 vm-${Tn} 192.168.1.`expr ${Tn} + 100` eth0:0 ${Tn}
        # Hopefully, you get startUML, startUMLAsDaemon, stop-uml-restore-lan, 3 files.
        # The first script is for starting UML in foreground.  For system repairing
        # or maintenance reasons, from time to time, we need to start our VMs in 
        # the foreground.  For common usage, whether it be special server, compute
        # slave in some high performance cluster, usually, we start it as a daemon.
        # After halting a VM, it is our job to restore the system network state to 
        # its origin. Otherwise, next time, we will have hard time to get our VM's
        # virtual network working again.
        # Nowadays, the smallest graphical system needs at least 1GB.  Otherwise, 
        # it would be almost impossible to upgrade system safely.  How I hate it!
        $ startUML 
        # login UML as cloud, and in UML
        $ su 
        ;; The first time online, take care of cached share object library database
        ;; so that the system know where they are, especially, some libraries are 
        ;; are from /usr/local/lib.
        # ldconfig
        # aptitude update; aptitude safe-upgrade 
        # aptitude clean 
        # deborphan 
        ;; If anything shows up,
        # dpkg -P `deborphan` ;; Do these two commands until deborphan return nothing
        # halt ;; shutdown uml
        ;; We need to restore lan to its original state 
        $ ifconfig -a ;; You see tap0, a virtual ethernet card
        $ stop-uml-restore-lan 
        $ ifconfig -a ;; The tap0 virtual ethernet card should disappear 
        # $ startUMLAsDaemon  
        # $ ps -C linux.uml 
        ;; Wait for two or three minutes 
        $ ssh -X cloud@192.168.1.`expr ${Tn} + 100` ;; check whether you can login or not.
        ;; After login, type exit.

  2. 建立 Kernel-based virtual machine, KVM, 虛擬機器.
  3. The procedures involved for starting and halting VMs are rather complicated, but the command lines need to accomplish the whole job are quite similar. If we are going to key-in all the commands by hands, it is rather error-prone. Our biggest contribution is that we unify the whole process, from starting to halting UML or KVM VMs, via rather similar procedures, such as startVM in foreground, startVM as daemon, stop-VM-restore-lan. And we write a shell script to prepare the above 3 scripts automatically.

    1. Kvm Compilation
    2. Kvm Template Creation

    1. Check hardware virtualization support, and then install KVM.
      註: 下列幾步已經由助教完成.
        $ egrep '(vmx|svm)' --color=always /proc/cpuinfo # Hardware Assisted Virtual 
        # $ sudo aptitude update
        # $ sudo aptitude install kvm qemu-utils vde2 socat
        # $ sudo modprobe kvm_amd
        ## Reboot
        # $ sudo adduser ctsp${Tn} kvm
    2. Prepare for VM.
        $ Tn={YOUR_TEAM_NUMBER}
        $ sudo mkdir /src4/KVM-ctsp${Tn}
        $ sudo chown ctsp${Tn}:ctsp${Tn} /src4/KVM-ctsp${Tn}
        $ cd /src4/KVM-ctsp${Tn}; mkdir Resize img
        $ cp /backup/KVM-tool/KVM-tool-*.tgz .
        # 系統中存有一份即可,不需要每個人都複製檔案系統樣板!
        # $ cp /backup/KVM-tool/Resize/Resize-*.tgz /src4/KVM/Resize
        # $ cd /src4/KVM/Resize; tar -zxvf Resize-*.tgz; rm Resize-*.tgz 
        $ cd ..; tar -zxvf KVM-tool-*.tgz; rm KVM-tool-*.tgz 
    3. Create KVM root-filesystem.
        $ cd img 
        $ qemu-img create DebSqz-Mini.img 2G 
        $ kvm -cdrom ../Resize/debian- DebSqz-Mini.img -m 512M
      Partition the image as: /boot and /

      Give 512M to boot, turn on its boot flag, use ext2 filesystem format. The rest for / partition, use ext4 filesystem format. Install only the minimal packages offered by the installation menu, but must include openssh-server, since we need to login the VM via ssh.

    4. Configure VM
        $ cd ../bin 
        $ ls -lia Config-Kvm* 
        # Notice the three files are the same. When invoke them by different names,
        # we can configure image files for different OSes.
        # Also, as far as I remembered, newer Debian VM need to run the command
        # ./recover70rules. Otherwise, next time eth0 is no longer available. 
        $ ./Config-Kvm 
        # We use this script to configure ../Resize/Debian-Gparted.img. 
        # VMIP 請參考 IP 分配表。
        $ Config-Kvm ../img/DebSqz-Mini.img kvm-${Tn} ${VMIP} eth1 kvm-${Tn}
        $ start-kvm*-1 
        # login KVM as cloud, and in KVM
        $ su
        # ./recover70rules 
        # halt 
        $ start-kvm*-1-AsDaemon
        $ ssh -X cloud@${VMIP}
        ;; After login, type exit.
        $ exit
        $ stop-kvm*-restore-lan-1

Lab 1: Assignment

複製 UML template
 $ cd /src3/UML-ctsp20
 $ cp /src3/UML/DebSqz-UltraLight.ext3.gz uml1.ext3.gz
 $ cp /src3/UML/DebSqz-UltraLight.ext3.gz uml2.ext3.gz
複製 KVM template
 $ cd /src4/KVM-ctsp20/img
 $ cp /src4/KVM/Resize/Debian-Mini.img.gz kvm1.img.gz
 $ cp /src4/KVM/Resize/Debian-Mini.img.gz kvm2.img.gz

註1:抱歉,由於我的疏失,造成大家複製了大量的 filesystem template($ cp /backup/KVM-tool/Resize/Resize-*.tgz Resize),
造成 /src4 空間銳減。因此,請大家使用以下指令清空各自目錄中之 Resize 目錄。

  $ rm /src4/KVM-ctsp${Tn}/*.img
爾後請從 /src4/KVM/Resize/ 中複製相關之樣板至各自目錄中 img 目錄,如前段 (複製 KVM template) 所述。

註2:Private LAN 已建置完成,所以不再使用 eth0:0 之虛擬網卡,請改用 eth1。