Saturday, July 21, 2012

OpenVZ HowTo

For installing openvz , first an OS need to be installed. So CentOS is first installed on bare machine, which is followed by Openvz installation

OpenVZ setup involves the basic two steps
  1. Installation of Openvz kernel and booting into the kernel
  2. Downloading of precreated OS template distribution of your choice(CentOS, Debian, Ubuntu etc) and installing them.

OpenVZ kernel installation

Get the OpenVZ repository. The repository has the openvz kernels
# cd /etc/yum.repos.d/
# wget http://download.openvz.org/openvz.repo
# rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
# yum update

# yum search vzkernel
======N/S Matched: vzkernel =============
vzkernel.i686 : The Linux kernel
vzkernel.x86_64 : The Linux kernel
vzkernel-devel.i686 : Development package for building kernel modules to match
                    : the kernel
vzkernel-devel.x86_64 : Development package for building kernel modules to match
                      : the kernel
vzkernel-firmware.noarch : Firmware files used by the Linux kernel
vzkernel-headers.i686 : Header files for the Linux kernel for use by glibc
vzkernel-headers.x86_64 : Header files for the Linux kernel for use by glibc

If the machine of x86_64 architecture and the base OS(CentOS) installed is 64-bit, install vzkernel.x86_64

# yum install vzkernel.x86_64 vzkernel-devel.x86_64 vzkernel-headers.x86_64

Check  /boot/grub/menu.lst. It must have the entry for the new kernel

title OpenVZ (2.6.32-042stab057.1)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-042stab057.1 ro root=UUID=fffff7aa-57b8-40aa-baa4-588c4eff7651 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
        initrd /initramfs-2.6.32-042stab057.1.img

# yum install vzctl vzquota

Ensure /etc/sysctl.conf has the following entries

net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1 

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp = 0
kernel.sysrq = 1
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth0.proxy_arp=1

Update the new kernel settings
# sysctl -p

Disable SELINUX . In the file, /etc/selinux/config, set
SELINUX=disabled

Reboot the machine
# shutdown -r now

In the grub menu, a new kernel will apppear(2.6.32-042stab057.1, in my case). Boot this kernel

Once booted, Check that a new network interface, venet0, exists

# ifconfig venet0
venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet6 addr: fe80::1/128 Scope:Link
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:374 errors:0 dropped:0 overruns:0 frame:0
          TX packets:454 errors:0 dropped:6 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:37637 (36.7 KiB)  TX bytes:39793 (38.8 KiB)

Also, check if the vz service is running
/etc/init.d/vz status
OpenVZ is running...

Installation of VPS with OS of our choice

For creating virtual machines(VPS) with OpenVZ, we need to have templates for the distribution (centos, debian, ubuntu) that we want to install


The precreated distribution templates can be downloaded from  http://wiki.openvz.org/Download/template/precreated



Download the precreated template distributions of your choice and store it under /vz/template/cache



# wget http://download.openvz.org/template/precreated/debian-6.0-x86_64.tar.gz -P /vz/template/cache



# wget http://download.openvz.org/template/precreated/centos-6-x86_64.tar.gz -P /vz/template/cache

I have downloaded the precreated template distributions of centos 64 bit and debian 64 bit.


To set up a VPS from the centos template, run the following command

# vzctl create 101 --ostemplate  centos-6-x86

Creating container private area (centos-6-x86)

Performing postcreate actions

CT configuration saved to /etc/vz/conf/101.conf

Container private area was created



Here, 101 is the uid of the newly created VPS.


To set a hostname and IP address for the vm, run:
# vzctl set 101 --hostname centos32 --save
# vzctl set 101 --ipadd 192.168.2.101 --save
The ipaddress 192.168.2.101 is assigned to venet0:0

The above steps can be done in a single step
# vzctl create 101 --ostemplate  centos-6-x86 --ipadd 192.168.2.101 --hostname centos32 

To get a list of all vms and their statuses, run
# vzlist -a

After the creation, initialize the created VPS via:
# vzctl start 101
Starting container ...
Container is mounted
Adding IP address(es): 192.168.2.101
Setting CPU units: 1000
Container start in progress...

If you want to have the vm started at boot, run
# vzctl set 101 --onboot yes --save

You can now enter into the VPS by simply SSH'ing into it or via the following command:
#vzctl enter 101

To set a root password for the vm, execute
#vzctl exec 101 passwd

Suppose we want to add an additional ip address, this
# vzctl set 101 --save --ipadd 192.168.2.102
The ipaddress 10.10.37.102 is assigned to venet0:1

Suppose we want to remove an ip address from the VPS
# vzctl set 101 --save --ipdel 192.168.2.102

To leave the vm's console, type
# exit

To stop a vm, run
# vzctl stop 101

To restart a vm, run
# vzctl restart 101

To delete a vm from the hard drive (it must be stopped before you can do this), run
# vzctl destroy 101

To find out about the resources allocated to a vm, run
# vzctl exec 101 cat /proc/user_beancounters
Version: 2.5
       uid  resource                     held              maxheld              barrier                limit              failcnt
      101:  kmemsize                  3409952              4366336  9223372036854775807  9223372036854775807                    0
            lockedpages                     0                    0  9223372036854775807  9223372036854775807                    0
            privvmpages                 10732                11784  9223372036854775807  9223372036854775807                    0
            shmpages                      129                  129  9223372036854775807  9223372036854775807                    0
            dummy                           0                    0                    0                    0                    0
            numproc                        17                   36  9223372036854775807  9223372036854775807                    0
            physpages                    5378                 6261                    0                65536                    0
            vmguarpages                     0                    0  9223372036854775807  9223372036854775807                    0
            oomguarpages                 1566                 1566  9223372036854775807  9223372036854775807                    0
            numtcpsock                      4                    4  9223372036854775807  9223372036854775807                    0
            numflock                        4                    5  9223372036854775807  9223372036854775807                    0
            numpty                          0                    1  9223372036854775807  9223372036854775807                    0
            numsiginfo                      0                   27  9223372036854775807  9223372036854775807                    0
            tcpsndbuf                   69760                69760  9223372036854775807  9223372036854775807                    0
            tcprcvbuf                   65536                65536  9223372036854775807  9223372036854775807                    0
            othersockbuf                 4624                59320  9223372036854775807  9223372036854775807                    0
            dgramrcvbuf                     0                 4360  9223372036854775807  9223372036854775807                    0
            numothersock                   30                   48  9223372036854775807  9223372036854775807                    0
            dcachesize                1341986              1363836  9223372036854775807  9223372036854775807                    0
            numfile                       319                  390  9223372036854775807  9223372036854775807                    0
            dummy                           0                    0                    0                    0                    0
            dummy                           0                    0                    0                    0                    0
            dummy                           0                    0                    0                    0                    0
            numiptent                      20                   20  9223372036854775807  9223372036854775807                    0

The failcnt column is very important, it should contain only zeros; if it doesn't, this means that the vm needs more resources than are currently allocated to the vm. Open the vm's configuration file in /etc/vz/conf and raise the appropriate resource, then restart the vm.


VPS disk space

By default, each VPS created is allocated 2GB disk space and 200000 inodes
# vzctl exec 101 df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/simfs            2.0G  570M  1.5G  28% /
none                  128M  4.0K  128M   1% /dev
none                  128M     0  128M   0% /dev/shm

# vzctl exec 101 df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/simfs            200000   21507  178493   11% /
none                   32768     151   32617    1% /dev
none                   32768       1   32767    1% /dev/shm

To increase the available disk space from the default 2GB to something more useful like 10GB: 

This will not immediately consume 10GB of space by the container, but will allocate a maximum of 10GB of hard drive space to it.

# vzctl set 101 --diskspace 10G:11G --save
CT configuration saved to /etc/vz/conf/101.conf

The above command increases the default 2GB drive space available to a barrier of 10GB and a maximum limit of 11GB. The upper limit allows for some grace; the disk space permitted will be 10GB but if it exceeds it, it won’t be restricted from that resource until the barrier is hit. This gives the container a 1GB “buffer.”

# vzctl exec 101 df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/simfs             10G  570M  9.5G   6% /
none                  128M  4.0K  128M   1% /dev
none                  128M     0  128M   0% /dev/shm

There are two ways to change settings for containers. 
  • The first is using vzctl as above (remember to use the –save option to make the changes persistent). 
  • The second is to edit the configuration file for the container. For a container with a CTID of 101, the file would be /etc/sysconfig/vz-scripts/101.conf. This file can be used to change options to the container and can also be used to see what existing configuration settings are.

Connecting VPS to Internet

In the main node(not VPS), on which the VPS is being run, run the follwoing command
# iptables -L

If such an entry is observed
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited

then add these two rules

iptables -A FORWARD -s xxx.xxx.xxx.xxx/xx -j ACCEPT
iptables -A FORWARD -d xxx.xxx.xxx.xxx/xx -j ACCEPT

NOTE: For example, if VPS IP address range is in 192.168.2.1 ~ 192.168.2.254, xxx.xxx.xxx.xxx/xx will be 192.168.2.0/24. So we shall add

# iptables -A FORWARD -s 192.168.2.0/24 -j ACCEPT
# iptables -A FORWARD -d 192.168.2.0/24 -j ACCEPT

Make sure that these two rules are placed above the rule 

"REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited"

Edit /etc/sysconfig/iptables and restart the iptables service (service iptables restart). Now iptables will be listed as follows

#iptables -L
...
ACCEPT     all  --  192.168.2.0/24       anywhere
ACCEPT     all  --  anywhere             192.168.2.0/24
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited

Add a POSTROUTING chain and a MASQUERADE target
# iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

Now set the nameserver of main node as the nameserver for the VPS too
# vzctl set 101 --nameserver 192.168.1.1 --save
CT configuration saved to /etc/vz/conf/101.conf







No comments:

Post a Comment