Puppet - How to set up - A simple Puppet Master and Puppet Client setup

Let me illustrate a simple set up of Puppet master and Puppet client, where a file /tmp/puppet-test-file will be automatically created in client once it is setup in Puppet master.

Puppet Master - 192.168.1.33

Puppet Client - 192.168.2.101

Setting up Puppet Master

Install puppet, puppet-server, facter 

      yum install puppet puppet-server facter

Puppet’s principal configuration file is puppet.conf

   In OpenSource Puppet, puppet.conf file is generally in the path

    /etc/puppet/puppet.conf

  In Puppet Enterprise:

    /etc/puppetlabs/puppet/puppet.conf

When running Puppet Master as normal user, puppet.conf file can be placed in

   /home/user/.puppet/puppet.conf

Here we are using OpenSource puppet. So edited /etc/puppet/puppet.conf file to add the following content

[master]

  certname=192.168.1.33

Create Puppet site.pp file

    touch /etc/puppet/manifests/site.pp

Initialize site.pp with following contents

    import ‘nodes.pp’

Now create a file nodes.pp for Node Definitions

   touch /etc/puppet/manifests/nodes.pp

Add default node definitions in nodes.pp, so that it becomes applicable to all the agents connecting to it.

node default {

       file { "/tmp/puppet-test-file":
         replace => "no", # this is the important property
         ensure  => "present",
         content => "From Puppet\n",
         mode    => 644,
       }

}

Start Puppet Master

  service puppet master start

Puppetmaster listens on port 8140

   netstat -anp | grep ruby

Puppet Client(192.168.2.101) configuration steps

Install puppet and facter

  yum install puppet facter

Edit /etc/puppet/puppet.conf to add master server details

   server=192.168.1.33

Before starting puppet service in client machine(192.168.1.33), run the command

    puppet agent —test

  Exiting: no certificate found and waitforcert is disabled

In the puppet master(192.168.1.33) run the command

  puppet cert list

  "centos32" (18:B9:34:16:B9:37:1C:59:7D:2B:DF:EE:FE:0F:C9:8A)



Now accept the cert request by running the following command in puppet master(192.168.1.33)

   puppet cert sign centos32

notice: Signed certificate request for centos32

notice: Removing file Puppet::SSL::CertificateRequest centos32 at '/var/lib/puppet/ssl/ca/requests/centos32.pem'

In puppet client(192.168.2.101, centos32) start the puppet client  

        service puppet start

On puppet client(agent - 192.168.2.101), now just observe the entries for puppet in /var/log/messages

Jan 19 11:41:07 centos32 puppet-agent[2027]: (/Stage[main]//Node[default]/File[/tmp/puppet-test-file]/ensure) created

Jan 19 11:41:07 centos32 puppet-agent[2027]: Finished catalog run in 0.04 seconds

Verification

Check if the file /tmp/puppet-test-file is present in puppet client machine 192.168.2.101

[root@centos32 ~]# ls -l /tmp/puppet-test-file 

-rw-r--r-- 1 root root 12 Jan 19 11:41 /tmp/puppet-test-file

Decrease reserved disk space in ext3/ext4 filesystems

When I setup ext3 partition in my 1TB hard disk and ran df -h command, I was in for a surprise

/dev/rootvg/scribe 992G  407M  941G   1% /scribe

Out of 992GB, only 941GB is available. Almost 41GB is missing. We lost 5% of 992GB (.05*992=49.60, 992-49.60=942.40)

It seems that ext filesystems by default will reserve about 5% disk space for superuser level processes and to prevent filesystem from fragmenting as it fills up. However, this reserved space can be claimed.

To reduce reserved blocks from 5% to 2% use the following command:
# tune2fs -m 2 /dev/rootvg/scribe

# umount /scribe

# tune2fs -m 2 /dev/rootvg/scribe
tune2fs 1.39 (29-May-2006)
Setting reserved blocks percentage to 2% (5281218 blocks)

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/rootvg-scribe  992G  407M  971G   1% /scribe

Thus we have reclaimed 30GB of space.

Extending LVM

I have a 1TB hard disk. The 400GB of the hard disk(/dev/cciss/c0d0p2) is configured as LVM, with the remaining 600GB left unutilized. Now I want to extend the logical volume(LVM) by 600GB by utilizing the unused 600GB.

The LV name is /dev/rootvg/scribe

Get the current size of LVM /dev/rootvg/scribe by using lvdisplay command. It should show LV Size 389.97 GB

Currently the disk hard disk consists of just two partitions

/dev/cciss/c0d0p1 - This is configured as /boot partition and not part of LVM
/dev/cciss/c0d0p2 - Configured as part of LVM /dev/rootvg/scribe

# fdisk -l

Disk /dev/cciss/c0d0: 1199.8 GB, 1199865640960 bytes
255 heads, 63 sectors/track, 145875 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14       65283   524281275   8e  Linux LVM

Now let us go about extending the Logical Volume /dev/rootvg/scribe by adding extra 600GB.

1) Create a partition of the unused 600GB

fdisk /dev/cciss/c0d0

p - to display the existing partition table
n - to create new partition

Once created

p - should display /dev/cciss/c0d0p3 as regular Linux partition
t - to convert newly created partition(/dev/cciss/c0d0p3) to lvm type
Enter partition number - 3(probably)
Hex code : 8e (Now the partition is converted to lvm type)

p - should show that /dev/cciss/c0d0p3 is of type lvm
w - to write the partition table to the disk

2) The newly added partition will not be visible on the system
   Check the same using
    ls /dev/cciss/   - We shall see that c0d0p3 is missing

   partprobe - Running partprobe reloads the partition table and brings the partition up. No need to reboot the system.

    ls /dev/cciss/ - We shall now see the c0d0p3

3) Now need to add /dev/cciss/c0d0p3 as part of physical volume(PV) 
     pvdisplay
This will show only the physical volume (/dev/cciss/c0d0p2)
   PV Name               /dev/cciss/c0d0p2

Now run the command
    pvcreate /dev/cciss/c0d0p3
This command creates a header on each partition so it can be used for LVM.

Now run pvdisplay to see your new PV.
   pvdisplay

It should now show
  PV Name               /dev/cciss/c0d0p2
  PV Name               /dev/cciss/c0d0p3

"VG Name" for /dev/cciss/c0d0p3 should be empty.
Since "/dev/cciss/c0d0p2 is already part of LVM, the "VG Name" for /dev/cciss/c0d0p2 will be displayed as rootvg

Need to add /dev/cciss/c0d0p3 as part of the volume group rootvg

4) Run the vgdisplay command to get the "VG Name"
    vgdisplay

5) Use command vgextend to add the PV(/dev/cciss/c0d0p3) to the VG rootvg
  vgextend  rootvg  /dev/cciss/c0d0p3

6)  Run   pvdisplay command to see the value of "VG Name" for PV /dev/cciss/c0d0p3. It shall display as rootvg
   
 7) Run command vgdisplay to see the new size of the VG rootvg.
     Mainly note the value of "VG size". It should be nearly 1TB.
     Also note "Free  PE / Size" and "Total PE". Earlier "Free  PE / Size" shall be "0 / 0". Now it shall show some values.

8) Now comes the extension of Logical Volume /dev/rootvg/scribe
   Run the command
    lvdisplay
   to get the names of existing Logical Volumes.

  From "vgdisplay" command get the value of "Free PE/ Size". It will show the available "Physical Extent / Disk Size".

  Now extend the Logical Volume "/dev/rootvg/scribe"
   lvextend -l +19755 /dev/rootvg/scribe
               (OR)
   lvextend -L+617.34 /dev/rootvg/scribe

  Run the command
    lvdisplay

Check the "LV Size" for "LV Name : /dev/rootvg/scribe". It should be more than current "LV Size 389.97 GB".

 --- Logical volume ---
  LV Name                /dev/rootvg/scribe
  VG Name                rootvg
  LV UUID                P08Qra-MnjE-5tgd-4Muw-c7r5-kY7o-6zUAu2
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1007.31 GB
  Current LE             32234
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2

9) Now need to grow the filesystem for the LV /dev/rootvg/scribe

  umount /scribe
  e2fsck -f /dev/rootvg/scribe
  resize2fs /dev/rootvg/scribe
  mount -a

 Use "df" command to see the new filesystem size

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p1      99M   13M   81M  14% /boot
tmpfs                  12G     0   12G   0% /dev/shm
/dev/mapper/rootvg-scribe
                      992G  407M  941G   1% /scribe

xargs prints file names even if there are no matching files

A problem I faced with xargs command is that, it just lists the files/directories in the current directory if there is no input for the xargs command.

Let me illustrate with an example

In my current directory, I have just *.txt files

$ ls
abc.txt  ver1.txt  ver1.txt.orig  ver2.txt

When I try to fetch files with names of type *.jpg, I expected to get no output when I ran xargs with ls. But it printed all the file names in the current directory.

$ find . -type f -iname "*.jpg" | xargs ls
abc.txt ver1.txt  ver1.txt.orig  ver2.txt

The reason seems to be that, by default the command executed by xargs is /bin/echo and it will simply display file/directory names.

Later I found that, there exists an option "-r" for xargs, which did the trick for me

$ find . -type f -iname "*.jpg" | xargs -r ls