Thursday, July 31, 2014

Friday, July 25, 2014

Tools for benchmarking Hard Disk performance(File system Read Write Performance)

dd

dd command can be used to read and write from any block device in Linux.
Using dd command we can perform simple sequential read and sequential write tests

Using dd command to check sequential write speed by writing a 1GB file to disk

dd if=/dev/zero of=/tmp/outputfile bs=1M count=1000

But the above test has a drawback.Any data written to disk is first cached in memory and then written to the disk. So in the above test, it gives the speed at which data was cached into RAM and not the speed at which data was written to the disk. So the above command is tweaked as follows

dd if=/dev/zero of=/tmp/outputfile bs=1M count=1000 conv=fdatasync

So now the dd command will report the write speed only after the data is synced to the disk

Alternative way,

  sync;time bash -c "(dd if=/dev/zero of=/tmp/outputfile bs=1M count=1000; sync)"

Using dd command to check sequential read speed

   dd if=/tmp/outputfile of=/dev/null bs=1M count=1000

hdparm 

It is a performance and benchmarking tool for SATA/IDE drives.

To measure how many MB/s, the hard drive can read
     hdparm -t --direct /dev/sda1
     (or) hdparm -tT /dev/sda1

  -t  : read from cache buffer
  -T : speed of reading without precached buffer

iozone

iozone is an open source file system benchmarking utility.

The default command line option is -a, which stands for full automatic mode. It tests with block sizes ranging from 4k to 16M and file sizes ranging from 64k to 512M.

     iozone -a /dev/sda1

iozone performs 13 types of tests. Some important tests are

Write : Indicates the speed of writing a new file to the filesystem. Creating a new file is always slower than rewriting an existing file owing to the metadata overhead involved while creating a new file (like creating a new inode entry).

ReWrite : Indicates the speed of writing to an existing file

Read : Indicates the speed of reading an existing file

ReRead : Indicates the speed of rereading a file that is already read.

RandomRead : Indicates the speed at which random areas of a single file are read.

RandomWrite : Indicates the speed at which we can write into random areas of a file





Tools for Benchmarking Network Performance


  • iperf : To measure bandwidth - Throughput between two nodes
  • sockperf ping-pong test : To measure network latency on TCP - https://code.google.com/p/sockperf/
  • PING : To measure network latency using ICMP

Wednesday, July 23, 2014

Difference between Incident and Problem

An incident as defined by ITIL is as follows: "An unplanned interruption to an IT Service or a reduction in the Quality of an IT Service. Failure of a Configuration Item that has not yet impacted Service is also an Incident. For example Failure of one disk from a mirror set." A configuration item is just about anything in ITIL Terms.  Everything from a NIC or Hard Drive to a Web Service can be declared a Configuration Item. It's just something to tie incidents and problems to.
A problem as defined by ITIL is as follows: "A cause of one or more Incidents. The cause is not usually known at the time a Problem Record is created, and the Problem Management Process is responsible for further investigation."
Here are some other quick hit differences:
Problems should be traced down to a root-cause level where as incidents should be resolved as quickly as possible to restore the service to operation.
Incidents like a known issue with a Java application memory leak need become a Problem when they repeatedly occur.  
Incidents can generally be resolved by the person doing the work.
Problems must be closed by a manager assigned in the problem management process.

Tuesday, July 22, 2014

How to find number of CPUs in Linux?

Nowadays, each CPU made up of multiple CORES and each CORE is now treated as a separate CPU.

Also, if HYPERTHREADING is enabled, then each CORE is treated as TWO SEPARATE LOGICAL UNITS and these two LOGICAL UNITS too are treated as separate CPUs.

dmidecode -t processor | egrep "Socket Designation|Core Count|Thread Count"
        Socket Designation: CPU1
        Core Count: 4
        Thread Count: 8
        Socket Designation: CPU2
        Core Count: 4
        Thread Count: 8

As per command dmidecode, there are
Two CPUs
Eight Cores - Each CPU has 4 cores
Sixteen Threads - Each Core has two threads

Hence, the number of CPUs in the system is 16

This can be verified by other means too

grep 'processor' /proc/cpuinfo
processor       : 0
processor       : 1
processor       : 2
processor       : 3
processor       : 4
processor       : 5
processor       : 6
processor       : 7
processor       : 8
processor       : 9
processor       : 10
processor       : 11
processor       : 12
processor       : 13
processor       : 14
processor       : 15



# Count the number of “physical processor(s)
grep "physical id" /proc/cpuinfo | sort -u | wc -l

# Count the number of “physical cores per CPU”
grep "cpu cores" /proc/cpuinfo |sort -u |cut -d":" -f2

# Count the number of “logical cores ” (including multi-threading cores)
grep -c "processor" /proc/cpuinfo





LoadBalancer or RoundRobin DNS

The advantage of using LoadBalancer over RoundRobin DNS are

  • LoadBalancer takes care of load on nodes behind it by directing traffic/requests to node with least load
  • LoadBalancer takes care of sessions/connections. For applications like shopping cart, which makes use of the sessions, in case if the node from which we are served goes down, the session is lost too. 
  • If one of the nodes behind the LoadBalancer goes down, then the LoadBalancer will immediately detect it and will not send any requests to that node which is down. This is not possible in RoundRobin DNS.