Wednesday, May 16, 2012

Buffer and Cache in free and vmstat commands


Buffer - Buffers are a type of cache associated with block devices(such as /dev/sda), and deals with caching of filesystem metadata. When a process needs to access data from a file, the kernel will bring the data into main memory where the process shall examine it, alter it, and request that the data be saved in the filesystem again. When the kernel brings the data into memory, it also needs to bring the auxillary data associated with that data from the disk into memory. The auxillary data is, for instance, 

  • the super block of the file system that describes the free space available on the file system
  • the inode which describes the layout of the file

Caching of filesystem metadata is done in the buffers. That is, the buffers remember what's in directories, what file permissions are, and keep track of what memory is being written from or read to for a particular block device.

Cache - Cache is also known as page cache. This cache only contains the contents of the files alone. When a file is read from disk or network, the contents are stored in pagecache. Also cache contains recently used (but currently unused) memory pages, in case they're needed again. When extra physical memory is not in use, the kernel attempts to put it to work as a cache. The cache stores recently accessed disk data(pages) in memory; if the same data is needed again it can be quickly retrieved from the cache, improving performance.

Both buffers and cache can be very quickly freed if needed by an application. Both buffer and cache help in preventing the kernel from having to read information from disk storage as much as possible.

Major and Minor Page Faults

When a process starts, the kernel searches the CPU caches and then physical memory(RAM). If the data does not exist in either, the kernel issues a major page fault (MPF). A MPF is a request to the disk subsystem to retrieve pages off disk and buffer them in RAM. Once memory pages are mapped into the buffer cache, the kernel will attempt to use these pages resulting in a minor page fault (MnPF). A MnPF saves the kernel time by reusing a page in memory as opposed to placing it back on the disk.

In the following example, the time command is used to demonstrate how many
MPF and MnPF occurred when an application started. The first time the evolution starts, there are many MPFs:

$ /usr/bin/time -v evolution

Major (requiring I/O) page faults: 43
Minor (reclaiming a frame) page faults: 1485

The second time evolution starts, the kernel does not issue any MPFs because the
application is in memory already:

$ /usr/bin/time -v evolution

Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1528

The filesystem cache is used by the kernel to minimize MPFs and maximize
MnPFs.

$ cat /proc/meminfo
MemTotal:        3078328 kB
MemFree:         2382324 kB
Buffers:           67444 kB
Cached:           256276 kB

The system has a total of 3 GB (MemTotal) of RAM available on it. There is
currently 2.3GB of RAM "free" (MemFree),  67 MB RAM that is allocated to disk
write operations (Buffers), and 256 MB of pages read from disk in RAM
(Cached).

Monitoring memory usage( especially, buffers and cache) using free, vmstat and sar commands

For example, while downloading a file, we can see the buffer and cache utilisation going up.
This can be observed using the free, vmstat and sar commands as follows

Before download

$ free -m
             total       used       free     shared    buffers     cached
Mem:          2996       1593       1402          0         89       1165
-/+ buffers/cache:        338       2657
Swap:         4999          0       4999

$ vmstat 2
procs -----------memory----------       ---swap-- -----io---- --system--   -----cpu-----
 r  b   swpd   free      buff     cache      si   so    bi    bo     in   cs       us sy   id wa st
 0  0      0 1435932  91856 1193196    0    0     0     0      13   26      0  0  100  0  0
 0  0      0 1435932  91856 1193196    0    0     0     0      16   30      0  0  100  0  0
 1  0      0 1435932  91856 1193196    0    0     0     0      20   30      0  0  100  0  0
 0  0      0 1435932  91856 1193196    0    0     0   138     20  32      0  0  100  0  0
 0  0      0 1435932  91856 1193196    0    0     0     0      14   33      0  0  100  0  0

# sar -r 2
Linux 2.6.32-042stab057.1 (dhcppc5)     10/08/2012      _x86_64_        (2 CPU)

12:07:51 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
12:07:53 PM   1432116   1635832           53.32     92024       1193432    561516      6.86
12:07:55 PM   1432116   1635832           53.32     92024       1193432    561516      6.86
12:07:57 PM   1432116   1635832           53.32     92024       1193432    561516      6.86
12:07:59 PM   1432116   1635832           53.32     92024       1193432    561516      6.86

During download

# vmstat 2
procs -----------memory----------       ---swap-- -----io----   --system--      -----cpu-----
 r  b   swpd   free      buff     cache      si   so    bi    bo      in   cs          us sy id wa st
 2  0      0 1391876  92552 1229088    0    0     3    11      31   60         0  0 100  0  0
 1  0      0 1388148  92560 1232688    0    0     0     6    4033 8093       6  3 91  1  0
 0  0      0 1384684  92560 1236272    0    0     0     0    4007 8073       6  3 91  0  0
 0  0      0 1380964  92568 1239888    0    0     0     6    3992 8021       6  3 91  0  0
 0  0      0 1377120  92568 1243456    0    0     0    12   4005 8049       6  4 91  0  0
 0  0      0 1373524  92568 1247008    0    0     0     0    3990 7984       6  3 90  0  0
 0  0      0 1369928  92576 1250432    0    0     0     6 3   833 7579       6  4 90  0  0
 3  0      0 1365960  92576 1253952    0    0     0 28880  4070 7955      7  5 88  0  0
 0  0      0 1362356  92584 1257520    0    0     0    10    4067 8111      7  4 89  1  0
 1  0      0 1358520  92584 1261120    0    0     0  4472   4023 8020      7  4 90  0  0
 1  0      0 1354800  92584 1264688    0    0     0     0     4041 8074      6  4 90  0  0
 1  0      0 1351204  92592 1268288    0    0     0    10    4056 8062       6  4 89  1  0
 1  0      0 1347484  92592 1271840    0    0     0     0     3985 8004      5  3 92  0  0
 0  0      0 1343764  92600 1275424    0    0     0     6     3993 8072      5  4 91  0  0

The following can be observed
  •  cache and buffer counters increasing
  • interrupts(in) and context switch(cs) counters increasing
  • disk activity observed in the form of bo counter under io
$ free -m
                   total       used       free     shared    buffers     cached
Mem:          2996       1722       1273          0         90       1285
-/+ buffers/cache:        346       2649
Swap:         4999          0       4999

# sar -r 2
Linux 2.6.32-042stab057.1 (dhcppc5)     10/08/2012      _x86_64_        (2 CPU)

12:35:55 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
12:35:57 PM   1415808   1652140          53.85       92536      1206036    565620      6.91
12:35:59 PM   1412088   1655860          53.97       92536      1209736    565620      6.91
12:36:01 PM   1408492   1659456          54.09       92544      1213132    565620      6.91
12:36:03 PM   1404780   1663168          54.21       92544      1216720    565620      6.91
12:36:05 PM   1400936   1667012          54.34       92552      1220396    565620      6.91

It can be observed that kbbuffers and kbcached counters are increasing

After download

# vmstat 2
procs -----------memory----------       ---swap-- -----io----   --system--       -----cpu-----
 r  b   swpd   free   buff  cache          si   so    bi    bo       in     cs        us sy id  wa st
 4  0      0  21540  88432 2565180    0    0     0     0       4034 8084      6  4 90  0  0
 0  0      0  22276  86700 2565996    0    0     0     0       4052 8066      7  4 90  0  0
 0  0      0  21036  86452 2567468    0    0     0     6       4005 7982      7  4 89  0  0
 0  0      0  21288  86456 2567392    0    0   142 26904  2145 4081      4  2 92  2  0
 0  0      0  21296  86464 2567420    0    0     0    16        30   47  0      0 100   0  0
 2  0      0  21300  86464 2567424    0    0     0     0         23   36  0      0 100   0  0
 0  0      0  21424  86464 2567424    0    0     0     0         28   38  0      0 100   0  0
 0  0      0  21424  86464 2567424    0    0     0     0         23   35  0     0  100   0  0
 0  0      0  21432  86464 2567424    0    0     0     0         20   33  0      0 100   0  0


The following can be observed

  •  The cache and buff counter values remaining the same
  • Decrease in interrupts(in) and context switches(cs)

$ free -m
                  total       used       free     shared    buffers     cached
Mem:          2996       2947         48          0         80       2482
-/+ buffers/cache:        384       2611
Swap:         4999          0       4999


No increase in kbbuffers and kbcached counters observed.

How to free up buffer and cache in memory

Pages in memory are classified as follows
  • Free : Pages in main memory that has not yet been allocated to any process.
  • Dirty : When a file is written to, the new data is stored in pagecache before being written back to a disk or the network. Also on request by a process, a page is copied from disk into memory and modified. When a page has a new or modified data not written back yet, it is called “dirty”. 
  • Clean : A page in memory that has not been modified since being read from disk or a page content that has been written to disk.
  • Active : Page is in use by a process. Both clean and dirty pages may be in use by a process. Clean and dirty pages not in use by a process are termed as inactive clean and dirty pages. 

Reclaim dirty buffers and pages

The sync command can be used to reclaim dirty buffers and cache. The sync() system call tells the kernel to immediately write all the dirty buffers in the buffer cache and all dirty pages in the page cache to disk. There is another system call named fsync(). The fsync() system call writes all dirty buffers and dirty pages associated with a specific file to disk.

Reclaim clean pages

To drop clean caches, buffer(dentries and inodes) from memory. This is the normal command used
    echo 3 > /proc/sys/vm/drop_caches

To free clean pages alone from page cache:
     echo 1 > /proc/sys/vm/drop_caches

To free clean pages in buffer(dentries and inodes):
      echo 2 > /proc/sys/vm/drop_caches

Hence to free up both dirty and clean pages from page cache and buffer, run the following
      sync && echo 3 > /proc/sys/vm/drop_caches

                           (OR)

      sync
      sysctl  -w vm.drop_caches=3

Here is an illustration

Before freeing up buffer and pagecache in memory
# free -m
                   total       used       free     shared    buffers     cached
Mem:          2996       2943         52          0         81       2479
-/+ buffers/cache:        382       2613
Swap:         4999          0       4999

Run the command to free up buffer and page cache in memory
   sync && echo 3 > /proc/sys/vm/drop_caches

After freeing up buffer and pagecache in memory by reclaiming dirty and clean pages in memory
# free -m
                   total       used       free     shared    buffers     cached
Mem:          2996        344       2651          0          0         83
-/+ buffers/cache:        260       2735
Swap:         4999          0       4999

No comments:

Post a Comment