Buffer - Buffers are a type of cache associated with block devices(such as /dev/sda), and deals with caching of filesystem metadata. When a process needs to access data from a file, the kernel will bring the data into main memory where the process shall examine it, alter it, and request that the data be saved in the filesystem again. When the kernel brings the data into memory, it also needs to bring the auxillary data associated with that data from the disk into memory. The auxillary data is, for instance,
- the super block of the file system that describes the free space available on the file system
- the inode which describes the layout of the file
Caching of filesystem metadata is done in the buffers. That is, the buffers remember what's in directories, what file permissions are, and keep track of what memory is being written from or read to for a particular block device.
Cache - Cache is also known as page cache. This cache only contains the contents of the files alone. When a file is read from disk or network, the contents are stored in pagecache. Also cache contains recently used (but currently unused) memory pages, in case they're needed again. When extra physical memory is not in use, the kernel attempts to put it to work as a cache. The cache stores recently accessed disk data(pages) in memory; if the same data is needed again it can be quickly retrieved from the cache, improving performance.
Both buffers and cache can be very quickly freed if needed by an application. Both buffer and cache help in preventing the kernel from having to read information from disk storage as much as possible.
Major and Minor Page Faults
When a process starts, the kernel searches the CPU caches and then physical memory(RAM). If the data does not exist in either, the kernel issues a major page fault (MPF). A MPF is a request to the disk subsystem to retrieve pages off disk and buffer them in RAM. Once memory pages are mapped into the buffer cache, the kernel will attempt to use these pages resulting in a minor page fault (MnPF). A MnPF saves the kernel time by reusing a page in memory as opposed to placing it back on the disk.
In the following example, the time command is used to demonstrate how many
MPF and MnPF occurred when an application started. The first time the evolution starts, there are many MPFs:
$ /usr/bin/time -v evolution
Major (requiring I/O) page faults: 43
Minor (reclaiming a frame) page faults: 1485
The second time evolution starts, the kernel does not issue any MPFs because the
application is in memory already:
$ /usr/bin/time -v evolution
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1528
The filesystem cache is used by the kernel to minimize MPFs and maximize
MnPFs.
$ cat /proc/meminfo
MemTotal: 3078328 kB
MemFree: 2382324 kB
Buffers: 67444 kB
Cached: 256276 kB
The system has a total of 3 GB (MemTotal) of RAM available on it. There is
currently 2.3GB of RAM "free" (MemFree), 67 MB RAM that is allocated to disk
write operations (Buffers), and 256 MB of pages read from disk in RAM
(Cached).
Monitoring memory usage( especially, buffers and cache) using free, vmstat and sar commands
For example, while downloading a file, we can see the buffer and cache utilisation going up.
This can be observed using the free, vmstat and sar commands as follows
Before download
$ free -m
total used free shared buffers cached
Mem: 2996 1593 1402 0 89 1165
-/+ buffers/cache: 338 2657
Swap: 4999 0 4999
$ vmstat 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 1435932 91856 1193196 0 0 0 0 13 26 0 0 100 0 0
0 0 0 1435932 91856 1193196 0 0 0 0 16 30 0 0 100 0 0
1 0 0 1435932 91856 1193196 0 0 0 0 20 30 0 0 100 0 0
0 0 0 1435932 91856 1193196 0 0 0 138 20 32 0 0 100 0 0
0 0 0 1435932 91856 1193196 0 0 0 0 14 33 0 0 100 0 0
# sar -r 2
Linux 2.6.32-042stab057.1 (dhcppc5) 10/08/2012 _x86_64_ (2 CPU)
12:07:51 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit
12:07:53 PM 1432116 1635832 53.32 92024 1193432 561516 6.86
12:07:55 PM 1432116 1635832 53.32 92024 1193432 561516 6.86
12:07:57 PM 1432116 1635832 53.32 92024 1193432 561516 6.86
12:07:59 PM 1432116 1635832 53.32 92024 1193432 561516 6.86
During download
# vmstat 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 0 1391876 92552 1229088 0 0 3 11 31 60 0 0 100 0 0
1 0 0 1388148 92560 1232688 0 0 0 6 4033 8093 6 3 91 1 0
0 0 0 1384684 92560 1236272 0 0 0 0 4007 8073 6 3 91 0 0
0 0 0 1380964 92568 1239888 0 0 0 6 3992 8021 6 3 91 0 0
0 0 0 1377120 92568 1243456 0 0 0 12 4005 8049 6 4 91 0 0
0 0 0 1373524 92568 1247008 0 0 0 0 3990 7984 6 3 90 0 0
0 0 0 1369928 92576 1250432 0 0 0 6 3 833 7579 6 4 90 0 0
3 0 0 1365960 92576 1253952 0 0 0 28880 4070 7955 7 5 88 0 0
0 0 0 1362356 92584 1257520 0 0 0 10 4067 8111 7 4 89 1 0
1 0 0 1358520 92584 1261120 0 0 0 4472 4023 8020 7 4 90 0 0
1 0 0 1354800 92584 1264688 0 0 0 0 4041 8074 6 4 90 0 0
1 0 0 1351204 92592 1268288 0 0 0 10 4056 8062 6 4 89 1 0
1 0 0 1347484 92592 1271840 0 0 0 0 3985 8004 5 3 92 0 0
0 0 0 1343764 92600 1275424 0 0 0 6 3993 8072 5 4 91 0 0
The following can be observed
- cache and buffer counters increasing
- interrupts(in) and context switch(cs) counters increasing
- disk activity observed in the form of bo counter under io
$ free -m
total used free shared buffers cached
Mem: 2996 1722 1273 0 90 1285
-/+ buffers/cache: 346 2649
Swap: 4999 0 4999
# sar -r 2
Linux 2.6.32-042stab057.1 (dhcppc5) 10/08/2012 _x86_64_ (2 CPU)
12:35:55 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit
12:35:57 PM 1415808 1652140 53.85 92536 1206036 565620 6.91
12:35:59 PM 1412088 1655860 53.97 92536 1209736 565620 6.91
12:36:01 PM 1408492 1659456 54.09 92544 1213132 565620 6.91
12:36:03 PM 1404780 1663168 54.21 92544 1216720 565620 6.91
12:36:05 PM 1400936 1667012 54.34 92552 1220396 565620 6.91
It can be observed that kbbuffers and kbcached counters are increasing
After download
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
4 0 0 21540 88432 2565180 0 0 0 0 4034 8084 6 4 90 0 0
0 0 0 22276 86700 2565996 0 0 0 0 4052 8066 7 4 90 0 0
0 0 0 21036 86452 2567468 0 0 0 6 4005 7982 7 4 89 0 0
0 0 0 21288 86456 2567392 0 0 142 26904 2145 4081 4 2 92 2 0
0 0 0 21296 86464 2567420 0 0 0 16 30 47 0 0 100 0 0
2 0 0 21300 86464 2567424 0 0 0 0 23 36 0 0 100 0 0
0 0 0 21424 86464 2567424 0 0 0 0 28 38 0 0 100 0 0
0 0 0 21424 86464 2567424 0 0 0 0 23 35 0 0 100 0 0
0 0 0 21432 86464 2567424 0 0 0 0 20 33 0 0 100 0 0
The following can be observed
- The cache and buff counter values remaining the same
- Decrease in interrupts(in) and context switches(cs)
$ free -m
total used free shared buffers cached
Mem: 2996 2947 48 0 80 2482
-/+ buffers/cache: 384 2611
Swap: 4999 0 4999
No increase in kbbuffers and kbcached counters observed.
How to free up buffer and cache in memory
Pages in memory are classified as follows
- Free : Pages in main memory that has not yet been allocated to any process.
- Dirty : When a file is written to, the new data is stored in pagecache before being written back to a disk or the network. Also on request by a process, a page is copied from disk into memory and modified. When a page has a new or modified data not written back yet, it is called “dirty”.
- Clean : A page in memory that has not been modified since being read from disk or a page content that has been written to disk.
- Active : Page is in use by a process. Both clean and dirty pages may be in use by a process. Clean and dirty pages not in use by a process are termed as inactive clean and dirty pages.
Reclaim dirty buffers and pages
The sync command can be used to reclaim dirty buffers and cache. The sync() system call tells the kernel to immediately write all the dirty buffers in the buffer cache and all dirty pages in the page cache to disk. There is another system call named fsync(). The fsync() system call writes all dirty buffers and dirty pages associated with a specific file to disk.
Reclaim clean pages
To drop clean caches, buffer(dentries and inodes) from memory. This is the normal command used
echo 3 > /proc/sys/vm/drop_caches
To free clean pages alone from page cache:
echo 1 > /proc/sys/vm/drop_caches
To free clean pages in buffer(dentries and inodes):
echo 2 > /proc/sys/vm/drop_caches
Hence to free up both dirty and clean pages from page cache and buffer, run the following
sync && echo 3 > /proc/sys/vm/drop_caches
(OR)
sync
sysctl -w vm.drop_caches=3
(OR)
sync
sysctl -w vm.drop_caches=3
Here is an illustration
# free -m
total used free shared buffers cached
Mem: 2996 2943 52 0 81 2479
-/+ buffers/cache: 382 2613
Swap: 4999 0 4999
Run the command to free up buffer and page cache in memory
sync && echo 3 > /proc/sys/vm/drop_caches
After freeing up buffer and pagecache in memory by reclaiming dirty and clean pages in memory
# free -m
total used free shared buffers cached
Mem: 2996 344 2651 0 0 83
-/+ buffers/cache: 260 2735
Swap: 4999 0 4999
No comments:
Post a Comment