Hadoop Namenode - Image, Checkpoint(fsimage), Journal(edits)

Image

HDFS Namenode keeps the entire namespace in RAM. The inode data and list of blocks belonging to each file comprise the metadata of the name system called the image.

Checkpoint (fsimage)

The persistent record of the image stored in the Namenode's native fielsystem is called a checkpoint.The locations of block replicas may change over time and are not part of the persistent checkpoint. checkpoint also called fsimage(filesystem image).

The fsimage file contains a serialized form of all the directory and file inodes in the filesystem. Each inode is an internal representation of a file or directory’s metadata and contains such information as the file’s

replication level, modification and access times, access permissions, block size, and the blocks a file is made up of. For directories, the modification time, permissions, and quota metadata is stored.

Note: The fsimage file does not record the datanodes on which the blocks are stored. Instead the namenode keeps this mapping in memory, which it constructs by asking the datanodes for their block lists when they join the cluster and periodically afterward to ensure the namenode’s block mapping is up-to-date.

Journal(edit log)

The NameNode also stores the modification log of the image called the journal in the local host’s native file system. journal also called edits. When a filesystem client performs a write operation (such as creating or moving a file), it is first recorded in the edit log. The namenode also has an in-memory representation of the filesystem metadata, which it updates after the edit log has been modified. The in-memory metadata is used to serve read requests.

Upon namenode startup, the fsimage file is loaded into RAM and any changes in the edits file are replayed, bringing the in-memory view of the filesystem up to date