Python : Using end=' ' to prevent new line after print

print("Tell me your name : ",  end=' ')
name = input()

Putting an end=' ' tells print not to end the line with a newline character

The output will be like

Tell me your name : <name>

Linux 7 Boot Process

1. As soon as the machine is powered on, the system firmware(either UEFI or BIOS) runs a Power On Self Test (POST) and starts initialising some hardware.
2. The system firmware then searches for a bootable device, which is either configured in UEFI boot firmware or by searching for Master Boot Record(MBR) on all disks in the order configured in BIOS.
3. The system firmware reads a boot loader from disk and passes control to the boot loader(grub2)
4. The boot loader loads it's configuration from disk and presents user with a menu of possible configurations to boot.
5. The boot loader then loads the kernel and initramfs from disk and place them in memory. An initramfs is a gziped cpio archive containing kernel modules for all hardware necessary at boot, init scripts and more. The initramfs containes an entire usable system by itself.
6. The bootloader hands control of the system to the kernel.
7. The kernel initializes all hardware for which it can find driver in the initramfs, then executes /sbin/init from initramfs as PID 1. The initramfs contains a working copy of systemd as /sbin/init, as well as udev daemon in RHEL 7.
8. The systemd instance from initramfs executes all units for initrd.target. This includes mounting the actual root file system on /sysroot
9. The kernel root file system is switched from the initramfs root file system to system root file system that was previously mounted on sysroot. systemd then re-executes itself using copy of systemd installed on the system.
10. systemd then looks for a default target, then starts(and stops) units to comply with the configuration for that target, solving dependancies between units automatically.

Knowing a rpm package

List Binaries

rpm -ql <package_name> | grep bin

Eg: # rpm -ql httpd | grep bin

/usr/sbin/apachectl

/usr/sbin/fcgistarter

/usr/sbin/htcacheclean

/usr/sbin/httpd

/usr/sbin/rotatelogs

/usr/sbin/suexec

/usr/share/httpd/icons/binary.gif

/usr/share/httpd/icons/binary.png

/usr/share/httpd/icons/binhex.gif

/usr/share/httpd/icons/binhex.png

/usr/share/httpd/icons/small/binary.gif

/usr/share/httpd/icons/small/binary.png

/usr/share/httpd/icons/small/binhex.gif

/usr/share/httpd/icons/small/binhex.png

/var/www/cgi-bin

List configuration files

rpm -qc <package_name>

Eg: # rpm -qc httpd

/etc/httpd/conf.d/autoindex.conf

/etc/httpd/conf.d/userdir.conf

/etc/httpd/conf.d/welcome.conf

/etc/httpd/conf.modules.d/00-base.conf

/etc/httpd/conf.modules.d/00-dav.conf

/etc/httpd/conf.modules.d/00-lua.conf

/etc/httpd/conf.modules.d/00-mpm.conf

/etc/httpd/conf.modules.d/00-proxy.conf

/etc/httpd/conf.modules.d/00-systemd.conf

/etc/httpd/conf.modules.d/01-cgi.conf

/etc/httpd/conf/httpd.conf

/etc/httpd/conf/magic

/etc/logrotate.d/httpd

/etc/sysconfig/htcacheclean

/etc/sysconfig/httpd

Tips for configuration files

man 5 <filename.conf>

List documentation

rpm -qd <package_name>

Will killing a parent process kill the child processes?

When parent process is sent SIGTERM signal, the parent process will execute it's own signal handlers to terminate it's child processes.

In the below case, sending SIGTERM to parent process ID 2086, kills all the child process too

# ps -eHf | grep http
root      2086     1  0 07:11 ?        00:00:00   /usr/sbin/httpd
apache    2088  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2089  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2090  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2091  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2092  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2093  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2094  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd
apache    2095  2086  0 07:11 ?        00:00:00     /usr/sbin/httpd

# kill 2086

# ps -eHf | grep http
root      2128  1932  0 07:12 pts/0    00:00:00         grep http

However, sending the parent process -9 signal (SIGKILL), will not allow to execute it's signal handlers. So parent process will not be able to send termination signal to it's child processes. So only parent process will get killed and child processes will live.

# ps -eHf | grep http
root      2157  1932  0 07:12 pts/0    00:00:00         grep http
root      2144     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2146  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2147  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2148  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2149  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2150  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2151  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2152  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd
apache    2153  2144  0 07:12 ?        00:00:00     /usr/sbin/httpd

# kill -9 2144

# ps -eHf | grep http
root      2190  1932  0 07:13 pts/0    00:00:00         grep http
apache    2146     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2147     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2148     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2149     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2150     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2151     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2152     1  0 07:12 ?        00:00:00   /usr/sbin/httpd
apache    2153     1  0 07:12 ?        00:00:00   /usr/sbin/httpd

Difference between fork() and exec() system calls

fork():

It creates a copy of running process. 

The running process is called parent process & newly created process is called child process.

• fork() returns the process identifier (pid) of the child process in the parent, and
• fork() returns 0 in the child.

exec():

It initiates a new process within a process. It loads a new program into the current process, replacing the existing one.

Enforce password policy in Linux

To enforce passwords

1) that expire after 90 days,
2) are at least 8 characters long and may not be a regular word

modify the files

1) /etc/login.defs -  In this file we can specify Expiry, Length with following entries
     PASS_MAX_DAYS 90
     PASS_MIN_LEN 8

2) /etc/pam.d/system-auth - In this file we can specify the complexity(Eg: not a regular word) and retries for password attempt with following entry

   password   required   pam_cracklib.so    

iptables examples

iptables has three chains - INPUT, OUTPUT, FORWARD

Set default policy using -P flag

iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP

Allow TCP traffic on port 80

iptables -A INPUT -p TCP --dport 80 -j ACCEPT

Allow TCP traffic on port 443

iptables -A INPUT -p TCP --dport 443 -j ACCEPT

Allow UDP traffic on port 53

iptables -A INPUT -p udp --dport 53 -j ACCEPT

Allow TCP traffic on port 3306 only from 192.168.0.2

iptables -A INPUT -p tcp --dport 3306 -s 192.168.0.2 -j ACCEPT

Allow outgoing TCP and UDP traffic on any port but only as part of existing connection state. DROP everything else

iptables -P OUTPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
iptables -P OUTPUT -p udp -m state --state ESTABLISHED -j ACCEPT
iptables -P OUTPUT DROP

Configure iptables to
(1) ACCEPT all TCP traffic on port 80 in the FORWARD chain
(2) ACCEPT all UDP traffic on port 53 if it comes from the IP 192.168.0.1 in the FORWARD chain
(3) Set the default FORWARD policy to DROP.

iptables -A FORWARD -p TCP --dport 80 -j ACCEPT
iptables -A FORWARD -p udp --dport 53 -j ACCEPT
iptables -P FORWARD DROP

Elastic search version

Fetch the elastic search version using

$ curl 'http://localhost:9200/?pretty'
{
  "status" : 200,
  "name" : "Midnight Sun",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "1.6.2",
    "build_hash" : "622039121e53e5f520b5ff8720fdbd3d0cb5326b",
    "build_timestamp" : "2015-07-29T09:24:47Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}

The elastic search version is 1.6.2

After upgrading to 5.x version of elastic search

$ curl -XGET 'http://localhost:9200'
{
  "name" : "3bsjSYM",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "zGjQ_4rBS1aoM-VzJ_hZCA",
  "version" : {
    "number" : "5.2.0",
    "build_hash" : "24e05b9",
    "build_date" : "2017-01-24T19:52:35.800Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.0"
  },
  "tagline" : "You Know, for Search"
}

The new elastic search version 5.2.0

Check if the file is on a NFS mount or on local filesystem mount

How to know if the file is available on a NFS mount or is part of the local filesystem mount?

df -PT <filename>

Eg: df -PT my_application.log
Filesystem    Type 1024-blocks      Used Available Capacity Mounted on
nfsbackup100.in.local:/my_app_log nfs 734003200 610560920 123442280      84% /a/nfsbackup30/my_app_log

So the above file my_application.log belongs to NFS mount

Elastic search Cluster: Identify Masternode

Use the following queries to identify the master node in an Elasticsearch cluster

There are many ways to identify the master node

Method1:

curl -XGET 'http://localhost:9200/_cat/nodes?pretty=true'

The entry of master node will be denoted by *

xyz.local 100.72.76.15 50 65 0.87 d * xyz

Method2:

1. Get the logical ID of the master node

curl -XGET 'http://localhost:9200/_cluster/state/master_node?pretty=true'

{
  "cluster_name" : "xxxxxx",
  "master_node" : "<Logical ID of master node>"
}

2. From the logical ID of the master_node obtained in step 1, identify the host

curl -XGET 'http://localhost:9200/_nodes/<logical ID of master node>/name?pretty=true'

{
  "cluster_name" : "xxxxxx",
  "nodes" : {
    "<Logical ID of master node" : {
      "name" : "<server_name>",
      "transport_address" : "inet[/100.72.76.19:9300]",
      "host" : "<server FQDN>",
      "ip" : "100.72.76.15",
      "version" : "1.7.1",
      "build" : "b88f43f",
      "http_address" : "inet[/100.72.76.19:9200]"
    }
  }
}

Elastic search: Shards and Replicas Performance

1. Having more shards enhances the indexing performance and allows to distribute a big index across machines.

2. Having more replicas enhances the search performance and improves the cluster availability

HTTP status code 200 and 204

The response code 200 literally means "OK" and is the code most often used when responding to a GET request. A POST request, however, may result in code 204 ("No Content") being sent back, meaning "Everything went OK but I don't really have anything to show you."

Ref: https://www.jeffknupp.com/blog/2014/03/03/what-is-a-web-framework/

Daemon process

Daemon process -

• detached from its parent, 
• all standard I/O redirected to /dev/null, and 
• the current directory changed to the root, / .

List files by their size - bash

To list files by their size

ls -Sl

To list top 5 files by their size

ls -S |  head -n 5

JVM Memory model and Garbage Collection

JVM Memory model and Garbage Collection (GC) is explained well in

http://www.journaldev.com/2856/java-jvm-memory-model-memory-management-in-java

Java JVM flags explained

To know about Java JVM flags,  a good read

https://blog.codecentric.de/en/2012/07/useful-jvm-flags-part-1-jvm-types-and-compiler-modes/
https://blog.codecentric.de/en/2012/07/useful-jvm-flags-part-2-flag-categories-and-jit-compiler-diagnostics/

Java Heap Space vs Stack – Memory Allocation in Java

The difference between Java Heap Space and Stack is explained well in the link

http://www.journaldev.com/4098/java-heap-space-vs-stack-memory

Why to set -Xms and -Xmx to the same value?

This query is answered well in

https://developer.jboss.org/thread/149559?_sscc=t

In a production environment, if you monitor the GC data, you will notice that is a relatively short period of time (usually less than an hour), the JVM will eventually increase the heap size to the -Xmx setting. Each time the JVM increases the heap size it must ask the OS for additional memory, which takes time (and thus adds to the response time of any requests that were is process when the GC hit). And usually the JVM will never let go of that memory. Therefore, since the JVM will eventually grab the -Xmx memory, you might as well set it to that at the beginning.

Another point is that with a smaller heap size (starting with -Xms), GCs will happen more often. So by starting with a larger heap initially the GCs will happen not as often.

Finally, in a production environment, you usually run only one app server per OS (or per VM). So since the app server is not competing for memory with other apps you might as well give it the memory up front.

Multiple python versions using pyenv

https://realpython.com/blog/python/python-virtual-environments-a-primer/

Splunk Indexing and data retention

http://www.learnsplunk.com/splunk-indexer-configuration.html
https://splunk-sizing.appspot.com

Capacity Planning:
http://docs.splunk.com/Documentation/Splunk/4.3.1/Installation/CapacityplanningforalargerSplunkdeployment