Linux: How to scan remote host for open ports ?

Helllo Team,

If you are trying to findout whether a remote server port is opened or closed, then there are so many ways. Here In this article I am posting on same using telnet and nc commands.

Using telnet

Telnet usually holds prompt, so to avoid that, we are using echo in front of the command.

> $ echo | telnet xxxxxx 22
Trying xxxxxx...
Connected to xxxxx.
Escape character is '^]'.
Connection closed by foreign host.

Using nc

with nc command, its direct.

> $ nc -z -v -u xxxxx 22
Connection to xxxx port 22 [udp/ssh] succeeded!

# -z: Just perform scans without sending any data to the respective service running on the given port.
# -v : Verbose
# -u: UDP packets, without this it will be default tcp.

Hope it helps. Thanks

OS Internals: Interprocess Communication ( IPC)

Interprocess Communication ( IPC )

• Interprocess communication allows a process to communicate with another process.

• Communications can be one of two types

  • Between related process ( parent and child )
  • Between unrelated processes ( one or more different processes)

• IPC can use

  • Pipes
  • First In First Out ( FIFO) - Queue
  • Message Queues
  • Shared Memory
  • Semaphores
  • Signals

Pipes

• Pipes are half-duplex, they can only read or write a process but only in one direction,

• ​x
  ls -lR /home | more
  # -R : List recursive subdirectory
  

•  

First In, First Out (Named Pipes)

• FIFO are also called as named pipes, and are half duplex as pipes.

• xxxxxxxxxx
  mkfifo mypipe
  # once data send to mypipe, till the other end ready to receive it, echo program will not exit.
  echo "Hello Pipe world" > mypipe
  # open another terminal window to receive the data
  cat mypipe
  

• 

• when you create a pipe, to identify it, execute ls -l command to the pipe, you can see if its a pipe or not

x
bash-3.2$ ls -l namedpipe00*
prw-r--r--  1 xxxx  xxxx  0 Jul 19 11:00 namedpipe001
prw-r--r--  1 xxxx  xxxx  0 Jul 19 11:06 namedpipe002
bash-3.2$

In the above output prw-r--r-- - p denotes that its a pipe.

Shared Memory

• shared memory is full duplex, either process can read and/or write.
• most efficient type of IPC, it does not require any kernel intervention once the shared memory has been allocated/deallocated.
• Required a program
• Any number of processes can read and write to the same shared memory segment.
• You can query shared memory with the ipcs command
• Processes must manage shared memory
• Process must protect shared memory being wirtten or race conditions will occur( if two process are trying to access same data at same time)

x
[xxxxxx:/root]# ipcs

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x020078cb 3          root       600        1

[xxxxxx:/root]#

Message Queues

• created by syscall
• managed by kernel
• kernel will delete the message from queue once it is read
• each read and write creates a syscall to the kernel
• the message queue helps eliminate the occurrences of race conditions but comes at the expense of performance due to the syscall interrupt(trap)
• data will remain in the queue until it is read and once read its gone.

 

Semaphores

• Semaphores are used to protect critical/common regions of memory shared between multiple processes.

• So other process cant use this shared memory till the process that owned the memory releases it.

• Semaphores are the atomic structures of operating systems

• Two Types of semaphores:

  • Binary: Only two states 0 and 1, locked/unlocked, or available, unavailable etc
  • Counting Semaphores : Allow arbitrary resource counters

• When a process 'allocates' a semaphore, it blocks the other processes from access until the first process issues a 'release' indicating it has completed its operation.

• The kernel then makes the semaphore available again for allocation.

Signals

• A signal is a notification of an event occurrence.

• A signal is also know as a trap, or software interrupt.

• if you perform 'kill -l' you can see list of killl siganls implemented in your kernel

  • for example in a linux machine

  x
  [xxxxx.xxxx:/home/users/xxxxxx]$ kill -l
   1) SIGHUP   2) SIGINT   3) SIGQUIT  4) SIGILL   5) SIGTRAP
   6) SIGABRT  7) SIGBUS   8) SIGFPE   9) SIGKILL 10) SIGUSR1
  11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM
  16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP
  21) SIGTTIN 22) SIGTTOU 23) SIGURG  24) SIGXCPU 25) SIGXFSZ
  26) SIGVTALRM 27) SIGPROF 28) SIGWINCH  29) SIGIO 30) SIGPWR
  31) SIGSYS  34) SIGRTMIN  35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
  38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
  43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
  48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
  53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
  58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
  63) SIGRTMAX-1  64) SIGRTMAX
  [xxxx.xxxxx:/home/users/xxxxx.xxxx]$
  

   

  • And in Mac,

x
bash-3.2$ kill -l
 1) SIGHUP   2) SIGINT   3) SIGQUIT  4) SIGILL
 5) SIGTRAP  6) SIGABRT  7) SIGEMT   8) SIGFPE
 9) SIGKILL 10) SIGBUS  11) SIGSEGV 12) SIGSYS
13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGURG
17) SIGSTOP 18) SIGTSTP 19) SIGCONT 20) SIGCHLD
21) SIGTTIN 22) SIGTTOU 23) SIGIO 24) SIGXCPU
25) SIGXFSZ 26) SIGVTALRM 27) SIGPROF 28) SIGWINCH
29) SIGINFO 30) SIGUSR1 31) SIGUSR2
bash-3.2$

• Signals can be generated by a user, process or kernel
• if its a process, it is supposed to be written to handle them
• Certain signals numeric 9-15 cannot be handled by the process they will immediately cause termination of the process and cannot be blocked, these are called 'process crash-outs' ExL CTRL+C, which cant be blocked by the process and terminates the process.

 

POSIX standards now govern the use and definition of IPC constructs and added support to modern features like threads etc.

 

Linux: How to validate /etc/fstab post changes ?

If You have done any changes to `/etc/fstab` file, and would like to check for any errors, then you can use the command `sudo findmnt --verify`.

Linux: How to remove directory / folder background color in terminal ?

Hello Everyone,

I moved to Ubuntu Linux recently from Arch Linux. I observed that my battery ( Lenovo Thinkpad E Series 3rd Gen) health getting low by using Arch Linux and there were some posts on that issue.

Though Arch Linux is great, I dont want to loose my $1000 laptop battery as I invested extra bucks especially for battery with extra capacity.

Any way, as I am using dual boot with Windows and Linux, I do have NTFS partitions.

As per color scheme of the terminal, if you list the items in the NTFS mount in terminal, they all come up with an ugly look as below

 so as you can see it, its completely not good.

So upon searching I came across below solution via Stackoverflow

If you are using zsh then open file ~/.zshrc, if bash then ~/.bashrc and paste following code at the end, save and exit from the file. Once exited, execute exec $SHELL  command, and it equal to source ~/.zshrc or source ~/.bashrc

eval "$(dircolors -p | \
   sed 's/ 4[0-9];/ 01;/; s/;4[0-9];/;01;/g; s/;4[0-9] /;01 /' | \
   dircolors /dev/stdin)"

So after applying the changes, my terminal output is clear as below

Hope it helps you.

Thank you.

How to take backup of your files regularly to external drive or to a new mount point ?

Hello Everyone,
As I have mentioned in other posts, I am using Arch based Endeavour distro in my ThinkPad.
No matter how stable the OS is, its always recommended to backup your important data.
So by doing very small tasks I achieve something like regular backup and OnDemand backup in my Arch Linux.

Step 1: Identify What folder you like to backup

So just to avoid any crazy permission related issues I got, like mentioned here , I set myself to use my home folder only for learnings.
So I have created a folder called `my_learning` and I am going to keep all my notes, code at that location. So if you want to create a folder you can simply do

mkdir ~/my_learning

Step 2: Install rsync ( If not installed )

If you dont know what rsync from Wikipedia

rsync is a utility for efficiently transferring and synchronizing files between a computer and an external hard drive and across networked computers by comparing the modification times and sizes of files

so what ever the distro you are using the binary name stays same to install rsync.
RedHat/CentOS/Fedora

sudo yum install rsync

Debian/Ubuntu

sudo apt-get install rsync

Arch

sudo pacman -S rsync

Step 3: Lets identify what Rsync options needed

So We need to backup to be happen in recusrive order and while its happening I need compress to happen just to save sometime. And verbose and human readable output format and pretty much needed anyway. So overall I need below options

-r, --recursive             recurse into directories
-z, --compress              compress file data during the transfer
-v, --verbose               increase verbosity
-h, --human-readable        output numbers in a human-readable format

But Rsync isnt limited to jus these 4 options, if you want to know more about rsync options, please check out its man page.

Step 4: source and target locations

So In one of my mount point I have created a folder with same name as source( make sure its mounted before creating the folder)

mkdir /run/media/username/ContinousImprovement/my_learning

.
and source is anyway my home folder

my_learning

location.
And the syntax of rsync is similar to cp command in linux i.e

cp [OPTIONS] source destination

rsync -zrvh /home/username/my_learning /run/media/username/ContinousImprovement/my_learning

Step 4: lets make the command handy

So I am using ZSH as my default shell, so I have opened my

.zshrc

file. If you are using bash, you can use

.bashrc

and I have added below function using shell scripting at very bottom

# backup home directory
function backup_home
  rsync -zrvh /home/username/my_learning /run/media/username/ContinousImprovement/my_learning

and execute

exec $SHELL

or

source ~/.zshrc

.
That's it, now if you call

backup_home

from terminal, your source directory will be backup to remote directory. if you want to automate it using a scheduler job, you can achieve same using

cron

, but make sure target is available during execution.

➜  my_learning backup_home 
sending incremental file list
my_learning/ansible/ansible.cfg
my_learning/ansible/inventory
my_learning/ansible/test.yaml
...
...
...

sent 51.93M bytes  received 141 bytes  103.86M bytes/sec
total size is 55.07M  speedup is 1.06
➜  my_learning 

Hope it helps.
Thank you.

What are the available Process Error Signals in Linux Kernel ?

As you all know like linux process uses error signals to communicate below is the list of errors 

Error Signal	Description
{{key}}	{{value}}

Arch Linux: Windows OS not apearing in Grub boot loader.

As I have installed Endeavour Linux operating system, on first boot I could see Windows OS option in boot menu. Later its gone. After searching a while, I came across below solution which worked.

Main reason is the os-prober is disabled by default so it wont be able to detect other OS available in the same system. So first we have to enable it.

Open file `/etc/default/grub` and add `GRUB_DISABLE_OS_PROBER=false` to the very end of the file. If you have already have this property, set it to `true`.

You can also execute below command, make sure you are in root shell or sudo user to execute this command.

echo "GRUB_DISABLE_OS_PROBER=false" >> /etc/default/grub

Then, execute below commands

sudo pacman -Syu
sudo pacman -S grub-tools
sudo pacman -S grub os-prober
sudo grub-mkconfig -o /boot/grub/grub.cfg

Actually if you `os-prober` command you can see the output that Windows detected right away

[agastya@agastya-thinkpad ~]$  sudo os-prober 
/dev/nvme0n1p1@/efi/Microsoft/Boot/bootmgfw.efi:Windows Boot Manager:Windows:efi
[agastya@agastya-thinkpad ~]$ 

Once grub configuration generated, restart your machine and you can see Windows in the grub menu.

Thank you.
Hope it helps.

Arch Linux - Frequent or Random Freezes

I have bought a new Leveno Thinkpad AMD Series laptop and installed EndeavourOS in it. Endeavour is a Arch flavour and Arch is known for stability. Usually I am CentOS fan but as Corporate made its choice with CentOS future, I did research thoughting of sticking with one of my favorite Arch Linux. But Arch, you have to setup everything and its an interesting process though time taking. So I have chosen its flavour which comes with everything setup from GUI to Stack OS

One problem kept me frustrated that its freezes randomly and started connecting Ethernet adapter suddenly and that hangs the entire OS and sometimes I have experienced restarts as well.

On searching around Arch Linux forums I came across installed a LTS kernel solving the issue.

If you are also facing the issue you can solve it by installing LTS kernel with below commands

sudo pacman -S linux-lts linux-lts-headers
sudo grub-mkconfig -o /boot/grub/grub.cfg

And reboot your machine. While Booting make sure you are selecting the LTS kernel instead of regular one.

Hope it helps.

MySQL Table Synchronization

MySQL Table Synchronization
=======================

Step: 1 Download the below package and sample script for table sysnchronization.

# cd /opt
# wget  https://static.spiceworks.com/images/how_to_steps/0000/3025/mysql-table-sync-0.9.3.tar.gz
# wget https://static.spiceworks.com/images/how_to_steps/0000/3026/syncTables.sh

Step: 2 Extract above package

# tar xzvf mysql-table-sync-0.9.3.tar.gz
# cd mysql-table-sync-0.9.3

Step: 3 Install perl-mysql modules

# yum install perl-ExtUtils-MakeMaker
# yum install "perl(DBD::mysql)"

Step: 4 Compile and install mysql-table-sync

# perl Makefile.PL
# make install

Step: 4 Then open syncTables.sh read configuration on top of the script. Change the configuration

Firewalld: Add Puppet master ports to firewalld in CentOS7.

1. Find out what are your server's active zones.

[root@vihitaatma ~]# firewall-cmd --get-active-zones
public
  interfaces: ens192
[root@vihitaatma ~]#

2 Puppet has different ports for different services.

3000: Web based installer 

8140: Communication port between Puppet Master & Agent. 

61613: Used by MCollective for orachestration requests by Puppet agents

443: Puppet Enterprise console web port.

5432: PostgreSQL 

8081: Puppet DB Request Port.

8142: Used by Orachestration services to accept inbound traffic/responses from Puppet Agents  

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --add-port=8140/tcp

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --add-port=61613/tcp

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --add-port=443/tcp

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --add-port=5432/tcp

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --add-port=8081/tcp

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --add-port=8142/tcp

success

[root@vihitaatma ~]# sudo firewall-cmd --reload

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --list-ports

 8140/tcp 61613/tcp 443/tcp 5432/tcp 8081/tcp 8142/tcp

3. And To Remove Ports

[root@vihitaatma ~]# firewall-cmd --zone=public --remove-port=3000/tcp

success

[root@vihitaatma ~]# firewall-cmd --runtime-to-permanent

success

[root@vihitaatma ~]# firewall-cmd --reload

success

[root@vihitaatma ~]# firewall-cmd --zone=public --permanent --list-ports

8140/tcp 61613/tcp 443/tcp 5432/tcp 8081/tcp 8142/tcp

[root@vihitaatma ~]#

Bash: fetch IP Address in a file using GNU grep

You can use below command

grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" filename.txt 

Root CA and Wildcard Certificate Generation in CentOS/RHEL 6&7

     Hi folks ! this is one of the best method to create your own RootCA server and generating self-signed wildcard certificates.The greatest advantage of following this method: this would not make any system level changes, as everything is stored in files mentioned in the commands. At any stage if something went wrong, clear all the files and perform the steps once again.

There are two sections

1. RootCA Server   --  Need to perform only once.

2. Generating wildcard certificates for xyz.com domain -- Need to perform once per domain to create wildcard certificates. Need to perform once per site per domain to create individual certificates per site.


RootCA Server
============
1. Install required packages.

# yum install openssl -y

2. Generate XYZRootCA certificates

# mkdir /opt/XYZRootCA
# cd /opt/XYZRootCA
# openssl genrsa -out XYZRootCA.key 2048
# openssl req -x509 -new -nodes -key XYZRootCA.key -sha256 -days 10950 -out XYZRootCA.pem

#Provide information as given below

Country Name (2 letter code) [XX]:IN
State or Province Name (full name) []:Karnataka
Locality Name (eg, city) [Default City]:Bengaluru
Organization Name (eg, company) [Default Company Ltd]:XYZ Solutions Pvt. Ltd
Organizational Unit Name (eg, section) []:IT
Common Name (eg, your name or your server's hostname) []:XYZRootCA
Email Address []:info@xyz.com

3. Convert .pem to .crt

# openssl x509 -outform der -in XYZRootCA.pem -out XYZRootCA.crt



Generating wildcard certificates for xyz.com domain
===========================================
1. Generate CSR for *.xyz.com

# openssl genrsa -out XYZWildcard.key 2048
# openssl req -new -key XYZWildcard.key -out XYZWildcard.csr

#Provide information as given below
Country Name (2 letter code) [XX]:IN
State or Province Name (full name) []:Karnataka
Locality Name (eg, city) [Default City]:Bengaluru
Organization Name (eg, company) [Default Company Ltd]:XYZ Solutions Pvt. Ltd
Organizational Unit Name (eg, section) []:Infra Support
Common Name (eg, your name or your server's hostname) []:*.xyz.com
Email Address []:infrasupport@xyz.com

2. Using CSR generated above (As Shown in Step no:1), generate a wildcard certificate for *.xyz.com also get it signed by XYZRootCA as well.

# openssl x509 -req -in XYZWildcard.csr -CA XYZRootCA.pem -CAkey XYZRootCA.key -CAcreateserial -out XYZWildcard.crt -days 3650 -sha256

3. Import XYZRootCA.crt to trusted root certificates

# yum install ca-certificates
# update-ca-trust force-enable
# cp XYZRootCA.crt /etc/pki/ca-trust/source/anchors/
# update-ca-trust extract



References:
###########

1. https://datacenteroverlords.com/2012/03/01/creating-your-own-ssl-certificate-authority/
2. http://linoxide.com/security/make-ca-certificate-authority/
3. https://blog.celogeek.com/201209/209/how-to-create-a-self-signed-wildcard-certificate/
4. http://stackoverflow.com/questions/13732826/convert-pem-to-crt-and-key
5. https://serversforhackers.com/self-signed-ssl-certificates
6. http://kb.kerio.com/product/kerio-connect/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html

A Go-Ready vagrant setup

Hello,

I hope you know what is a Vagrant. If you don't , please visit https://www.vagrantup.com/intro/index.html.

For a Vagrant setup Vagrantfile is important. In the Vagrantfile we can define so many parameters for the VM we would like to build, for example IP Address, Hostname, Port Forwarding and while spinning the VM itself we can install the new packages and make our development or testing environment ready when ever we want.

I dont write Installation of Vagrant here, because their documentation is excellent and I dont want to duplicate it. If you want to install Vagrant , look at https://www.vagrantup.com/downloads.html and VirtuboxVM is one of pre-requisite for Vagrant and I hope you know how to install VirtualBox, if you dont know then look at https://www.virtualbox.org/wiki/Downloads.

Now I assume , you have installed Vagrant and VirtulBox.  Whether you are using Windows or Linux commands are same , just use your senses at path format which is different for Windows and Linux.

Here I will show you how to generate a new Vagrant file and how to use it as per our requirement.
Open you command prompt or powershell prompt depends on the OS and type as

mkdir vagrant_1
cd vagrant_1
vagrant init

Here vagrant_1 is the directory name where I have initialized the vagrant and you can replace it with anything you want. If you see contents of vagrant_1 directory after executing vagrant init , you will see a file with name Vagrantfile.

This is our main file. Open it with your favorite editor like Sublime, Atom , Brackets or any other you like.

Now observe carefully. Remove everything between

Vagrant.configure("2") do |config|
end

Now lets define some configuration.

Vagrant.configure("2") do |config|
config.vm.define "vagrant-centos01" do |vc01|
vc01.vm.box = "centos/7"
vc01.vm.hostname = "vagrant-centos01"
vc01.vm.network "private_network", ip: "192.168.20.20"
end
end

Lets go through each of them ,

1. With Vagrant.configure("2") do |config| , we saying to Vagrant that use Vagrant from 1.1+ to 2.0.X.

2. With config.vm.define "vagrant-centos01" do |vc01| we are saying as define or create a new VM with name as vagrant-centos01

3. With vc01.vm.box = "centos/7" , we are saying as for use centos 7 box. if you centos-7 not available locally , Vagrant download it from Hashicorp.

4. vc01.vm.hostname = "vagrant-centos01" , says what is the hostname

5. vc01.vm.network "private_network", ip: "192.168.20.20" , says what is the private IP or static IP and this is similar to Host-Only adapter at VirtualBox.

Thats it, A very basic VM setup is ready with hostname and Private IP. After saving this configuration use command as

vagrant up

and then

vagrant ssh 

To login into that machine. And this is very basic Vagrantfile setup. The deeper you dive the complex and beautiful it will.

For more information on building a Vagrantfile , check https://www.vagrantup.com/docs/vagrantfile/


Hope that helps.


=======================THIS IS NOT THE END===========================