Saturday 27 September 2014

SED Command Basics

SED also known as Stream Editor, Here we will discuss the basic functionality of the sed that will help in simple scripts with bash.

General Syntax:

# sed OPTIONS ' ADDRESS ACTION' FILENAME

in the above syntax,

OPTIONS: We generally use some options with commands. Options also applies with sed commands. List of sed options can be found in the man page of the sed.

ADDRESS: address here signifies to the location where our operation will apply. There are two types of addressing techniques available.
1. content addressing
2. line addressing

ACTION: action is the operation that we want to apply on the selected lines using address.

FILENAME: it is the path of the file that we want to manipulate with sed.

EXAMPLES:

1. # sed '3,q' filename.txt

In the example 1 here, 3 is the line no and q is the operation applied. It means that from the filename.txt 3rd line will be printed and the quit operation will result in closing the file.

2. # sed -n '1,2p' filename.txt

In the example 2 here, 1 and 2 is the line no and p is the operation. p will print the 1st and 2nd line. Option -n will ensure printing of result only once. If -n option is not used, the text will be printed twice.

3. # sed -n '$p' filename.txt

In the example above, we use $ to print last line of the file. The $ will always used to refer to the end of the file or end of the line.

4. # sed -n '9,11p' filename.txt

We can also refer a range of lines from a file. here we have given the range from 9 to 11 lines. Sed allows to select lines from anywhere in the file.

5. # sed -n '1,4p'
> '7,10p'
> '12,15p' filename.txt

We can select the multiple range groups in one command and apply different operations in the single command.

6. # sed -n -e '1,4p' -e '7,10p' -e '12,15p' filename.txt

If we don't follow the example 5, we can also use the example 6. We can use -e option to specify the different ranges in the address.

7. # sed -n '/echo/p' filename.txt

In the above example we used content addressing. Sed will select the line that has echo in the line and print that line.

8. # sed -n '/echo/, /printf/, /print/p' file.txt

This is the multiple addressing in content addressing.

We have following operation available in the SED command.
1. Inserting Text (i)
2. Deleting text (d)
3. Substitution (s)
4. Writing lines (w)
5. Append Text (a)

9. # sed -n '/echo/w newfile' filename.txt

The lines in the filename.txt, having echo will be copied to newfile and -n option will stop from printing.

10. # sed -n '/echo/w echo.txt,
> '/print/w print.txt' filename.txt

This will write line from filename.txt, the lines having echo will copied to echo.txt and lines having print will be copied to print.txt.

11. # sed -n '1,50w file1.txt
> '51,$w file2.txt' original.txt

This will divide a file original.txt into two files file1.txt and file2.txt, first file will have 50 lines and second file will have rest of files.

12. # sed '1i\
> first line goes here\
> second line goes here\
> third line goes here
> ' original.txt > tmpfile.txt

This will insert lines beginning from first line in the temp file. then we can move the tmp file into original file.

13. # sed '/echo/d' emp.list > newfile.txt

The above example will delete the lines having echo and create a new file with the changes. We can move the new file with original filename.

14. # sed '/^[ /t]*$/d' originalfile.txt > tmpfile.txt

This will delete all the empty lines from original file and create a tempfile that have no empty lines.

Subtitution(s):

SED allows a useful utility in subtitution of strings.
Below example will help to understand these operations.

Basic syntax:

# sed [address] s/exp1/exp2/flags filename.txt

15. # sed 's/echo/printf/' filename.txt > newfile.txt

this will change the echo with printf in the beginning of the file filename.txt and save the changes to newfile.txt. Since the above command don't specify the address, hence applied on the complete file. We can use g flag to change all the occurrences.

16. # sed 's/echo/printf/g' filename.txt > newfile.txt

change all the existance of the echo with printf.

17. #sed '1,5s/echo/printf/g' filename.txt > newfile.txt

This command will look for occurences of echo in first five lines, any echo found here will be changed to printf. Echo after these lines will not be changed.

18. # sed 's/^/#/' file.txt

19. # sed 's/$/#/' file.txt

The above examples will be used to remove the # sign from beginning and the end of the lines.

Friday 26 September 2014

MySQL Master-Slave Replication

Install a MySQL in Master Server

First, proceed with MySQL installation using YUM command. If you already have MySQL installation, you can skip this step.
# yum install mysql-server mysql
Configure a MySQL in Master Server
Open my.cnf configuration file with VI editor.
# vi /etc/my.cnf
Add the following entries under [mysqld] section and don’t forget to replace replicationdb with database name that you would like to replicate on Slave.
server-id = 1 
binlog-do-db=replicationdb
relay-log = /var/lib/mysql/mysql-relay-bin
relay-log-index = /var/lib/mysql/mysql-relay-bin.index
log-error = /var/lib/mysql/mysql.err
master-info-file = /var/lib/mysql/mysql-master.info  
relay-log-info-file = /var/lib/mysql/mysql-relay-log.info
log-bin = /var/lib/mysql/mysql-bin
Restart the MySQL service.
# /etc/init.d/mysqld restart
Login into MySQL as root user and create the slave user and grant privileges for replication. Replace replicationuser with user and redhat with password.
# mysql -u root -p
mysql> GRANT REPLICATION SLAVE ON *.* TO 'replicationuser'@'%' IDENTIFIED BY 'redhat';  
mysql> FLUSH PRIVILEGES;
mysql> FLUSH TABLES WITH READ LOCK;
mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000003 | 11128001 | replicationdb |                  |
+------------------+----------+--------------+------------------+
1 row in set (0.00 sec)
mysql> quit;
Please write down the File (mysql-bin.000003) and Position (11128001) numbers, we required these numbers later on Slave server. Next apply READ LOCK to databases to export all the database and master database information with mysqldump command.
#  mysqldump -u root -p --all-databases --master-data > /root/dbdump.db
Once you’ve dump all the databases, now again connect to mysql as root user and unlcok tables.
mysql> UNLOCK TABLES;
mysql> quit;
Upload the database dump file on Slave Server (192.168.3.93) using SCP command.
#scp /root/dbdump.db root@192.168.3.93:/root/
That’s it we have successfully configured Master server, let’s proceed to Phase II section.
Phase II: Configure Slave Server (192.168.3.93) for Replication
In Phase II, we do the installation of MySQL, setting up Replication and then verifying replication.

Install a MySQL in Slave Server

If you don’t have MySQL installed, then install it using YUM command.
# yum install mysql-server mysql
Configure a MySQL in Slave Server
Open my.cnf configuration file with VI editor.
# vi /etc/my.cnf
Add the following entries under [mysqld] section and don’t forget to replace IP address of Master server, replicationdb with database name etc, that you would like to replicate with Master.
server-id = 2
master-host=192.168.3.71
master-connect-retry=60
master-user=replicationuser
master-password=redhat
replicate-do-db=replicationdb
relay-log = /var/lib/mysql/mysql-relay-bin
relay-log-index = /var/lib/mysql/mysql-relay-bin.index
log-error = /var/lib/mysql/mysql.err 
master-info-file = /var/lib/mysql/mysql-master.info
relay-log-info-file = /var/lib/mysql/mysql-relay-log.info
log-bin = /var/lib/mysql/mysql-bin
Now import the dump file that we exported in earlier command and restart the MySQL service.
# mysql -u root -p < /root/dbdump.db
# /etc/init.d/mysqld restart
Login into MySQL as root user and stop the slave. Then tell the slave to where to look for Master log file, that we have write down on master with SHOW MASTER STATUS; command as File (mysql-bin.000003) and Position (11128001) numbers. You must change 192.168.3.71 to the IP address of the Master Server, and change the user and password accordingly.
# mysql -u root -p
mysql> slave stop;
mysql> CHANGE MASTER TO MASTER_HOST='192.168.3.71', MASTER_USER='replicationuser', MASTER_PASSWORD='redhat', MASTER_LOG_FILE='mysql-bin.000003', MASTER_LOG_POS=11128001; 
mysql> slave start;
mysql> show slave status\G
  •  1. row *
Slave_IO_State: Waiting for master to send event
                Master_Host: 192.168.3.71
                Master_User: replicationuser
                Master_Port: 3306
              Connect_Retry: 60
            Master_Log_File: mysql-bin.000003
        Read_Master_Log_Pos: 12345100
             Relay_Log_File: mysql-relay-bin.000002
              Relay_Log_Pos: 11381900
      Relay_Master_Log_File: mysql-bin.000003
           Slave_IO_Running: Yes
          Slave_SQL_Running: Yes
            Replicate_Do_DB: replicationdb
        Replicate_Ignore_DB:
         Replicate_Do_Table:
     Replicate_Ignore_Table:
    Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
                 Last_Errno: 0
                 Last_Error:
               Skip_Counter: 0
        Exec_Master_Log_Pos: 12345100
            Relay_Log_Space: 11382055
            Until_Condition: None
             Until_Log_File:
              Until_Log_Pos: 0
         Master_SSL_Allowed: No
         Master_SSL_CA_File:
         Master_SSL_CA_Path:
            Master_SSL_Cert:
          Master_SSL_Cipher:
             Master_SSL_Key:
      Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
              Last_IO_Errno: 0
              Last_IO_Error:
             Last_SQL_Errno: 0
             Last_SQL_Error:
1 row in set (0.00 sec)
Verifying MySQL Replication on Master and Slave Server
It's really very important to know that the replication is working perfectly. On Master server create table and insert some values in it.
On Master Server
mysql> use replicationdb;
mysql> CREATE TABLE employee (c int);
mysql> INSERT INTO employee (c) VALUES (1);
mysql> SELECT * FROM employee;
+------+
|  c  |
+------+
|  1  |
+------+
1 row in set (0.00 sec)
On Slave Server
Verifying the SLAVE, by running the same command, it will return the same values in the slave too.
mysql> use replicationdb;
mysql> SELECT * FROM employee;
+------+
|  c  |
+------+
|  1  |
+------+
1 row in set (0.00 sec)
That's it, finally you've configured MySQL.

Setup Two instances of OTRS on same server

This setup will describe how to run multiple instances of OTRS on one host.

We will install OTRS and Apache from sources.

Download OTRS from following link:
http://ftp.otrs.org/pub/otrs/

Download APACHE from following link: http://httpd.apache.org/download.cgi

Check for the following perl modules on your system. These perl modules are needed by the otrs and must be installed on your system.

CGI Date::Pcalc
DBI DBD::mysql
Digest::MD5
LWP::UserAgent
IO::Scalar
IO::Wrap
MIME::Base64
MIME::Tools
Mail::Internet
Net::DNS
Net::POP3
Net::LDAP (for directory authentication - not required)
Net::SMTP
Authen::SASL
GD (for stats - not required)
GD::Text (for stats - not required)
GD::Graph (for stats - not required)
GD::Graph::lines (for stats - not required)
GD::Text::Align (for stats - not required) XML::Parser

Installation Procedure

First Instance of OTRS

Step-1: Login to the system as root user


Step-2: Move the downloaded source on to /usr/local/src

root@otrs#mv otrs-2.0.4-01.tar.gz /usr/local/src/

Step-3: Unpack the tar.gz


root@otrs#cd /usr/local/src/
root@otrs#tar xzvf otrs-2.0.4-01.tar.gz

Step-4:


root@otrs#mv otrs /opt/otrsone
root@otrs#cd /opt/otrsone/Kernel root@otrs#cp Config.pm.dist Config.pm root@otrs#cd Config
root@otrs#cp GenericAgent.pm.dist GenericAgent.pm

Step-5: Add a user “otrsone” in the group “otrsone”


root@otrs#useradd -d /opt/otrsone/ -c 'First OTRS Instance user' otrsone root@otrs#groupadd otrsone
root@otrs#usermod -G otrsone otrsone

Step-6: Configure the Home Directory of OTRS in Config.pm


root@otrs#cd /opt/otrsone/Kernel
root@otrs#vi Config.pm

Change the otrs Home to be “/opt/otrsone”, I believe it is in line number 61# ---------------------------------------------------- # # fs root directory # ---------------------------------------------------- #


 $Self->{Home} = '/opt/otrsone';

Step-7: Assign appropriate permissions on to the otrs install directory


root@otrs#cd /opt/otrsone/bin/
root@otrs#./SetPermissions.sh /opt/otrsone/ otrsone otrsone otrsone otrsone

First Instance of httpd

Step-1: Move the downloaded tar.gz of httpd on to /usr/local/src


root@otrs#mv httpd-2.2.2.tar.gz /usr/local/src

Step-2: Unpack the tar.gz


root@otrs#cd /usr/local/src
root@otrs#tar xzvf httpd-2.2.2.tar.gz

Step-3: Create a target directory for first apache’s installation


root@otrs#mkdir /opt/apache-one

Step-4: Configure


root@otrs#cd /usr/local/src/httpd-2.2.2
root@otrs#./configure --prefix=/opt/apache-one

Step-5: Compile


root@otrs#make

Step-6: Install


root@otrs#make install

Step-7: Create a “conf.d” Directory


root@otrs#cd /opt/apache-one
root@otrs#mkdir conf.d

Step-8: Insert the “OTRS” Configuration File in the “conf.d” directory


root@otrs#cd /opt/apache-one/conf.d/
root@otrs#vi otrs.conf

#basic apache configurations file for OTRS
ScriptAlias /otrs/ ”/opt/otrsone/bin/cgi-bin/”
Alias /otrs-web/ ”/opt/otrsone/var/httpd/htdocs/”
# Directory settings
#
<Directory ”/opt/otrsone/bin/cgi-bin/”>
AllowOverride None
Options +ExecCGI -Includes
Order allow,deny
Allow from all
</Directory>
<Directory ”/opt/otrsone/var/httpd/htdocs/”>
AllowOverride None
Order allow,deny
Allow from all
</Directory>

Step-9: Modify the httpd.conf file to do the following:

* Listen to the desired port number
* Run as user otrsone of group otrsone
* Load the configurations from “Conf.d” during server
startup

root@otrs#cd /opt/apache-one/conf/

root@otrs#cp httpd.conf httpd.conf.orig [Backup the Original Configuration]


diff of my httpd.conf and httpd.conf.orig 40c40 < Listen 9000 --- > Listen 80 65,66c65,66 < User otrsone < Group otrsone --- > User daemon > Group daemon 414,417d413 < < <

 # include legacy conf.d during a transition period < Include conf.d/*.conf


Its simple, I have made the “Listen Port” to be 9000, the apache user to be “otrsone” and group to be “otrsone”, further I have added a line to include scripts in “Conf.d” directory during server startup

Step-10: Change the ownership of the apache install directory


root@otrs#chown otrsone:otrsone -R /opt/apache-one


Step-11: Start the Web Server


root@otrs#su otrsone
otrsone@otrs$/opt/apache-one/bin/apachectl start


Step-12: Remove the Apache Install Directory


root@otrs#rm –rf /usr/local/src/httpd-2.2.2

Second Instance of OTRS

Step-1: Unpack the tar.gz


root@otrs#cd /usr/local/src/
root@otrs#tar xzvf otrs-2.0.4-01.tar.gz


Step-2:


root@otrs#mv otrs /opt/otrstwo
root@otrs#cd /opt/otrstwo/Kernel
root@otrs#cp Config.pm.dist Config.pm
root@otrs#cd Config
root@otrs#cp GenericAgent.pm.dist GenericAgent.pm


Step-3: Add a user “otrstwo” in the group otrstwo


root@otrs#useradd -d /opt/otrstwo/ -c 'Second OTRS Instance user' otrstwo root@otrs#groupadd otrstwo
root@otrs#usermod -G otrstwo otrstwo


Step-4: Configure the Home Directory of OTRS in Config.pm


root@otrs#cd /opt/otrstwo/Kernel
root@otrs#vi Config.pm


Change the otrs Home to be “/opt/otrstwo”# ---------------------------------------------------- # # fs root directory # ---------------------------------------------------- #

 $Self->{Home} = '/opt/otrstwo';


Step-5: Assign appropriate permissions on to the otrs install directory


root@otrs#cd /opt/otrstwo/bin/
root@otrs#./SetPermissions.sh /opt/otrstwo/ otrstwo otrstwo otrstwo otrstwo

Second Instance of httpd

Step-1: Unpack the tar.gz


root@otrs#cd /usr/local/src
root@otrs#tar xzvf httpd-2.2.2.tar.gz

Step-2: Create a target directory for first apache’s installation


root@otrs#mkdir /opt/apache-two


Step-3: Configure


root@otrs#cd /usr/local/src/httpd-2.2.2
root@otrs#./configure --prefix=/opt/apache-two

Step-4: Compileroot@otrs#make

Step-5: Install


root@otrs#make install

Step-6: Create a “conf.d” Directory


root@otrs#cd /opt/apache-two
root@otrs#mkdir conf.d


Step-7: Insert the “OTRS” Configuration File in the “conf.d” directory


root@otrs#cd /opt/apache-two/conf.d/
root@otrs#vi otrs.conf


#basic apache configurations file for OTRS
ScriptAlias /otrs/ ”/opt/otrstwo/bin/cgi-bin/”
Alias /otrs-web/ ”/opt/otrstwo/var/httpd/htdocs/”
# Directory settings
#
<Directory ”/opt/otrstwo/bin/cgi-bin/”>
AllowOverride None
Options +ExecCGI -Includes
Order allow,deny
Allow from all
</Directory>
<Directory ”/opt/otrstwo/var/httpd/htdocs/”>
AllowOverride None
Order allow,deny
Allow from all
</Directory>


Step-8: Modify the httpd.conf file to do the following:


  • Listen to the desired port number
  • Run as user otrstwo of group otrstwo
  • Load the configurations from “Conf.d” during server startup


root@otrs#cd /opt/apache-two/conf/
root@otrs#cp httpd.conf httpd.conf.orig [Backup the Original Configuration]


diff of my httpd.conf and httpd.conf.orig40c4 < Listen 9010 --- > Listen 80 65,66c65,66 < User otrstwo < Group otrstwo --- > User daemon > Group daemon 414,417d413 < < < # includes legacy conf.d during a transition period < Include conf.d/*.conf


Step-9: Change the ownership of the apache install directory


root@otrs#chown otrstwo:otrstwo -R /opt/apache-two


Step-10: Start the Web Server


root@otrs#su otrstwo
otrstwo@otrs$/opt/apache-two/bin/apachectl start


Step-11: Remove the Apache Install Directory


root@otrs#rm –rf /usr/local/src/httpd-2.2.2

Boot Up Scripts

Here is the way to write the boot scripts on apache servers. I don’t know if this is the right way. But it works!

First Instance

Step-1: Insert the following script in /etc/init.d


root@otrs#vi otrsone

#!/bin/sh

# # otrsone: Starts the First copy of otrs
# # chkconfig: 2345 96 20
# description: Starts and stops the First Copy of OTRS at boot time and shutdown.
#
/opt/apache-one/bin/apachectl start ==============================================================


Step-2:


root@otrs#chkconfig –add otrsone
root@otrs#chkconfig --level 2345 otrsone on

Second Instance

Step-1: Insert the following script in /etc/init.d


root@otrs#vi otrstwo

#!/bin/sh

# # otrstwo: Starts the Second copy of otrs #
# chkconfig: 2345 97 21
# description: Starts and stops the Second Copy of OTRS at boot time and shutdown.
#
/opt/apache-two/bin/apachectl start =========================================================

Step-2:


root@otrs#chkconfig –add otrstwo
root@otrs#chkconfig --level 2345 otrstwo on

Test the Installation:


http:<ip_address>:9000/otrs/installer.pl http:<ip_address>:9010/otrs/installer.pl

Refer OTRS Manual to proceed with the rest of the installation

Wednesday 24 September 2014

Virtual IP Configuration

Configuring multiple IP addresses on the single interface is known as virtual IP configuration. Some time also known as the IP alias. IP aliasing is very useful in case of creating different websites on same server using virtual hosting (IP based virtual hosting) using a single network interface card.
The main advantage of using Virtual IP is that we can create as many virtual IPs as required without the need of additional interface cards.

Here we will discuss how to setup virtual IP configuration on Linux:

If we want to temporarily setup the virtual IP we can use the below command:

# ifconfig eth0:0 10.14.153.23 netmask 255.255.255.0

Now you can check the interface using ifconfig command.

If we want to permanently setup the virtual IP, We need to setup the configuration in a file.

CentOS, RedHat, Fedora.

Suppose we have eth0 physical interface available and we want to create 3 new virtual interface based on this interface. We will follow the below steps:

1. Move to the network files directory:

# cd /etc/sysconfig/network-scripts/

# ls -l

2. Open the contents of the ifcfg-eth0 file using cat command.

DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
IPADDR=10.14.153.31
NETMASK=255.255.255.0
GATEWAY=10.14.153.254

BOOTPROTO=static

3. We can see the configuration file of eth0 interface as ifcfg-eth0. Copy this file to below names in the same directory.

# cp ifcfg-eth0 ifcfg-eth0:0 

# cp ifcfg-eth0 ifcfg-eth0:1

# cp ifcfg-eth0 ifcfg-eth0:2

4. Now edit the individual file and set the IP address and Device name as shown below.

# vim ifcfg-eth0:0

DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
IPADDR=10.14.153.23
NETMASK=255.255.255.0
GATEWAY=10.14.153.254

BOOTPROTO=static

# vim ifcfg-eth0:1

DEVICE=eth0:1
TYPE=Ethernet
ONBOOT=yes
IPADDR=10.14.153.24
NETMASK=255.255.255.0
GATEWAY=10.14.153.254

BOOTPROTO=static

# vim ifcfg-eth0:2

DEVICE=eth0:2
TYPE=Ethernet
ONBOOT=yes
IPADDR=10.14.153.25
NETMASK=255.255.255.0
GATEWAY=10.14.153.254

BOOTPROTO=static

5. After saving the files, restart the network service.

# /etc/init.d/network restart

6.  We can also specify the range of virtual IP addresses. create a file and specify the range.

# vim /etc/sysconfig/network-scripts/ifcfg-eth0-range0

TYPE=Ethernet
IPADDR_START=10.14.153.23
IPADDR_END=10.14.153.26


7. Save the file and restart the service.

# /etc/init.d/network restart

IP Bonding

In linux IP bonding generally refers to aggregate multiple network interfaces into single logical interface called as bonded interface. The activity of the bonded interface depends on the mode chosen. There are different modes available in IP bonding that provides necessary integrity and redundancy. Here we will discuss how to bond the two interfaces using this technique.

1. Create configuration file bond0:

# vim /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
IPADDR=192.168.0.10
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
USERCTL= no
BOOTPROTO= none
ONBOOT= yes

2. Replace the text of your first ethernet interface with following:

DEVICE= eth0
USERCTL= no
ONBOOT= yes
MASTER= bond0
SLAVE= yes
BOOTPROTO= none

3. Replace the text of your second ethernet interface with following:

DEVICE= eth1
USERCTL= no
ONBOOT= yes
MASTER= bond0
SLAVE= yes
BOOTPROTO= none

Make sure to add interface name you are using in DEVICE and bond configuration file in MASTER.

4. Make sure Loading bond kernel modules while using bonding. By default this module in not loaded. Append the configuration in configuration file of modules.

# vim /etc/modprobe.conf

alias bond0 bonding
options bond0 mode=6 miimon=100

miimon: This specifies how frequently MII link is monitored. The state of each slave is monitored every 100 milli seconds.

mode: Specifies one of the bonding facilities. Default is round robin. The list of all the modes is given in the end of this tutorial.

5. Now load the module using below command.

# modprobe bonding

6. Restart the service of network and you are done.

# /etc/init.d/network restart

7. Check using ifconfig command and manually test the setup.

List of all the modes available in bonding:

1. balance-rr or 0

Distribute packets in alternate sequence.

2. active backup or 1

Only one interface will be active at a time. Due to failure if one goes down, then other comes up automatically.

3. balance-xor or 2

Packet transmission is based on the specific hash policy, provides load balancing and fault tolerance.

4. broadcast or 3

broadcast policy, transmitted through all the available interfaces.

5. 802.3ad or 4

Dynamic Link aggregation, the interfaces that share the same link speed.

6. balance-tlb or 5

The outgoing traffic is distributed according to current load. adaptive transmit load balancing.

7. balance-alb or 6

The outgoing traffic is distributed according to current load. adaptive transmit and receive load balancing.

Packet Capturing using TCPDUMP

Packet Capturing using TCPDUMP

TCPdump has a powerful language you can use to describe and filter packets, ranging from matching semantic attributes of the packets, protocols, hosts, and ports being used right down to filtering attributes in the TCP and UDP headers. In this section, we’re going to go over how the packet filter language works and how you filter packets for certain attributes.

# tcpdump host 10.14.148.91

The above command will matches all the packets that will have 10.14.148.91 as source or destination.

# tcpdump src host 10.14.153.81 or dst host 10.14.153.91

The above command will match if 10.14.153.81 in source or either 10.14.153.91 in destination.

We can also match the whole subnet, for example:

# tcpdump net 10.14.153.0/24

We can also apply the filter on the ports:

# tcpdump net 10.14.153.0/24 and tcp port 80

# tcpdump port 80

# tcpdump tcp

# tcpdump portrange 0-1024

For Ping responses, we can use some regular expressions:

# tcpdump ‘icmp[icmptype] & icmp-echo!= 0’

# tcpdump –i eth0

The above command will scan packets arriving under eth0.

# tcpdump –c 100

Will capture only 100 packets only.

# tcpdump –n

The above command will display IP addresses and port number instead of domain and services when capturing packets.

Below are some attributes that we can use in tcpdump.

dst, src, host, net, portrange, udp, tcp, icmp, arp etc.

# tcpdump –v icmp

# tcpdump –n “dst host 10.14.153.212 and (dst port 80 or dst port 443)

# tcpdump –n dst net 10.14.153.0/24

Network Discovery using NMAP

Network Discovery using NMAP

Nmap is a network scanning and host detection tool that is very useful during several steps of penetration testing. It can be used for following purposes:

1. Detect the live host on the network.

2. Detect the open ports on the host.

3. Detect the software and the version to the respective port.

4. Detect the operating system, MAC address, software version etc.

Below are some commands to demonstrate the use of the NMAP.

# nmap ip/hostname

# nmap 10.14.153.31






The above command will show the information about the given host.

We can also specify the whole subnet for this purpose.

# nmap 10.14.153.0/24




Or

# nmap 10.14.153.142 10.14.153.212 10.14.153.25

We can also specify the range of IP address,

# nmap 10.14.153.10-100

We can also refer the hosts from a list of IPs.

# nmap –iL hosts.txt

Or

#nmap –sL hosts.txt

We can also use - - exclude option,

# nmap –sL 10.14.153.0/24 – exclude

We can specify the port number to scan them manually.

# nmap –p80,443,22 10.14.153.1-100

# nmap –sT 10.14.153.142

The above command can be used to check the open ports, the command will connect to every open port on the target machine and lists the open ports.

# nmap –sS 10.14.153.142

The above command is used to check open ports again, however the difference is that the T option will establish a real connection but S option will not make a full TCP connection to target machine. This type of scanning is logged to very less extent.

# nmap –sP 10.14.153.1/24

This is a simple ping scan to the complete network.

# nmap –PT80 10.14.153.5

This is an example for TCP ping scan.

RAID Concept

 
RAID CONCEPTS

How to calculate the various sizes after RAID implementations?

RAID 0(Stripe Set)

RAID 0 splits data across drives, resulting in higher data throughput. The performance of this configuration is extremely high, but a loss of any drive in the array will result in data loss. This level is commonly referred to as striping.





Space efficiency = 1 (100%)
Fault tolerance = 0 disk(s) (none)
Total Disks X Size of DISK = Total Size
Size after implementation = Total Size X Space Efficiency

Suppose we are using 3 disks of 250GB size, after RAID 0 implementation we get the total size out to be:


3 * 250 = 750
750 * 1 = 750

Hence we get 750 as final disk size.

RAID 1(Mirror)

RAID 1 writes all data to two or more drives for 100% redundancy: if either drive fails, no data is lost. Compared to a single drive, RAID 1 tends to be faster on reads, slower on writes. This is a good entry-level redundant configuration. However, since an entire drive is a duplicate, the cost per megabyte is high. This is commonly referred to as mirroring.




Space efficiency = 0.5 (50%)
Fault tolerance = 1 disk(s)
Total Disks X Size of DISK = Total Size
Size after implementation = Total Size X Space Efficiency

Suppose we are using 3 disks of 250GB size, after RAID 1 implementation we get the total size out to be:

3 * 250 = 750
750 * 0.5 = 375

Hence we get 375 as final disk size.

RAID 5

RAID 5 stripes data at a block level across several drives, with parity equality distributed among the drives. The parity information allows recovery from the failure of any single drive. Write performance is rather quick, but because parity data must be skipped on each drive during reads, reads are slower. The low ratio of parity to data means low redundancy overhead.





Space efficiency = 0.666666666666667 (66.6666666666667%)
Fault tolerance = 1 disk(s)
Total Disks X Size of DISK = Total Size
Size after implementation = Total Size X Space Efficiency

Suppose we are using 3 disks of 250GB size, after RAID 5 implementation we get the total size out to be:

3 * 250 = 750
750 * 0.667 = 500

Hence we get 500 as final disk size.

RAID 6

RAID 6 is an upgrade from RAID 5: data is striped at a block level across several drives with double parity distributed among the drives. As in RAID 5, parity information allows recovery from the failure of any single drive. The double parity gives RAID 6 additional redundancy at the cost of lower write performance (read performance is the same), and redundancy overhead remains low.





Space efficiency = 0.5 (50%)
Fault tolerance = 2 disk(s)
Total Disks X Size of DISK = Total Size
Size after implementation = Total Size X Space Efficiency

Suppose we are using 4 disks of 250GB size, after RAID 6 implementation we get the total size out to be:

4 * 250 = 1000
1000 * 0.5 = 500

Hence we get 500 as final disk size.

RAID 10

RAID 10 is a striped (RAID 0) array whose segments are mirrored (RAID 1). RAID 10 is a popular configuration for environments where high performance and security are required. In terms of performance it is similar to RAID 0+1. However, it has superior fault tolerance and rebuild performance.





Space efficiency = 0.5 (50%)
Fault tolerance = 1 disk (min) to 2 disks (max)
Total Disks X Size of DISK = Total Size
Size after implementation = Total Size X Space Efficiency

Suppose we are using 4 disks of 250GB size, after RAID 10 implementation we get the total size out to be:

4 * 250 = 1000
1000 * 0.5 = 500

Hence we get 500 as final disk size.

RAID 50

RAID 50 combines RAID 5 parity and stripes it as in a RAID 0 configuration. Although high in cost and complexity, performance and fault tolerance are superior to RAID 5.

RAID 60

RAID 60 combines RAID 6 double parity and stripes it as in a RAID 0 configuration. Although high in cost and complexity, performance and fault tolerance are superior to RAID 6.





RAID 01

RAID 0+1 is a mirror (RAID 1) array whose segments are striped (RAID 0) arrays. This configuration combines the security of RAID 1 with an extra performance boost from the RAID 0 striping.


Tuesday 23 September 2014

What to do when you have deleted /etc/fstab file from Linux and rebooted the server?

/etc/fstab is one of the most critical file in the Linux system. It contains the filesystem tables and information of the mount points. We need this file information at the time of reboot. Partition information is required to mount / partition and other partition table. When this is deleted or missed, our PC will definitely ran into some problem. But we can rather save our PC, if we miss our file.

This blog will tell how to recover from a missing /etc/fstab file.

When you rebooted the server without the /etc/fstab file, we will get some error on the screen on the booting time. But it is important to debug if it is really a fstab error or something else.
For this we have to know the boot process also. As we know that the fstab information is required by the kernel at the time of the reboot. So you will see BIOS is loading, MBR is loading and then GrUB loaded successfully but after that at the time of kernel starting the services, we will ran to an error screen.



Read the error carefully, as the error itself gives the detailed description of the problem. It will say, filesystem table is missing or /etc/fstab not found etc.

At this stage, you know that the error is related to the filesystem table.

This problem can be resolved by a number of ways. We will discuss them one by one.

1. By creating the required file in single user mode.
2. By booting the system in rescue mode and then creating the file.


1. Using single user mode.

1. Reboot the server.
2. Press e when kernel listing is there on the grub screen, add 1 to the kernel parameters to boot the system in single user mode.
3. When the server boots into single user mode, check the UUID using command, blkid as shown in the snapshot.
# blkid
4. Now try to create the file /etc/fstab using vi. But you will not be able to create the file as the server is mounted in the read only mode. So remount the server in rw mode again.
# mount -o loop rw,remount /dev/sda2 /
5. Now create file /etc/fstab and write all the mount points required to boot the server.






6. Unmount the / again and reboot the server. The server will surely boot and come up.


2. Using Rescue Mode.

1. Reboot the server using a rescue CD or bootable media.
2. When menu appears for installation from CD, select from menu the rescue the previous installation.
3. Follow the instruction and it will finally give a command prompt.
4. Try mounting the / partition on /tmp or /mnt of your server in read write mode using above method.
5. After that follow the same above procedure and create /etc/fstab file on the server. If we don't want to go in all this mess, you can also copy the file /proc/mounts to /etc/fstab.
# cat /proc/mounts > /etc/fstab
6. This option will let you reboot the server next time, but your filesystem may or may not be mounted at this time. So after rebooting the server you can mannually add the partition information and then reboot the server.