What is shell Script?
A shell script is a text file that contains a sequence of commands for a UNIX-based operating system.
ex: server_details.sh
[root@ip-172-31-42-179 ShellScript]# cat server_details.sh
#!/bin/bash
hostname
uname -a
uptime
cat /etc/os-release
Output:
[root@ip-172-31-42-179 ShellScript]# ./server_details.sh
ip-172-31-42-179.us-east-2.compute.internal
Linux ip-172-31-42-179.us-east-2.compute.internal 5.10.118-111.515.amzn2.x86_64 #1 SMP Wed May 25 22:12:19 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
15:56:41 up 12:57, 1 user, load average: 0.00, 0.00, 0.00
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
-----------------------------------------------------------------------------------------------------------------------------
Write Your First Shell Script.
#touch HelloWorld.sh
#vi HelloWorld.sh
#!/bin/bash
echo "Hello World"
Execute Shell Script
#bash shellscript_name.sh
#./shellscript_name.sh
What is shebang?
The shebang is the combination of the # (pound key) and ! (exclamation mark).
This character combination has a special meaning when it is used in the very first line of the script.
It is used to specify the interpreter with which the given script will be run by default.
So, if the first line of a script is:
#!/bin/bash
echo command: To print output
echo "Hello World"
comments in shell script
#single line comment
: '
this is multi
line comment'
what is comment?
Instruction which present in file but its not performing any task.. its gives us information about code.
Loops:
1)While Loop:
[root@ip-172-31-42-179 ShellScript]# cat while_example.sh
#!/bin/bash
i=1
while [ $i -lt 11 ]
do
echo $i
i=`expr $i + 1`
done
[root@ip-172-31-42-179 ShellScript]# ./while_example.sh
1
2
3
4
5
6
7
8
9
10
2) For Loop:
[root@ip-172-31-42-179 ShellScript]# cat for_loop.sh
#!/bin/bash
for (( number=1; number<10; number++ ))
do
echo $number
done
OUTPUT:
[root@ip-172-31-42-179 ShellScript]# ./for_loop.sh
1
2
3
4
5
6
7
8
9
For Vs While Loop?
For loop
The initialization, condition checking, and the iteration statements are written at the beginning of the loop.
It is used only when the number of iterations is known beforehand.
If the condition is not mentioned in the 'for' loop, then the loop iterates infinite number of times.
The initialization is done only once, and it is never repeated.
The iteration statement is written at the beginning.
Hence, it executes once all statements in loop have been executed
While condition
The initialization and the condition checking is done at the beginning of the loop.
It is used only when the number of iterations isn’t known.
If the condition is not mentioned in the 'while' loop, it results in a compilation error.
If the initialization is done when the condition is being checked, then initialization occurs every time the loop is iterated through.
The iteration statement can be written within any point inside the loop.
User Input:
In Shell script if want to take any input from user then we are using read command
[root@ip-172-31-42-179 ShellScript]# cat user_input.sh
#!/bin/bash
echo "Enter Username"
read user
echo "Welcome $user to Bash Shell Scripting Program"
[root@ip-172-31-42-179 ShellScript]# ./user_input.sh
Enter Username
kishor
Welcome kishor to Bash Shell Scripting Program
[root@ip-172-31-42-179 ShellScript]#
If statement:
You can use if condition with single or multiple conditions. Starting and ending block of this statement is define by ‘if’ and ‘fi’.
[root@ip-172-31-42-179 ShellScript]# cat if_statement.sh
#!/bin/bash
i=30
if [ $i -gt 20 ];
then
echo "$i is greator than 20"
fi
[root@ip-172-31-42-179 ShellScript]# ./if_statement.sh
30 is greator than 20
if else statement:
#!/bin/bash
n=2
if [ $n -eq 1 ]
then
echo "This is true"
else
echo "N is no longer 1"
fi
If statement with AND , OR logic
if with AND
#!/bin/bash
echo "Enter username"
read username
echo "Enter password"
read password
if [[ ( $username == "admin" && $password == "secret" ) ]]; then
echo "valid user"
else
echo "invalid user"
fi
if with OR
#!/bin/bash
echo "Enter any number"
read n
if [[ ( $n -eq 15 || $n -eq 45 ) ]]
then
echo "You won the game"
else
echo "You lost the game"
fi
else if statement:
#!/bin/bash
echo "Enter your lucky number"
read n
if [ $n -eq 101 ];
then
echo "You got 1st prize"
elif [ $n -eq 510 ];
then
echo "You got 2nd prize"
elif [ $n -eq 999 ];
then
echo "You got 3rd prize"
else
echo "Sorry, try for the next time"
fi
Case Statement
Case statement is used as the alternative of if-elseif-else statement. The starting and ending block of this statement is defined by ‘case’ and ‘esac’.
syntax:
case value in
pattern1)
statement1
...
statementN
;;
pattern2)
statement1
...
statementN
;;
*)
statement1
...
statementN
;;
esacEx:
#!/bin/bash
echo "Enter your lucky number"
read n
case $n in
101)
echo echo "You got 1st prize" ;;
510)
echo "You got 2nd prize" ;;
999)
echo "You got 3rd prize" ;;
*)
echo "Sorry, try for the next time" ;;
esac
Get Arguments from Command Line
Bash script can read input from command line argument like other programming language. For example, $1 and $2 variable are used to read first and second command line arguments.
#!/bin/bash
echo "Total arguments : $#"
echo "1st Argument = $1"
echo "2nd argument = $2"
Combine two strings in a variable:
#!/bin/bash
string1="Linux"
string2="Hint"
echo "$string1$string2"
string3=$string1+$string2
string3+=" is a good tutorial blog site"
echo $string3
Get Substring of Strings
Like other programming language, bash has no built-in function to cut value from any string data. But you can do the task of substring in another way in bash that is shown in the following script.
#!/bin/bash
Str="Learn Linux from LinuxHint"
subStr=${Str:6:5}
echo $subStr
Add 2 numbers into a variable
#!/bin/bash
echo "Enter first number"
read x
echo "Enter second number"
read y
(( sum=x+y ))
echo "The result of addition=$sum"
Create a Function
#!/bin/bash
function myfunction()
{
echo 'I like bash programming'
}
myfunction
Use Function Parameters
#!/bin/bash
Rectangle_Area() {
area=$(($1 * $2))
echo "Area is : $area"
}
Rectangle_Area 10 20
Make directory
#!/bin/bash
echo "Enter directory name"
read newdir
`mkdir $newdir`
Make directory by checking existence
#!/bin/bash
echo "Enter directory name"
read ndir
if [ -d "$ndir" ]
then
echo "Directory exist"
else
`mkdir $ndir`
echo "Directory created"
fi
Pass Return Value from Script
#!/bin/bash
function return_val() {
str="Hello, $name"
echo $str
}
echo "Enter your name"
read name
val=$(return_val)
echo "Return value of the function is $val"
Read a file:
#!/bin/bash
file='book.txt'
while read line; do
echo $line
done < $file
#!/bin/bash
read -p "Enter file name : " filename
while read -n1 character
do
echo $character
done < $filename
Delete a File:
#!/bin/bash
echo "Enter filename to remove"
read file_name
rm -i $file_name
Append to file:
#!/bin/bash
echo "Before appending the file"
cat file_name.txt
echo "Learning Laravel 5">> file_name.txt
echo "After appending the file"
cat file_name.txt
Test if File Exists
#!/bin/bash
filename=$1
if [ -f "$filename" ]; then
echo "File exists"
else
echo "File does not exist"
fi
Send Email Example
#!/bin/bash
Recipient=”kishoraswar@gmail.com”
Subject=”MailDemo”
Message=”Welcome to our site”
`mail -s $Subject $Recipient <<< $Message
Get Parse Current Date:
#!/bin/bash
Year=`date +%Y`
Month=`date +%m`
Day=`date +%d`
Hour=`date +%H`
Minute=`date +%M`
Second=`date +%S`
echo `date`
echo "Current Date is: $Day-$Month-$Year"
echo "Current Time is: $Hour:$Minute:$Second"
Wait Command:
wait is a built-in command of Linux that waits for completing any running process. wait command is used with a particular process id or job id. If no process id or job id is given with wait command then it will wait for all current child processes to complete and returns exit status.
#!/bin/bash
echo "Wait command" &
process_id=$!
wait $process_id
echo "Exited with status $?"
Sleep Command
When you want to pause the execution of any command for specific period of time then you can use sleep command. You can set the delay amount by seconds (s), minutes (m), hours (h) and days (d).
#!/bin/bash
echo “Wait for 5 seconds”
sleep 5
echo “Completed”
grep:
The grep filter searches a file for a particular pattern of characters, and displays all lines that contain that pattern. The pattern that is searched in the file is referred to as the regular expression (grep stands for global search for regular expression and print out).
Options Description
-c : This prints only a count of the lines that match a pattern
-h : Display the matched lines, but do not display the filenames.
-i : Ignores, case for matching
-l : Displays list of a filenames only.
-n : Display the matched lines and their line numbers.
-v : This prints out all the lines that do not matches the pattern
-e exp : Specifies expression with this option. Can use multiple times.
-f file : Takes patterns from file, one per line.
-E : Treats pattern as an extended regular expression (ERE)
-w : Match whole word
-o : Print only the matched parts of a matching line,
with each such part on a separate output line.
-A n : Prints searched line and nlines after the result.
-B n : Prints searched line and n line before the result.
-C n : Prints searched line and n lines after before the result.
root@ip-172-31-42-179 ~]# grep -C2 "localhost" /var/log/messages
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Running in initial RAM disk.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: No hostname configured.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Set hostname to <localhost>.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Initializing machine ID from random generator.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Reached target Local File Systems.
[root@ip-172-31-42-179 ~]# grep -A2 "localhost" /var/log/messages
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Set hostname to <localhost>.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Initializing machine ID from random generator.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Reached target Local File Systems.
[root@ip-172-31-42-179 ~]# grep -B2 "localhost" /var/log/messages
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Running in initial RAM disk.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: No hostname configured.
Jun 26 02:58:49 ip-172-31-42-179 systemd[1]: Set hostname to <localhost>.
[root@ip-172-31-42-179 ~]#
SED VS AWK
Use sed to automate tasks you would do otherwise in a text editor manually. That's why it is called stream editor. (You can use the same commands to edit text in vim). Use awk if you want to analyze text, meaning counting fields, calculate totals, extract and reorganize structures etc.
- Sed - when you need to do simple text transforms on files.
- Awk - when you only need simple formatting and summarisation or transformation of data
awk :Aho Weinberger Kernighan
AWK is an interpreted programming language. It is very powerful and specially designed for text processing. Its name is derived from the family names of its authors − Alfred Aho, Peter Weinberger, and Brian Kernighan.
Uses:
- Text processing,
- Producing formatted text reports,
- Performing arithmetic operations,
- Performing string operations, and many more.
Syntax:
awk option pattern action source_file
#ls -l |awk '{print $1}'
#ls -l|awk '{print $1,$4,$6}'
#ls -l|awk '{awk $0}'
List Directories
#ls -l|awk /^d/ {print $0}
List only Files:
#ls -l|awk /^-/ {print $0}
#ls -l /* nl --> We can see multiple files and Directories
Link Files:
#ls -l /* |awk '/^l/ {print $0}'
Match with specific Field...
ls -l /* |awk '$2 ~ /^l/ {print $0}
ls -l /* |awk '$2 ~ /^2/ {print $0} '
ls -l /* |awk '$2 ~/60/ {print $0}' -- In 2nd number field it will check for 60
ls -l /* |awk '$2 ~/1/ {print $0}'
ls -l /* |awk '$2 ~/^1$/ {print $0}' --> Start and end with 1
ls -l /* |awk '$4 ~ /root/ {print $0}'
ls -l /* |awk '$6 ~ /Nov/ {print $0}'
ls -l /* |awk '$9 ~ /mail/ {print $0}'
ls -l /*|awk '$9 ~ /^mail/ {print $0}'
Print Files which created in year 2021
ls -l /* |awk
List files which size is 1kb
ls -l /*|awk '$5 > 1024 {print $0}'
ls -l /* |awk '$5 == 1024 {print $0}'
ls -l /*|awk '$5 == 6 && $2 == 2 {print $0}'
ls -l /* |awk '$5 == 6 || $2 == 2{print $0}'
ls -l /* |awk '{print NR,$0}' --> Record Number
ls -l /*|awk '{print NR,$0,NF}' -->Number of record and Number of Field
ls -l /*|awk '{print NF,$0,NR}' --> It will display NF first and Numbe rof record last
ls -l /* |awk 'NR == 750, NR == 800 {print NR, $0, NF}'
ls -l /* |awk '$1 ~ /rwxr-xr-x/ {print $0}' -->Display Files with Perticular Permission
ls -l /*|awk 'BEGIN {count=0;} {if ($6 == "Nov") count+=1} END {print count}' ==> Diplay total number of Nov word
Set Header and Footer:
ls -l /* | awk 'BEGIN {print "Welcome To Shell Scripting"} NR == 700 , NR ==750 {print NR, $0, NF} END {Good Bye Shell Script}'
ls -l /* | awk 'BEGIN {print "**************"} NR == 700 , NR ==750 {print NR, $0, NF} END {print "Good Bye Shell Script"}'
#df -h
Create File With Below Data:
vi awkdemo
ename depart desgination Salary
kishor IT DevOps 10000
satish IT Developer 9000
ganesh admin avp 11000
john hr manager 6000
#awk '{print $1}' awkdemo
#awk '{print $1,$2,$4}' awkdemo
#awk '{print $1,$2,$4}' OFS=' ' awkdemo
#awk '{print $1,$2,$4}' OFS=' \t ' awkdemo
#awk '{print $1,$2,$4}' OFS='-' awkdemo
#awk '{print $1,$2,$4}' OFS='-' awkdemo|column -t
#awk '/kishor/ {print $0}' awkdemo
#awk '$2 ~ /^kishor/ {print $0}' awkdemo
#awk '$1 ~ /^ganesh$/ {print $0}' awkdemo
#awk '$4 == 9000' awkdemo
#awk '$4 > 6000' awkdemo
#awk '$4 < 5000' awkdemo
#awk '{print NR,$0}' awkdemo
#awk '{print NR,$0,NF}' awkdemo
===========
awk '{print $1}' /etc/passwd
Use delimeter in awk
awk -F':' '{print $1,$3}' /etc/passwd
awk -F':' '{print $1,$3}' /etc/passwd|column
awk -F':' '{print $1,$3}' OFS='-' /etc/passwd
work on logs files:
#awk "{print $1}' /var/log/messages
#awk "{print $1,$2,$3,$4}' /var/log/messages
Find localhost word how many times repeated in /var/log/message file...
#awk 'BEGIN {count=0;} {if ($4 == "localhost") count +=1} END {print count}' /var/log/messages
Shell Scripting Question:
Write a command delete all line except first line?
Write a command to print zero byte size files?
Write a command remove all empty lines?
Write a command to print the line number before each line?
How add a first record and last record to the current file in linux?
How to get only zero byte files which are present in the directory?
How to display even number of records into one file and odd number of records into another file?
How to print only blank line of file?
Write a command to print the second and third line of a file without using nr?
Write a command to find the total number of lines in a file without using nr?
Write a command to print all line except first line?
Write a command to rename the files in a directory with "_new" as postfix?
In the text file, some lines are delimited by colon and some are delimited by space. Write a command to print the third field of each line.?
1) Display the 10th line of the file?
[root@ip-172-31-42-179 log]# tail -n 10 /var/log/messages |tail -n 1
Jul 2 18:05:42 ip-172-31-42-179 dhclient[2884]: XMT: Solicit on eth0, interval 124840ms.
2)How to remove header from file?
Reference Link:
ex: ls -l |cut -d' ' -f1 --Work perfect where actual row and column
