This document provides an introduction to useful Linux commands. It begins with a brief history of UNIX and Linux and describes the Linux directory structure. It then discusses commands for navigating directories, searching for files, manipulating text, basic networking, system information, disk usage monitoring, and more. The goal is to compile some essential Linux commands for newcomers to learn and get comfortable with.
1. A Quick Introduction toA Quick Introduction to
LinuxLinux
(A compilation of some useful Linux commands)(A compilation of some useful Linux commands)
Tusharadri Sarkar
June 28, 2010
IBM
June 28, 2010 1Tusharadri Sarkar
2. A Brief History of UNIXA Brief History of UNIX
Bell Labs : 1970
The C programming language
Support for simultaneous users
Minimum effort for platform compatibility
Simple, elegant, and easy* to use
* Of course, we are talking about the CLI folks !!
June 28, 2010 2Tusharadri Sarkar
3. A Brief History of LinuxA Brief History of Linux
The GNU project - Richard Stallman
1984
MINIX - Professor Andrew Tanenbaum
Linus Torvalds - inspired by MINIX
April 1991
Linux 0.01 released
September 1991
June 28, 2010 3Tusharadri Sarkar
5. Main DirectoriesMain Directories
/ The parent (root) directory
bin Compiled executable files (ls, grep)
boot Linux Kernel images and configs
etc System configuration files
dev Device files for all system H/Ws
home Stores home directories of users
lib System library files
June 28, 2010 5Tusharadri Sarkar
6. Main DirectoriesMain Directories
usr
Stores files and executables that all
users can access
var Stores files whose size might vary
tmp Stores temporary files
sbin
Stores executables for system
administrator, user can also access
proc
Provides Kernel windows to view
system status (memory, IO etc.)
June 28, 2010 6Tusharadri Sarkar
7. Users and GroupsUsers and Groups
Each “user” is a member of at least one
“group” in Linux
Each user has a home directory
What are there in /home and /usr/users?
UNIX permissions: both users and
groups
> ls –l : see all permissions
June 28, 2010 7Tusharadri Sarkar
Note: From now on, the > at the beginning of each command
represents the command prompt. The command starts after that
8. Notes on File PermissionNotes on File Permission
In UNIX/Linux system everything is a file
File has 3 permissions :
read (r), write (w) and execute (x)
File has 3 sets of permissions:
user – group – others
For each, permissions have weights:
r = 4 w = 2 x = 1
So, what does permissions 755 means ?
June 28, 2010 8Tusharadri Sarkar
9. Alias: To make life easy...Alias: To make life easy...
Pipes |: to chain multiple commands
Alias: Write commands once and
execute them whenever you need
An example:
> alias jtest=`find <JAVA_TEST_PATH> -name *.java |
grep 'Test' | sed 's/Test/TestCase/g'`
What does it do?
The .bashrc file: Your own set of aliases
June 28, 2010 9Tusharadri Sarkar
10. Directory NavigationDirectory Navigation
June 28, 2010 10Tusharadri Sarkar
pwd Present Working Directory
cd Login (home) dir. of the user
cd – User’s last (previous) dir.
cd ~/<path> Here, ~ stands for ‘home’
cd /<path> Using absolute path
cd .. One dir. level up
'.' (dot) and '..' (dot dot) indicate current
and parent directories, just like Windows
11. Creating & Removing DirectoriesCreating & Removing Directories
Check your permissions carefully:
write permission – to create and remove
directories
Check your permissions again:
not only user but group permissions
also matters
> mkdir <dname> : Create directory
> rmdir <dname> : Remove directory
June 28, 2010 11Tusharadri Sarkar
12. File Searching (ls)File Searching (ls)
ls: Stands for “List”
> ls –a : Expose the hidden files
> ls –lrt : Very useful when searching by
last modification dates
Use regular expressions: ls –lrt lib*.so
Create alias of your own listing
commands to make things easy for you
again:
Example: > alias l=`ls –l --color=auto`
June 28, 2010 12Tusharadri Sarkar
13. File Searching (find)File Searching (find)
find: A very powerful command for file
searching operations
> find <path> -name *.txt
Typical usage with regular expressions
> find <path> -type d
when you are after the directories!!
> find <path> -type f
when you are after the regular files!!
June 28, 2010 13Tusharadri Sarkar
14. File Searching (find)File Searching (find)
find is more powerful when chained with
other commands like xgrep or grep
For example:
> find <path> -name "*.*" -exec grep
"<pattern>" '{}' ; -print 2>/dev/null
What does the above command do?
June 28, 2010 14Tusharadri Sarkar
15. File Searching (grep)File Searching (grep)
grep: When you are interested in the
contents of the files
Typical usage for search operations:
> grep '<pattern>' <files>
When the pattern position also matters:
> grep –a N '<pattern>' <file>
> grep –b N '<pattern>' <file>
> grep –C N '<pattern>' <file>
June 28, 2010 15Tusharadri Sarkar
16. File Searching (grep)File Searching (grep)
When you are looking for the line
numbers: > grep –c '<pattern>' <file>
Remember: C and c are different options
for 'grep'
The --color option: Highlight <pattern>
Enhance power of 'grep' : 'find' and 'sed'
Extremely useful if you are interested in
Shell scripting
June 28, 2010 16Tusharadri Sarkar
17. File Operations (Copy Move Delete)File Operations (Copy Move Delete)
cp – mv – rm: Straight forward and simple
They have similar syntax and options:
> cp <source> <destination>
> mv <source> <destination>
> rm <filename>
-i & -r : Do it 'interactively' or 'recursively'
Be careful while using rm and move. Use
> alias rm=`rm –i`; alias mv=`mv –i`
June 28, 2010 17Tusharadri Sarkar
18. Archives and CompressionsArchives and Compressions
Need to create archive and compress
files? It’s easy with tar and gzip:
> tar –cf demo.tar <file1> <file2> …
> tar –xvf demo.tar - like verbose?
Zip and Unzip a file:
gzip & gunzip :: bzip2 & bunzip2
Combine archiving and zipping together:
> tar –czvf myfiles.tar.gz *.sh
> tar –xzvf myfiles.tar.gz
June 28, 2010 18Tusharadri Sarkar
19. Text Manipulation (sed)Text Manipulation (sed)
sed: Stands for “Stream Editor”
Powerful but also complex
When it comes to text manipulation:
Combine powers of 'sed' and 'grep'
General format of 'sed' for substitution:
> sed 's/<pat1>/<pat2>/<switch>' file1 > file2
How does the <switch> influence the
output?
June 28, 2010 19Tusharadri Sarkar
20. Text Manipulation (sed)Text Manipulation (sed)
9 typical ‘sed’ usage examples
Pattern substitutions:
1. > sed 's/string1/string2/g' – very basic
2. > sed 's/(.*)1/12/g' – a little tricky !
3. > sed '/ *#/d; /^ *$/d' – trickier !!
4. > sed ':a; /$/N; s/n//; ta' – too cryptic !!!
What are all those '' required for?
June 28, 2010 20Tusharadri Sarkar
21. Text Manipulation (sed)Text Manipulation (sed)
5. > sed 's/[ t]*$//'
If you don't like spaces...
6. > sed 's/([`"$])/1/g'
Use escape judiciously with meta-characters
7. > sed 10 | sed 's/^/ /; s/ *(.{7,})/1/'
Handy for number alignments
8. > sed –n '1000{p;q}'
9. > sed –n '10,20p;20q'
Printing with sed
June 28, 2010 21Tusharadri Sarkar
22. Text Manipulation (tr)Text Manipulation (tr)
tr: Stands for “Translate”
> echo 'Test' | tr '[:lower:]' '[:upper:]'
Case conversion
> tr –dc '[:print:]' < /dev/urandom
Handle non printable characters
> tr –s '[:blank:]' 't' < /proc/diskstats | cut –f4
Combine with 'cut' to manipulate fields
June 28, 2010 22Tusharadri Sarkar
23. Set Operations (sort)Set Operations (sort)
sort: is helpful with text files
uniq helps refining you sorting
> sort file1 file2 | uniq
> sort file1 file2 | uniq –d
> sort file1 file1 file2 | uniq –u
> sort file1 file2 | uniq –u
What are the differences in output?
June 28, 2010 23Tusharadri Sarkar
24. Set Operation (join)Set Operation (join)
join: works best on previously sorted
files:
> join –t'0' –a1 –a2 file1 file2
> join –t'0' file1 file2
> join –t'0' –v2 file1 file2
> join –t'0' –v1 –v2 file1 file2
On unsorted files join works just like
concatenating files
June 28, 2010 24Tusharadri Sarkar
25. Basic NetworkingBasic Networking
ifconfig: An equivalent of 'ipconfig' on
DOS, but it does much more
> ifconfig
> ifconfig –< interface>
iwconfig: 'w' for wireless network
hostname: When you want to know
about the host/ the system you are on
> hostname
> hostname –i
June 28, 2010 25Tusharadri Sarkar
27. Basic System InformationBasic System Information
> uname –a
Which OS, Kernel version, hardware and
system architecture am I running on? For
specific information use the following:
-s, -o, -r, -v, -m, or -n
Know who all are working with you:
> w
> who
> whoami
> who am i (Yes, the output is different!!)
June 28, 2010 27Tusharadri Sarkar
28. Basic System InformationBasic System Information
> head –n1 /etc/issue
What distribution version and release
am I using?
> cat /proc/partitions
What about my partitions?
> grep 'MemTotal' /proc/meminfo
Am I running out of RAM?
> mount | column –t
What file systems is currently in use?
June 28, 2010 28Tusharadri Sarkar
29. Disk Space UsageDisk Space Usage
> ls –lSr
'ls' command coming to rescue again!!
du – Stands for “Disc Usage”
> du –s * | sort –k1,1rn | head
> du –hk .
> du –h <file>
df – Stands for “Disc Free”
> df –h
> df –i
June 28, 2010 29Tusharadri Sarkar
30. Basic Monitoring & DebuggingBasic Monitoring & Debugging
If you want to monitor a file continuously:
> tail –f <filename> You will need Ctrl+C too!!
When you want to deal with processes:
> ps –ef <processname>
> ps –p pid1, pid2,... Be sure of PID & PPID !!
> ps –e –o pid, args --forest Hierarchical list
Some miscellaneous commands:
> free –m
> last reboot
June 28, 2010 30Tusharadri Sarkar
31. Basic Monitoring & DebuggingBasic Monitoring & Debugging
3 Advance commands (for sysadmin)
Every CPU cycle is costly...
> ps –e –o pcpu, cpu, nice, state, cputime,
args --sort pcpu | sed '/^ 0.0 /d'
And every byte of memory too...
> ps –e –orss=, args= | sort –b –k1, 1n | pr –
TW$COLUMNS
Locked threads can cause troubles...
> ps –C <process> -L –o pid, tid, pcpu, state
June 28, 2010 31Tusharadri Sarkar
32. MiscellaneousMiscellaneous
Get familiar with the time keepers:
cal: “Calendar”, no one can live without it...
> cal (current month’s calendar)
> cal <mm> <yyyy> (for specific month)
date: Of course you want to know this...
> date (Date with IST)
> date “%d/%m%y %H:%M:%S”
(A more familiar date format)
June 28, 2010 32Tusharadri Sarkar
33. MiscellaneousMiscellaneous
Where are your commands resting?
> which <command>
> whereis <command>
Read a file without opening it:
> cat <filename>
Finally, the grand “Manual” of Linux:
> man <item>
The <item> list is really large!! Try it...
June 28, 2010 33Tusharadri Sarkar
Created at Bell Labs in 1970 Written in the C programming language which was developed at the same time Supports large numbers of simultaneous users Runs with few alterations on many hardware platforms Simple, elegant, and easy to use (at least compared to its predecessors)
1984 - Richard Stallman started the GNU project to create a free operating system Professor Andrew Tanenbaum wrote the operating system MINIX from scratch to teach his students the inner workings of a real operating system April 1991 - Linus Torvalds starts working on a new operating system inspired by MINIX September 1991 - Linux version 0.01 released.
Each user has a home directory, which is where personal files and preferences are stored. These are typically in /home or in /usr/users Use the 'ls -l' or long listing command to see all the permissions assigned to a file. It is similar to the 'dir' command in DOS.
Each permission is assigned a different weight r=4, w=2 & x=1 If a file is assigned permissions 755 it means the current user has all permissions while group and others have only read and execute permissions
For example: alias jtest=`find . -name *.java | grep 'Test' | sed 's/Test//'` The entire command will execute when you run 'jtest’ To create permanent aliases add them in the .bashrc file in you home or login directory.
pwd: shows the current directory path cd: go to the home directory of the user cd - : go to the previous directory cd ~/<path> : go to any directory <path> under the home directory cd /<path> : go to any directory <path> under root (/). This is the absolute path name cd ../ : go one directory up
mkdir <dname> : Creates directory <dname> rmdir <dname> : Removes directory <dname> Remember: In Unix everything is 'file'
ls : Lists directory contents (The command that you will probably use the most) ls -a : Lists directory contents including hidden files ls -lrt : Lists directory contents by date of modification in reverse order Regular expressions blends efficiently with ls: For example: ls -lrt lib*.so --color Option lists the filenames with different color according to filetypes
find <path> -name *.txt : lists all the .txt files under the directory <path> find <path> -type d : lists all the sub-directories under <path> find <path> -type f: lists all the regular files under <path> You can get the same result of the above commands by using 'ls' with some tricks, but 'find' is far more powerful and easy to use
find <path> -name "*.*" -exec grep "<pattern>" '{}' \; -print 2>/dev/null The above example will look for every instance of <pattern> even inside file contents under <path> recursively
grep '<pattern>' *.* : Searches for <pattern> inside all regulare files in current directory grep -a/-b N '<pattern>' *.* : Same as above but prints N lines including the matching lines either after or before it grep -C N '<pattern>' *.* : Same as above but prints N lines including the matching lines both before and after it
grep -c '<pattern>‘ *.* Searches for <pattern> inside all the regular files in the current directory and prints the file names and the line numbers at which <pattern> is present
cp <source> <destination> : To copy a file <source> to <destination> move <source> <destination> : To move a file <source> to <destination>. This also doubles as the command to rename the file rm <fname> : To remove / delete file <fname>
tar -cf demo.tar <files>: Create an archive demo.tar from <files> tar -xvf demo.tar : Extracts the files from archive demo.tar (-v gives verbose output) gzip & gunzip : To Zip and Unzip a file bzip2 & bunzip2 : Same as above with a different compression algorithm tar -czvf tushar.tar.gz *.sh and tar -xzvf tushar.tar.gz (-z refers to zipping)
General format of 'sed' for substitution operations: sed 's/<pat1>/<pat2>/<switch>' file1 > file2 It means, in file file1 replace occurrences of <pat1> with <pat2> depending on <switch> and put the outcome in file2 <switch> = ‘g’ or ‘c’ which stands for ‘global’ and ‘conditional’ substitution
sed 's/string1/string2/g' : Replace all occurrence of string1 with string2 sed 's/\(.*\)1/\12/g' : Modify anystring1 to anystring2. '\' is used to escape special chars sed '/ *#/d; /^ *$/d' : Remove comments and blank lines. ';' joins two sed operations sed ':a; /\\$/N; s/\\\n//; ta' : Concatenate lines with trailing \
sed 's/[ \t]*$//' : Remove trailing spaces from lines sed 's/\([`"$\]\)/\\\1/g' : Escape shell meta characters active within double quotes seq 10 | sed 's/^/ /; s/ *\(.\{7,\}\)/\1/' : Right align numbers (Not applicable for all patterns) sed -n '1000{p;q}' : Print 1000th line sed -n '10,20p;20q' : Print lines 10 to 20
echo 'Test' | tr '[:lower:]' '[:upper:]' : Convert uppercase letters to lower case letters tr -dc '[:print:]' < /dev/urandom : Filter non printable characters tr -s '[:blank:]' '\t' < /proc/diskstats | cut -f4 : cut fields separated by blanks
sort file1 file2 | uniq : Shows union of unsorted files sort file1 file2 | uniq -d : Shows intersection of unsorted files sort file1 file1 file2 | uniq -u : Shows difference of unsorted files sort file1 file2 | uniq -u : Shows symmetric Difference of unsorted files
join -t'\0' -a1 -a2 file1 file2 : Shows union of sorted files join -t'\0' file1 file2 : Shows intersection of sorted files join -t'\0' -v2 file1 file2 : Shows the difference of sorted files join -t'\0' -v1 -v2 file1 file2 : Shows the symmetric Difference of sorted files
ifconfig : Lists details of all the available network interfaces of your system ifconfig -<IFACE> : Shows details about interface <IFACE> iwconfig : Same as 'ifconfig' but lists only the wireless network interfaces hostname : Shows the system host name hostname -i : Shows host IP address
netstat -r : Lists the routing table of the kernel netstat -i : List interface details netstat -l: Lists all the binded listening ports netstat -tup : Lists all active connections route add : To add a new route to the table route delete : To delete a route from the table
uname -a : Shows kernel version and the system architecture information To get specific information use following options: -r (release) -n (distribution) -m (Hardware ) -v (version) -s (OS name) -o (OS category) who : Lists all the users currently logged in into the system whoami : Gives the current user name who am i : Lists current user login info and time
head -n1 /etc/issue : Shows the name and the version of distribution cat /proc/partitions : Lists all the partitions registered on the system grep 'MemTotal' /proc/meminfo : Shows the total RAM registered to the system Mount | column -t : Lists all the mounted file systems on the system (Aligns the output)
ls -lSr : Show files by size, biggest last du -s * | sort -k1,1rn | head : Show top disk users in current directory du -h : Shows the file sizes in easy to interpret disk usage (-h stands for Human Readable) df -h : Show free space on mounted filesystems df -i : Show free inodes on mounted filesystems
tail -f <fname> : Continuously monitors file <fname> until terminated by Ctrl+C ps -ef : Lists all the processes in the system ps -p id1,id2,... : Lists information for particular process IDs id1,id2,… ps -e -o pid,args --forest : List processes and displays them arranged in a hierarchy free -m : Shows amount of free memory in MB last reboot : Show system reboot history
ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d' : Lists processes by % cpu usage ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS : Lists processes by mem (KB) usage. See also ps_mem.py ps -C <process> -L -o pid,tid,pcpu,state : Lists all threads for a particular <process>