SlideShare a Scribd company logo
1 of 29
grep | sed | awk | xargs | etc Practical *nix utilities(for text processing)
whoami
/ awk cat grep tac sed echo ls du test mv split tail dir join wc head vim tr sort sum cut expr uniq paste kill tee
/ Log Mega App Files DB
pwd | ls | find | tee GNU  Coreutils http://www.gnu.org/software/coreutils/ The takeaway command:man > info coreutils
pwd | ls | find | tee List of files: ls –l ls–1 ls –latr find . –name *.txt
pwd | ls | find | tee Seek for a string in a file: grep“cat” file.txt grep –v “dog” file.txt grep –i “PaTtErN” file.txt egrep“cat|dog” file.txt zgrep“cat” file.txt.gz
for / xargs Do something with each file: for file in `find . –name *tmp`  do rm$file  done find . –name *tmp| xargsrm
pwd | ls | find | tee find + grep find . -name '*txt' -exec grep-l aaa{}  find . -name '*txt' | xargsgrep-l aaa
pwd | ls | find | tee cat grep tac echo ls du test mv split tail dir join wc head tr sort sum cut expr uniq paste kill tee
paste
join
sort | uniq
wc
cut
csplit
awk sed
sed sfor substitution sed‘s/cat/dog/’  # cat -> dog sed ‘s/ab//’ # ab-> ba
sed pfor printing sed –n ‘/dog/p’  # print lines that match ‘dog’ sed–n ‘/start/,/end/p’ # print range
sed dto delete sed ‘/dog/d’ # delete lines that match ‘dog’ sed ‘1,/pattern/d’ # delete range
sed | and –e for invocation sed ‘s/a/A/’ | sed ‘s/b/B/’ # sed –e ‘s/a/A/’ –e ‘s/b/B/’  #
sed { .. } to group the commands sed ‘/pattern/ {        s/p/P/        s/e/E/     }’ #pattern -> PattErn
sed rto read a file sed ‘/include/ r file.txt’ # insert file.txt after include wto write to a file sed‘/pattern/ w file.txt’ # write matched lines to a file
sedtris
awk aaabbb ccc aaabbbzzz awk '/zzz/'  1.txt   grepzzz    1.txt aaabbbzzz
awk awk 'BEGIN{<initializations>} <pattern 1> {<actions>} <pattern 2> {<actions>} ...  END{<final actions>}'
awk awk 'BEGIN{a=0, b=0}  /aaa/{a++}  /bbb/   {b++} END{printf “%d%d”,a,b}'
awk awk '{arr[$2]+=$1}  END  { for (id in arr)  printf "%s%d",id,arr[id]}'
exit @antonarhipov

More Related Content

What's hot

Fast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambdaFast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambdaMila Frerichs
 
2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekingeProf. Wim Van Criekinge
 
Why Python (for Statisticians)
Why Python (for Statisticians)Why Python (for Statisticians)
Why Python (for Statisticians)Matt Harrison
 
Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities Ahmed El-Arabawy
 
Parse, scale to millions
Parse, scale to millionsParse, scale to millions
Parse, scale to millionsFlorent Vilmart
 
Introduction to Perl
Introduction to PerlIntroduction to Perl
Introduction to PerlSway Wang
 
Value protocols and codables
Value protocols and codablesValue protocols and codables
Value protocols and codablesFlorent Vilmart
 
Advanced linux chapter ix-shell script
Advanced linux chapter ix-shell scriptAdvanced linux chapter ix-shell script
Advanced linux chapter ix-shell scriptEliezer Moraes
 
7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)Steven Francia
 
Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話dcubeio
 
Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017Shuji Watanabe
 
JIP Pipeline System Introduction
JIP Pipeline System IntroductionJIP Pipeline System Introduction
JIP Pipeline System Introductionthasso23
 
Logrotate sh
Logrotate shLogrotate sh
Logrotate shBen Pope
 
Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout source{d}
 
File handling in c language
File handling in c languageFile handling in c language
File handling in c languageHarish Gyanani
 
Rotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App EngineRotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App Enginegeehwan
 

What's hot (20)

Cscope and ctags
Cscope and ctagsCscope and ctags
Cscope and ctags
 
Fast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambdaFast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambda
 
2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge
 
Txjs
TxjsTxjs
Txjs
 
Why Python (for Statisticians)
Why Python (for Statisticians)Why Python (for Statisticians)
Why Python (for Statisticians)
 
Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities
 
Parse, scale to millions
Parse, scale to millionsParse, scale to millions
Parse, scale to millions
 
Introduction to Perl
Introduction to PerlIntroduction to Perl
Introduction to Perl
 
Value protocols and codables
Value protocols and codablesValue protocols and codables
Value protocols and codables
 
Advanced linux chapter ix-shell script
Advanced linux chapter ix-shell scriptAdvanced linux chapter ix-shell script
Advanced linux chapter ix-shell script
 
7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)
 
Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話
 
Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017
 
JIP Pipeline System Introduction
JIP Pipeline System IntroductionJIP Pipeline System Introduction
JIP Pipeline System Introduction
 
Logrotate sh
Logrotate shLogrotate sh
Logrotate sh
 
Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout
 
My life as a beekeeper
My life as a beekeeperMy life as a beekeeper
My life as a beekeeper
 
File handling in c language
File handling in c languageFile handling in c language
File handling in c language
 
A Python Crash Course
A Python Crash CourseA Python Crash Course
A Python Crash Course
 
Rotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App EngineRotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App Engine
 

Viewers also liked

Defeating The Network Security Infrastructure V1.0
Defeating The Network Security Infrastructure  V1.0Defeating The Network Security Infrastructure  V1.0
Defeating The Network Security Infrastructure V1.0Philippe Bogaerts
 
Unix Command Line Productivity Tips
Unix Command Line Productivity TipsUnix Command Line Productivity Tips
Unix Command Line Productivity TipsKeith Bennett
 
Sed & awk the dynamic duo
Sed & awk   the dynamic duoSed & awk   the dynamic duo
Sed & awk the dynamic duoJoshua Thijssen
 
Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...Thoughtworks
 
Web Application Security with PHP
Web Application Security with PHPWeb Application Security with PHP
Web Application Security with PHPjikbal
 
Learning sed and awk
Learning sed and awkLearning sed and awk
Learning sed and awkYogesh Sawant
 
Practical Example of grep command in unix
Practical Example of grep command in unixPractical Example of grep command in unix
Practical Example of grep command in unixJavin Paul
 
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAPVirtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAPMichael Coates
 
Secure Shell(ssh)
Secure Shell(ssh)Secure Shell(ssh)
Secure Shell(ssh)Pina Parmar
 
Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014iimjobs and hirist
 
RHCE FINAL Questions and Answers
RHCE FINAL Questions and AnswersRHCE FINAL Questions and Answers
RHCE FINAL Questions and AnswersRadien software
 
Introduction to SSH
Introduction to SSHIntroduction to SSH
Introduction to SSHHemant Shah
 
Linux Systems Performance 2016
Linux Systems Performance 2016Linux Systems Performance 2016
Linux Systems Performance 2016Brendan Gregg
 

Viewers also liked (20)

Defeating The Network Security Infrastructure V1.0
Defeating The Network Security Infrastructure  V1.0Defeating The Network Security Infrastructure  V1.0
Defeating The Network Security Infrastructure V1.0
 
class12_Networking2
class12_Networking2class12_Networking2
class12_Networking2
 
Unix Command Line Productivity Tips
Unix Command Line Productivity TipsUnix Command Line Productivity Tips
Unix Command Line Productivity Tips
 
PHP Secure Programming
PHP Secure ProgrammingPHP Secure Programming
PHP Secure Programming
 
Sed & awk the dynamic duo
Sed & awk   the dynamic duoSed & awk   the dynamic duo
Sed & awk the dynamic duo
 
Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...
 
Secure SHell
Secure SHellSecure SHell
Secure SHell
 
Secure shell protocol
Secure shell protocolSecure shell protocol
Secure shell protocol
 
How to Setup A Pen test Lab and How to Play CTF
How to Setup A Pen test Lab and How to Play CTF How to Setup A Pen test Lab and How to Play CTF
How to Setup A Pen test Lab and How to Play CTF
 
Web Application Security with PHP
Web Application Security with PHPWeb Application Security with PHP
Web Application Security with PHP
 
Learning sed and awk
Learning sed and awkLearning sed and awk
Learning sed and awk
 
Practical Example of grep command in unix
Practical Example of grep command in unixPractical Example of grep command in unix
Practical Example of grep command in unix
 
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAPVirtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
 
SSH - Secure Shell
SSH - Secure ShellSSH - Secure Shell
SSH - Secure Shell
 
Secure Shell(ssh)
Secure Shell(ssh)Secure Shell(ssh)
Secure Shell(ssh)
 
SSH
SSHSSH
SSH
 
Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014
 
RHCE FINAL Questions and Answers
RHCE FINAL Questions and AnswersRHCE FINAL Questions and Answers
RHCE FINAL Questions and Answers
 
Introduction to SSH
Introduction to SSHIntroduction to SSH
Introduction to SSH
 
Linux Systems Performance 2016
Linux Systems Performance 2016Linux Systems Performance 2016
Linux Systems Performance 2016
 

Similar to Practical unix utilities for text processing

Unit 12 finding and processing files
Unit 12 finding and processing filesUnit 12 finding and processing files
Unit 12 finding and processing filesroot_fibo
 
Linux commands
Linux commandsLinux commands
Linux commandsshekhar70
 
Basic shell commands by Jeremy Sanders
Basic shell commands by Jeremy SandersBasic shell commands by Jeremy Sanders
Basic shell commands by Jeremy SandersDevanand Gehlot
 
Linux Basic Commands
Linux Basic CommandsLinux Basic Commands
Linux Basic CommandsHanan Nmr
 
Old school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX WildcardsOld school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX WildcardsUNIDEES Algérie
 
Productivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformaticsProductivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformaticsBITS
 
Unit 8
Unit 8Unit 8
Unit 8siddr
 
Linux advanced privilege escalation
Linux advanced privilege escalationLinux advanced privilege escalation
Linux advanced privilege escalationJameel Nabbo
 
Unix Basics 04sp
Unix Basics 04spUnix Basics 04sp
Unix Basics 04spDr.Ravi
 
101 3.3 perform basic file management
101 3.3 perform basic file management101 3.3 perform basic file management
101 3.3 perform basic file managementAcácio Oliveira
 
3.1.a linux commands reference
3.1.a linux commands reference3.1.a linux commands reference
3.1.a linux commands referenceAcácio Oliveira
 

Similar to Practical unix utilities for text processing (20)

Unit 12 finding and processing files
Unit 12 finding and processing filesUnit 12 finding and processing files
Unit 12 finding and processing files
 
Linux commands
Linux commandsLinux commands
Linux commands
 
unix_commands.ppt
unix_commands.pptunix_commands.ppt
unix_commands.ppt
 
Basic shell commands by Jeremy Sanders
Basic shell commands by Jeremy SandersBasic shell commands by Jeremy Sanders
Basic shell commands by Jeremy Sanders
 
Linux Commands.pptx
Linux Commands.pptxLinux Commands.pptx
Linux Commands.pptx
 
Linux Basic Commands
Linux Basic CommandsLinux Basic Commands
Linux Basic Commands
 
Old school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX WildcardsOld school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX Wildcards
 
Linux Commands
Linux CommandsLinux Commands
Linux Commands
 
Productivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformaticsProductivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformatics
 
Linux
LinuxLinux
Linux
 
Unit 8
Unit 8Unit 8
Unit 8
 
Unix for Librarians
Unix for LibrariansUnix for Librarians
Unix for Librarians
 
Linux midterm quiz
Linux midterm quizLinux midterm quiz
Linux midterm quiz
 
Linux advanced privilege escalation
Linux advanced privilege escalationLinux advanced privilege escalation
Linux advanced privilege escalation
 
Unix Basics 04sp
Unix Basics 04spUnix Basics 04sp
Unix Basics 04sp
 
101 3.3 perform basic file management
101 3.3 perform basic file management101 3.3 perform basic file management
101 3.3 perform basic file management
 
Unix Basics Commands
Unix Basics CommandsUnix Basics Commands
Unix Basics Commands
 
3.1.a linux commands reference
3.1.a linux commands reference3.1.a linux commands reference
3.1.a linux commands reference
 
Red Hat Linux cheat sheet
Red Hat Linux cheat sheetRed Hat Linux cheat sheet
Red Hat Linux cheat sheet
 
Unix 2 en
Unix 2 enUnix 2 en
Unix 2 en
 

More from Anton Arhipov

JavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdfJavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdfAnton Arhipov
 
TechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервьюTechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервьюAnton Arhipov
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCityAnton Arhipov
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCityAnton Arhipov
 
Devoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hourDevoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hourAnton Arhipov
 
GeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hourGeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hourAnton Arhipov
 
Build pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSLBuild pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSLAnton Arhipov
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCityAnton Arhipov
 
JavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainersJavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainersAnton Arhipov
 
GeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainersGeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainersAnton Arhipov
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassleJavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassleAnton Arhipov
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloadingJavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentationJUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentationAnton Arhipov
 
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloadingRiga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
GeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassleGeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassleAnton Arhipov
 
JEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloadingJEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with JavassistJEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with JavassistAnton Arhipov
 

More from Anton Arhipov (20)

JavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdfJavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdf
 
Idiomatic kotlin
Idiomatic kotlinIdiomatic kotlin
Idiomatic kotlin
 
TechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервьюTechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервью
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCity
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCity
 
Devoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hourDevoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hour
 
GeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hourGeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hour
 
Build pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSLBuild pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSL
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCity
 
JavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainersJavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainers
 
GeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainersGeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainers
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
 
JavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassleJavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassle
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
 
JavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloadingJavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloading
 
JUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentationJUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentation
 
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloadingRiga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloading
 
GeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassleGeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassle
 
JEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloadingJEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloading
 
JEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with JavassistJEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with Javassist
 

Recently uploaded

Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 

Recently uploaded (20)

Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 

Practical unix utilities for text processing

  • 1. grep | sed | awk | xargs | etc Practical *nix utilities(for text processing)
  • 3. / awk cat grep tac sed echo ls du test mv split tail dir join wc head vim tr sort sum cut expr uniq paste kill tee
  • 4. / Log Mega App Files DB
  • 5. pwd | ls | find | tee GNU Coreutils http://www.gnu.org/software/coreutils/ The takeaway command:man > info coreutils
  • 6. pwd | ls | find | tee List of files: ls –l ls–1 ls –latr find . –name *.txt
  • 7. pwd | ls | find | tee Seek for a string in a file: grep“cat” file.txt grep –v “dog” file.txt grep –i “PaTtErN” file.txt egrep“cat|dog” file.txt zgrep“cat” file.txt.gz
  • 8. for / xargs Do something with each file: for file in `find . –name *tmp` do rm$file done find . –name *tmp| xargsrm
  • 9. pwd | ls | find | tee find + grep find . -name '*txt' -exec grep-l aaa{} find . -name '*txt' | xargsgrep-l aaa
  • 10. pwd | ls | find | tee cat grep tac echo ls du test mv split tail dir join wc head tr sort sum cut expr uniq paste kill tee
  • 11. paste
  • 12. join
  • 14. wc
  • 15. cut
  • 18. sed sfor substitution sed‘s/cat/dog/’ # cat -> dog sed ‘s/ab//’ # ab-> ba
  • 19. sed pfor printing sed –n ‘/dog/p’ # print lines that match ‘dog’ sed–n ‘/start/,/end/p’ # print range
  • 20. sed dto delete sed ‘/dog/d’ # delete lines that match ‘dog’ sed ‘1,/pattern/d’ # delete range
  • 21. sed | and –e for invocation sed ‘s/a/A/’ | sed ‘s/b/B/’ # sed –e ‘s/a/A/’ –e ‘s/b/B/’ #
  • 22. sed { .. } to group the commands sed ‘/pattern/ { s/p/P/ s/e/E/ }’ #pattern -> PattErn
  • 23. sed rto read a file sed ‘/include/ r file.txt’ # insert file.txt after include wto write to a file sed‘/pattern/ w file.txt’ # write matched lines to a file
  • 25. awk aaabbb ccc aaabbbzzz awk '/zzz/' 1.txt grepzzz 1.txt aaabbbzzz
  • 26. awk awk 'BEGIN{<initializations>} <pattern 1> {<actions>} <pattern 2> {<actions>} ... END{<final actions>}'
  • 27. awk awk 'BEGIN{a=0, b=0} /aaa/{a++} /bbb/ {b++} END{printf “%d%d”,a,b}'
  • 28. awk awk '{arr[$2]+=$1} END { for (id in arr) printf "%s%d",id,arr[id]}'