Editors and file processing commands

 

Editors

The most used editor is gvim. It works just like our word or notepad that we use in windows OS.

Gvim creates a file if it doesn’t exist. It opens a file if it exists.

Syntax: gvim filename

In gvim we have 3 modes of operation:

1. Command mode

2. Insert mode

3. Visual mode

1. Command mode: This mode is main as most of the operations are done in this mode. By default the gvim opens in command mode but if one has to change to command mode from other mode we need to use “esc” key on the keyboard.

Few commands used in gvim are:

1. :w – to save

2. :q – to quit

3. :wq – to save and quit

4. :q! – forceful quit , it does not save any changes done.

5. :qa – quit all

6. :wa – save all

7. :vsp – split the gvim in vertical

8. :sp – split the gvim in horizontal

9. :u – undo

10. /searchtext – to search text from top to bottom

11. ?searchtext – to search text from bottom to top

12. N : move to  highlighted text from bottom to top

13. n : move to highlighted text from top to bottom

14. o : insert line below the cursor

15. O : insert line above the cursor

16. ^ : to go to start of line

17. $ : to go to end of line

18. gg : to go to start of gvim

19. G : to go to end of gvim

20. w : move word by word in forward direction

21. b : move word by word in backward direction

22. %s/searchtext/replacetext/gc : to search and replace a text , if g is not mentioned then only 1st occurrence of each line will be replaced, If g is mentioned then all occurrences will be searched and replaced.

If c is not mentioned then all the searched text will be replaced at once, if c is mentioned to each searched text before replacing the gvim prompts if we are ok in replacing if yes we have to give y, if no we have to give n.

23. /<\searchtext\> : to exclusively search only that particular text

24. ggVG: This selects complete gvim file

25. h : move cursor left by one character

26. j : move cursor down by one character

27. k : move cursor up by one character

28. l : move cursor right by one character

29. :!ls /path/ : to use tcsh shell commands in gvim

30. #! /path of installed shell : to run gvim script using any shell scripting

2. Insert mode:

One can edit the content in gvim only in insert mode. We have to change to insert mode from command mode only by using “i” in keyboard.

3. Visual mode: 

One can use visual mode to read by highlighting a text, to copy, paste, we can enter into visual mode by pressing “v” key in the keyboard.

Select a text using navigation keys and press “y” in keyboard which means yanking i.e. copying the selected text and enter into command mode using “esc” key and press “p” by keeping the cursor at required place, this pastes the copied content.

One cannot switch between insert mode and visual mode.

File processing commands

1. Grep: Global Regular Expression 

Grep is a file processing command where one can search for a pattern of a record.

When we search for a pattern in a file the whole record containing that particular pattern will be printed on the terminal or can be redirected.

Syntax: grep –option “pattern” filename

Options: -i:  Case insensitive.

-v: Prints the unmatched pattern.

-E: To match more than one pattern.

Let’s say I have a file with file name as file and the data in it is:

Name  id  domain

Kailash 675 PD

Veena768  DV

Keerthi  789 CL

Geetha876  pd

Let me search the record kailash using: grep “kailash” file  : output ->Kailash 675 PD

Let me search the record pd using grep: grep –i “pd” file : output ->kailash 675 PD

Geetha 876 pd

Let me search all the records which donot have DV using grep: grep –v “DV” file

Output: Name id domain

Kailash 675 PD

Keerthi 789 CL

Geetha 876 pd

Let me search all the records with DV and CL: grep –E “DV\|CL” file

Output: Veena 768 DV

Keerthi 789 CL

To print complete file using grep: grep –i “/ / /”  file

2. Stream editor: sed

Sed command is used to search, replace, modify, delete, insert, append to a file without altering the content of the original file. It only prints the output on the terminal but does not alter the original file.

Flags used in sed:

r: to read another file

a: to append

s: to search

g: global search

A: line number or number of occurrence

i: to insert

d: to delete

c: to modify

w: to write

p: to print 

options used in sed are:

-n : to suppress the line

-f: to read another file

-e: to search multiple patterns

Let’s discuss about each flag and options that can be used with those flags in detail:

• To search and replace:

Sed ‘s/searchtext/replacetext/g’file: It searches and replaces the text globally.

• To search and replace text of particular occurrence of each line.

Sed ‘s/search text/replace text/3’ file : it searches and replaces the 3 occurrence of each record.

• To search and replace text of particular lines and particular occurrence.

Sed ‘2s/search text/replace text/g’ file : it searches all the occurrence of a pattern in 2nd record and replace.

Sed ‘2,5s/search text/replace text/4’ file : it searches and replaces only 4th occurrence of pattern from 2nd to 5th line .

Sed –e ‘2s/search text/replace text/5 ; 5s/search text/replace text/5’ file : it search and replace all 5th occurrences of  only 2nd and 5th line.

• To search and replace using a particular word of a record.

Sed ‘/check word/s/search word/replace word/g’ file : It checks for a check word in each line of a file and once it finds that word it then checks if search word is present, If present it will replace it with the replace word If not it will iterate over all the lines of the line.

• To print replaced text using p flag and –n option:

Sed ‘s/search text/replace text/p’ file

It prints me the original line of file and even the replaced line and as well unmatched line also.

Sed –n ‘s/search text/replace text/p’ file

It prints me only the replaced text line of a file.

• To  append a line to a file:

Sed ‘a\hi’ file

It appends hi to all the records of a file.

Sed ‘2a\hi’ file

It appends hi to 2nd record of a file.

Sed ‘2,5a\hi’ file

It appends hi from 2nd line to 5th line of a file.

Sed –e ‘2a\hi ; 5a\hi’ file

It appends hi to 2nd and 5th line of file.

• To insert a line to a file:

Sed ‘i\hi’ file

It inserts hi to all the records of a file.

Sed ‘2i\hi’ file

It inserts hi to 2nd record of a file.

Sed ‘2,5i\hi’ file

It inserts hi from 2nd line to 5th line of a file.

Sed –e ‘2i\hi ; 5i\hi’ file

It inserts hi to 2nd and 5th line of file.

•  To modify a file:

Sed ‘c\hi’ file

It replaces each record with hi of the file.

Sed ‘2c\hi’ file

It replaces 2nd record with hi.

Sed ‘2,5c\hi’ file

It replaces 2nd line to 5th line by hi.

Sed –e ‘2c\hi ; 5c\hi’ file

It replaces 2nd and 5th line by hi.

• To delete a line of a file.

Sed ‘d’ file

It deletes to all the records of a file.

Sed ‘2di’ file

It deletes 2nd record of a file.

Sed ‘2,5d’ file

It deletes from 2nd line to 5th line of a file.

Sed –e ‘2d ; 5d’ file

It deletes 2nd and 5th line of file.

• To read another file in the current file.

Sed ‘rnewfile’ file : It reads new file content below each line of the content of file called file.

Sed ‘7rnewfile’ file : It reads new file content below 7th line of the file named file.

Sed ‘/checkword/r newfile’ file : It checks for the check word in each line and if the word is present then the content of new line is printed below the found check word of the file called file.

• To write the sed command output to another file.

Sed ‘s/search text/replace text/w file2’ file

The original lines and the replaced lines are printed as output on the terminal but the replaced lines are even copied onto file2.

• To use sed command in a file.

Sed –f file1 file2

Lets say file1 has :sed ‘s/this/THIS/g’ file2

Lets say file2 has: this is India, this is a vast country, this is a secular country.

Output of sed –f file1 file2 is :

THIS is India,THIS is a vast country, THIS is a secular country.

• To print line numbers:

Sed ‘=’ file

3. Awk:

It is used for pattern scanning and processing, it searches one or more files to see if they contain lines that matches with specified patterns, transforms data files, produce formatted reports, performs arithmetic operations, conditional and loop statements, and then perform the associated actions.

AWK is the abbreviated from the names of the developers – Aho, Weinberger and Kernighan.

Syntax: awk<option> ‘selection criteria{action}’  <input file>

Built – in variables of awk:

1. $n : Gvies the fields of records 

Eg: $0 : prints the complete record

      $1: prints the 1st field of the record.

2. NR: Number of records:

Prints line numbers of the file.

Eg: awk ‘{print NR, $0}’ file

It prints me each line number and the total content.

Eg: awk ‘NR==2, NR==5{print $0}’ file

It prints me lines from 2 to 5 including the line numbers and the total content.

Eg: awk ‘NR==2; NR==5{print $0}’ file

It prints me only 2nd and 5th line of the line with the line number and the total content.

3. NF: Number of fields:

Prints number of fields of each record.

Eg: awk ‘{print NF, $0}’ file

It prints me number of fields of each line and as well the content.

Eg: awk ‘NR==2, NR==5{print NF, $0}’ file

It prints me the number of fields and the line content only from 2nd line to 5th line.

Eg: awk ‘NR==2;NR==5{print NF, $0}’ file

It prints me the number of fields and the line content of only 2nd and 5th line.

4. FS: Field separators:

It contains the field separator character which is used divide the record of the file into separators. By default field separator is space.

Eg: awk‘BEGIN{FS=”%”} {print NF}’ file

Let say the file has :

orand%xor

or2%nand nor

xnor% not %mux

ouput : 2 or and%xor

              2 or2%nand nor

3 xnor%not%mux

So FS separates the fields with %.

5. RS: Record separators:

It has a character which divides the records of file, by default record separators is new line.

Eg: awk‘BEGIN{RS=”%”} {print NF}’ file

Let say the file has :

orand%xor

or2%nand nor

xnor% not %mux

ouput :1 or and

2.xor or2

3. nand nor xnor

4. not

5. mux

So RS separates the records with %.

6. OFS: Output field separators:

It uses a special character to separate the fields of the file and print in terminal as output but by default it is space.

7. ORS: Output record separators:

It uses a special character to separate the records of the file and print in terminal as output but by default it is new line.

Types of processing in awk:

1. Pre-processing:

This does not require input file, it uses the keyword “BEGIN” where we can do loops, conditional statements, arithmetic operations using this keyword.

Syntax: awk‘BEGIN{print “text to be printed”}’

Eg: awk‘BEGIN{print “hello”}’

In awk we have arguments and can be represented using ARGC, ARGV[i]

Eg: awk‘BEGIN{print ARGC}’ and or xor nor

Ouput: 5 ; ARGC is used to print total number of arguments that the awk has and the awk itself is an argument.

Eg: awk‘BEGIN{print ARGV[0], ARGV[1], ARGV[2],  ARGC}’ and or 

Output: awk, and, or, 3 ;awk is the 0th index, and is 1st index and or is the 2nd index and we have total 3 arguments. So ARGV[i] is used to access the arguments using index.

Mathematical operations:

Awk‘BEGIN{print ARGV[1] * ARGV[2]}’ 10 20

Output: 200

To access external variable using awk:

Eg: set a = 10

      Set b = 20

Awk –vA=$a  -vB=$b ‘BEGIN{print A B}’

Output: 10 20

To access external variables using awk and do arithmetic operation:

Eg: awk –vA=$a –vB=$b ‘BEGIN{print A+B}’ 

Eg: awk ‘{print ARGV[2]/ARGV[1]}’ $a $b

Output: 2

Accessing external variables without using –v:

Eg: awk ‘{print ARGV[1] ARGV[2]}’ $a $b

10 20

2. Record – processing:

It requires input file. It does search operation, it could replace file.

Eg: awk ‘/Hello/{print}’ file

Let say file has:

Hello this is xyz.

Hi this is abc.

Hello world.

Hey dear.

Output: Hello this is xyz.

Hello world.

So this prints me only the records which “Hello” in it.

Eg: awk ‘{print $1 $2}’ file

Output:  Hello this

Hi this

Hello world

Hey dear.

Eg: awk ‘{print “scripting”}’  file

Output: scripting

Scripting

Scripting

Scripting

So this replaces all the records of the file with scripting.

3. Post-Processing:

This requires input as file. It used keyword “END” and prints only the last line of file, if NR is used it prints even the last record number.

Eg: awk‘END{print NR, $0}’ file

Let’s say file has:

And or nand

Xor or

Nor not xnor

Ouput: 3 nor not xnor

Loops in awk:

1. For loop:

Syntax: awk‘BEGIN{for(initialization;condition;increment){){print variable}}’

2. Nested loops:

Syntax: awk‘BEGIN{for(initialization;condition;increment){for(initialization;condition;increment){print variables}}}’

3. While loops:

Syntax: awk‘BEGIN{initialization; while(condition) {print increment}}’

Conditional statements:

1. If:

Syntax: if (condition) {action}

2. If-else:

Syntax: if(condition)(action}else{action}

3. If-elseif-else

Syntax: if(condition){action}elseif (condition) {action } else{action}

Control statements:

1. Break:

If(condition){break} 

Breaks the loop at that condition.

2. Continue:

If(condition){continue}

Continues the loop skipping that particular condition.

3. Exit:

If(condition){exit}

Exits the loop at that condition.