Safe search&replace on linux

Question

Let's consider I have files located in different subfolders and I would like to search, test and replace something into these files.

I would like to do it in three steps:

1. Search of a specific pattern (with or without regexp)
2. Test to replace it with something (with or without regexp)
3. Apply the changes only to the concerned files
   

My current solution is to define some aliases in my .bashrc in order to easily use grep and sed:

alias findsrc='find . -name "*.[ch]" -or -name "*.asm" -or -name "*.inc"'
alias grepsrc='findsrc | xargs grep -n --color '
alias sedsrc='findsrc | xargs sed '

Then I use

1. grepsrc <pattern> to search my pattern
2. (no solution found yet)
3. sedsrc -i 's/<pattern>/replace/g'

Unfortunately this solution does not satisfy me. The first issue is that sed touch all the files even of no changes. Then, the need to use aliases does not look very clean to me.

Ideally I would like have a workflow similar to this one:

Register a new context:

$ fetch register 'mysrcs' --recurse *.h *.c *.asm *.inc 

Context list:

$ fetch context
 1. mysrcs --recurse *.h *.c *.asm *.inc
 Extracted from ~/.fetchrc

Find something:

$ fetch files mysrcs /0x[a-f0-9]{3}/
./foo.c:235     Yeah 0x245
./bar.h:2       Oh yeah 0x2ac hex

Test a replacement:

$ fetch test mysrcs /0x[a-f0-9]{3}/0xabc/
./foo.c:235     Yeah 0xabc
./bar.h:2       Oh yeah 0xabc hex 

Apply the replacement:

$ fetch subst --backup mysrcs /0x[a-f0-9]{3}/0xabc/
./foo.c:235     Yeah 0xabc
./bar.h:2       Oh yeah 0xabc hex 

Backup number: 242

Restore in case of mistake:

$ fetch restore 242

This kind of tools look pretty standard to me. Everybody needs to search and replace. What alternative can I use that is standard in Linux?

Answer 1

Yes there is a tool doing exactely that you are looking for. This is Git. Why do you want to manage the backup of your files in case of mistakes when specialized tools can do that job for you?

You split your request in 3 subquestions:

• How quickly search into a subset of my files?
• How to apply a substitution temporarly, then go back to the original state?
• How to substitute into your subset of files?

We first need to do some jobs in your workspace. You need to init a Git repository then add all your files into this repository:

$ cd my_project
$ git init
$ git add **/*.h **/*.c **/*.inc 
$ git commit -m "My initial state"

Now, you can quickly get the list of your files with:

$ git ls-files

To do a replacement, you can either use sed, perl or awk. Here the example using sed:

$ git ls-files | xargs sed -i -e 's/search/replace/'

If you are not happy with this change, you can roll-back anytime with:

$ git checkout HEAD

This allows you to test your change and step-back anytime you want to.

Now, we did not simplified the commands yet. So I suggest to add an alias to your Git configuration file, usually located here ~/.gitconfig. Add this:

[alias]
sed = ! git grep -z --full-name -l '.' | xargs -0 sed -i -e

So now you can just type:

$ git sed s/a/b/

It's magic...

Answer 2

#!/bin/ksh

# Call the batch with the 2 (search than replace) pattern value as argument

# assuming  the 2 pattern are "sed" compliant regex
SearchStr="$1"
ReplaceStr="$2"

# Assuming it start the search from current folder and take any file
# if more filter needed, use a find before with a pipe
grep -l -r "$SearchStr" . | while read ThisFile
 do
    sed -i -e "s/${SearchStr}/${ReplaceStr}/g" ${ThisFile}
 done

should be a base script to adapt to your need

Answer 3

I often have to perform such maintenance tasks. I use a mix of find, grep, sed, and awk.

And instead of aliases, I use functions.

For example:

# i. and ii.
function grepsrc {
    find . -name "*.[ch]" -or -name "*.asm" -or -name "*.inc" -exec grep -Hn "$1"
}

# iii.
function sedsrc {
    grepsrc "$1" | awk -F: '{print $1}' | uniq | while read f; do
        sed -i s/"$1"/"$2"/g $f
    done
}

Usage example:

sedsrc "foo[bB]ar*" "polop"

Answer 4

for F in $(grep -Rl <pattern>) ; do sed 's/search/replace/' "$F" | sponge "$F" ; done

• grep with the -l argument just lists files that match
• We then use an iterator to just run those files which match through sed
• We use the sponge program from the moreutils package to write the processed stream back to the same file

This is simple and requires no additional shell functions or complex scripts.

If you want to make it safe as well... check the folder into a Git repository. That's what version control is for.