xargs string used as an input for agrep
Question
Guys I am using xargs to pass the input for a agrep.I am using xargs like the below
Script:
xargs -L 1 -I string echo "RequestId="string | xargs -L 1 -I string zcat FILEB | agrep -dEOE string
Output till "RequestId="string
RequestID=543214324
RequestID=512121221
RequestID=521212121
My goal is to agrep this Request ID's in another file "File B". Since this file is zipped I need to use zcat before doing agrep...But if I do this I am getting incorrect results.. The final "string" which I use in agrep is not proper.. Any suggestions regarding this would be highly helpful.
No correct solution
OTHER TIPS
It is not completely clear what you want. I assume you have a file FILEA containing:
543214324
512121221
521212121
and that you want to agrep FILEB.gz for:
RequestID=543214324
RequestID=512121221
RequestID=521212121
FILEBB.gz is gzipped so you need zcat
to decompress it, and you cannot use zgrep because you depend on a feature only found in agrep
.
Using GNU Parallel you can do:
cat FILEA | parallel zcat FILEB.gz \| agrep -dEOE RequestID={}
One of the advantages of doing it in parallel is that if FILEB.gz is too big to fit in memory, then this will make sure to search in parallel thus avoiding reading FILEB.gz from disk for each search. Another advantage is that you can use multiple CPU cores which your computer is almost guaranteed to have these days.
You can install GNU Parallel simply by:
wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem
Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1