Question

I'm trying to write a Bash script that finds the occurrence of multiple items in a file, in one loop. This is a small piece of the file I'm trying to read:

112/14-13:21:18 - write(FD 1) = 82 bytes
112/14-13:21:18 - read(FD 0, 16384 bytes buffer) = 146 bytes
112/14-13:21:18 - unlink(/data/547/07/17/nmsa000000000005/archives/9589fecf-779b-4fc3-8bb3-4cd6b1932d19/ar16f8888285fb4989b817589b0219d08b) = 0
112/14-13:21:18 - __xstat64(1, /data/547/07/17/nmsa000000000005/archives/9589fecf-779b-4fc3-8bb3-4cd6b1932d19/ar16f8888285fb4989b817589b0219d08b, ...) = 0
112/14-13:21:18 - write(FD 1) = 86 bytes
112/14-13:21:18 - read(FD 0, 16384 bytes buffer) = 138 bytes

What I want is that I can display how many times there's a "read" and a "write" and how long it took.

Currently I've got this script:

#!/bin/bash

READ=0
WRITE=0

while read -r line || [[ -n $line ]]
do
    READ=$(grep -c 'read(')
    WRITE=$(grep -c 'write')
done < $1

echo "Read $READ times"
echo "Wrote $WRITE times"

But I'm receiving this output on a small testfile:

Read 20 times
Wrote 0 times

The script doesn't seem to count the number of times "write" is in the input file.

I'm new to Bash and it's my first script. Can anyone help or tell what I'm doing wrong? And how can I implement the time part of the script? Also, can this script be optimized since the input will be quite large (+10GB file)?

Thanks in advance!

EDIT: Extra second of data

112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Reverse/Common.php, ...) = 0
112/14-13:21:17 - open(/usr/share/pear/MDB2/Driver/Reverse/Common.php, 0) = FD 8
112/14-13:21:17 - close(FD 8) = 0
112/14-13:21:17 - __lxstat(1, /usr/local/lib/php/MDB2/Driver/Manager/pgsql.php, ...) = -1
112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Manager/pgsql.php, ...) = 0
112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Manager, ...) = 0
112/14-13:21:17 - open(/usr/share/pear/MDB2/Driver/Manager/pgsql.php, 0) = FD 8
112/14-13:21:17 - close(FD 8) = 0
112/14-13:21:17 - __lxstat(1, /usr/local/lib/php/MDB2/Driver/Manager/Common.php, ...) = -1
112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Manager/Common.php, ...) = 0
112/14-13:21:17 - open(/usr/share/pear/MDB2/Driver/Manager/Common.php, 0) = FD 8
112/14-13:21:17 - close(FD 8) = 0
112/14-13:21:17 - __lxstat(1, /usr/local/lib/php/MDB2/Driver/Datatype/pgsql.php, ...) = -1
112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Datatype/pgsql.php, ...) = 0
112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Datatype, ...) = 0
112/14-13:21:17 - open(/usr/share/pear/MDB2/Driver/Datatype/pgsql.php, 0) = FD 8
112/14-13:21:17 - close(FD 8) = 0
112/14-13:21:17 - __lxstat(1, /usr/local/lib/php/MDB2/Driver/Datatype/Common.php, ...) = -1
112/14-13:21:17 - __lxstat(1, /usr/share/pear/MDB2/Driver/Datatype/Common.php, ...) = 0
112/14-13:21:17 - open(/usr/share/pear/MDB2/Driver/Datatype/Common.php, 0) = FD 8
112/14-13:21:17 - close(FD 8) = 0
Was it helpful?

Solution

Maybe try awk then you'll only need one pass over the file during which you can count both reads and writes:

awk '/write\(/{w++;next}  /read\(/{r++;next}  END{print "Read: ",r, "Write: ",w}' file
Read:  2 Write:  2

That says... every time you find a line with write( in it, increment w. Every time you find a line with read( in it, increment r. At the end, print r and w.

If you want the number of times each second, you can use this:

awk 'FNR==1{r=0;w=0;lasttime=$1} $1!=lasttime{print lasttime,"Read: ",r, "Write: ",w;r=0;w=0;lasttime=$1} /write\(/{w++} /read\(/{r++}' file

Basically, it outputs the count of reads and writes every time the second changes.

OTHER TIPS

Why do you loop ?

This script work fine :

#!/bin/bash
READ=$(grep -c 'read(' $1)
WRITE=$(grep -c 'write(' $1)
echo "Read $READ times"
echo "Wrote $WRITE times"

In your script, you pass all the data inside the file $1 to the stdin of the while loop. Your initial read command consumes the first line of this data, then the first grep command consumes the remainder. This leaves no data left for the second grep command, and hence it does not return anything. As a side-effect, if the first line of the file contains read(, it will not be counted by grep as it has already been consumed by read. Your script can be improved thus (similar to @John's answer):

#!/bin/bash

READ=$(grep -c 'read(' "$1")
WRITE=$(grep -c 'write' "$1")
echo "Read ${READ:-0} times"
echo "Wrote ${WRITE:-0} times"

As the OP is concerned about reading the file twice, so this could be done:

#!/bin/bash
R=0
W=0
while read -r line
do
        R=$((  $(grep -c 'read('  <<< "$line" )  + R ))
        W=$((  $(grep -c 'write(' <<< "$line" )  + W ))
done

printf "Read  %i times\n" $R
printf "Wrote %i times\n" $W

Cheers,

Paul

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top