Update: Apologies for posting a duplicate of steveha's much earlier answer. Speaks volumes about my reading skills. Still leaving this answer online for now, just because of my musings about the i/o/buffering/runtime effects.
Original post:
I cannot believe that it takes Python longer to apply one regular expression, and calculate one square root, than it does take to read one line from standard input and output a result on standard output (or any I/O for that matter).
Since I/O at one point in time will come from a hardrive and will either go to another hardrive or to a user's eye, that should be the limiting factor.
I/O is usually buffered for speedup. Usually a buffer is filled in bursts, then the cpu idles while waiting for the device to provide more data.
That leads to a generator for your application. Write a generator that reads input line by line and immediate provides a sqrt number on demand. I doubt that this will be any slower than the overall I/O speed on any reasonable modern hardware. If you're on a special device (like embedded, uController, Raspberry Pi, etc let us know)
The one optimization you can do is precompile the regular expression. As you're using the same regexp for each test, let's do the parsing of the regexp only once. Your example in the question is fine because you're doing a re.findall()
. I'm just elaborating for other readers.
import sys, re, math
pattern = re.compile(r'\b\d+\b')
def fh_numbers_to_sqrt(fh):
for line in fh:
for i in re.findall(pattern, line):
yield math.sqrt(float(i))
numbers_g = fh_numbers_to_sqrt(sys.stdin)
for num in numbers_g:
print('%.4f' % num)
This allows all the regexp and math operations to interleave with the I/O times.
Now, the one thing we simply cannot really optimize and integrate is the reverse
. The algorithm must wait until the last element to be able to reverse.
So we could change the calling code to:
numbers_g = fh_numbers_to_sqrt(sys.stdin)
for num in reverse(list(numbers_g)):
print('%.4f' % num)
And hope that this is faster what you originally had.
Again, the only reason why this should be faster is because we've hidden the runtime of regexp parsing and calculation within the wall clock time it takes to read data from standard input. This should still be I/O limited. Actually the reverse
might not really add to overall runtime, because it just might interleave with the I/O happening on standard output. Looking at a wall clock, this algorithm might use up no time at all. :-)
To prove or negate my whole post, you could measure with time.time()
how long it takes from the start of your script to just before the line Data = re.findall
, and from then on to the end. If I'm correct then the data reading will take most of the time. If not, it's worthwhile to measure also the time required for all the regular expression searches. Let us know. I'm curious...