Why does Ruby open-uri's open return a StringIO in my unit test, but a FileIO in my controller?

Question

I inherited a Rails 2.2.2 app that stores user-uploaded images on Amazon S3. The attachment_fu-based Photo model offers a rotate method that uses open-uri to retrieve the image from S3 and MiniMagick to perform the rotation.

The rotate method contains this line to retrieve the image for use with MiniMagick:

temp_image = MiniMagick::Image.from_file(open(self.public_filename).path)

self.public_filename returns something like

http://s3.amazonaws.com/bucketname/photos/98/photo.jpg

Retrieving the image and rotating it work just fine in the running application in production and development. However, the unit test fails with

TypeError: can't convert nil into String
    /Users/santry/Development/totspot/vendor/gems/mini_magick-1.2.3/lib/mini_magick.rb:34:in `initialize'
    /Users/santry/Development/totspot/vendor/gems/mini_magick-1.2.3/lib/mini_magick.rb:34:in `open'
    /Users/santry/Development/totspot/vendor/gems/mini_magick-1.2.3/lib/mini_magick.rb:34:in `from_file'

The reason is that when the model method is called in the context of the unit test, open(self.public_filename) is returning a StringIO object that contains the image data. The path method on this object returns nil and MiniMagick::Image.from_file blows up.

When this very same model method is called from the PhotosController, open(self.public_filename) returns a FileIO instance tied to a file named, eg, /tmp/open-uri7378-0 and the file contains the image data.

Thinking the cause must be some environmental difference between test and development, I fired up the console under the development environment. But just as in the unit test, open('http://...') returned a StringIO, not a FileIO.

I've traced my way through open-uri and all the relevant application-specific code and can find no reason for the difference.

Answer 1

The code responsible for this is in the Buffer class in open-uri. It starts by creating a StringIO object and only creates an actual temp file in the local filesystem when the data exceeds a certain size (10 KB).

I assume that whatever data your test is loading is small enough to be held in a StringIO and the images you are using in the actual application are large enough to warrant a TempFile. The solution is to use methods which are common to both classes, in particular the read method, with MiniMagick::Image#from_blob:

temp_image = MiniMagick::Image.from_blob(open(self.public_filename, &:read))

Answer 2

The open-uri library uses a constant to set the 10KB size limit for StringIO objects.

> OpenURI::Buffer::StringMax
=> 10240 

You can change this setting to 0 to prevent open-uri from ever creating a StringIO object. Instead, this will force it to always generate a temp file.

Just throw this in an initializer:

# Don't allow downloaded files to be created as StringIO. Force a tempfile to be created.
require 'open-uri'
OpenURI::Buffer.send :remove_const, 'StringMax' if OpenURI::Buffer.const_defined?('StringMax')
OpenURI::Buffer.const_set 'StringMax', 0

You can't just set the constant directly. You need to actually remove the constant and then set it again (as above), otherwise you'll get a warning:

warning: already initialized constant StringMax

UPDATED 12/18/2012: Rails 3 doesn't require OpenURI by default, so you need to add require 'open-uri' at the top of the initializer. I updated the code above to reflect that change.