ruby - IO.popen not working lame stdin and stdout encoding

Question

I've been working with pipes and IO.popen specifically in Ruby and have come across a problem that I can't figure out. I am trying to write binary data from the flac process to the lame process into a file. The code structure I am using is below.

# file paths
file = Pathname.new('example.flac').realpath
dest = Pathname.new('example.mp3')

# execute the process and return the IO object
wav = IO.popen("flac --decode --stdout \"#{file}\"", 'rb')
lame = IO.popen("lame -V0 --vbr-new - -", 'r+b')

# write output from wav to the lame IO object
lame << wav.read

# close pipe for writing (should indicate to the
# process that input for stdin is finished).
lame.close_write

# open up destiniation file and write from lame stdout
dest.open('wb'){|out|
    out << lame.read
}

# close all pipes
wav.close
lame.close

However, it doesn't work. After flac has run, the script hangs and lame remains idle (no processor usage at all). No errors or exceptions occur.

I am using cygwin on Windows 7, with the cygwin ruby package (1.9.3p429 (2013-05-15) [i386-cygwin]).

I must be doing something wrong, any help is much appreciated. Thanks!

EXTRA #1

I am wanting to pipe in and out the binary data from the lame process because I am trying to create a platform independent (ruby support limited of course) to transcode audio files, and the Windows binary of lame only supports Windows' path names, and not cygwin's.

EDIT #1

I read in some places (I did not save the URLs, I'll try looking for them in my browser history) that IO.popen has known issues with blocking processes in Windows and that this could be the case.

I have played around with other libraries including Ruby's Open3.popen3 and Open4, however following a very similar code structure to the one above, the lame process still hangs and remains unresponsive.

EDIT #2

I found this article which talked about the limitations of of Windows's cmd.exe and how it prevents the use of streamed data from files to stdin.

I refactored my code to look like shown below to test this out, and as it turns out, lame freezes on stdin write. If I removed (comment out) that line, the lame process executes (with an 'unsupported audio format' warning). Perhaps what the article said could explain my problem here.

# file paths
file = Pathname.new('example.flac').realpath
dest = Pathname.new('example.mp3')

# some local variables
read_wav = nil
read_lame = nil

# the flac process, which exits succesfully
IO.popen("flac --decode --stdout \"#{file}\"", 'rb'){|wav|
    until wav.eof do
        read_wav = wav.read
    end
}

# the lame process, which fails
IO.popen("lame -V0 --vbr-new --verbose - -", 'r+b'){|lame|
    lame << read_wav # if I comment out this, the process exits, instead of hanging
    lame.close_write
    until lame.eof do
        read_lame << lame.read
    end
}

EDIT #3

I found this stackoverflow which (in the first answer) mentioned that cygwin pipe implementation is unreliable. This could perhaps not actually be related to Windows (at least not directly) but instead to cygwin and its emulation. I have instead opted to use the following code, based upon icy's answer, which works!

flac = "flac --decode --stdout \"#{file}\""
lame = "lame -V0 --vbr-new --verbose - \"#{dest}\""

system(flac + ' | ' + lame)

Answer 1

Did you try the pipe | character?
Tested this on windows with ruby installer

require 'open3'

command = 'dir /B | sort /R'  # a windows example command
Open3.popen3(command) {|stdin, stdout, stderr, wait_thr|
  pid = wait_thr.pid
  puts stdout.read  #<a list of files in cwd in reverse order>
}

Other ways: Ruby pipes: How do I tie the output of two subprocesses together?

EDIT: using IO::pipe

require 'open3'

command1 = 'dir /B'
command2 = 'sort /R'

reader,writer = IO.pipe
Open3.popen3(command1) {|stdin, stdout, stderr, wait_thr|
  writer.write stdout.read
}
writer.close

stdout, stderr, status = Open3.capture3(command2, :stdin_data => reader.read)
reader.close

puts "status: #{status}"   #pid and exit code
puts "stderr: #{stderr}"   #use this to debug command2 errors
puts stdout

Embedding the two also appears to work, yet, as the blog you referred to said, one must wait for the first command to finish (not real-time -- test with a ping command)

stdout2 = ''
Open3.popen3(command1) {|stdin, stdout, stderr, wait_thr|
  stdout2, stderr2, status2 = Open3.capture3(command2, :stdin_data => stdout.read)
}
puts stdout2

Answer 2

Use pipeline from Open3:

require "open3"

wavCommand = "flac --decode --stdout \"#{file}\""
lameCommand = "lame -V0 --vbr-new - -"

Open3.pipeline(wavComamnd, lameCommand)

The last line spawns two processes and connects stdout of the first process to stdin of the second one. Alternatively you can have access to stdin of the first process using pipeline_w or you can obtain stdout of the last command using pipeline_r or you can have both, using pipline_rw.