Send a file over websocket with pharo, ZnWebsocket, and Fuel in Pharo 2.0?

Question

I am trying to send fairly large video files over websocket using Pharo 2.0, and Zinc Websocket. This is a pharo to pharo transfer.

I have tried in vain to get this to work. I'm sure there's a very simple solution I'm missing. Thanks for any help.

Edit: Update: I did get this to work, through the following code:

myStream :=   
(FileStream oldFileNamed: 'big_buck_bunny_480p_surround-fix.avi') .
bytes := myStream next: myStream size.
myStream close.
myStream := nil.
server distributeMessage: bytes.

testConnect
| websocket1 receivedFile|
websocket1 := ZnWebSocket to: 'ws://192.168.1.102:1701/ws-chatroom'.
testProcess := [ 
[ 
| mes |
mes := [ websocket1 readMessage ]
    on: ConnectionTimedOut
    do: [ nil ].
mes
    ifNotNil: [ 
        Transcript
            show: 'Message received in client';cr .
            receivedFile := FileStream newFileNamed: 'receivedFile3.avi'. 
receivedFile nextPutAll: mes.
receivedFile close.
             ] ] repeat ] fork

However, it doesn't work for files bigger than ~500 megabytes, because Pharo runs out of memory.

Unless someone has a great suggestion to get this to work, I'm going to switch gears and try to serve the files using ZincHTTPComponents, and perhaps send a message to the remote computer containing the location of the file to download on the webserver.

Edit: Important Warning/Update:

I discussed this issue with Sven who wrote the Zinc websocket packages for pharo smalltalk. He told me that sending large files over websocket is not really a feasible idea. Indeed, even though I got the solutions below implemented, the final file would be off by a few bytes every time.

I solved my issue by doing my plan B: Serving the files over HTTP using Zinc HTTP Components and using a client get such as:

FileStream 
    newFileNamed: '/tmp/cog.tgz' 
    do: [ :stream | | entity |
            stream binary.
            entity := ZnClient new
                    streaming: true;
                    get: 'https://ci.lille.inria.fr/pharo/job/Cog%20Git%20Tracker/lastSuccessfulBuild/artifact/cog.tar.gz';
                    entity.
            entity writeOn: fileStream ]

Answer 1

The reason you're running out of memory is that you read the entire file into memory before processing it. Since you can only send chunks of data over the wire anyway, you should also only read one chunk at a time (that's what streams are for). So instead of this:

myStream :=   
(FileStream oldFileNamed: 'big_buck_bunny_480p_surround-fix.avi').
bytes := myStream next: myStream size.
myStream close.

you should use something like this:

blockSize := 128.
myStream :=   
(FileStream oldFileNamed: 'big_buck_bunny_480p_surround-fix.avi') .
[ myStream atEnd ] whileFalse: [
    bytes := myStream next: blockSize.
    "process your bytes here" ].
myStream close.

Or even better: use the convenience block to automatically close the stream:

blockSize := 128.  
FileStream 
    oldFileNamed: 'big_buck_bunny_480p_surround-fix.avi'
    do: [ :stream |
        [ stream atEnd ] whileFalse: [
            bytes := stream next: blockSize.
        "process your bytes here" ].

Edit

To answer the question from your comment: if you look at FileStream>>next: you will see the following lines:

...
[self atEnd ifTrue:
    [(howManyRead + 1) to: anInteger do: [:i | newCollection at: i put: (self next)].
    ^newCollection].
...

This means that if you ask for more than is available, you will simply get the rest of the stream up to its end.