Out of memory with NetStream.appendBytes() or How to Play Large Video?

Question

The following is a fully-functioning pure actionscript project for AIR

When run, try opening a large FLV (I'm testing it with a 3GB file)

With DEBUG_UNUSED_BUFFER and DEBUG_APPEND_VIDEO set to false, it works fine- reads through the entire file without a problem.

However, with either of those set to true, it crashes with an OUT OF MEMORY error.

For practical purposes, I'm more interested in why appendBytes() fails, but for the sake of interest, the DEBUG_UNUSED_BUFFER only makes it to like 6% of the file while DEBUG_APPEND_VIDEO makes it to around 46% or so.

Question: How then are we supposed to play a large video?!

package
{
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.events.IOErrorEvent;
    import flash.events.MouseEvent;
    import flash.events.ProgressEvent;
    import flash.events.TimerEvent;
    import flash.filesystem.File;
    import flash.filesystem.FileMode;
    import flash.filesystem.FileStream;
    import flash.net.FileFilter;
    import flash.net.NetConnection;
    import flash.net.NetStream;
    import flash.net.NetStreamAppendBytesAction;
    import flash.text.TextField;
    import flash.text.TextFieldAutoSize;
    import flash.text.TextFormat;
    import flash.text.TextFormatAlign;
    import flash.utils.ByteArray;
    import flash.utils.Timer;

    public class MEMORY_TEST extends Sprite
    {
        //Set this to throttle data processing to once every DEBUG_THROTTLE_TIME milliseconds
        // 0 = no throttling at all
        // Note this this seems to make little difference, other than making it easier to see what's happening
        private static const DEBUG_THROTTLE_TIME:Number = 100;

        //Set this to write all bytes to an unused buffer.
        //THIS FAILS (at around 237912064 bytes)!!!!!
        private static const DEBUG_UNUSED_BUFFER:Boolean = false;

        //Set this to write the video data via appendBytes.
        //THIS FAILS (at around 1360003072 bytes)!!!!
        private static const DEBUG_APPEND_VIDEO:Boolean = true;

        /****************************************************************/
        /******* Nothing else to configure below this line **************/
        /****************************************************************/
        private var openButton:Sprite;
        private var statusTextField:TextField;

        private var inputFile:File = null;
        private var inputFileStream:FileStream = null;
        private var netStream:NetStream = null;
        private var netConnection:NetConnection = null;
        private var readBytes:ByteArray = null;
        private var totalBytesRead:Number = 0;

        private var throttleTimer:Timer = null;
        private var unusedBuffer:ByteArray = null;

        private static const READSIZE:uint = 2048;




        public function MEMORY_TEST()
        {
            this.addEventListener(Event.ADDED_TO_STAGE, onStage);
        }

        /*************************
         * 
         *  UI SETUP
         * 
         **************************/

        private function onStage(evt:Event) {
            this.removeEventListener(Event.ADDED_TO_STAGE, onStage);

            makeButtonAndStatus();
            updateStatus('Click the button to begin');
        }

        private function makeButtonAndStatus(buttonText:String = 'Open File') {
            var textField:TextField = new TextField();
            var fmt:TextFormat = new TextFormat();
            var padding:Number = 20;
            var halfPadding:Number = padding/2;

            //Button
            fmt.color = 0xFFFFFF;
            fmt.size = 24;
            fmt.font = "_sans";
            fmt.align = TextFormatAlign.LEFT;

            textField.autoSize = TextFieldAutoSize.LEFT;
            textField.multiline = false;
            textField.wordWrap = false;
            textField.defaultTextFormat = fmt;
            textField.text = buttonText;

            openButton = new Sprite();
            openButton.graphics.beginFill(0x0B8CC3);
            openButton.graphics.drawRoundRect(-halfPadding,-halfPadding,textField.width + padding, textField.height + padding, 20, 20);
            openButton.graphics.endFill();
            openButton.addChild(textField);

            openButton.buttonMode = true;
            openButton.useHandCursor = true;
            openButton.mouseChildren = false;

            openButton.addEventListener(MouseEvent.CLICK, selectFile);

            openButton.x = (stage.stageWidth - openButton.width)/2;
            openButton.y = (stage.stageHeight - openButton.height)/2;

            addChild(openButton);

            //Status
            statusTextField = new TextField();
            fmt = new TextFormat();

            fmt.color = 0xFF0000;
            fmt.size = 17;
            fmt.font = "_sans";
            fmt.align = TextFormatAlign.CENTER;

            statusTextField.defaultTextFormat = fmt;
            statusTextField.multiline = true;
            statusTextField.wordWrap = false;
            statusTextField.width = stage.stageWidth;
            statusTextField.text = '';

            statusTextField.x = 0;
            statusTextField.y = openButton.y + openButton.height + padding;
            statusTextField.mouseEnabled = false;

            addChild(statusTextField);
        }

        private function selectFile(evt:MouseEvent) {
            var videoFilter:FileFilter = new FileFilter("Videos", "*.flv");
            var inputFile:File = File.desktopDirectory;

            inputFile.addEventListener(Event.SELECT, fileSelected);
            inputFile.browseForOpen('Open', [videoFilter]);
        }

        private function fileSelected(evt:Event = null) {
            inputFile = evt.target as File;

            openButton.visible = false;

            startVideo();
            startFile();

            if(DEBUG_THROTTLE_TIME) {
                startTimer();
            }
        }

        private function updateStatus(statusText:String) {
            statusTextField.text = statusText;
            trace(statusText);
        }

        /*************************
        * 
        *   FILE & VIDEO OPERATIONS
        * 
        **************************/

        private function startVideo() {
            netConnection = new NetConnection();
            netConnection.connect(null);

            netStream = new NetStream(netConnection);

            netStream.client = {};

            // put the NetStream class into Data Generation mode
            netStream.play(null);

            // before appending new bytes, reset the position to the beginning
            netStream.appendBytesAction(NetStreamAppendBytesAction.RESET_BEGIN);

            updateStatus('Video Stream Started, Waiting for Bytes...');
        }

        private function startFile() {
            totalBytesRead = 0;
            readBytes = new ByteArray();
            if(DEBUG_UNUSED_BUFFER) {
                unusedBuffer = new ByteArray();
            }

            inputFileStream = new FileStream();
            inputFileStream.readAhead = READSIZE;
            inputFileStream.addEventListener(ProgressEvent.PROGRESS, fileReadProgress);
            inputFileStream.addEventListener(IOErrorEvent.IO_ERROR,ioError);
            inputFileStream.openAsync(inputFile, FileMode.READ);    
        }

        private function fileReadProgress(evt:ProgressEvent = null) {
            while(inputFileStream.bytesAvailable) {
                inputFileStream.readBytes(readBytes, readBytes.length, inputFileStream.bytesAvailable);
                if(!DEBUG_THROTTLE_TIME) {
                    processData();
                }
            }
        }

        private function processData(evt:TimerEvent = null) {
            var statusString:String;

            if(readBytes.length) {

                if(DEBUG_APPEND_VIDEO) {
                    //Here's where things get funky...
                    netStream.appendBytes(readBytes);
                }


                totalBytesRead += readBytes.length;

                statusString = 'bytes processed now: ' + readBytes.length.toString();
                statusString += '\n total bytes processed: ' + totalBytesRead.toString();
                statusString += '\n percentage: ' + Math.round((totalBytesRead / inputFile.size)  * 100).toString() + '%';

                if(DEBUG_UNUSED_BUFFER) {
                    //Here too....
                    unusedBuffer.writeBytes(readBytes);
                    statusString += '\n Unused Buffer size: ' + unusedBuffer.length.toString();
                }

                updateStatus(statusString);

                readBytes.length = 0;

                if(totalBytesRead == inputFile.size) {
                    fileReadComplete();
                }
            }

        }

        private function fileReadComplete(evt:Event = null) {
            closeAll();
            updateStatus('Finished Reading! Yay!');
        }

        private function ioError(evt:IOErrorEvent) {
            closeAll();
            updateStatus('IO ERROR!!!!');
        }

        /*************************
         * 
         *  TIMER OPERATIONS
         * 
         **************************/

        private function startTimer() {
            throttleTimer = new Timer(DEBUG_THROTTLE_TIME);
            throttleTimer.addEventListener(TimerEvent.TIMER, processData);
            throttleTimer.start();              
        }

        /*************************
         * 
         *  CLEANUP
         * 
         **************************/

        private function closeAll() {

            if(inputFile != null) {
                inputFile.cancel();
                inputFile = null;
            }

            if(inputFileStream != null) {
                inputFileStream.removeEventListener(ProgressEvent.PROGRESS, fileReadProgress);
                inputFileStream.removeEventListener(IOErrorEvent.IO_ERROR,ioError);
                inputFileStream.close();
                inputFileStream = null;
            } 

            if(readBytes != null) {
                readBytes.clear();
                readBytes = null;
            }

            if(unusedBuffer != null) {
                unusedBuffer.clear();
                unusedBuffer = null;
            }

            if(throttleTimer != null) {
                throttleTimer.removeEventListener(TimerEvent.TIMER, processData);
                throttleTimer.stop();
                throttleTimer = null;
            }

            if(netConnection != null) {
                netConnection.close();
                netConnection = null;
            }

            if(netStream != null) {
                netStream.close();
                netStream = null;

            }

            openButton.visible = true;
        }
    }
}

---

UPDATE:

NetStream.seek() will flush the content appended by appendBytes()... in other words, it seems appendBytes() will just keep adding whatever data you throw at it, which makes sense. However- the core of this question still stands...

In theory, I guess calling seek() at every 10 seconds worth of keyframes would work... That is really kindof weird, not at all what "seek" typically is used for, and it would require a whole bunch of manual calculations to make it work right since in Data Generation Mode using seek to continue playing would require calling appendBytesAction(NetStreamAppendBytesAction.RESET_SEEK), and that in turn requires that the next call to appendBytes() needs to begin on the next byte location for an FLV tag (which hopefully exists in the metadata).

Is this the right solution? Adobe Team, is this what you had in mind? Any sample code?!

Help! :)

Answer 1

I'm pretty sure that once data leaves the playout buffer of the NetStream that the memory is released. In your sample you are not attaching the NetStream to a Video object, so I'm wondering if the NetStream is actually playing (and therefore releasing) the bytes you push in.

Try adding a trace on a timer and check netStream.bufferLength. This should be an ever changing value if the video is actually playing out. If bufferLength just goes forever, the bytes are never playing and never released. If you find that happening, try attaching the NetStream to a Video so that the content actually plays and run the same test on bufferLength.

I would also recommend using the Flash Builder Profile or Adobe Scout to watch the memory usage. With NetStream the memory usages should go up and down as bytes are played out and released.

My other thought is that you may be reading bytes in too quickly. You push bytes basically as fast as they are loaded. NetStream cannot play them out that quickly and so the bytes get stuck in memory until it's time to play that data. You could read out the data in chunks; the video should be a set of discrete FLV tags that you can read one by one. You can figure out how long the chunk is (there should be a property of the chunk that tells it's length, or you could figure it out via the timestamps) and then only load more data when you have to.

Lastly, I believe that AIR is still only 32bit. At least that's what some googling is telling me. So that means that it can only get a limited amount of memory from the OS.. So I'm betting that you're hitting the ceiling and crashing the process.

Answer 2

Try this: Check out System.totalMemory to see how much memory AIR is using before it crashes. In theory it should be the same between thetwo use cases, but I bet NetStream is throwing away a lot more bytes sooner.

I suspect the underlying problem is having multiple copies of a 3GB file in memory.