What is a good approach to develop a synchronous/blocking and an asynchrounous/non-blocking library-api in parallel? (JavaScript)

StackOverflow https://stackoverflow.com/questions/20557857

문제

I rewrote this question, because the old version was obviously misleading. Please read the text and make shure you understood what I'm asking for. If there is still anything left in the dark I'll modify this question for clarity. Just inform me.

One of my projects is to port a library from Python to JavaScript. The Python library is entirely blocking/synchronous when it comes to I/O and such. This is of course perfectly normal for Python code.

I plan to port the synchronous/blocking methods as they are to JavaScript. This has several reasons and whether or not it's worth the effort is a good but different question.

Additionally I wan't to add an asynchronous/non-blocking api.

Think of it like the fs module in node where there are i.e. fs.open and fs.openSync coexisting.

The library is pure JavaScript and will run in Node and in the Browser.

The question is what a good/the best approach for the development of these two coexisting APIs would be.

I believe its good to have the same thing happening in one place only. Hence an approach where some parts of the implementation could be shared would be preferable. Not at any price of course, that's why I'm asking.

I had a proposal for an approach in here, but I'm going to post it as a possible answer. However, I'm waiting for some serious discussion to happen before I decide what I accept as an answer.

So far approaches are:

  • implement both apis separately and definetly use promises for the asynchronous functions.
  • use something like the obtain api proposal - beeing a more integrated approach
도움이 되었습니까?

해결책 3

I implemented a library that does what I'm asking for ObtainJS.

(Yes, the Library uses Promises BUT not as others proposed in their ansewers here)

Reposting the Readme.md:

ObtainJS

ObtainJS is a micro framework to bring together asynchronous and synchronous JavaScript code. It helps you to Don't Repeat Yourself (DRY) if you are developing a library with interfaces for both blocking/synchronous and non-blocking/asynchronous execution models.

As a USER

of a library that was implemented with ObtainJS you won't have to learn a lot. Typically a function defined using ObtainJS has as first argument the switch, that lets you choose the execution path, followed by its normal arguments:

// readFile has an obtainJS API:
function readFile(obtainAsyncExecutionSwitch, path) { /* ... */ }

execute synchronously

If the obtainSwitch is a falsy value readFile will execute synchronously and return the result directly.

var asyncExecution = false, result;
try {
    result = readFile(asyncExecution, './file-to-read.js');
} catch(error) {
    // handle the error
}
// do something with result

execute asynchronously

If the obtainSwitch is a truthy value readFile will execute asynchronously and always return a Promise.

See Promises at MDN

var asyncExecution = true, promise;

promise = readFile(asyncExecution, './file-to-read.js');
promise.then(
    function(result) {
        // do something with result
    },
    function(error){
        // handle the error
    }
)

// Alternatively, use the returned promise directly:

readFile(asyncExecution, './file-to-read.js')
    .then(
        function(result) {
            // do something with result
        },
        function(error){
            // handle the error
        }
    )

You can use a callback based api, too. Note that the Promise is returned anyways.

var asyncExecution;

function unifiedCallback(error, result){
    if(error)
        // handle the error
    else
        // do something with result
}

asyncExecution = {unified: unifiedCallback}

readfile(asyncExecution, './file-to-read.js');

or with a separate callback and errback

var asyncExecution;

function callback(result) {
    // do something with result
}

function errback(error) {
    // handle the error
}

var asyncExecution = {callback: callback, errback: errback}
readfile(asyncExecution, './file-to-read.js');

```

As a smart ;-) LIBRARY AUTHOR

who's going to implement a API using with ObtainJS the work is a bit more. Stay with me.

The behavior above is achieved by defining a twofold dependency tree: one for the actions of the synchronous execution path and one for the actions of the asynchronous execution path.

Actions are small functions with dependencies on the results of other actions. The asynchronous execution path will fallback to synchronous actions if there is no asynchronous action defined for a dependency. You wouldn't define an asynchronous action if its synchronous equivalent is non-blocking. This is where you DRY!

So, what you do, for example, is splitting your synchronous and blocking method in small function-junks. These junks depend on the results of each other. Then you define a non-blocking AND asynchronous junk for each synchronous AND blocking junk. The rest does obtainJS for you. Namely:

  • creating a switch for synchronous or asynchronous execution
  • resolving the dependency tree
  • executing the junks in the right order
  • providing you with the results via:
    • return value when using the synchronous path
    • promises OR callbacks (your choice!) when using the asynchronous path

Here is the readFile function

from above, taken directly from working code at ufoJS

define(['ufojs/obtainJS/lib/obtain'], function(obtain) {

    // obtain.factory creates our final function
    var readFile = obtain.factory(
        // this is the synchronous dependency definition
        {
            // this action is NOT in the async tree, the async execution
            // path will fall back to this method
            uri: ['path', function _path2uri(path) {
                return path.split('/').map(encodeURIComponent).join('/')
            }]
            // synchronous AJAX request
          , readFile:['uri', function(path) {
                var request = new XMLHttpRequest();
                request.open('GET', path, false);
                request.send(null);

                if(request.status !== 200)
                    throw _errorFromRequest(request);

                return request.responseText;
            }]
        }
      ,
      // this is the asynchronous dependency definition
      {
            // aynchronous AJAX request
            readFile:['uri', '_callback', function(path, callback) {
                var request = new XMLHttpRequest()
                  , result
                  , error
                  ;
                request.open('GET', path, true);

                request.onreadystatechange = function (aEvt) {
                    if (request.readyState != 4 /*DONE*/)
                        return;

                    if (request.status !== 200)
                        error = _errorFromRequest(request);
                    else
                        result = request.responseText
                    callback(error, result)
                }
                request.send(null);
            }]
        }
      // this are the "regular" function arguments
      , ['path']
      // this is the "job", a driver function that receives as first
      // argument the obtain api. A method that the name of an action or
      // of an argument as input and returns its result
      // Note that job is potentially called multiple times during
      // asynchronoys execution
      , function(obtain, path){ return obtain('readFile'); }
    );
})

a skeleton

var myFunction = obtain.factory(
    // sync actions
    {},
    // async actions
    {},
    // arguments
    [],
    //job
    function(obtain){}
);

action/getter definition

// To define a getter we give it a name provide a definition array.
{
    // sync

    sum: ['arg1', 'arg2',
    // the last item in the definition array is always the action/getter itself.
    // it is called when all dependencies are resolved
    function(arg1, arg2) {
        // function body.
        var value = arg1 + arg2
        return value
    }]
}

// For asynchronous getters you have different options:
{
    // async

    // the special name "_callback" will inject a callback function
    sample1: ['arg1',  '_callback', function(arg1, callback) {
            // callback(error, result)
        }],
    // you can order separate callback and errback when using both special
    // names "_callback" and "_errback"
    sample2: ['arg1',  '_callback', '_errback', function(arg1, callback, errback) {
            // errback(error)
            // callback(result)
        }],
    // return a promise
    sample3: ['arg1', function(arg1) {
            var promise = new Promise(/* do what you have to*/);

            return promise
        }]
}

The items in the definition array before the action are the dependencies their values are going to be injected into the call to action, when available.

If the type of an dependency is not a string: It's injected as a value directly. This way you can effectively do currying.

If the type of the value is a string: It's looked up in the dependency tree for the current execution path(sync or async).

  • If its name is defined as an caller-argument (in the third argument of obtain.factory) the value is taken from the invoking call.
  • If its name is defined as the name of another action, that action is executed and its return value is used as a parameter. An action will executed only once per run, later invocations will return a cached value.
    • If the execution path is asynchronous obtain will first look for a asynchronous action definition. If that is not found it falls back to a synchronous definition.

If you wish to pass a String as value to your getter you must define it as an instance of obtain.Argument: new obtain.Argument('mystring argument is not a getter')

A more complete example

from ufoLib/glifLib/GlyphSet.js

Note that: obtainJS is aware of the host object and propagates this correctly to all actions.

    /**
     * Read the glif from I/O and cache it. Return a reference to the
     * cache object: [text, mtime, glifDocument(if alredy build by this.getGLIFDocument)]
     *
     * Has the obtainJS sync/async api.
     */
GlypSet.prototype._getGLIFcache = obtain.factory(
    { //sync
        fileName: ['glyphName', function fileName(glyphName) {
            var name = this.contents[glyphName];
            if(!(glyphName in this.contents) || this.contents[glyphName] === undefined)
                throw new KeyError(glyphName);
            return this.contents[glyphName]
        }]
      , glyphNameInCache: ['glyphName', function(glyphName) {
            return glyphName in this._glifCache;
        }]
      , path: ['fileName', function(fileName) {
            return [this.dirName, fileName].join('/');
        }]
      , mtime: ['path', 'glyphName', function(path, glyphName) {
            try {
                return this._io.getMtime(false, path);
            }
            catch(error) {
                if(error instanceof IONoEntryError)
                    error = new KeyError(glyphName, error.stack);
                throw error;
            }
        }]
      , text: ['path', 'glyphName', function(path, glyphName) {
            try {
                return this._io.readFile(false, path);
            }
            catch(error) {
                if(error instanceof IONoEntryError)
                    error = new KeyError(glyphName, error.stack);
                throw error;
            }
        }]
      , refreshedCache: ['glyphName', 'text', 'mtime',
        function(glyphName, text, mtime) {
            return (this._glifCache[glyphName] = [text, mtime]);
        }]
    }
    //async getters
  , {
        mtime: ['path', 'glyphName', '_callback',
        function(path, glyphName, callback) {
            var _callback = function(error, result){
                if(error instanceof IONoEntryError)
                    error = new KeyError(glyphName, error.stack);
                callback(error, result)
            }
            this._io.getMtime({unified: _callback}, path);
        }]
      , text: ['path', 'glyphName', '_callback',
        function(path, glyphName, callback){
            var _callback = function(error, result) {
                if(error instanceof IONoEntryError)
                    error = new KeyError(glyphName, error.stack);
                callback(error, result)
            }
            this._io.readFile({unified: _callback}, path);
        }
      ]
    }
    , ['glyphName']
    , function job(obtain, glyphName) {
        if(obtain('glyphNameInCache')) {
            if(obtain('mtime').getTime() === this._glifCache[glyphName][1].getTime()) {
                // cache is fresh
                return this._glifCache[glyphName];
            }
        }
        // still here? need read!
        // refreshing the cache:
        obtain('refreshedCache')
        return this._glifCache[glyphName];
    }
)

다른 팁

If you're talking I/O in node.js then most I/O methods have a synchronous version.

There is no direct conversion from Asynchronicity To Synchronicity. I can think of two approaches:

  1. Have each asynchronous method run a polling loop waiting for the async task to complete before returning.
  2. Drop the idea of mimicking synchronous code and instead invest in better coding patterns (such as promises)

To illustrate I will assume option 2 is a better choice. The following example uses Q promises (easily installed with npm install q.

The idea behind promises is that although they are asynchronous the return object is a promise for a value as if it was a normal function.

// Normal function
function foo(input) {
  return "output";
}

// With promises
function promisedFoo(input) {
  // Does stuff asynchronously
  return promise;
}

The first function takes an input and returns a result. The second example takes an input and immediately returns a promise which will eventually resolve to a value when the async task finishes. You then manage this promise as follows:

var promised_value = promisedFoo(input);
promised_value.then(function(value) {
  // Yeah, we now have a value!
})
.fail(function(reason) {
  // Oh nos.. something went wrong. It passed in a reason
});

Using promises you no longer have to worry when something will happen. You can easily chain promises so things happen synchronously without insane nested callbacks or 100 named functions.

It well worth learning about. Remember promises are meant to make async code behave like sync code even though it isn't blocking.

Write lower level API using promises that takes async/sync flag.

Higher level async API returns these promises directly (while also working with async callbacks like it's 1970).

Higher level sync API unwraps the value synchronously from the promise and returns the value or throws the error.

(Examples use bluebird which is orders of magnitude faster and has more features at the cost of file size compared to Q, although that might not be ideal for browsers.)

Low level api that is not exposed:

//lowLevelOp calculates 1+1 and returns the result
//There is a 20% chance of throwing an error

LowLevelClass.prototype.lowLevelOp = function(async, arg1, arg2) {
    return new Promise(function(resolve, reject) {
        if (Math.random() < 0.2) {
            throw new Error("random error");
        }
        if (!async) resolve(1+1);
        else {
        //Async
            setTimeout(function(){
                resolve(1+1);
            }, 50);
        }
    });
};

High level exposed API that works synchronously, using promises or callbacks:

HighLevelClass.prototype.opSync = function(arg1, arg2) {
    var inspection = 
        this.lowLevel.lowLevelOp(false, arg1, arg2).inspect();

    if (inspection.isFulfilled()) {
        return inspection.value();
    }
    else {
        throw inspection.error();
    }
};

HighLevelClass.prototype.opAsync = function(arg1, arg2, callback) {
    //returns a promise as well as accepts callback.
    return this.lowLevel.lowLevelOp(true, arg1, arg2).nodeify(callback);
};

You can automatically generate the high level api for synchronous methods:

var LowLevelProto = LowLevelClass.prototype;

Object.keys(LowLevelProto).filter(function(v) {
    return typeof LowLevelProto[v] === "function";
}).forEach(function(methodName) {
    //If perf is at all a concern you really must do this with a 
    //new Function instead of closure and reflection
    var method = function() {
        var inspection = this.lowLevel[methodName].apply(this.lowLevel, arguments);

        if (inspection.isFulfilled()) {
            return inspection.value();
        }
        else {
            throw inspection.error();
        }
    };

    HighLevelClass.prototype[methodName + "Sync" ] = method;
});
라이센스 : CC-BY-SA ~와 함께 속성
제휴하지 않습니다 StackOverflow
scroll top