How to create a framework in Express.js

Question

I'm trying to figure out how to create a Node.js/Express.js application that is a framework for hosting and running third-party code in my application, and what are the appropriate JavaScript/Node/Express terms for these concepts if they're different from other language I've used in the past (C++/C#/Java).

In developing the back-end for an Android/iOS app using some or all MEAN technologies--at least Express.js and Node.js--I'm getting requirements to set it up as a framework so third-parties can code to a published API to develop plugins/modules for my back-end that I might have running on some server somewhere.

This portion might get outsourced to someone else, in which case I would be responsible for ensuring that I communicate the requirements accurately to the assignee.

I'm relatively new to Node and Express, and I'm having difficulty finding information on how to go about writing code for this if I were to do it myself. When I try to search for information on how to create what I described above, I end up getting hits for Express itself or Node itself. Searching for "Express" and "frameworks", I basically get information on Express itself since Express.js is a framework; similarly searching for "Node" and "modules" basically just gives me tons of NPM information since NPM hosts Node modules.

Coming from a C++/C#/Java background, I envision this application would use the equivalent of something like "dlopen" to manually open the equivalent of a .so, .dll, or ,.jar file, find a class that implements an expected interface, and then call methods on that interface.

If a third-party developer creates a separate plug-in module to run in my application, my understanding is there is no concept of a DLL or JAR in JavaScript, so all that's available would be some special .js file.

If I have ten of these or a hundred of these from numerous third-party developers, as the owner of the back-end, would I be responsible for hooking each of these into my application so that my application runs with all of these--with the end user potentially enabling/disabling/configuring zero or more of these for his/her own account via his/her own app?

I'm wondering if I should account for three separate cases for the outside code:

1. Fully-trusted code developed in-house, so no protection needed. If nothing else, this would help for debugging if ever necessary as a layer of complexity can be removed just to get into the code to get it to run with no frills.

2. Untrusted code from an arbitrary third party, so it might be beneficial to run it in its own dedicated process at the very least. Regarding the badly designed code, we might want to run our in-house modules in this environment in production anyway just for stability so if a module goes down, it doesn't take the entire Express application down with it.

3. Remotely-hosted code. It might be the case that a third party might not want to provide their JavaScript code to us to run, or they might want to develop their module in something other than JavaScript. If something like a REST API is available to them and they expose a fixed REST API, they could host their own code themselves and hook-into my back-end with web service calls.

I'm now wondering if #3 would be the way to go for all cases of #1 and #2 as well. Even for trusted code, simply provide the ability to put in web hooks and stand up separate, compartmentalized services perhaps in isolated Docker containers on the same host for one standardized interface, and don't even bother with "modules" and "plugins".

Answer 1

If I have ten of these or a hundred of these from numerous third-party developers, as the owner of the back-end, would I be responsible for hooking each of these into my application so that my application runs with all of these (...).

In a way. Remember that while a framework is similar to a library (they both provide some core functionality, utilities, etc.) it is entirely a different beast. When you're writing a library it's users' code who is in charge:

function users_function(x) {
    yourlibrary.your_fn(x);
}

When you're writing a framework, it's your code who is in charge of the overall execution:

function users_function(x) {
    // do some interesting stuff...
}

YourFramework.register_module(users_function);

And users do explicitly affect the control flow.

A framework abstracts a typical scenario for a specific domain, e.g. a Web application. You have to model that scenario - define how it is handled under your framework, what steps are performed (and possibly in what order), what data you need on input, and what data you provide as output (if any). After you've done that, allow your users to plug-in at certain points by providing a way to register modules they wrote - your framework is responsible for invoking these user-provided chunks of functionality at appropriate points (somewhat like programming against an event-driven interface, only here you are the source of events - you may find it enlightening to read up on event-driven programming, writing event loops in particular).

So, to answer your question succinctly: no, you will not be responsible for hooking each plugin into your framework (that's the user's job), but you will be responsible for triggering them at appropriate points.