Principles of an extensible data proxy

Question

There is a growing industry now with more than 30 companies playing in the Backend-As-A-Service (BaaS) market.

The principle is simple: give companies a secure way of exposing data housed on premises and behind the firewall publicly.

This can include database data, as well as Legacy PC data through established connectors; SAP for example provides a connector for transacting with their legacy systems.

Early attempts were fixed providers for specific systems like SAP, IBM or Oracle, but the new breed is extensible, allowing Channel Partners and Consultants to build robust integration applications that can consume whatever data sources the client wants to expose.

I just happen to be close to finishing a Cloud Based HTML5 application platform that provides robust integration services, and I would like to break ground on an extensible data proxy to complete the system.

From what I can gather, I need to provide either an installable web service of some kind, or a Cloud service which the client can configure with VPN for interactions.

Then I can build in connectors, which can be activated with a service account, and expose those transactions via web services of some kind (JSON, SOAP, etc).

I can also provide a framework that allows people to build in their own connectors, and use some kind of schema to hook those connectors into the proxy.

The end result is some kind of public facing web service that could securely be consumed by applications to show data through HTML5 on any device.

My gut is, this isn't as hard as it sounds. Almost all of the 30+ companies (With more popping up almost weekly) have all come into existence in the last 18 months or so, which tells me either the root technology, or the skillset to create the technology is in abundance right now.

Where should I start on this? Are there some open source projects I can leverage? A specific group of developers I can hire? I'm confident someone here can set me on the right path and save me some time.

You don't see this many companies spring up this rapidly if they are all starting from scratch with proprietary technology.

The Register: WTF is BaaS

One Minute Video from Kony on their BaaS

Answer 1

Have a look at the Usergrid stack, maintained (and subsequently owned) by Apigee. Apigee actually does most of this already. https://github.com/apigee/usergrid-stack

You may find yourself diving into a world of competition where the competitors have more capital and more resources than you, which would make it a difficult start.

What sets these large providers apart from the smaller firms, is that they can (and do) handle the support, scalability, and security concerns that are critical to a successful BaaS.

Answer 2

The principle is simple: give companies a secure way of exposing data housed on premises and behind the firewall publicly.

From the definition of Back end as a Service in one of this article that you mentioned

a backend-as-a-service gives you an API into a common 
infrastructure for sending data between applications

So in principle, BaaS is simply providing APIs to allow applications to communicate with each other. Not necessarily restricted to items behind corporate firewalls but even items available publicly, look at firebase for example. Firebase doesn't do the authentication of users, another part of your app does that. What firebase does do is to provide your application with an api for real time communication with other instances of your application as well as a real-time database storage service.

The end result is some kind of public facing web service that could securely be consumed by applications to show data through HTML5 on any device. My gut is, this isn't as hard as it sounds.

The difficulty lies on what kind of communication to what kind of things will you be offering and how general purpose or specialized it is. In Meteor's case, it doesn't seem too complicated to do what its doing, very well defined and specialized to connections with other social media/social network apis. In Firebase' case, they had to make a general purpose real time communication API from ground up(though they heavily made use of nodejs), that should speak for itself.

Ok, so back to your question.

Where should I start on this?

You could start by defining the specifics of your BaaS. Specifically, what kind of communication are you offering. Who will communicate with who, communication security etc.

Are there some open source projects I can leverage?

Open source projects? Sadly, I got none.

A specific group of developers I can hire?

I believe API developers would be a good fit here. They should be able to create ways for your end points to communicate with each other for the sake of your BaaS