How to create a full-duplex communication between an API and a Rails app?

Question

I have a video transcoding API and a CMS application which consumes it.

Workflow is as follows:

• CMS sends a GET request to to the API to get a of list video files to transcode and it gets the answer in JSON format.
• CMS user selects which file to be transcoded and clicks on the "Transcode" button.
• As user click on the link, a POST request goes to API and API creates necessary Progresses internally and responds with initial status of the progresses in JSON format.
• After creating Progresses at the API, I make GET ajax requests to API asking about the current process status periodically. Because it's a requirement for me to show user current status of the transcoding processes (like percentage..).
• Additionally, I have some other resources that I should inform CMS about the current status.

At this time, I'm making an ajax call to API once in 5 seconds.

Here is the question:

Is this the best way to do it all?

Is there a way to create full-duplex communication between an API and a Rails app?

I tried pusher and push notifications but seems like they're only for front-end communication? Is this true?

I had heard about websocket but I couldn't find a good resource to fully understand it.

What is the approved way of Rails community to do what I ask for?

Thank you

Answer 1

To clarify: you need more than just server -> client communication because you need to inform the user/browser and also other components in the CMS about progress.

Note: not sure what technology the API is written in so I'll assume Ruby

Based on this Faye may be a very good solution. Both the web browser and the CMS server can be Faye clients and can subscribe to updates from the API. The API can publish updates to a channel appropriately named based on the video file that is being transcoded.

Browser <-> API Faye server communication will be over WebSocket or fallback transport. CMS <-> API Faye server comms can be over the same transport types or they can use an Engine such as redis.

• CMS sends a GET request to to the API to get a of list video files to transcode and it gets the answer in JSON format.
• CMS user selects which file to be transcoded and clicks on the "Transcode" button.
• As user click on the link, a POST request goes to API and API creates necessary Progresses internally and responds with initial status of the progresses in JSON format and the channel for progress updates.
• After creating Progresses at the API, the client subscribes to the updates channel. As progress changes the API will publish the update progress to the channel and the client will receive the updates.
• The CMS somehow needs to know the progress channel too. It could do this by having an channel that it always subscribes to. The API can then publish all transcoding information on that channel e.g. video-transcoding. When it sees a new transcoding has started it can subscribe to the channel for specific video transcoding updates.

It may be that bi-directional communication isn't 100% necessary. It seems like the API needs to push updates to the CMS and to the client. If that's the case EventSource/Server-Sent Events may be an option. And if you don't want the CMS to have a persistent connection to the API and would rather have updates pushed to it via HTTP you could add WebHook support to the API; on progression it makes an HTTP request to the CMS to inform it of progress.