To clarify: you need more than just server -> client communication because you need to inform the user/browser and also other components in the CMS about progress.
Note: not sure what technology the API is written in so I'll assume Ruby
Based on this Faye may be a very good solution. Both the web browser and the CMS server can be Faye clients and can subscribe to updates from the API. The API can publish updates to a channel appropriately named based on the video file that is being transcoded.
Browser <-> API Faye server communication will be over WebSocket or fallback transport. CMS <-> API Faye server comms can be over the same transport types or they can use an Engine such as redis.
- CMS sends a GET request to to the API to get a of list video files to transcode and it gets the answer in JSON format.
- CMS user selects which file to be transcoded and clicks on the "Transcode" button.
- As user click on the link, a POST request goes to API and API creates necessary Progresses internally and responds with initial status of the progresses in JSON format and the channel for progress updates.
- After creating Progresses at the API, the client subscribes to the updates channel. As progress changes the API will publish the update progress to the channel and the client will receive the updates.
- The CMS somehow needs to know the progress channel too. It could do this by having an channel that it always subscribes to. The API can then publish all transcoding information on that channel e.g.
video-transcoding
. When it sees a new transcoding has started it can subscribe to the channel for specific video transcoding updates.
It may be that bi-directional communication isn't 100% necessary. It seems like the API needs to push updates to the CMS and to the client. If that's the case EventSource/Server-Sent Events may be an option. And if you don't want the CMS to have a persistent connection to the API and would rather have updates pushed to it via HTTP you could add WebHook support to the API; on progression it makes an HTTP request to the CMS to inform it of progress.