Finally this could be solved. I am posting the answer here if it could help someone. We did lot of research to find that for streaming the urls in our case we had to explicitly provide the domain in the Origin: header of CORS. For example: Normally while adding CORS support we provide Origin header as follows
Access-Control-Allow-Origin: *
which means that the resource can be accessed by any domain in a cross-site manner.
But may be due to Streaming CDNs restricting the source of videos to be from particular domains, we couldn't access it with the above setting. On changing the Origin in the CORS header to the domain name where the videos were hosted, We could finally stream the videos.
EX: If the resource owners wished to restrict access to the resource to be only from http://foo.example, they would send back:
Access-Control-Allow-Origin: http://foo.example
in the CORS header.
This might not be the case in all the scenarios but was in our case.