Accessing GPU via web browser

Question

I came across this proof of concept earlier today (on TechCrunch.com) and was blown away and intrigued as to how they had managed to accomplish the end result. They state that they don't use webGL or any plugins yet they are able to interact directly with the GPU and render 3D visuals at up to 60 fps using nothing but Javascript. Any ideas how this could be done, or how to access the GPU from Javascript in general without the use of plugins?

Site Address is: famo.us

ps: Try using your arrow keys to shift orientation its far out!!

Answer 1

They're using standard HTML5 Javascript APIs to achieve this.

I saw several references to requestAnimationFrame in their code. This method allows one that uses a compatible browser, to display more fluid and optimized animations, with far better frame rates than setInterval based animations will ever allow you to have. This is certainly achieved by using the GPU and the available hardware background.

GPU or every other hardware component cannot be accessed directly using any Javascript routines. Instead, the browser, based on the called JS directives and the execution context, will use the GPU if possible to optimize some specific treatments, calculus and renderings.

EDIT

For future references, I recently found out (7 years after the original answer) that a new initiative - the W3C GPU for the Web Community Group, has recently been created in 2020 to do just that. It describes itself as follow.

The mission of the GPU on the Web Community Group is to provide an interface between the Web Platform and modern 3D graphics and computation capabilities present in native system platforms. The goal is to design a new Web API that exposes these modern technologies in a performant, powerful and safe manner. It should work with existing platform APIs such as Direct3D 12 from Microsoft, Metal from Apple, and Vulkan from the Khronos Group. This API will also expose the generic computational facilities available in today's GPUs to the Web, and investigate shader languages to produce a cross-platform solution.

In the long-term, this initiative might allow developers to directly interact with the GPU from all web browsers. You can track the implementation status of the WebGPU API Spec on Github here.

Answer 2

Concerning famo.us: they did analysed the bottlenecks of the Webkit rendering pipe and then found a way to bypass them while building the pages. Basically: the DOM tree construction, the Render tree construction, the Layout of Render Tree are bypassed. Take a look on this article for a whole explanation.

Answer 3

They're using CSS 3D transforms. Web browsers are increasingly using hardware acceleration to do these kinds of CSS3 things.

Answer 4

i think the webGL glsl.js library might be good for this, though i havnt seen benchmarks... https://github.com/gre/glsl.js/

also this approach seems viable;

basically to use the gpu in the way we like to, hardware optimised functions (have a little look into "blas") are used, you do not want to write these! strangely it seems that people still use the old fortran blas.... there is some stuff with compiling via llvm and then using emscripten to turn it into javascript. Use Emscripten with Fortran: LAPACK binding

the emscripten way seems the most versatile, im just about to check it out, but it looks like a mountain, also this topic seems to be somewhat of a call to arms, emscripten only works with fortran if you hack it (see links from the second link). i think what we are looking for is blas support in javascript, this is not a closed issue by any means, and for a few of us out here it is very frustrating! hopefully someone has a link to these blas libraries we cant find... please let me know if i dont quite have my head round this issue, js is new to me.

also, suggesting html5 is sufficiently "optimised" to use the hardware, how can we trust this!? we need gpu optimised blas (basic linear algebra subprograms) e.g. dot product.

i think maybe also that these old fortran blas arnt actually the right thing for a modern gpu? node-cuda seems very likely the best thing i have found...