Render HTML via WebGL

[SoapSeller]

This project(at least for me) is a little bit misleading. Because it isn't rendering HTML using WebGL.

It using html2canvas[0] to rasterize the browser rendering into a canvas, upload it to WebGL, apply effects and patch-back events(i.e. mouse clicks) back to the hidden browser dom elements. (On the other side, font hinting and such are not really a problem)

Implementing full UI framework over WebGL has it merits, and hopefully someone will make a useful(and not html+css based) one some day.

[0] https://github.com/niklasvh/html2canvas

[macspoofing]

>Implementing full UI framework over WebGL has it merits, and hopefully someone will make a useful(and not html+css based) one some day

I'm surprised it hasn't been done already (in similar spirit to the Feathers framework for Flash's Stage3D). Now that WebGL is supported in Mobile WebView, maybe you'll see a really good component set released? That would give devs a really slick option for building smooth phone-gap based apps.

[fapjacks]

Yet another instance of html2canvas being used to do something interesting. It's the library I have been surprised by more than any other, for any language.

[Polarity]

u know http://famo.us ?

[phn]

Make it feature complete and you'll get the same speed the browser has.

If we're going to build a new rendering engine for the web, maybe we could try to create something nicer than HTML and CSS, whose only advantage at the present is the number of developers familiar with it. I know I wouldn't miss them.

[paraboul]

FWIW, I'm working in this : http://www.nidium.com/ since a few years. (It's almost ready and about to be released)

Previous HN thread : https://news.ycombinator.com/item?id=6314961

video : https://www.youtube.com/watch?t=63&v=BSzEZXUTXlQ

[perdunov]

Yep, the model of HTML-CSS-DOM-JS implies extreme overhead, and only heroic efforts and thousands of man-years of work and optimization have made it bearable.

I'm saying this as I have compared the performance of a simple local web page with a simple view with the same exact functionality rendered with my pet naive unoptimized UI engine and the latter was blazingly fast compared to HTML.

So I guess you are right, we definitely need something less cluttered and more efficient than HTML+CSS+JS.

[PixelsCommander]

It is not a new rendering engine. It is just one custom tag which makes WebGL texture of your HTML content and allows to apply shaders. That is it...

[macNchz]

>HTML and CSS, whose only advantage at the present is the number of developers familiar with it

Given the sheer number of people familiar with it, I'd say this is a pretty huge advantage.

[PixelsCommander]

The only bottleneck here is rasterization which we are trying to solve by making native Rasterization API live. You can read about here http://pixelscommander.com/en/javascript/state-of-html-conte...

[macspoofing]

>Make it feature complete

That's the opposite direction you want to go. At that point, you might as well just use HTML. I think there are benefits to providing a subset of HTML features that you can tailor for super fast performance.

[shimonamit]

Not necessarily. Looks like it is an immediate mode, HTML-like syntax but without the DOM.

[emodendroket]

What is there that's so much better?

[aout]

Wait wait wait... Handling textures via CG Buffers is quite fast you are right and you can basically do whatever you want in terms of image manipulation using the shaders but you'll lose efficiency as soon as you introduce complex picking (text selection for example) and / or computations (aliasing etc...) between JS and your pseudo-DOM.

In the end the only difference you'll have compared to the real browser renderer is that you do less computations from CG(GL) to RAM (JS), hence the performance.

You can check this very old project of mine where we tried to link DOM and WebGL Nodes: https://github.com/aout/SAGE

edit: just saw the repo wasn't entirely up to date. Done now.

[bsenftner]

Could you speak a bit more on the issues you encountered?

[PixelsCommander]

Nice, we may use it as a renderer for HTML GL. Shaders and fast rendering wirth it I believe. Try the demo in the bottom http://htmlgl.com/

[pierrec]

Very cool. This could be useful for some applications, but I don't see it becoming a thing for general use cases unless it's implemented and exposed directly by the browsers.

Even then, the main problem is that the internet is still mostly text, and if you can't implement or access font hinting, your text will be blurry and ugly on any display that isn't ultra-high DPI. You can't really render text to a texture and then manipulate that texture with sub-pixel precision. It must be done the other way around: text rendering must be done last, and with the help of a hinting library so it will look sharp and readable.

[PixelsCommander]

There is an intent to make rasterization native http://pixelscommander.com/en/javascript/state-of-html-conte...

[methyl]

So we have browser in the browser, we have to go deeper...

[PixelsCommander]

It is basicaly not. Just converting HTML to texture with some tricky proxying. The reason is to be able to do that http://pixelscommander.com/polygon/htmlgl/demo/filters.html

[moron4hire]

This creates an interesting change in how applications are deployed to users. Instead of the user choosing the HTML rendering engine, the developer now gets to choose it. Instead of having to design sites with multiple rendering engines in mind, this makes it possible to manage only one. And it's not like the user cares that content is being rendered by Firefox or Chrome or whatever, the user just cares to see the page, leaving the selection of the actual rendering engine only of concern to the developer.

I think that's important enough to warrant going through the paces and implementing this. Don't listen to the naysayers. Reinvent wheels. The wheels we currently have largely suck.

[perdunov]

C++ with asm.js + WebGL effectively bring back desktop programming to the web.

[slaction]

"I really wish I could use bump mapping on this div" - No one ever

[bsenftner]

Really great work. I've had in the back of my mind that this was possible, but now you've gone and done it, and quite well!

I'm curious if there would be any conflict with usage of Three.js on the same site? Could they end up clobbering each other's textures? (I've not gotten into webgl's texture memory management, so that question is a stab in the dark.)

[danbruc]

This was at least kind of possible (not sure if it worked only for images or also other elements) with Internet Explorer 4 [1] in 1997, almost two decades ago. It was obviously non-standard.

[1] https://msdn.microsoft.com/en-us/library/ms532849.aspx

[mandeepj]

Then all the hard work that flipboard did to achieve 60 fps rendering speed was not required?

Reference - https://news.ycombinator.com/item?id=9029159

[bsenftner]

Would forms and input fields still work? Could one push their wysiwyg rich text editor into this? (I'm thinking game / VR content where an in-3D-world web browser on some in-3D-world device screen is fully operational.)

[PixelsCommander]

It depends on how far it will go. At the moment there is a plan how to do that, but not enough time to implement... You may use it for editor, but keep in mind that text itself would be sent to GPU every time it updates. Anyhow you may ping me in skype lp_funky if interested in doing this, would be glad to help.

[cturhan]

What if we merge it with React and use its virtual Dom management?

[PixelsCommander]

I think could be nice. There is an intention to create React-GL, but nothing serious yet. All you basically need is to call something like new GLElement(this.getDOMNode()). Or how is it in 0.13?...

[josephmx]

Very minor bug on the demo: The alert on click reads "clicked X" (eg, "Clicked Avatar") on all of the titles except for Quantum of Solace

[heldrida]

Holy cow! This is a great idea : D the responsiveness is a bit sloppy but it's really good you guys have it working! haha really cool : D

[PixelsCommander]

Thanks! We are working on improving responsiveness a bit. Will be better.

[kolodny]

fiddle: http://jsfiddle.net/409gxzm6/

[bshimmin]

The text is blurred on my retina MacBook Pro. (And not selectable, as others have mentioned.)

[jeremiep]

It looks great, but I don't like the fact that I can't select the text anymore!

[kluck]

Recreating existing desktop GUI frameworks one useless HTML/JS library at a time.

[PixelsCommander]

Come on http://pixelscommander.com/polygon/htmlgl/demo/filters.html

[angersock]

Massively underrated post, honestly.

I wonder what exactly the use of this is, beyond Compiz for web pages. What's next, jiggly divs? :P

moron4hire made a good point about letting devs pick the layout engine, but I am not sure we want to ignore all the nice features browsers give us here.

[MrDosu]

I don't get why people do this without looking at least once at all the existing open source html rendering implementations.

Also, css shaders.

[outworlder]

Now we need a WebGL demoscene.

[theandrewbailey]

There is. It's been around since WebGL was a thing.

http://www.pouet.net/prodlist.php?platform[]=JavaScript&page...

[PixelsCommander]

It is already there https://www.chromeexperiments.com/webgl , but it starts to be even more exciting if you think about effects which could be applied to web sites and HTML content... Bumb mapping, reflections, motion blur, elastic elements when dragging. Whole bunch of shaders https://www.shadertoy.com/browse

[MrDosu]

Demoscene is not about flashy graphics. It's about doing something beautiful with code (this includes the way the code uses the machine to do things that go beyond, like mode 13h).

[perdunov]

As far as I understand, this is roughly how iOS UI works (UIView - CALayer).

Obviously, this can be done at the browser engine level.

I have also tried to do this in my pet UI engine, but found out that it was tricky to figure out caching rules to avoid memory blowup and performance degradation for all possible cases.

[designml]

Ahh Nice! I was going to start working on something similar to this.