Their technique for generating the HTML representation of deep-link into a Meteor app is to run the entire client app in a headless browser and serialize the generated DOM?!
This is an area of vital importance to public, JS-based RIAs, and needs some real innovation. Why even bother delivering this half-baked solution? The processing cost makes it untenable for all but the tiniest of URL-spaces.
Yeah, it's definitely the kind of solution that makes you twitch a bit. The reason we wrote it is because we needed it now for meteor.com, and it wasn't too much engineering work (I think Nick wrote the first version over a weekend, and then it was a couple days of packaging it up). The reason we released it is, well, why not? :)
The "real solution" is coming, and we will get it right. It's connected to URL routing, and sending down initial HTML on page load. We're all excited for the day when Meteor apps initialize the client session on the server. It will look at first glance like a traditional server-side app. :)
> Why even bother delivering this half-baked solution?
Because we used it for our funding announcement at the end of last month -- a full press cycle, where the Andreessen Horowitz and Matrix Partners press machine pushed our story out to all of the tech blogs, with all of the traffic that that implies -- and not only did it work fine, it got us to #1 on Google for "meteor." Above, you know, actual meteors :)
It works fine for us as a stopgap measure and we wanted to share it with others. We put it in a optional smart package that isn't included in new projects by default.
We're near the end of a major rewrite of Meteor's page update engine. You can see the latest progress on the 'spark' branch. One thing that happened during this rewrite was the conversion of Meteor's templating to be 100% string-based. Check it out: go to meteor.com/faq, open your browser console, and evaluate "Template.faq()".
This means that the server can render the templates for your app without having a DOM implementation of any kind (much less a headless client.) 'spiderable' doesn't do this yet, but it will by Meteor 1.0 (if 1.0 even has a separate package.)
For the record, the ratio of our investment in auth and accounts, to our investment in Spark, is about 2:1.
Never mind the cost. They are proposing that you run your app, with its 3rd-party jquery/facebook/twitter/google code loaded dynamically over non-SSL connections on a platform with filesystem-write-access to your server.
It actually is possible to render those pages in phantom at scale, but it's not very elegant. It seems like something like env.js would be sufficient to render the page, though. I think other people have done that, although I haven't seen a framework that makes it natural. Meteor and others would be in a great position to build such a framework.
Agreed that it needs some real innovation, something better than this. Consider it just a start: the fact that the spiderable package is one of the first ones out means that it is something the Meteor team cares about trying to solve. It's certainly not perfect, but it paves the way for future iterations.
Who really needs this for their web app? Nearly 99% of heavy web apps require a login, so Google is out of the picture anyway.
Anyone who is building a content site with DOM-manipulating Javascript doing all the work have completely lost their way. Seriously, just render your templates on the server and deliver them to the client. Why does the world want app-ify everything?
Fat client apps are fantastic for making some kinds of UI interactions trivial.
The simplest example is the checkout form, or any kind of wizard linking multiple pages together. In a fat client app, all state for all N pages of your checkout cart are in the same page, with 5 lines of code to switch between them. Doing that in standard MVC "fat server" model is annoying, and about 10x more code.
> Why does the world want app-ify everything?
Think about GUI applications pre-web. They weren't written as servers that generate PDFs, with an embedded scripting language. The current webapp technology stack is a complete accident, and if we had actually sat down to design the "optimal" stack, it would look nothing like what we have.
I have a site with lots of statistics, tables, charts and similar stuff. As you can understand, fetching all that data from the database to render the analytical charts is fairly expensive. My solution is to first render the page template and send it back to the client as quickly as possible with all statistics widgets empty. Then the page makes a few ajax calls to fetch the data from the server to render the pie and bar charts widgets.
It works very well for a human visitor. The page loads extremely quickly and the widgets rendered using javascript and ajax are below the fold anyway so it doesn't matter that they are visible 500 ms later than the initial page load. Unfortunately it is crap for Googlebot which never runs the ajax calls and never "sees" my pretty graphs which leads it to think I have a much more boring site than what it really is.
So that is my need to "app-ify" my site and my, as of yet unfulfilled, need for a framework that is able to provide Googlebot with an accurate view of my appified site.
Because server architecture and rendering is really expensive to scale whereas a client-side-dependent app could be supported for much less. It is a trade-off.
The presentation layer has been moving to the client-side for the past few years, where have you been?
Why is this interesting? Because 1) search engine crawlability matters and 2) the more AJAXy web apps get, the harder it is to make them crawlable.
The more we move away from traditional web "pages" to rich web apps that do everything through DOM manipulations on a single page, the harder it is for the search engine robots to crawl what we build.
Google's making moves towards having their crawler essentially be a headless Chrome instance. Crawling AJAXy apps is rapidly going to get easier for Google et al.
So after year of developing this product, they finally realized that each page should have an unique URL.
It's called lack of vision.
ps. ajax content does not rank in Google SERPs at all, it's a typical band aid solution, so websites made with meteor will have some serious issues with monetization and stuff
If they can tell which pages are public vs private (which I think they can) they could just tell the client that they need a copy of the page when they are done rendering it, have it post the serialized DOM back to the server and then cache and serve that until the next redeploy.
[+] [-] mcgwiz|13 years ago|reply
This is an area of vital importance to public, JS-based RIAs, and needs some real innovation. Why even bother delivering this half-baked solution? The processing cost makes it untenable for all but the tiniest of URL-spaces.
[+] [-] dgreensp|13 years ago|reply
The "real solution" is coming, and we will get it right. It's connected to URL routing, and sending down initial HTML on page load. We're all excited for the day when Meteor apps initialize the client session on the server. It will look at first glance like a traditional server-side app. :)
[+] [-] geoffschmidt|13 years ago|reply
Because we used it for our funding announcement at the end of last month -- a full press cycle, where the Andreessen Horowitz and Matrix Partners press machine pushed our story out to all of the tech blogs, with all of the traffic that that implies -- and not only did it work fine, it got us to #1 on Google for "meteor." Above, you know, actual meteors :)
It works fine for us as a stopgap measure and we wanted to share it with others. We put it in a optional smart package that isn't included in new projects by default.
We're near the end of a major rewrite of Meteor's page update engine. You can see the latest progress on the 'spark' branch. One thing that happened during this rewrite was the conversion of Meteor's templating to be 100% string-based. Check it out: go to meteor.com/faq, open your browser console, and evaluate "Template.faq()".
This means that the server can render the templates for your app without having a DOM implementation of any kind (much less a headless client.) 'spiderable' doesn't do this yet, but it will by Meteor 1.0 (if 1.0 even has a separate package.)
For the record, the ratio of our investment in auth and accounts, to our investment in Spark, is about 2:1.
[+] [-] jcoglan|13 years ago|reply
[+] [-] tomku|13 years ago|reply
[+] [-] mnutt|13 years ago|reply
[+] [-] audreyr|13 years ago|reply
[+] [-] trung_pham|13 years ago|reply
[+] [-] jarcoal|13 years ago|reply
Anyone who is building a content site with DOM-manipulating Javascript doing all the work have completely lost their way. Seriously, just render your templates on the server and deliver them to the client. Why does the world want app-ify everything?
[+] [-] arohner|13 years ago|reply
The simplest example is the checkout form, or any kind of wizard linking multiple pages together. In a fat client app, all state for all N pages of your checkout cart are in the same page, with 5 lines of code to switch between them. Doing that in standard MVC "fat server" model is annoying, and about 10x more code.
> Why does the world want app-ify everything?
Think about GUI applications pre-web. They weren't written as servers that generate PDFs, with an embedded scripting language. The current webapp technology stack is a complete accident, and if we had actually sat down to design the "optimal" stack, it would look nothing like what we have.
[+] [-] bjourne|13 years ago|reply
It works very well for a human visitor. The page loads extremely quickly and the widgets rendered using javascript and ajax are below the fold anyway so it doesn't matter that they are visible 500 ms later than the initial page load. Unfortunately it is crap for Googlebot which never runs the ajax calls and never "sees" my pretty graphs which leads it to think I have a much more boring site than what it really is.
So that is my need to "app-ify" my site and my, as of yet unfulfilled, need for a framework that is able to provide Googlebot with an accurate view of my appified site.
[+] [-] mattacular|13 years ago|reply
The presentation layer has been moving to the client-side for the past few years, where have you been?
[+] [-] audreyr|13 years ago|reply
The more we move away from traditional web "pages" to rich web apps that do everything through DOM manipulations on a single page, the harder it is for the search engine robots to crawl what we build.
[+] [-] ceejayoz|13 years ago|reply
[+] [-] mortice|13 years ago|reply
[+] [-] scottkrager|13 years ago|reply
[+] [-] Xyzodiac|13 years ago|reply
[+] [-] debergalis|13 years ago|reply
https://github.com/meteor/meteor/wiki/Getting-Started-with-A...
[+] [-] mikebannister|13 years ago|reply
[+] [-] encoderer|13 years ago|reply
[+] [-] wavephorm|13 years ago|reply
[deleted]
[+] [-] dotborg2|13 years ago|reply
It's called lack of vision.
ps. ajax content does not rank in Google SERPs at all, it's a typical band aid solution, so websites made with meteor will have some serious issues with monetization and stuff
[+] [-] spullara|13 years ago|reply
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] AznHisoka|13 years ago|reply