Unlike a lot of comments here, I don't need to be sold on the concept. As luck would have it, I am actively building an app that runs mostly in the browser using CouchDB, Kanso, and PouchDB. It's a pretty amazing tech stack; I get a local DB in the browser, which I can sync with the main DB in the cloud the client is connected. Plus I get some nice APIs for stuff like sessions, registering users, security, validation, etc.
I see you've already mentioned switching to PouchDB on your backend, and you're already using CouchDB for the cloud, so it seems like you're basically competing straight up against Kanso. So.... How is what you're doing different from what Kanso does? Why should I use Hoodie over Kanso?
users.create('testuser', 'testing', {roles: ['example']}, function (err){});
session.login('testuser', 'testing', function (err, response){}});
Seems like six of one, half a dozen of the other. What's your killer feature? Kanso has shaky documentation and is currently more aimed at couchapps (they strongly assume that you'll serve your frontend code from CouchDB), but I'm not sure how that really differs from Hoodie. :)
If we take the overall architecture as a given (browser based app, PouchDB or equivalent for local storage, syncing with CouchDB in the cloud, and node workers handling the backend work), what's the focus of Hoodie that sets it apart from Kanso?
(In any case, nice stuff. I think the concept is great, and there's definitely room for several competing frameworks in this space.)
Spot on! We stole a lot of ideas from the whole CouchDB application space. @caolan, the Kan.so dev is currently looking into Hoodie as well.
As far as originality, we choose a different design approach. We want to cater to people who barely get jQuery and give them a tool to build larger apps. We are building the technology downwards from that idea towards the bits and pieces we have.
One of the fun things about Hoodie is that there is nothing technically new in it. Existing ideas and tools are just put together a different way.
I am trying very hard not to be dumb, but this is beating me
seemingly Kanso? and Hoodie both have JS stubs at the client and server end, these synch with each other, then synch with whatever you are using at client (say backbone) and server (node? CouchDB? My Postgres server behind my python app?)
If thats roughly right, why do I do this? I control my server apps no? Why wrap them in another layer? Json->backbone is hardly a difficult synch operation - I mean its called synch().
I really am interested in understanding the philosophy and driving ideas here - everyone has a moment of clarity where they can see how to build an app or a framework, and how it will come together - I am not attacking you (yet !) - please describe your moment of clarity.
Key difference: kan.so is Couch specific. Hoodie is not. We do not try to make another nice library to wrap CouchDB. We try to make a nice JavaScript API for the most common backend tasks.
Forgive my ignorance, but CouchDB does not run in the browser, right? So am I correct that the "mostly in the browser" statement doesn't apply to that particular data store?
Looks lovely, but I wish people would focus less on fast development and more on something that better maintainability 2 years down the road.
When it comes to the lifecycle of most applications, the speed of developing something new has a marginal impact on the overall costs, and most tools and frameworks than enable rapid initial development tend to suck once you have a big app on top of them. This is why we fawn over a tool when it's shiny and new, and bitch about how much it sucks 3 to 5 years later.
I haven't had time to look at try out the walkthrough, so here's a kind of dumb question that wasn't answered explicitly in the docs: how does it work with existing front-end frameworks, such as Backbone, Angular, Ember, etc?
hoodie is a JavaScript library that runs in your browser.
It gives you
user authentication
data storage and sync
sharing
emails
and so much more
None of these things are really handled by the client-side frameworks mentioned above, so I'm wondering if Hoodie is basically drop-in easy for existing projects.
at first, Hoodie is just a Store for data. So instead of Backbone, Angular etc storing their data via AJAX on a server, or in LocalStorage, we can very simply build adapters for all of the MV* frameworks to use Hoodie as their store.
This way, you get accounts and data synchronization for free. And offline ;-)
Emails and other future modules like payments are just extensions of the available JavaScript API in the browser.
I have a question for experts in Hoodie and/or other similar projects (like Meteor):
How are database connections managed? If every client (browser) basically talks nearly directly to the database, don't you end up with of thousands of database connections?
Large number of connections is not a problem by itself, and servers like nginx handle them well, but I am not aware of a database which will feel comfortable being exposed this way. MongoDB, for example, really starts struggling with 2K+ connections. PostgreSQL can barely handle a hundred.
Is this why CouchDB is used? Is connection pooling used on the server? Or maybe the connections are short-lived and never persistent? Basically, how does this work with large number (say, 10K) of concurrent visitors?
As long as the underlying OS can provide file descriptors, CouchDB can handle the concurrent connections through the magic of Erlang, both persistent and shorter lived.
There are obvious limits to this, but the Hoodie architecture allows easy scale out (more DB servers and manual sharding, or a dynamo-like BigCouch, more workers etc.) that we’ll get to making use of once Hoodie apps become that big.
Never attribute to hipsterism what could adequately be explained with picking priorities :)
The hoodie backend works anywhere node and Couch work (anywhere, really), it’s just that the local dev setup with the fancy domains and everything is tailored to Mac OS X. We have Linux support in the works.
are you on Windows or Linux? Would be great if you could help us to migrate it.
hoodie is basically CouchDB + node.js, nothing fancy. We just do local DNS magic, creating *.dev domains like http://pow.cx/, that is only compatible with Mac atm
What do you think the probability is that someone considering using this framework is also running OS X? I'd say it's pretty high. BTW, wouldn't you say associating OS X with hipsters is a bit outdated?
Nice presentation. In the description of back end modules, you mention payments as a future capability. What kind of payment service would you be able to work with? (I don't see how you could handle card details in the Hoodie architecture). Thanks.
How do you handle permissions by the way? is it the same concept as Firebase's? how do you handle data validation? (e.g. preventing users gaming the data, e.g. saving all their stolen ebooks in your system by using a console, or just cheating in a game by doing illegal moves?)
I’m a Unix dev too, check out my beard in the linked video :D
Like mentioned elsewhere in this thread, other *nixen support is in the works. Hoodie core is platform agnostic, we just chose Mac as the dev environment because we figured that was what most people use. We are definitely not exclusive and would love to see any help towards getting it run on other systems :)
so sorry that we didn't get to it yet. Hoodie is certainly not OS X exclusive, we just have some DNS magic that helps developing locally, that's Mac only at the moment.
Hoodie is just Node.js & CouchDB, you should be able to run it without too much trouble. Try to follow the instructions here: https://github.com/hoodiehq/hoodie-app/issues/35 Were happy to help
This looks pretty optimal for a little app I'm building. I've been bolting bits to Sinatra and using it as a half-API-half-template engine for a frontend I built, this looks far cleaner. I'm curious about security though, what's stopping a user from plugging in random data via the console?
hood.ie has a different approach. Meteor brings backend logic to the frontend. hood.ie tries to hide the backend entirely and provide an API that feels natural for the frontend environment.
The JavaScript API is what we care about most. We do currently have the hood.ie backend implemented in Node/CouchDB, but it could be everything, really. The frontend developer doesn't care.
Berlin and Zurich seem to be doing great lately. I'm also planning on building a startup in Zurich in the coming months. How about a Hacker News Zurich meetup?
It's awesome when you read posts on HN about interesting things and realize your friends were involved. Congrats gr2m!
(P.S... +1 for an Angular adapter =))
I think promising "we want to enable you to build complete web apps in days" might be promising too much. Programming is difficult, there is no silver bullet, and a different data store certainly won't change that.
Interesting choice of couchdb. Any reasons for that compared to the alternatives?
We have our own little lightweight version of PouchDB that isn’t really a version of PouchDB. It works well for now, but we might migrate to Pouch later.
we currently use localStorage. But we <3 PouchDB, just didn't get to it yet. Shouldn't be hard to change that, or to build your own store wrapper providing the same API
I know everyone hates that question.. (I hate it too), but what about non-js users or, maybe more importantly, SEO or crawlers? Honestly, this is what is holding me back to go pure client-side for most of my projects.. I know there's the phantomJS server-side hack but how does that work in practice?
Also, something else that really trouble me with javascript client-side is that, often, when something bugs, everything just stop working. I.e. links don't work anymore, button obviously don't.. Is there a better solution than a hard-refresh? Would a first-level try/catch solve this problem?
> but what about non-js users or, maybe more importantly, SEO or crawlers
Hoodie is more of a tool for applications, with user authentication etc, so SEO is not relevant here. I wouldn't build a public website with Hoodie.
> Also, something else that really trouble me with javascript client-side is that, often, when something bugs, everything just stop working
Yes. That's something we have to handle if we want dynamic web apps build on web technologies. But there are great tools today that help you 1. prevent JavaScript by automated testing in all the browsers / OS 2. track JavaScript errors, e.g. with errorception. And 3. todays browsers are more relaxed about errors, they try to continue running the app, even if one function errored out.
> Is there a better solution than a hard-refresh? Would a first-level try/catch solve this problem
If you can keep the state of an app and store user's data immediately, there is no big problem to reload the page, I do that in several occasions at minutes.io, the user usually doesn't realize, it's very fast.
while (true) {
hoodie.account.signUp(_.uniqueId() + '@gmail.com', _.uniqueId())
}
great effort though, but I still don't think you've got client/server separation right.
we've built something very similar over a year ago: an offline client db backed by memory/websql/indexed-db that replicates with dynamo-db so the app works offline. Looked at Couch/Pouch solution but decided against Couch for performance reasons.
Might open source it later once we figure out the user management/security bit. I still find the existing solutions (including Firebase) not ideal.
You have the option of requiring user confirmation via email (right now users are auto-confirmed by default). Non-confirmed users only write to their localStorage (so they can still use the app pre signup), but nothing (apart from their tiny user object) is synced to the remote DB, nothing is parsed by workers, etc. Then write a worker to cull unconfirmed user objects after a while, and you should be fine. I don't know about additional throttling though, but it seems very doable.
Disclaimer: this isn't my area of expertise, but the rest of the Hoodie team are quite confident about this issue, and I trust them. The other two are currently travelling, but if you have more questions, I'm sure one of them will get back to you with more in-depth info.
I was thinking there might be some sort of throttling/banning mechanism within the signUp function itself, but then being client based that could be easily circumvented unless there's some server side logic aswell. I wonder whether there's a sane way to handle this client side short of minify and/or obfuscate the code and hope for the best.
Apologies for an off-topic question - how did you go about registering a .ie domain name? I've been wanting to register one for a while, but the last time I looked there was a requirement to show proof of residency (or a business address) in Ireland.
Edit: The reason I ask is that it seems like the developers are living in Zurich and Berlin, not Ireland. I live in SF, so can't fulfil the residency requirement.
No problem :) There are actually more than three of us working on Hoodie, one (actually two by now) of them an Irish national. The about section just lists the core team, if you will.
I'm very interested in using this together with emberjs, but honestly I'm having trouble understanding what Hoodie actually does. Why do I need Hoodie to go with my CouchDB? Is it only for the local storage?
[+] [-] Lazare|13 years ago|reply
I see you've already mentioned switching to PouchDB on your backend, and you're already using CouchDB for the cloud, so it seems like you're basically competing straight up against Kanso. So.... How is what you're doing different from what Kanso does? Why should I use Hoodie over Kanso?
I see this code in your docs:
Very nice. But Kanso has: Seems like six of one, half a dozen of the other. What's your killer feature? Kanso has shaky documentation and is currently more aimed at couchapps (they strongly assume that you'll serve your frontend code from CouchDB), but I'm not sure how that really differs from Hoodie. :)If we take the overall architecture as a given (browser based app, PouchDB or equivalent for local storage, syncing with CouchDB in the cloud, and node workers handling the backend work), what's the focus of Hoodie that sets it apart from Kanso?
(In any case, nice stuff. I think the concept is great, and there's definitely room for several competing frameworks in this space.)
[+] [-] janl|13 years ago|reply
As far as originality, we choose a different design approach. We want to cater to people who barely get jQuery and give them a tool to build larger apps. We are building the technology downwards from that idea towards the bits and pieces we have.
One of the fun things about Hoodie is that there is nothing technically new in it. Existing ideas and tools are just put together a different way.
[+] [-] lifeisstillgood|13 years ago|reply
I am trying very hard not to be dumb, but this is beating me
seemingly Kanso? and Hoodie both have JS stubs at the client and server end, these synch with each other, then synch with whatever you are using at client (say backbone) and server (node? CouchDB? My Postgres server behind my python app?)
If thats roughly right, why do I do this? I control my server apps no? Why wrap them in another layer? Json->backbone is hardly a difficult synch operation - I mean its called synch().
I really am interested in understanding the philosophy and driving ideas here - everyone has a moment of clarity where they can see how to build an app or a framework, and how it will come together - I am not attacking you (yet !) - please describe your moment of clarity.
[+] [-] rgbrgb|13 years ago|reply
https://github.com/kanso/kanso
[+] [-] gr2m|13 years ago|reply
We start with a dream API, and build from there.
[+] [-] sopooneo|13 years ago|reply
[+] [-] onemorepassword|13 years ago|reply
When it comes to the lifecycle of most applications, the speed of developing something new has a marginal impact on the overall costs, and most tools and frameworks than enable rapid initial development tend to suck once you have a big app on top of them. This is why we fawn over a tool when it's shiny and new, and bitch about how much it sucks 3 to 5 years later.
[+] [-] danso|13 years ago|reply
I ask this because of the description found on the Github page: https://github.com/hoodiehq/hoodie.js
None of these things are really handled by the client-side frameworks mentioned above, so I'm wondering if Hoodie is basically drop-in easy for existing projects.[+] [-] gr2m|13 years ago|reply
This way, you get accounts and data synchronization for free. And offline ;-)
Emails and other future modules like payments are just extensions of the available JavaScript API in the browser.
[+] [-] old-gregg|13 years ago|reply
How are database connections managed? If every client (browser) basically talks nearly directly to the database, don't you end up with of thousands of database connections?
Large number of connections is not a problem by itself, and servers like nginx handle them well, but I am not aware of a database which will feel comfortable being exposed this way. MongoDB, for example, really starts struggling with 2K+ connections. PostgreSQL can barely handle a hundred.
Is this why CouchDB is used? Is connection pooling used on the server? Or maybe the connections are short-lived and never persistent? Basically, how does this work with large number (say, 10K) of concurrent visitors?
Thanks!
[+] [-] janl|13 years ago|reply
There are obvious limits to this, but the Hoodie architecture allows easy scale out (more DB servers and manual sharding, or a dynamo-like BigCouch, more workers etc.) that we’ll get to making use of once Hoodie apps become that big.
[+] [-] tempaccount9473|13 years ago|reply
> Hoodie currently only runs on OS X
Oh, a hipster framework.
[+] [-] janl|13 years ago|reply
Never attribute to hipsterism what could adequately be explained with picking priorities :)
The hoodie backend works anywhere node and Couch work (anywhere, really), it’s just that the local dev setup with the fancy domains and everything is tailored to Mac OS X. We have Linux support in the works.
[+] [-] larrydavid|13 years ago|reply
[+] [-] gr2m|13 years ago|reply
hoodie is basically CouchDB + node.js, nothing fancy. We just do local DNS magic, creating *.dev domains like http://pow.cx/, that is only compatible with Mac atm
[+] [-] sbochins|13 years ago|reply
[+] [-] evilhackerdude|13 years ago|reply
Pull requests > Witty comments
[+] [-] coldtea|13 years ago|reply
[+] [-] janl|13 years ago|reply
And a presentation from earlier at @berlinjs including a live demo: https://www.youtube.com/watch?v=X3Ttb0BD8pg
[+] [-] jufo|13 years ago|reply
[+] [-] eranation|13 years ago|reply
How do you handle permissions by the way? is it the same concept as Firebase's? how do you handle data validation? (e.g. preventing users gaming the data, e.g. saving all their stolen ebooks in your system by using a console, or just cheating in a game by doing illegal moves?)
[+] [-] beefsack|13 years ago|reply
[+] [-] janl|13 years ago|reply
Like mentioned elsewhere in this thread, other *nixen support is in the works. Hoodie core is platform agnostic, we just chose Mac as the dev environment because we figured that was what most people use. We are definitely not exclusive and would love to see any help towards getting it run on other systems :)
[+] [-] gr2m|13 years ago|reply
Hoodie is just Node.js & CouchDB, you should be able to run it without too much trouble. Try to follow the instructions here: https://github.com/hoodiehq/hoodie-app/issues/35 Were happy to help
[+] [-] ZenJosh|13 years ago|reply
[+] [-] janl|13 years ago|reply
[+] [-] hayksaakian|13 years ago|reply
How does this compare to, say, Meteor?
[+] [-] gr2m|13 years ago|reply
The JavaScript API is what we care about most. We do currently have the hood.ie backend implemented in Node/CouchDB, but it could be everything, really. The frontend developer doesn't care.
[+] [-] m_mueller|13 years ago|reply
[+] [-] thomas11|13 years ago|reply
See http://techup.ch and meetup.com for what's going on in the tech meetup space in Zurich. I'm often at zhgeeks, jszurich, web tuesday.
[+] [-] dylanz|13 years ago|reply
[+] [-] taybin|13 years ago|reply
Interesting choice of couchdb. Any reasons for that compared to the alternatives?
[+] [-] bsaul|13 years ago|reply
[+] [-] janl|13 years ago|reply
[+] [-] gr2m|13 years ago|reply
[+] [-] d0m|13 years ago|reply
Also, something else that really trouble me with javascript client-side is that, often, when something bugs, everything just stop working. I.e. links don't work anymore, button obviously don't.. Is there a better solution than a hard-refresh? Would a first-level try/catch solve this problem?
Thanks!
[+] [-] gr2m|13 years ago|reply
Hoodie is more of a tool for applications, with user authentication etc, so SEO is not relevant here. I wouldn't build a public website with Hoodie.
> Also, something else that really trouble me with javascript client-side is that, often, when something bugs, everything just stop working
Yes. That's something we have to handle if we want dynamic web apps build on web technologies. But there are great tools today that help you 1. prevent JavaScript by automated testing in all the browsers / OS 2. track JavaScript errors, e.g. with errorception. And 3. todays browsers are more relaxed about errors, they try to continue running the app, even if one function errored out.
> Is there a better solution than a hard-refresh? Would a first-level try/catch solve this problem
If you can keep the state of an app and store user's data immediately, there is no big problem to reload the page, I do that in several occasions at minutes.io, the user usually doesn't realize, it's very fast.
[+] [-] verbalist|13 years ago|reply
the web is no longer just text. if you need to consume text then you need a text-api client, which is what web browsers used to be.
[+] [-] akrymski|13 years ago|reply
while (true) { hoodie.account.signUp(_.uniqueId() + '@gmail.com', _.uniqueId()) }
great effort though, but I still don't think you've got client/server separation right.
we've built something very similar over a year ago: an offline client db backed by memory/websql/indexed-db that replicates with dynamo-db so the app works offline. Looked at Couch/Pouch solution but decided against Couch for performance reasons.
Might open source it later once we figure out the user management/security bit. I still find the existing solutions (including Firebase) not ideal.
[+] [-] espylaub|13 years ago|reply
Disclaimer: this isn't my area of expertise, but the rest of the Hoodie team are quite confident about this issue, and I trust them. The other two are currently travelling, but if you have more questions, I'm sure one of them will get back to you with more in-depth info.
[+] [-] Mahn|13 years ago|reply
[+] [-] vosper|13 years ago|reply
Edit: The reason I ask is that it seems like the developers are living in Zurich and Berlin, not Ireland. I live in SF, so can't fulfil the residency requirement.
[+] [-] espylaub|13 years ago|reply
[+] [-] jodonovan|13 years ago|reply
[+] [-] janl|13 years ago|reply
See you again tomorrow CEST. <3
[+] [-] cupofjoakim|13 years ago|reply
[+] [-] donjaime_hn|13 years ago|reply
A few questions. What is the strategy is for concurrent modification of data? What are the consistency guarantees for the sync operation?