top | item 10056033

Open Sourcing Our SDKs

141 points| jamesjyu | 10 years ago |blog.parse.com | reply

41 comments

order
[+] mmastrac|10 years ago|reply
I really wanted to like Parse, but I saw a startup get burned badly by it recently. It works great as a fast way to get your app up and running, but it absolutely falls flat on its face as soon as you do anything more complicated than a basic read/write interface.

The real problems show up when you've all of a sudden got these old apps out in the field with no way to upgrade them to a newer version of the DB. If you aren't being diligent and making all your Parse DB calls through their cloud functions, you are guaranteed to end up in a situation where your older mobile clients are holding back schema and feature evolution. You can try to hack around it using triggers, but these are too buggy to be reliable.

Other examples of where Parse is painful:

- beforeSave/afterSave triggers are completely unusable! They are unbelievably limited and each one you add slows your app down by as much time as it takes to run. If you want to do something like add users to Mailchimp, you have now tied the length of time it takes to save a user to the length of time it takes to talk to mailchimp!

- There is absolutely no way to test your parse code other than uploading it to a parse app and hitting it. You can hack together some node code that sort-of tests it, but it's nowhere near representative of what happens on the server.

- You can't manage Parse.Config programatically. There's a web interface and that's that. None of your config can live in your git repo because there's no way to read or write it from the command-line.

- The schema tools are pretty bad on the website. IIRC they introduced REST APIs to make changes to schema recently, but unless you plan on writing tooling around those, you'll be manually syncing schema between prod/staging at some point in your life and you'll probably also miss a critical column that got added.

- Parse has a bunch of weird datatypes that act strangely in their cloud code. Want to make a pointer and store it in a parse field? Construct its internal representation and set it! I wish I was kidding. If you construct a pointer incorrectly, it can actually create a phantom object in another table. Also wish I was kidding about this.

- createdAt and updatedAt are special and Parse won't let you touch them or set them. Heaven forbid you want to actually set these to a meaningful time other than the actual last time the record was written. Nope. So you end up making your own updatedAt columns and just using those.

For your own sanity: stay away from Parse. The range of use cases where it is actually useful is so narrow that you'll leave that very quickly unless you are building a toy app.

[+] roddylindsay|10 years ago|reply
I think one can go pretty far using Parse, and it gives you a lot out of the box for your mobile clients...but it definitely has its quirks, and you have to use it in the right way.

1) Don't even bother with their hosted Cloud Code, spin up your own backend and use the webhooks. As of the JS SDK 1.5 you can finally run a node backend securely.

2) Push complex logic off the client and into Cloud Functions.

3) Put plenty lot of thought into your schemas before creating them.

That said, there are a few big areas of need that I see that I hope the team will address:

1) Development/Staging environments are a pain, particularly around schemas. Tooling around migrations would help a lot here.

2) Testing tools leave a lot to be desired (I had to roll my own backend mock library for node, https://github.com/HustleInc/parse-mockdb )

3) Limited exposure of MongoDB feature set (e.g. aggregation framework)

4) Downtime (and notification speed / transparency thereof)

5) Security/ACLs leave a lot to be desired, particularly for complex schemas. For instance, it would be great to specify object-level access controls in terms of Parse queries (e.g. Users that satisfy a query built around the object in question).

[+] shortstuffsushi|10 years ago|reply
Aside from the awesome writeup that Hector provided, I just wanted to call this out:

> It works great as a fast way to get your app up and running

It's a really neat and useful tool for prototyping. Obviously as you start to get more and more business logic that should be in your backend, you may want to migrate to your own backend, but if all you need is a simple key:value store, it's great. Heck, it even does files, auth, email, etc. You can take a prototype pretty far before reaching the edge of its usefulness.

[+] bmir-alum-007|10 years ago|reply
One of my friends likes node+mongo-based deployd [0]

To me, it's just as easy/fast and more powerful to use RoR/Sinatra or Django/Flask from the get-go and for limiting longer-term tech-debt (migrating from one to the other).

0. http://deployd.com/

[+] HectorRamos|10 years ago|reply
Hey,

I'm Héctor, a Parse developer advocate for the past 3.5 years. I've probably seen every question related to Parse, spend most days supporting apps with millions of users, and hopefully I can address some of these.

You bring up some valid points, some specific to Parse, others related to backend systems in general.

- "The real problems show up when you've all of a sudden got these old apps out in the field with no way to upgrade them to a newer version of the DB. If you aren't being diligent and making all your Parse DB calls through their cloud functions, you are guaranteed to end up in a situation where your older mobile clients are holding back schema and feature evolution."

Maintaining compatibility with a long tail of old, stale versions of your clients in the field is something all of us need to deal with as app developers. Our examples do place more emphasis on client-originated object updates and queries due to our approach of making it very easy for developers to get started. Some of our more sophisticated developers end up using the approach you've used as an example of moving as much of the operations over to Cloud Code. This is, in fact, one of the reasons we decided to work on our Cloud Code server-side compute environment.

- "beforeSave/afterSave triggers are completely unusable! They are unbelievably limited and each one you add slows your app down by as much time as it takes to run. If you want to do something like add users to Mailchimp, you have now tied the length of time it takes to save a user to the length of time it takes to talk to mailchimp!"

A beforeSave, by definition, will block a save until the operation has finished. Save operations on the client side are performed asynchronously, so this should generally not block your app. An afterSave can be used for things such as sending an email through Mailchimp, which in general you would not block a user operation on, but rather these would be queued up. There are some sensible default timeouts in place to discourage lengthy operations that could lead to a unsatisfying user experience (e.g. creating a new row should not take more than 3 seconds). If you are OK with having the user wait more than 3 seconds, you can use Webhooks to redirect the beforeSave/afterSave/cloud-function computation to a third party server of your choosing. In that case, you would have up to 30 seconds to return a response to the client.

- "There is absolutely no way to test your parse code other than uploading it to a parse app and hitting it. You can hack together some node code that sort-of tests it, but it's nowhere near representative of what happens on the server."

This is a big pain point many of our users have highlighted. Cloud Code is not node, so the best way to test code is by running it on Cloud Code itself. We do not have a good solution for satisfying this need at this time, so I would suggest using node.js on a third party server and taking advantage of Webhooks. This effectively allows you to Bring Your Own Server and provides you with limitless flexibility on what can be done as part of a save hook or cloud function call.

- "You can't manage Parse.Config programatically. There's a web interface and that's that. None of your config can live in your git repo because there's no way to read or write it from the command-line."

I think that everything you can do on Parse should have a REST API equivalent. We've recently launched a Hooks API as well as a Schema API. Perhaps a Config API can be considered. I personally use Config to store credentials that I would not want to be checked in to version control, and I use Parse Objects for anything that changes often enough to require programmatic updates.

- "The schema tools are pretty bad on the website. IIRC they introduced REST APIs to make changes to schema recently, but unless you plan on writing tooling around those, you'll be manually syncing schema between prod/staging at some point in your life and you'll probably also miss a critical column that got added."

You can clone an app and its entire schema from your Settings. This is the best way to ensure there is a 1 to 1 mapping between prod and staging. One thing that is missing from this is cloning data from prod to staging. Bigger apps, with gigabytes of data, could take a very long time to clone. In this case, I would suggest having some sort of seed script that populates your staging database with placeholder data. I understand this does not solve the problem of having an exact copy of your production database, but this is not a simple problem to solve (would you want to keep both apps in sync as the prod app gets updated? would you be willing to wait hours/days for your an initial sync from prod to staging?). With that said, this is something we're aware of and hope to be able to address soon. Stay tuned.

- "Parse has a bunch of weird datatypes that act strangely in their cloud code. Want to make a pointer and store it in a parse field? Construct its internal representation and set it! I wish I was kidding. If you construct a pointer incorrectly, it can actually create a phantom object in another table. Also wish I was kidding about this."

Mucking with JSON to successfully create a valid pointer sounds like an easy way to introduce bugs in your code. You shouldn't need to to do this, use YourClass.createWithoutData(objectId) instead.[1]

- "createdAt and updatedAt are special and Parse won't let you touch them or set them. Heaven forbid you want to actually set these to a meaningful time other than the actual last time the record was written. Nope. So you end up making your own updatedAt columns and just using those."

createdAt and updatedAt are automatic fields set by the underlying mongo database, and you can depend on these being accurate as no user or admin operation can force them to use arbitrary values. You're definitely welcome to create your own Date fields if your use case requires arbitrary values. This shouldn't be too much of a burden, IMHO.

Hopefully this helps! As always, if you have any questions, feel free to join us on our developer community[2].

[1]: http://parse.com/docs/js/api/symbols/Parse.Object.html#.crea...

[2]: https://groups.google.com/forum/#!forum/parse-developers

[+] snambi|10 years ago|reply
Absolutely right. Parse is only good for prototyping. Especially prototyping the UI part of the App, without dealing with data.
[+] andrewljohnson|10 years ago|reply
Freaking awesome! So much to learn from the code from these guys... plus easier to use, more trustworthy... and you don't have to wait for the vendor to fix critical bugs when needed.

I'd really like to see all infrastructure SDKs open sourced (obviously a tough sell, but maybe Parse paves the way). In building our core stack, we don't tolerate closed source. But we end up allowing it for things like Crashlytics, MixPanel, Segment, etc - and it's not a free lunch.

[+] jamesjyu|10 years ago|reply
Thanks Andrew! Transparency and community involvement is our top priority here.

Also, we're doing a series of blog posts that will cover each part of the SDK stack. Lots of learnings to share from our kick ass SDK team.

Here's the first: http://blog.parse.com/learn/the-parse-sdk-whats-inside/

[+] roddylindsay|10 years ago|reply
What would it take to have these Parse client SDKs support a generic GraphQL server (and for the Parse backend itself to conform to GraphQL)?

It seems like that would be a promising direction to give Parse apps flexibility to swap backends down the road, without having to change the entirety of their client code base...and preserving the out-of-the-box speed and convenience of using Parse for brand new and prototype apps.

[+] GeneralTspoon|10 years ago|reply
Last time I used Parse the Android SDK had a crippling bug that made it unusable. This was version 1.8.0 I believe.

Essentially Disk I/O was using the same synchronized lock as the main thread. On some devices this caused a reproducible ANR (App Not Responding). I ended up writing a small REST Api client using Retrofit to replace the SDK. It worked well but as mmastrac mentioned there are just too many other critical issues with Parse to make it suitable for anything other than quick prototypes.

Besides, it's super expensive if you need have any sort of scale. You could almost hire a part-time engineer to build and maintain a backend for those prices.

[+] grantland|10 years ago|reply
It's unfortunate to hear you had a bad experience with our SDK :(

We've actually fixed a few ANR deadlocks in this last release [1], as well as a few others in the releases since 1.8.0 [2].

If you can supply reproduction steps, we'd love to be able to fix this issue as I'm sure you're not the only one experiencing it. In addition, we're more than happy to accept fixes as contributions :D

[1]: https://github.com/ParsePlatform/Parse-SDK-Android/releases/...

[2]: https://parse.com/docs/downloads

[+] evv|10 years ago|reply
I wouldn't write off Parse's value because one client was rough around the edges. I suspect the SDKs will become much more stable and usable now that the community can help contribute to them.

If you can find a part-time engineer that can build and maintain a backend with all of Parse's features, you should absolutely do that.

[+] pbreit|10 years ago|reply
Has anyone reverse engineered the server side such that another provider could step in or you could run it on your own servers?
[+] dguaraglia|10 years ago|reply
Wouldn't that be more work than just reimplementing whatever CRUD you would implement in Parse directly?
[+] sdpurtill|10 years ago|reply
This is awesome. Huge fan of what you guys are doing at Parse!
[+] grinich|10 years ago|reply
Why weren't these open sourced from the start?
[+] csmajorfive|10 years ago|reply
The initial magic of Parse was in the tight coupling of native mobile SDKs with standard cloud services. We released our first barebones beta SDKs long before we had a publicly documented (not to mention properly versioned and stable) server API. So for a while we iterated quickly underneath the hood and dragged people over to new versions of our cloud APIs with new SDK releases. It only became feasible to do this after we productized our server side APIs. And then it took some time to prioritize the project and execute on it.
[+] GFK_of_xmaspast|10 years ago|reply
Just the last couple days I tried to get the parse "embedded c" SDK working (on a non-embedded platform and mixing in with existing c++ code), I found the "embedded c" docs to be not at all helpful and had to fall back to libcurl and fumbling out how things worked from the general-purpose API docs.
[+] listic|10 years ago|reply
What kind of device is this, displayed on Parse's homepage? http://i298.photobucket.com/albums/mm249/hrenistic/parse_wtf...
[+] gk3|10 years ago|reply
Haha good question! It's nothing specific, but I intended it to look kinda like a thermostat. When we launched this new homepage animation, we had just announced our IoT support so the goal of this animation was to convey that Parse runs on a lot of different things :)