top | item 26355779

Bring Your Own Client

533 points| goranmoomin | 5 years ago |geoffreylitt.com | reply

249 comments

order
[+] stupidcar|5 years ago|reply
The competitive advantages of owning both the application and the data are so high that don't see widespread BYOC ever happening without government intervention.

I'd like to see an enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider. And that storage provider has an obligation to make my data accessible and manageable by me.

This wouldn't be a panacea, but it at least makes it theoretically possible to write a replacement client for Google Docs that could be a drop-in replacement.

[+] dec0dedab0de|5 years ago|reply
I'd like to see an enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider. And that storage provider has an obligation to make my data accessible and manageable by me.

I think that is a bit much, I would much rather make reverse engineering/screen scraping/whatever for interoperability be 100% legal with zero grey area. Including a bit that says TOS/Eula/NDA/NonCompete/any contract can not give away this right.

Edit: basically let asshole companies use technical means to try to stop us, but give them no legal recourse if we manage to get the cheese out of their trap.

[+] gklitt|5 years ago|reply
(post author here)

While I agree that commercial incentives are critical to consider, two caveats I'd add:

1) I think the nature and depth of the incentives problem varies a lot by industry. Social media is a really complex case, for example. But I'm most interested in collaborative productivity tools, where I think many companies are okay being incentivized to build the best product rather than build data moats.

2) Cutthroat commercial incentives aren't the only thing that got us here. There are also tech barriers.

If I was starting a Google Docs competitor today, even if I wanted to make it open, I think it would be hard to pull off. What kind of API would I expose to enable realtime editing with good offline mode and conflict resolution? How would I deal with clients that have slightly different rich text representations than my own?

In an alternate universe where we already had a user-owned "web filesystem" with good answers to these questions, I could just hook up my client to that existing system and not even worry about persistence at all. It needs to be _easier_ for devs to build the right thing for users, not harder. And it needs to be a _better_ experience for end users, not a sacrifice. The convenient thing will win.

[+] kjrose|5 years ago|reply
Well it can happen if a significant enough open source project establishes a protocol early enough and doesn't get co-opted.

That or a consortium of companies agree on some sort of standard. (See the work on pc buses in the early pc days)

But you are right. When companies run under trying to get as many users locked in as possible during a paid for investors period. This model is precisely the opposite outcome expected as the whole point is to build a moat around the sales model.

[+] TheRealDunkirk|5 years ago|reply
> without government intervention

Well that's the real trick, isn't it? </Han Solo>

Our government is "captured" by campaign financing. Now we have 2 problems: ineffective governmental oversight, AND Citizens United. We have to solve the latter before we can ever solve the former.

[+] rozab|5 years ago|reply
I think it's not completely inconceivable that this could be regulated some day with music streaming. There are zero technical hurdles (unlike with complex apps like Google docs), and because there is such potential for decoupling the client the consumer benefits would likely be much greater.

I've been meaning to switch from Apple Music to Spotify for a while now (I'm on Android). I just had a look around the Spotify app today and the interface is terrible, it won't let me play and view my library like I want to. It would be so much easier if I could just use both from a single open source client, and then I wouldn't have to switch off to soundcloud and bandcamp for certain artists.

Wouldn't it be in the interests of players like Google and Apple to create this service/protocol, even before the regulatory net starts closing in?

[+] CyberRabbi|5 years ago|reply
I generally support the spirit of requiring cloud apps to provide equivalently powerful API access to their service. The (solvable) problem is how do you regulate this without stifling development of new cloud apps?

Another more practical problem is how do you even get something like this on the agenda of a lawmaker? This isn’t a readily apparent problem to most people but it likely would result in less monopolistic outcomes for cloud providers and potentially better and/or more efficient products.

[+] tomxor|5 years ago|reply
I think the focus here is on the wrong thing, the "app", which makes ideas like legal intervention and other distractions feel relevant.

I think client interchangeability and ultimately user freedom is fundamentally about data driven design, look at all the examples: text is text is text... git is a file based graph of merkle trees. They are both highly abstract, portable, unopinionated data structures that don't describe APIs, UI or implementation - don't like git CLI? you can literally write a completely different one with different porcelain and use the exact same git repos as everyone else, people have done this. Text is the same, it's just more obvious because so many examples exist.

If you start with data and make it successful clients will emerge around it without the choice of making it interchangable.

The problem with things like google docs isn't the app, it's the opaque data format. And then there are formats posing as being open and portable like DOCX which is essentially a proprietary microsoft format forced open due to reverse engineering - all of it's details will no doubt be closely coupled to the history of MS Word features and implementation.

[+] visarga|5 years ago|reply
Separation of application and storage is a great idea!

How about web search and social news feeds? Can there be a similar frontend / back-end separation for Google, FB and Twitter? I am thinking - different search front ends with different UIs, rankings and filters. We could have an EFF-Google and a Conservative-Google. Users could pick their preferred flavor, as I don't think it's fair for Google to decide for us all.

[+] nickez|5 years ago|reply
We just have to do what the phone manufacturers did for the mobile phone connectivity. GSM was a huge hit thanks to interoperability.
[+] 3np|5 years ago|reply
OTOH, give it time. Open alternatives exist for every private walled garden. I believe the decentralization/federalization and FLOSS movements have a real chance, even if they move at 0.05 velocity. Private platforms are subject to closure, but the light of an open source project will shine as long as there's someone spending time on it.

In due time we'll have networks of small providers catering for BYOC hosting. Cryptocurrencies can play a part in efficient and fair compensation.

But then again, maybe by the time we catch up to current date, everything happens in walled VR moats.

Maybe I'll be the oddball in the metaphorical cabin in the woods, but given the progress we're making so far I'll refuse to enter a closed VR/AR ecosystem.

[+] Vinnl|5 years ago|reply
There's been some of that already, not in the sense of governments forcing the separation, but in the sense of governments making the keeping of data a liability, e.g. with the EU's GDPR.

There's certainly still a whole class of applications when owning the data is a competitive advantage, but there are also lots of organisation whose core competency is not selling data, even though they do have to work with user data. When storing data yourself is a risk, it becomes a lot more attractive to just let the user store that data somewhere themselves and have them give your application access to it.

I'm sure this one of the reasons we're seeing a lot more interest from Europe than elsewhere in Solid [1].

Then of course, there are also lots of organisations struggling to maintain correct and up-to-date data. If multiple organisations access the same data as controlled by the user, the user is more likely to have kept that data up-to-date.

(Disclaimer: views are my own.)

[1] https://solidproject.org/

[+] polote|5 years ago|reply
> The competitive advantages of owning both the application and the data are so high

Honestly this is wrong. Google drive has API for everything, Notion is working on an API, Confluence has API for everything, Trello ... Any serious text editor (and all the ones wrongly cited in the article) software has API for everything. Want to export your Confluence data to Notion ? you can. In the documentation tools space owning the data is not something looked for, because company pay for the service so the data belong to them

BYOC can exists today, the only reason it doesn't. Is that the documents format is super complex and require complex editors. But the format themselves are not secret, the application is (and not even always, for example the Confluence editor is open source).

[+] istjohn|5 years ago|reply
Even better: They have to use the storage provider of your choosing.
[+] simonebrunozzi|5 years ago|reply
> The competitive advantages of owning both the application and the data are so high that don't see widespread BYOC ever happening without government intervention.

This.

[+] pjc50|5 years ago|reply
Interesting proposal, but I see it greatly reducing development and also further entrenching the cloud providers as Lords Of Everything.
[+] ryanmarsh|5 years ago|reply
don't see widespread BYOC ever happening without government intervention

This is why people don't like nerds, it's like you actually want to fuck up this industry. The number of users that care about this is a rounding error. What you're talking about is increasing the complexity and flakiness of every product through modularization. Have you ever actually shipped a product for the general public?

[+] deepstack|5 years ago|reply
>I'd like to see an enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider. And that storage provider has an obligation to make my data accessible and manageable by me.

Same can be said about gmail. In the old days, one can access gmail via imap/pop3. I believe that was removed. All email ought to provide an alternative way of getting the email other than company's web interface.

[+] blfr|5 years ago|reply
This idea suffers from the same problem federation does. It's harder to move a whole ecosystem. Makes it harder to innovate. Leads to centralized platforms outperforming these BYOC/federated/open options. Basically how Reddit replaced Usenet with upvotes and basic spam filtering.

There's more on this from 'moxie

https://www.youtube.com/watch?v=Nj3YFprqAr8

https://signal.org/blog/the-ecosystem-is-moving/

[+] nickez|5 years ago|reply
Can you imagine a world where you could only use a specific phone model with a specific operator? or where you could only send text messages to people that is on the same network? If we can regulate phones we should be able to regulate social networks.
[+] cyberlab|5 years ago|reply
> Reddit replaced Usenet

Many things replaced Usenet. Usenet is more of a protocol (that uses NNTP) than a social media platform. If you're any way a systems thinker, you would know that protocols are more resilient than services.

[+] platz|5 years ago|reply
> Leads to centralized platforms outperforming these BYOC/federated/open options. Basically how Reddit replaced Usenet with upvotes and basic spam filtering.

That is a nice insight.

[+] rapnie|5 years ago|reply
> This idea suffers from the same problem federation does.

Yes, this is a problem you also see with the Fediverse (based on W3C ActivityPub). You get innovators and laggers, and the latter when they are popular can hold the former back (you see this with Mastodon, which as early adopter created their own client API's, but are now lagging to implement the Client-to-Server part of the AP specification).

At the same time standardization of federated protocols can also be an advantage in that it allows many different projects and applications [0] to be developed and mature independently. Innovation can come from unexpected corners here (heads up to openEngiadina with ERIS [1], DREAM [2] and Spritely [3] project).

[0] https://git.feneas.org/feneas/fediverse/-/wikis/watchlist-fo...

[1] https://gitlab.com/openengiadina/eris/-/blob/main/doc/eris.a...

[2] https://dream.public.cat/

[3] https://spritelyproject.org

[+] dredmorbius|5 years ago|reply
There can be considerable use-value to ecosystems which don't move as a whole, or do so with deliberation and in a component-wise manner.

As with, say, LaTeX, or Unix/Linux.

[+] drjgk45839|5 years ago|reply
>...Reddit replaced Usenet...

According to Wikipedia, Reddit was created the same year Usenet service was discontinued by AOL. Reddit "replaced" Usenet in the same sense that cell phones "replaced" telegrams.

[+] lifeisstillgood|5 years ago|reply
tl;dr

>>> So while it’s nice that I’m able to host my own email, that’s also the reason why my email isn’t end-to-end encrypted, and probably never will be. By contrast, WhatsApp was able to introduce end-to-end encryption to over a billion users with a single software update.

[+] bombcar|5 years ago|reply
This strikes at something that people often WANT when they say "text files are better than the registry/binary formats" - they want to easily bring their own tooling to bear.

Standard APIs let you do this - even if you have a "binary format" like MySQL or PostgreSQL (on disk) - nobody really complains about that because they have a defined API you can interact with.

[+] vbsteven|5 years ago|reply
And this works both ways. Like the Postgres standard API you mention: Not only does it allow you to bring your own client, you can swap the underlying implementation and still use all the clients that speak the protocol.

Like the author I would really like to see some open protocols (and adoption) for todo's and calendars. I mention adoption because for calendars there are some standards but popular software like Google Calendar and Exchange do not implement them properly or fully.

[+] zomglings|5 years ago|reply
The problem (currently) is that most people simply don't care about being forced to use a particular client - as long as that client looks nice.

I say this with a lot of love for the idea. My company sells a collaborative knowledge base that supports Bringing Your Own Client. Out of the hundreds of users I have spoken to, only ONE has ever asked me if they could use their favorite markdown editor to access their knowledge base.

On the positive side of things, having the capability to bring your own client makes it really easy for us to support many different use cases. We just have to write those clients ourselves.

[+] allenu|5 years ago|reply
Immutability and mutability of content, and expectations or conventions surrounding it, seems to play a large role in feasibility here. The successful examples listed are generally cases where content is immutable to the general audience (someone produces content and then publishes it to others) or the way content is mutable is "understood" by the participants. That is, text editors mutate content, but no one expects there to be multiple editors of a text file, and the convention of a file is understood and encapsulates the "state" of the data.

In the wishful cases listed, collaboration is the norm, so to make it BYOC, you'd have to expose core "mutation" API or else use a general convention that is understood across the board.

In my dream world, we'd be use CRDTs for data and the "schema" of the data for a given "service" (say something like Google Docs) is open. The data storage layer would be a commodity and you can swap in different providers as you see fit. Of course, there is no benefit to the creators of such services to do this, and I don't think CRDTs are quite there yet with defining mutation efficiently with respect to multiple collaborators. However, from a data portability standpoint, it feels like the ideal to me.

[+] eric4smith|5 years ago|reply
The main reason for using google docs is not it’s superlative editor... /sarcasm

It’s the collaboration features. And no I don’t mean the multiple people typing the doc at the same time.

It’s the ability to quickly share a document with a set of people that you want.

Sure I use Vim every day, but without that git push and a trip to GitHub to invite collaborators I’m a man on a desert island waiting for that next weekly ferry.

Code editors and that ecosystem is good for code, but too slow for people who just want to share some recipes with their brother and not think about it too much.

Until we have some sharing infrastructure easy enough for a any client to sit on top of, it’s all a dream.

[+] dnissley|5 years ago|reply
I've got lots of small tweaks I'd love to incorporate into a Twitter client. Unfortunately, I think many of them would violate the developer TOS.

For example: hiding avatars completely or generating replacement avatars using the username to remove any chance of internal bias associated with an account's avatar. Another one: hiding images by default and forcing you to click to expand/open to see them (no previews). A lot of these ideas are intending to modify Twitter's default salience landscape.

However disappointing it might be though that I can't do this, I get it. These kinds of modifications could totally change the ways in which a user experiences Twitter, and how Twitter would be able to monetize those users. Simply forcing Twitter (via legislation) to allow BYOC doesn't seem like a good idea because of this. It doesn't seem right to force them to run a service with a reduced potential for monetization -- e.g. many clients could just not show any ads, which would totally remove Twitter's ability to monetize at all.

An idea might be to charge users money in order to allow BYOC, so Twitter could focus on building out the core offering, which is just basically the public messaging substrate of the internet. But I'm not at all sure how well that would work out in practice.

[+] jasode|5 years ago|reply
>What would it look like to have a thriving ecosystem of third-party clients for Google Docs style word processing, which can all interoperate with each other, even supporting realtime collaboration?

Well, based on decades of history of other complex file formats such as pdf, zip, and MS Office formats of doc, docx, xls, xlsx, etc... it would be a buggy mess. It didn't matter whether the format was reverse-engineered or an officially open specification.

The issue is that a plain text file used for programming code is linear from top to bottom and so the low complexity for universal editors is just parsing CR/LF to interpret lines. (Yes, there can be extra complexity of syntax highlighting but the base level complexity of opening the file for display is still just parsing CR/LF.)

The complexity is higher for pdf/zip/xls because a common theme is each have a internal hashmap/dictionary/directory that has byte pointers backwards and forwards to other parts of the file. And they have internal hierarchies of data structures. And changing from binary representation to XML such as Microsoft's OOXML doesn't change the base complexity which is why LibreOffice has constant bug reports from users unable to open their particular docx/xlsx file.

When I collaborate with others in MS Word/Excel, a best practice is to make sure everybody is using the same version of MS Office. If somebody is using MS Office 2007 while others are on Office 2013, a roundtrip save & open between different versions will eventually corrupt the file or lose data. Even staying with just one vendor like MS can get unreliable. The wild west has lots of utilities/libraries that write incorrect zip files and broken pdf files.

>Some successful existing examples of client ecosystems built around open standards: [...] email clients

I'm not totally sold on this example. Yes, the open standard is SMTP for network communication... but there's another aspect that isn't standard: the on-disk file format for email archives. Microsoft Outlook uses binary PST files but Mozilla Thunderbird uses text-based MBOX files. But Mozilla's MBOX is slightly different from other tools that use MBOX.

[+] phren0logy|5 years ago|reply
I'm in medicine, and I desperately want this model for electronic health records.

The current state of lock-in has been effectively maintained by the entrenched players, and it's really bad.

[+] derefr|5 years ago|reply
Re: Notion, Trello, etc. — wouldn’t enabling interoperability on top of a presumably-free “open core”, completely commoditize these services? (As, for anything they offered on top of their open core, you could instead substitute a third-party service that does the same thing by operating against the service’s API.)

Which is to say — in a world where interoperability is a social norm or legal requirement, how would these services exist? (I would suspect they wouldn’t.) And, without them, would there be any money in advancing the state of the art in these verticals?

It’d be a lot like if there were a legal requirement for every drug to have a generic available from the start. Would there still be an incentive for drug research?

[+] benjamir|5 years ago|reply
That's why I hate forums without a proper E-mail gateway -- for me plain text E-Mail is the most essential API between humans. My mail client is the main reason and I really hate everyone changing something that disables using either my preferred editor.

Everytime some modern hype takes that from me I cannot stop thinking of goose fattening: forcefully stuffing it down the throat and not for good.

[+] o_m|5 years ago|reply
I feel Reddit is another thing that has "Bring your own client". I'm using the old design and Reddit Enhancement Suite to tweak it to whatever I want. Reddit also gives you everything as JSON if you just add ".json" behind any URL.
[+] luplex|5 years ago|reply
It's a situation like parents: There should be an incentive for a company to build a closed, innovative system. But once that system converges on features, and they reaped their rewards, it should be replaced by an open standard.

Instant messaging was fine, WhatsApp and Slack brought a lot of innovation and drove adoption, and I would argue it has gotten to a stable place where we should all use Matrix.

We need to figure out that process of standardization and opening up.

[+] lewisjoe|5 years ago|reply
I work on a popular Google Docs alternative product in the market.

I gave some thoughts around this idea of experimenting a standard for collaborative rich text, that any client can implement. The problem though is that people want easy collaboration, more than the freedom to bring their own clients.

To simply put how can we deal with assigning people (for @mentions, comments, document ownership and content locking for a group of people) in such file formats? SaaS universe has a concept of a userbase with unique ids. So when a person is assigned/mentioned the product knows who is relevant and what to do. This implies we need a universal userbase standard (which is already hugely complicated) and is adopted by the SaaS that your target users belong to.

This is one huge roadblock that I don't see any practical solution for.

[+] hirundo|5 years ago|reply
This applies well to the remote work environment. My employer makes no requirement about which editor or terminal or operating system or anything else we use to access their resources. The deliverables are (mostly code) files, and they just don't care what we use to manipulate those. They don't provide any hardware either.

What reads to some as worker exploitation is actually empowerment in practice.

[+] throwaway316943|5 years ago|reply
I’m reading Thinking Forth at the moment and there’s a part in the description of Forth about how it decouples words from their parameters and return types by only allowing stack manipulation. This means that all words share a common interface and so they’re infinitely composable. The reason I started reading this book is due to learning how WebAssembly works which turns out to be quite similar to Forth. It got me thinking if there might be a way to make all code modules interoperable. Being able to compose an application or client out of many independent parts would be a dream come true.
[+] megous|5 years ago|reply
BYOC for shopping online would be nice. Exchange formats for product aggregators sort of solve one part of the equation. But ordering is still an unsolved issue.

It shouldn't be that hard to do. You may need maybe 5 endpoints. Get all product data for the whole stock, get individual product images, get order requirements (can include terms and conditions, desired information for the order, payment options, human contact info, etc.), post an order, get order status.

On this API you can easily create a fully featured 3rd party online shopping clients. Some shopping experiences these days are atrocious.

Some hugely popular platforms have product lists hidden like 4 screens bellow the fold, separated by tons of crap, that just gets in the way. Category selection is placed such, so that categories slide away, when you get to the product list after selecting one of the categories, and if you want to switch category, you just have to scroll all the way up again. No options for selecting alternative views, like in a file manager (large icons with a ton of detail, list view, etc.)

Each new eshop you have to learn how to navigate it, and collecting information about the ToC and payment/delivery methods is a hassle.

Could be so much better if there was some simple web protocol for this, and it could be exposed in addition to existing website, too.

In the simplest implementation, everything but the POST could just be some static json files uploaded to the http server. And POST /order could just be sending an e-mail to someone.

[+] ndespres|5 years ago|reply
I think a major omission from the authors' wishlist of BYOC-capable applications is chat. Maybe in their case they are lucky enough to have a choice in the matter, but I think overall the walled gardens and closed APIs for the major chat programs is a shame. Every day that I have to use Microsoft Teams in an attempt to communicate with coworkers is an exercise in patience. Basic chat functionality should be open and interoperable.
[+] vhanda|5 years ago|reply
disclaimer: Markdown Notes App Author

At least for simpler text documents, where formatting isn't a big concern, markdown seems to be wining as the standard. Sure, each of these have added a few features on top of the markdown standard, but even those features are slowly getting quite standardized. [0][1][2][3][4][5][6]

[0] https://obsidian.md

[1] https://zettlr.com/

[3] https://dendron.so/

[4] https://foambubble.github.io/foam/

[5] https://logseq.com/

[6] https://gitjournal.io

I wish using Git as the VCS would also become more standardized, but we're still lacking good clients which hide the complexity [6] (my project, fails at hiding all of the complexity, but hopefully it'll get there)