Fergi's comments

From Craft: https://m.youtube.com/watch?v=vBkzm4a7iY4&feature=youtu.be

In 2015 Razorpay pitched themselves as “Stripe for India.” Look at them now!

This is exactly correct. Private blockchains create efficiency between large institutions like JP Morgan and other banks who are constantly moving large volumes of money around by eliminating the traditional processes in place required to move that money. A private blockchain is essentially a shared database (distributed ledger) used by an arbitrary number of private institutions, each of whom has an interest in participating to reap efficiency gains by using a shared database that they can trust, more than they would trust a traditional database owned by a single party. It's not game changing innovation in the same way that crypto futurists think of Bitcoin as a new type of decentralized money, it's incremental efficiency for existing processes. It's more accurate to think about this as blockchain technology for JP Morgan than as cryptocurrency in the sense that Bitcoin is an application of blockchain technology. JPM coin is not like Bitcoin, or ripple for that matter.

The important thing to understand here is that JP Morgan thinks about blockchain technology completely differently than libertarian futurists. Blockchain is not a movement for them - it's a technology that reduces friction for things they are already doing.

Yes, that is my opinion. And if it's possible to build something minimal that helps demo or describe the product to users, it's totally reasonable to build that thing quickly and take it to customers for feedback.

The pitfall to avoid is investing large amounts of time, energy, and money building products in a vacuum based on assumptions about what people want and will pay for. I've heard this referred to as committing "assume-icide."

(Jeff from PipelineDB & post author here)

Thanks for this comment. I think you articulated the thought process that this post aims to speak to beautifully. Builders do want to build, and finding an audience first and doing the type of tedious customer development work described in this post IS an impediment to building, which is precisely the point I wanted to make.

Assuming that the goal of building a product is to ultimately generate revenue, having a temporary impediment between conceptualizing a product idea and building the product is a good thing. This impediment allows for the builder to pause and objectively scrutinize his own idea, using feedback from potential customers as data about the extent to which the product hypothesis is correct.

You're right that stagnation is the worst outcome. And the inverse of stagnation is momentum, which will exist to the extent that people want what we're making for them, something that can be determined in advance of building simply by talking to potential users and customers.

I'll also add here that the process mentioned in this post in no way inhibits the type of creative and inspired thinking that developers use to envision game-changing products. It's quite the opposite - rigorous and merciless scrutiny of our own ideas is the distillation process that allows us to refine our ideas into their essence, then confidently build things with conviction, and be right.

(Jeff from PipelineDB / author here)

Thanks for the honest feedback. You are definitely correct that there are many successful projects and businesses that did not start with a clear sense that people would use or buy their products. But there are a disproportionately large number of projects and businesses that have failed, precisely because they did not have a clear sense that people wanted what they were making and would pay for it.

Also, this post isn't saying that builders shouldn't begin with a strong, fundamental conviction about how things should be dramatically different. I, as the author, would actually argue the exact opposite. The point I'm making is that once we have our convictions, we should test and measure the extent to which those convictions are correct before investing large amounts of time, money, and energy into productizing them.

Lastly, the point about asking questions and then listening to customer feedback would only result in building minimally better products if the product hypothesis we start with is itself unimaginative. But that hypothesis can be literally anything. Customer feedback simply teaches us if market demand lines up with our assumptions about market demand.

I think this is a smart strategy, and is essentially using marketing in the same way that this post suggests using sales, to gauge customer interest in a potential product. The extent to which manual sales outreach or the kind of marketing technique described here should be used likely depends on the kind of product in question. B2B products with higher contract prices and a smaller number of total potential customers will likely require a sales strategy, where B2C products with lots of users and lower contract values may be better served with this kind of marketing tactic.

Great point, though! This is an important topic to consider with marketing and sales strategy development. Thanks for the comment.

(Jeff from PipelineDB / author here)

Good questions!

I would definitely suggest being straightforward and honest with the potential customers you're talking to. And ideally, it would be better to have a MVP you can demo than nothing, but the main point here is that talking to potential customers before building anything is a better strategy than building a product and then talking to customers.

If you talk to a bunch of potential customers about a product idea and discover that there is demand for the product, then building a MVP that you can demo would be a logical next step. And if the MVP is something you can build quickly, then it probably makes sense to do this sooner than later.

The trap to avoid here is building products in a vacuum, based on assumptions about what people want and will pay for. Doing the customer development (pre-sales) work to prove, or disprove, customer demand for a product before building the product is wise.

No, not currently; however, we offer hosted deployments of PipelineDB as well as a SaaS product called Stride (stride.io), which is based on PipelineDB.

Just out of curiosity, do you have a specific use case that necessitates stream-stream JOINs, or were you just exploring the docs and wondering about this?

Fundamentally different approaches. Citus focuses on horizontal scaling for PostgreSQL and PipelineDB focuses on continuous aggregation for large volumes of streaming time-series data.

This only applies to continuous views, not all PG tables. Think of continuous views in PipelineDB as very high throughput, incrementally updated materialized views. Raw data hits continuous queries in PipelineDB (continuous views) and only the output of the continuous queries is stored. So 1 billion events ingested could be distilled down into a single row that incrementally counts up from 1 => 1 billion as each data point arrives, instead of storing all of the 1 billion raw data points and counting them up later.

PostgreSQL 11 support is imminent =)

(Jeff, PipelineDB Co-Founder here) - thanks, Chucky! We appreciate your support!

We are about to release an open-source extension for PostgreSQL that will help with the bloat problem you mentioned here, at least for continuous updates like high volume aggregation (see: pipelinedb.com). PipelineDB will be refactored as a standard PostgreSQL extension this month. Continuous aggregation can help with the bloat problem for use cases where you only need aggregated / summary data, like realtime reporting & monitoring.

Perhaps the most ironic thing I've read in a while:

"Suicides would also decrease, as the most of them are result of depression, which is mostly caused by anxiety about survival."

The storage engine for PipelineDB is PostgreSQL and the output of continuous SQL queries (continuous views in PipelineDB) is stored in what are essentially incrementally updated, realtime tables. You can think of PipelineDB as very high throughput, incrementally updated materialized views, also.

see: http://docs.pipelinedb.com/continuous-views.html

Powering real-time reporting dashboards is definitely the #1 use case we see for PipelineDB from open source users and customers of our new SaaS product powered by PipelineDB, called Stride (stride.io).

Not for streaming analytic workloads, because PipelineDB fundamentally adds continuous queries to PostgreSQL, so data is continuously distilled and aggregated as it arrives, before it is stored, which drastically reduces the amount of data stored in PipelineDB (or soon, in PostgreSQL via the extension refactor).

PipelineDB also offers a clustering extension for large workloads (see: http://enterprise.pipelinedb.com/docs/)

But in terms of ad hoc, exploratory analytics workloads, yes - the scaling limitations would be the same, since for ad hoc, exploratory analytics PipelineDB and PostgreSQL are the same. But with that said, the processed, aggregated data that gets stored is generally much smaller than large volumes of granular data, so there is much less data to comb through with PipelineDB.

PipelineDB will become a standard, open-source PostgreSQL extension this year, enabling anybody using PostgreSQL to leverage continuous SQL queries in combination with all other PostgreSQL functionality.

https://www.pipelinedb.com/blog/pipelinedb-0-9-8-postgresql-...