ahmedNarrator's comments

Yeah DynamoDB is very much aligned with a single table. We do it in the customers warehouse so they have access to all their data.

The real magic is how do you assemble the data into tables that can answer any question when the output table is billions of rows. Unfortunately, that requires a lot of SQL magic that depends on taking advantage of a columnar warehouse.

We would love that but Narrator works on any warehouse. To support that we built a query abstraction later that compiles to the flavor of SQL used by the customers warehouse.

(We will open source that query abstraction later with a demo where you can translate Redshift Queries to Snowflake Queries).

Maybe in the future we can get that project into dbt so that dbt models can work on any warehouse as well.

This is so great! You see exactly what we see and clearly you have shared similar experiences with dashboards not matching because of wrong table. (The good old "spent 3 weeks debugging an analysis using sales_data and then finally found that sales_data_v2 was built to solve it).

Yeah we do something very similar to dbt for taking restructuring the data into a single time-series table. We add things like identity resolution, diffing, incremental update and computing some cache columns.

Your Crystal Ball is SPOT ON!!! We get 3 kinds of data people. The ones who are like: "THIS WILL NEVER WORK", "Too bad I already built all this" or the "THIS IS THE FUTURE, HOW IS EVERYONE NOT USING IT".

I would love to chat and show you what we have (schedule a demo on our site and it will go to me and we can chat!)

Also, Teaser... When you standardize all of data and you create a consistent way to relating that standardized structure then analysis become very consistent. Imagine a world where your email attribution deep dive can be run by loading a template and point it to your "opened email" activity and your "order activity".... coming soon ... a Narrative Library.

Activity, Customer, Time, Activity, 3 features, Revenue impact. source and source_id (for identity resolution) and a couple of meta data columns.

You can see some example transformations here: https://docs.narrator.ai/docs/by-data-source

Yeah, Entity modeling was one of the big inspirations to our approach. The main difference is how do you reassemble the single time-series table to create any table.

This was quite a challenge and I think what makes the traceability and source of truth problem a lot simpler.

In Narrator, the data team writes small SQL to create single customer centric business concepts that we call activities. These are around 25 lines and decided to be understood by anyone in the company (i.e. "viewed page", "called us",...).

Now, every question you or a stakeholder has will simply be a rearrangement of these activities. If you can describe what you want, then Narrator can assemble a table that represent it.

Source of truth - What ever is in the activity stream? Tracebility - always Dataset (activities and how they relate), then activities (~25 SQL). Coherent Model - Customers doing actions in time.

Does that make sense? Some of these things are easier to show in a demo then describe in text.