[neighbour]

The guy you're arguing with in the reply thread has a point but I think you're talking past each other because you're coming at it from two different sides. I understand his point about syncing everything if you're a small enough company or the data is small enough because the cost is immaterial.

I agree with most of what you're saying and have seen the downfalls of "just sync all our data" first hand. At a late stage start-up I worked at, we would often have teams (finance, CX, etc.) implement a shiny new tool and immediately request to have that data accessible for analytics/reporting. If we did blindly agree, the data would often never be used and it would run up our Fivetran costs to sync the data and our Github actions bill to build the dbt models and include it in our modeling.

We eventually landed on an approach that considered the following:

- Does the tool already have reporting baked in? If so, just use that until you need more.

- How often are you really going to be using it and what makes this data so special that you need to join it to our other data?

- How much work will be involved to implement a sync for this data? Does Fivetran or Segment have a connector readily available?

- If the tool doesn't have reporting built-in and the team did have a genuine business case for the data being synced, we would figure out which specific objects from the source that needed to be synced and sync them at a fairly infrequent cadence (once a day).

So while I offer no reasonable rebuttal, I wanted to explain a happy medium that worked for us.

[williamcotton]

Thanks for the insight!

As for communicating the issue to administration I feel that putting things in terms of costs in dollars, even if the approach is as arbitrary as any other form of internal accounting, can help make the case.

Management is as susceptible to the marketing of panaceas as much as engineers! They must be informed of a more true cost.