top | item 45736681

(no title)

maxdemarzi | 4 months ago

The current benchmark for real world text to sql is https://spider2-sql.github.io/ and the SOTA is at 64%

Anyone selling you 99% accuracy can prove it there first.

discuss

order

zurfer|4 months ago

Cofounder of one of those analytics agents here (https://getdot.ai).

The promise of the technology is not that it can deal with any arbitrarily complex Enterprise setup, but rather that you expose it with enough guidance on a controlled and sufficiently good data model.

Depending on your use case this can be super valuable as it enables a lot more people to use data and get relevant recommendations.

But yeah it's work to make nice roads and put up signs everywhere.

macinjosh|4 months ago

I am working with Databricks' Genies. I have a _very_ complex Enterprise data schema(s). Genies, and from what I can tell, your product work on a set of tables ~20 and expect a well thought out and documented data model.

I have hundreds of tables designed by several different teams. I do have decent documentation on the tables but if I had a nice, organized data model I wouldn't need an AI assistant. If I had a perfect data model my team could write simple SQL queries or give chatgpt a schema dump + a natural language query and it would get the answer most of the time.

IMHO, the big value in this space will be when these tools can wrangle realistic databases.

XCSme|4 months ago

I don't see Grok there