I think the comparison was Snowflake vs Databricks SQL. Databricks SQL is a PaaS service just like Snowflake. Also, it uses their Photon engine, which is a proprietary engine written in C++. It is not Spark.
I'm aware that Databricks is a PaaS service, but what Databricks runs under the hood is Spark (with a few proprietary extensions). So your jobs/queries do require some tuning just like with OS Spark.
Spark has had SQL engines (SparkSQL/Hive on Spark) for a long time. Photon is just a new, faster one. Photon tasks also run on Spark executors only, so it's not independent of Spark[1]. Also, while it's proprietary now, I wouldn't be surprised if Databricks open-sources it in the future, like they did with Delta Lake.
hiyer|4 years ago
Spark has had SQL engines (SparkSQL/Hive on Spark) for a long time. Photon is just a new, faster one. Photon tasks also run on Spark executors only, so it's not independent of Spark[1]. Also, while it's proprietary now, I wouldn't be surprised if Databricks open-sources it in the future, like they did with Delta Lake.
1. https://databricks.com/blog/2021/06/17/announcing-photon-pub...