I'm thoroughly convinced that software engineering efficiency can be increased by at least one order of magnitude by letting domain experts and product owners directly modify the product through a well defined DSL.
To create that ideal dsl one has to both know the domain well enough and have the technical skills which is a rare combination so i'm rather sceptical of this claim.
Besides any dsl can only help with well understood repeatable problems, for problems that aren't covered by a dsl, software engineers are still required.
The fundamental role of software engineers is to build easy to use and insightful interfaces to understand complex data generated/collected from the real world.
To do that one needs to have the skill to organise information and control complexity by data hiding not exactly the skills product owners and domain experts are known for.
We built the first open-source feature store for ML, https://github.com/logicalclocks/hopsworks , when every existing proprietary feature store (Uber Michelangelo and Bighead at AirBnb) were shouting about how their DSL for feature engineering was the future.
Fast-forward 2 years and it is clear that Data Scientists want to work with Python, not with a DSL. We based our Feature Store on a Dataframe API for Python/PySpark. The DSL can never evolve at the same rate as libraries in a general-purpose programming language. So, your DSL is great for show-casing a Feature Store, but when you need to compute embeddings or train a GAN or done any type of feature engineering that is not a simple time-window aggregation, you pull out Python (or Scala/Java). I am old enough to have seen many DSLs in different domains (GUIs, aspect-oriented programming, feature engineering) have their day in the sun only to be replaced by general-purpose programming languages due to their unmatched utility.
The article (controversely, maybe) classifies libraries for general-purpose programming languages as internal DSLs. One could argue that Data Scientists working with libraries in Pythin are already using a DSL, just with an escape hatch into the general-purpose world.
I don't think proper vertical DSLs should be made marketed towards people who are comfortable working in a general-purpose language. I see them as a way to help non-technical domain experts work on code instead of specification. Limiting the possibilities of what one can write, like with MPS' projectional editor, is a feature here and not a bug.
aszen|5 years ago
Besides any dsl can only help with well understood repeatable problems, for problems that aren't covered by a dsl, software engineers are still required.
The fundamental role of software engineers is to build easy to use and insightful interfaces to understand complex data generated/collected from the real world. To do that one needs to have the skill to organise information and control complexity by data hiding not exactly the skills product owners and domain experts are known for.
jamesblonde|5 years ago
Fast-forward 2 years and it is clear that Data Scientists want to work with Python, not with a DSL. We based our Feature Store on a Dataframe API for Python/PySpark. The DSL can never evolve at the same rate as libraries in a general-purpose programming language. So, your DSL is great for show-casing a Feature Store, but when you need to compute embeddings or train a GAN or done any type of feature engineering that is not a simple time-window aggregation, you pull out Python (or Scala/Java). I am old enough to have seen many DSLs in different domains (GUIs, aspect-oriented programming, feature engineering) have their day in the sun only to be replaced by general-purpose programming languages due to their unmatched utility.
bnr|5 years ago
I don't think proper vertical DSLs should be made marketed towards people who are comfortable working in a general-purpose language. I see them as a way to help non-technical domain experts work on code instead of specification. Limiting the possibilities of what one can write, like with MPS' projectional editor, is a feature here and not a bug.
layoutIfNeeded|5 years ago