[sgaur]

Several data engineers that I know in this space (AI for business applications) are doing the usual ETL + mapping pipeline related work. Nowadays a lot of them are also having to deal with unstructured data such as textual reviews of products, service quality feedback, policy documents. Data engineers are developing the pipelines for chunking, vectorization, ingestion into RAG pipeline and for LLM training fine tuning. So it's still collecting, cleaning data, but quite different at the next level of detail, in my opinion.

[akudha]

Thank you. How does one get into this field? Web dev isn’t that exciting anymore

[sevensor]

The data engineers I know started out as data scientists. They discovered that their ability to make models that did anything useful was hampered by low-quality or absent data. Then they discovered that there was a lot more job satisfaction (not to mention much better pay) in making useful data available than there was in clicking the "train" button over and over in slightly different configurations.

[gaws]

> How does one get into this field?

Network and meet someone who will give you, a web developer (a profession that is heavily stigmatized in the industry), a chance in a very competitive field.