[hhimanshu]

I am wondering how libraries like DSPY [0] fits in your factor-2 [1]

As I was reading, I saw mention of BAML > (the above example uses BAML to generate the prompt ...

Personally, in my experience hand-writing prompts for extracting structured information from unstructured data has never been easy. With DSPY, my experience has been quite good so far.

As you have used raw prompt from BAML, what do you think of using the raw prompts from DSPY [2]?

[0] https://dspy.ai/

[1] https://github.com/humanlayer/12-factor-agents/blob/main/con...

[2] https://dspy.ai/tutorials/observability/#using-inspect_histo...

[dhorthy]

interesting - I think I have to side with the Boundary (YC W23) folks on this one - if you want bleeding edge performance, you need to be able to open the box and hack on the insides.

I don't agree fully with this article https://www.chrismdp.com/beyond-prompting/ but the comparison of punchards -> assembly -> c -> higher langs is quite useful here

I just don't know when we'll get the right abstraction - i don't think langchain or dspy are the "C programming language" of AI yet (they could get there!).

For now I'll stick to my "close to the metal" workbench where I can inspect tokens, reorder special tokens like system/user/JSON, and dynamically keep up with the idiosyncrasies of new models without being locked up waiting for library support.

[chrismdp]

It's always true that you need to drop down a level of abstraction in order to extract the ultimate performance. (eg I wrote a decent-sized game + engine entirely in C about 10 years ago and played with SIMD vectors to optimise the render loop)

However, I think the vast majority of use cases will not require this level of control, and we will abandon prompts once the tools improve.

Langchain and DSPY are also not there for me either - I think the whole idea of prompting + evals needs a rethink.

(full disclaimer: I'm working on such a tool right now!)