[ericlewis]

ChatGPT is in-fact that. GPT-4 the API is not that. There is clearly no RAG, you bring that yourself, same with toxicity filtering, again - a different api, up to you to implement. etc. There is probably something going on before the input reaches GPT-4 and we are definitely not seeing the raw output from the LLM. But the layers over GPT-4 the API are thinner than ChatGPT.

The Assistants API is closer to ChatGPT & what you are describing.

Edit: it should be noted that the Assistants API is somewhat model agnostic as well, so the product part of this isn't part of the inference system.