top | item 45687480 (no title) svachalek | 4 months ago Is Claude caching by whole message only? Pretty sure OpenAI caches up to the first differing character. discuss order hn newest jasonjmcghee|4 months ago Interesting. Claude places breakpoints. Afaik - no way to do mid message.I believe (but not positive) there are 4 breakpoints.1. End of tool definitions2. End of system prompt3. End of messages thread4. (Least sure) 50% of the way through messages thread?This is how I've seen it done in open source things / seems optimal based on constraints of anthropic API (max 4 breakpoints)
jasonjmcghee|4 months ago Interesting. Claude places breakpoints. Afaik - no way to do mid message.I believe (but not positive) there are 4 breakpoints.1. End of tool definitions2. End of system prompt3. End of messages thread4. (Least sure) 50% of the way through messages thread?This is how I've seen it done in open source things / seems optimal based on constraints of anthropic API (max 4 breakpoints)
jasonjmcghee|4 months ago
I believe (but not positive) there are 4 breakpoints.
1. End of tool definitions
2. End of system prompt
3. End of messages thread
4. (Least sure) 50% of the way through messages thread?
This is how I've seen it done in open source things / seems optimal based on constraints of anthropic API (max 4 breakpoints)