top | item 43609554

(no title)

transformi | 11 months ago

Interesting idea of <secret>...maybe extend it to several <secret_i>....to form a groups of secretes with different persons.

In Addition it will be interesting to extend a variation of the game that the players can use tools and execute code to take their preparation one step further.

discuss

order

wongarsu|11 months ago

Most models do pretty well with keeping state in XML if you ask them to. You could extend it to <secret><content>[...]</content><secret_from>P1</secret_from><shared_with>P2, P3</shared_with></secret>. Or tell the model that it can use <secret> tags with xml content and just let it develop a schema on the fly.

At that point, I would love to also see sub-benchmarks how each models's score is affected by being given a schema vs having it make one up, and if the model does better with state in text vs xml vs json. Those don't tell you which model is best, but they are very useful to know for actually using them.