[renchuw]

I designed 2 modes in the project, exploration mode and exploitation mode.

Exploration mode uses entropy search to explore the latent space (used for evaluating the LLM on the selected corpus to evaluate), and eploitation mode is used to figure out how well / bad the model is performing on what regions of the selected corpus.

For accurate evaluations, exploration is used. However, I'm also working on a visualization too s.t. users can see how well the model is performing at what region (courtesy of gaussian process models built in by bayesian optimization) and that is where exploitation mode can come in handy.

Sorry for the slightly messy explanation. Hope it clarifies things!

[abhgh]

Thanks for the explanation!

I don't entirely understand what two models mean here, because typically the search strategy (or acquisition function) in bayesopt - which in your case seems to be some form of entropy search (ES) - decides the explore-vs-exploit tradeoff for itself (possibly with some additional hyperparams ofc). For ex., ES would do this one way, Expected Improvement (EI) would do it differently, etc. - all this in the service of the bayesopt objective you want to maximize (or minimize).

Assuming that you mean this objective when you mention exploitation, which here is based on the model performing well, wouldn't it just pick queries that the model can (or is likely to) answer correctly? This would be a very optimistic evaluation of the LLM.