This seems like a really interesting usecase of LLMs. I’m curious how you’ve validated the outputs and what gives you confidence that the forecasts are good and not affected by hallucinations or incomplete information?
The most interesting aspect of this is backtesting. Quant models get run on past data to see if their predictions work.
When you use LLMs agents, though, you run into their memorized knowledge of the world. And then there's the fact that they do their research on the open internet. It makes backtesting hard - but not impossible.
ddp26|3 months ago
The most interesting aspect of this is backtesting. Quant models get run on past data to see if their predictions work.
When you use LLMs agents, though, you run into their memorized knowledge of the world. And then there's the fact that they do their research on the open internet. It makes backtesting hard - but not impossible.
We wrote about how we do our pastcasting validation here: https://stockfisher.app/backtesting-forecasts-that-use-llms