Show HN: Steiner – An open-source reasoning model inspired by OpenAI o1

[peakji]

Steiner is a series of reasoning models trained on synthetic data using reinforcement learning. These models can explore multiple reasoning paths in an autoregressive manner during inference and autonomously verify or backtrack when necessary, enabling a linear traversal of the implicit search tree.

Blog: https://medium.com/@peakji/a-small-step-towards-reproducing-...

Hugging Face: https://huggingface.co/collections/peakji/steiner-preview-67...

[schmeichel]

This seems promising! Great work! Any chance there will be a Ollama Modelfile for the masses?

[peakji]

GGUF files are available on HF: https://huggingface.co/peakji/steiner-32b-preview-gguf

I haven't personally used Ollama Modelfile, but I think it should be relatively easy to convert from GGUF?

[swyx]

advice to OP - you hurt your own credibility posting on medium dot com. just blog on huggingface or substack or hashnode.

[peakji]

I'm new here. Just curious, why avoid Medium? Is it a Hacker News thing, or did I miss something?

[mdaniel]

For those similarly allergic to medium: https://scribe.rip/@peakji/a-small-step-towards-reproducing-...

[nxobject]

As someone without specific background in the subfield (I do embedded programming) – thanks for spelling out what people "in the know" seem to understand about o1's functioning!

[zby]

Can it be mixed with the sampling based approaches from optillm (https://github.com/codelion/optillm)?

[peakji]

Approaches like best of n sampling and majority voting are definitely feasible. But I don't recommend trying things related to CoT, as it might interfere with the internalized reasoning patterns.

[nwnwhwje]

Silly question time.

Is this a fined tuned LLM, for example drop in replacement for Llama etc.

Or is it some algorithm on top of an LLM, doing some chain of reasoning?

[peakji]

It is an LLM fine-tuned using a new type of dataset and RL reward. It's good at reasoning, but I would not recommend to replace Llama for general tasks.

[Mr_Bees69]

Really hope this goes somewhere, o1 without openai's costs and restrictions would be sweet.

[peakji]

The model can already answer some tricky questions that other models (including GPT-4o) have failed to address, achieving a +5.56 improvement on the GPQA-Diamond dataset. Unfortunately, it has not yet managed to reproduce inference-time scaling. I will continue to explore different approaches!

[ActorNightly]

OpenAIs o1 isnt really going that far though. Its definitelly better in some areas, but not overall better.

Im wondering if we can abstract chain of thought further down into the computation levels to replace a lot of matrix multiply. Like smaller transformers with less parameters and more selection of which transformer to use through search.