01-AI/Yi: A series of large language models trained from scratch

[niel]

Previous discussion: https://news.ycombinator.com/item?id=38158554

[dang]

Thanks! Macroexpanded:

Kai Fu Lee's New AI Company: Yi-Open Source - https://news.ycombinator.com/item?id=38158554 - Nov 2023 (66 comments)

[brucethemoose2]

Not sure if its worth a new HN post, but a 200K context version of the base model just dropped:

https://huggingface.co/01-ai/Yi-34B-200K

Even untuned, I am liking the 34B 4K model so far. I never really use raw llamav2 70b though.

[popinman322]

When I saw "from scratch" instead of "pretrained" I assumed this was trained using some novel RL setup-- there's value to using the same verbiage as everyone else.

[zapnuk]

These pedantic “when I saw X instead of Y I assumed Z” are the most annoying comments on this website.

“From scratch” might be non-specific, but a fine phrasing in this case.

[GaggiX]

"Trained from scratch" is perfectly fine terminology to report that the model they published is not a finetuned, but was trained from randomly initialized weights.

[Mougatine]

"From scratch" is commonly used in the field.

[synarchefriend]

They probably want to emphasize that it's not another llama derivative.

[leoff]

It's scratch, the programming language

https://scratch.mit.edu/about

[seydor]

I would have liked to see how it compares with Mistral

[logicchains]

China's catching up fast in the open source model space, I wonder how long it'll take until they have a commercial model competitive with ChatGPT3.5 or Claude 2?

[antupis]

Who cares if it is open source model and weights are available. Nightmare scenario is that AI is behind some paywall and some entity can decide what goes in and what goes out.

[zone411]

They do already. Ernie 4.0.

[ilaksh]

Looks amazing. Too bad they don't allow commercial use of the model (without a license agreement).

[gs17]

Non-commercial use is also limited.

> Your use of the Yi Series Models must comply with the Laws and Regulations as well as applicable legal requirements of other countries/regions, and respect social ethics and moral standards

("Laws and Regulations" is specifically mainland China's)

[dvh]

I downloaded the repository and it is 700kB (1900 LOC). It clearly doesn't contains what it claims to have. How is this considered "open source"?

[TheBlapse]

Weights are available on huggingface: https://huggingface.co/01-ai/Yi-34B/

[knapcio]

Please read the project description (Point 2 - 2. Download the model).

[brucethemoose2]

I'd posit you didn't install git-lfs.

You probably want to download a quant instead anyway, even if you are on a very fast PC.

[ChrisArchitect]

[dupe]

https://news.ycombinator.com/item?id=38158554

[anon23432343]

Another hour another new AI model.

Which can't solve bubble sort correct and will output you a bad performing version of it.

AI is the future.

[esafak]

It's a language model. You don't expect your car to fly.

[postalrat]

I'm a programmer and I couldn't tell you how I could "solve" a bubble sort.