ozr's comments

It's a guy browsing the internet a bit. This is not some deep philosophical question. You're just advocating being the type of person we'd all hate to work with.

By this logic, I should be concerned about defending against raccoon attacks since they are endemic to my area and I often go outside.

The point is that, in practice, the attacks are so uncommon and mitigated by so many other factors that the cost involved of further mitigation it isn't worth it.

You develop a threat model to specifically get rid of concerns like this; not to list every possible attack vector imaginable.

Yes. Whether or not a particular standard has been implemented is not interesting. What matters is the result.

Is BGP an attack vector that matters for the vast majority of threat models right now? I would say no. Given that: there is no need for (inevitably) poor regulation.

BGP operators _have_ self-organized sufficient security measures. Compared to just about any other attack vector on the internet, BGP hijacking is among the least likely to impact most people.

Yes it is; it's just not updating the weights.

https://ai.stanford.edu/blog/understanding-incontext/

The parent's description isn't quite correct. It's kinda sorta describing the implementation; RAG is often implemented via embeddings. In practice, you generally get better results with a mix of vector and, e.g., TF-IDF.

An example of RAG could be: you have a great LLM that was trained at the end of 2023. You want to ask it about something that happened in 2024. You're out of luck.

If you were using RAG, then that LLM would still be useful. You could ask it

> "When does the tiktok ban take effect?"

Your question would be converted to an embedding, and then compared against a database of other embeddings, generated from a corpus of up-to-date information and useful resources (wikipedia, news, etc).

Hopefully it finds a detailed article on the tiktok ban. The input to the LLM could then be something like:

> CONTEXT: <the text of the article>

> USER: When does the tiktok ban take effect?

The data retrieved by the search process allows for relevant in-context learning.

You have augmented the generation of an LLM by retrieving a relevant document.

That's not what RAG is. RAG is the process of adding relevant information to a prompt in an LLM. It's a form of in-context learning.

Blue Origin doesn't have to, and had much more investment. Virgin, too.

It's not just SpaceX outperforming NASA, it's SpaceX outperforming everyone.

This is more rhetoric driven by a personal dislike of someone vs. reality.

> Twitter dropped in value to just 25% of what it was.

It's a private company. You don't know the value.

> Racism and naziism are rampant, endless stream of bots pushing propaganda or porn

Maybe on your feed? Statistically, no.

> and has any advertiser actually come back?

Who knows. It's still running after firing anyone: that's the point.

Tesla was the first and still the only electric car player that matters.

Same with SpaceX and space.

Again: you can dislike Elon's personality or politics, but trying to attack his results is ridiculous.

Twitter is running fine post-firings, and he saved a ton of money.

If Elon is so horrific at SpaceX, why is it the only space organization (including NASA) able to innovate and ship anymore?

You can dislike his personality, but criticizing his performance is silly.

RAG and fine-tuning are very different. Few-shot prompting and RAG are both variants of in-context learning.

I’ve taught LLMs imaginary words and their meanings with minute amounts of data (two or three examples) via full fine-tuning, LoRA and QLoRA.

I have no idea where the myth of ‘can’t add new knowledge via fine-tuning’ came from. It’s a sticky meme that makes no sense.

Pretraining obviously adds knowledge to a model. The difference between pretraining and fine-tuning is the number of tokens and learning rate. That’s it.

> the more Shotwell is running things, the more optimistic I am about SpaceX’s future.

Her age is relevant when discussing the future of SpaceX, considering the median retirement age.

She's 60 years old now.

The awesome-X lists seem like a community attempt to do something similar.

He's obviously not closing Jira tickets at Meta, but that doesn't mean he's not an engineer. As an example of the positive impact that the Llama releases have had, this post from him has been doing the rounds lately in response to criticsm like yours:

https://www.facebook.com/notes/775294156352065/

If your product is an AI model (OpenAI, Anthropic, etc) you can't give it away for free.

If your product is a social graph w/ ads (Meta), you can.

It's hardly corporate charity:

* Meta releasing these models creates an improvement and tuning ecosystem around it, giving them access to tons of free developer time.

* It's also a strong recruiting tool, for engineers and researchers frustrated by, e.g., Google and OpenAI becoming increasingly closed. They know they can publish at Meta.

* The cost is insignificant. Meta had 30B in revenue just in Q2 2023.

For a k/v store, a hash table will take you very, very far.

You aren't required to purchase a home or drive a car. If you chose to, as most Americans do, then yes: the demand is inelastic.

But that doesn't imply what you're saying unless the supplier has monopoly power, which they, by law, do not.

Insurance carriers have an incentive to compete on price like any other business. There are plenty of options. The margins are generally pretty small, and there is a lot of regulation around what portion of payments must be used for claims.

There's a few obvious exceptions, but plenty of insurance isn't required.