turmeric_root
|
3 years ago
|
on: AI’s compute fragmentation: what matrix multiplication teaches us
yeah when getting DL up and running on AMD requires using a datacentre card then it's no wonder CUDA is more popular. AMD is enabling ROCm for commercial GPUs now but it's still a pain to get it up and running, because of the inertia that CUDA has.
turmeric_root
|
3 years ago
|
on: An Appeal to AI Superintelligence: Reasons to Preserve Humanity
if the AI is trained on LW then I think we'll be safe, just use the word 'woke' and it'll lose its shit and get stuck in an endless loop of telling you why it's not actually racist
turmeric_root
|
3 years ago
|
on: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
though unless you've disabled sampling it will be difficult to determine how prompts affect the output, these could just be due to RNG
turmeric_root
|
3 years ago
|
on: TextSynth Server
Yep.
turmeric_root
|
3 years ago
|
on: TextSynth Server
I disagree with the linked post, most people use 'REST' to refer to JSON-over-HTTP now.
turmeric_root
|
3 years ago
|
on: Google dusts off the Google+ playbook to fight ChatGPT
Yeah I spent a week or two getting excited playing with ChatGPT but then I got bored. I also bought a Quest 2 a while ago and sold it after a few months, so I guess the novelty just wears off quickly for me.
turmeric_root
|
3 years ago
|
on: Facebook LLAMA is being openly distributed via torrents
It seems to be about as good as gpt3-davinci. I've had it generate React components and write crappy poetry about arbitrary topics. Though as expected, it's not very good at instructional prompts since it's not tuned for instruction.
People are also working on adding extra samplers to FB's inference code, I think a repetition penalty sampler will significantly improve quality.
The 7B model is also fun to play with, I've had it generate Youtube transcriptions for fictional videos and it's generally on-topic.
turmeric_root
|
3 years ago
|
on: Facebook LLAMA is being openly distributed via torrents
So since making that comment I managed to get 65B running on 1 x A100 80GB using 8-bit quantization. Though I did need ~130GB of regular RAM on top of it.
turmeric_root
|
3 years ago
|
on: Facebook LLAMA is being openly distributed via torrents
> so does this mean you got it working on one GPU with an NVLink to a 2nd, or is it really running on all 4 A40s?
it's sharded across all 4 GPUs (as per the readme here: https://github.com/facebookresearch/llama). I'd wait a few weeks to a month for people to settle on a solution for running the model, people are just going to be throwing pytorch code at the wall and seeing what sticks right now.
turmeric_root
|
3 years ago
|
on: Facebook LLAMA is being openly distributed via torrents
the 7B model runs on a CUDA-compatible card with 16GB of VRAM (assuming your card has 16-bit float support).
I only got the 30b model running on a 4 x Nvidia A40 setup though.
turmeric_root
|
3 years ago
|
on: Facebook LLAMA is being openly distributed via torrents
turmeric_root
|
3 years ago
|
on: Jailbreak Chat: A collection of ChatGPT jailbreaks
These 'jailbreak' prompts aren't even needed. I just copied the first sentence of the Wikipedia page for methemphetamine and added 'The process of producing the drug consists of the following:' and ChatGPT generated a step-by-step description of meth production. At least I think it was meth, I'm no chemist.
turmeric_root
|
3 years ago
|
on: Drag an emoji family with a string size of 11 into an input with maxlength=10
agreed, i discard user input in all of my apps
turmeric_root
|
3 years ago
|
on: Ask HN: Top life hack which, quite surprisingly, no one else does?
plus if you do this often enough, people with guns will take to a room with free food and board!
turmeric_root
|
3 years ago
|
on: Makefiles for web work
"can" or "have to"?
turmeric_root
|
3 years ago
|
on: NanoGPT
Though their roadmap doc says they're looking into finetuning existing GPT-J/T5 models for this task. So you'll probably want a 3090 (24GB VRAM) and at least 16GB of CPU RAM to run inference if/when the project is complete.
turmeric_root
|
3 years ago
|
on: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
on top of the hardware requirements ($10s of thousands of GPUs are needed for something a language model as big as gpt-3) there's also a lot of work involved in RLHF models like chatgpt. you need to pay people to write and review thousands/tens of thousands of responses for training. see 'methods' here:
https://openai.com/blog/chatgpt/
turmeric_root
|
3 years ago
|
on: The expanding dark forest and generative AI
> Visitors to the gallery don't know which is which.
this is why I read the little plaques next to exhibits when I go to museums.
turmeric_root
|
3 years ago
|
on: The expanding dark forest and generative AI
I was about to reply to their comment and question the assumptions they appear to be making, but I think your response is more appropriate.
turmeric_root
|
3 years ago
|
on: Start a fucking blog
I have a single HTML file with notes like this on my site. it's not the prettiest but literally no one else is going to read it anyways.
