top | item 37818631

Ask HN: LLM in Lab Environment

4 points| Santisco | 2 years ago

Hi folks, AI newbie here,

How to run LLM-based experiments in lab environment? I'm a PhD student focusing on system security, and recently we are trying to catch the trend and find some security issues in LLM training. However, it seems tricky to me on how to set up the environment or simulations. We can buy few A100/H800, is there a standard way to get started?

3 comments

speedgoose|2 years ago

Hi, it depends on your lab.

Because you can afford a few A100/H800, it sounds like you have a descent budget. Do the maths first though. It may be cheaper to rent.

Also you may have access to large scale cluster through your lab. If you are located in Europe and work on European research projects, you may be able to play with one of the EuroHPC supercomputers for example.

https://eurohpc-ju.europa.eu/supercomputers/our-supercompute...

Santisco|2 years ago

Thanks so much for the info!

Another thing is that I'm a little bit confused about all kinds of "LLMs". I looked into hugging face but there are so many different models and they may have variant size or structure. Which model can be a good start? I guess gpt4 is too large to handle in a lab environment (sorry I don't really know much about LLM :).