Ask HN: Anybody using Amazon Machine Image for AWS Deep Learning?

[DrNuke]

Hello, I do not own GPU hardware and am trying to understand if public amis on AWS are worth the fuss at professional level or at least good enough for the likes of course.fast.ai moocs. Newest here https://aws.amazon.com/marketplace/pp/B06VSPXKDX for Ubuntu. Any experience? Thanks!

[bhouston]

GPUs in the cloud right now are so pricy, in part I think because NVIDIA is making people put in K40s and K80 - the Quadro and Telsa class cards that are super expensive.

We started to put on premise machines that have GPUs in them to reduce our costs. We can get a desktop class Core i7 machine with a GeForce 1060 6GB for around $800 all in, where as on Amazon you have to have a G2 2.xlarge on demand it is $400/month, or $3478 one year term (that is for 1/4 of a G2 8.xlarge achine, thus you get one of its 4 Quadro K5000 4GB cards to yourself.) I could put 2 1060s in that machine for another $200 or so. This just beats AWS pricing in <4 months of use.

My expectation is that these 1060s will be good for two years, maybe even three.

Because our GPU tasks are low bandwidth and non-critical we can run them on our existing office internet. Our rent also includes power. So there are not too many additional costs over the bare machine costs.

[mabbo]

I think you're definitely winning on price- provided you plan to run things continuously for that time period. The AWS GPU instances are probably not aimed at you though.

It's far more likely to be aimed at the Master's student in ML who has this new awesome model but it takes goddamn forever to train on his laptop and he just needs 3 days of time with a big-ass GPU instance.

He spins up the instances, runs his model, spins it down and is out what, $100 maybe? That fits into his limited budget.

Additional benefit: if I buy a kickass GPU, I'm going to reduce my overall productivity in research because now I can finally play some of these hot new games on full graphics!

[jph00]

To answer your question about the http://course.fast.ai MOOC: the course provides an AMI that has everything you need pre-installed. Just follow the directions in the setup video, which includes a script that sets up the instance for you.

To save around 80% (but slightly less convenient) there are two ways to use spot instances provided on http://forums.fast.ai that are largely automated and work well.

In part 2 of the course you'll learn how to set up your own server. As others have mentioned that's much better value if you're using deep learning somewhat regularly.

(I'm the founder/teacher of this course.)

[shoshin23]

I can totally vouch for the ease with which you can setup your deep learning AMI. I followed the instructions on the site and the scripts provided by Jeremy abstract a lot of the stuff away. Super cool work!

[rocketcity]

I have been using it a good bit for some of the deep learning assignments on Siraj Raval's channel (https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A). I have had a really good experience so far. The trick is to find the AMI id for whichever region you want to use the image in and then just put in a spot request for your instance. I just checked the current going rate for a P2.XL in us-east-1 and it was at 0.2355 per hour.

[ShirsenduK]

Siraj is _the_ bomb! Fellow wizard here :)

[torbjorn]

Siraj has great material.

[Fripplebubby]

I've been using this AMI for CNTK since the AMI was made available recently. It's been working well for me - they seem to update the AMI for big tagged releases of the libraries, and all the CUDA drivers are good to go out of the box. Personally I found it easier to use than Docker images, and both are easier than manual install.

With a spot request you can get a p2.xlarge for around $0.22/hr, which for me is sufficient for moocs and side projects, but for beefier instances the prices get high.

[KaiserPro]

I'm using them at $work. But they are expensive, but so is EC2.

Unless you are able to get a decent deal (far better than reserved instances) then for raw CPU/GPU power hosting your own is better. Its the whole package that makes AWS attractive. (RDS, S3, instant spinup)

Spot prices is a good place to start if you are doing batches. just make sure you checkpoint little and often.

However if you are on your own, with limited funds, Get a second hand workstation (dell or hp z620/600) they tend to have decent PSUs, loads of ram and decent support, and a couple of Geforces with more Vram. (Quadros you are paying for the double float performance and ram. transfer Bandwidth and speed are the same or slightly better on geforce class cards.)

[minimaxir]

A helpful on-topic note: Amazon has 1x K80 cloud GPU instances for $0.90/hr (not prorated), but very recently, Google announced GPU instances of the same caliber (https://cloud.google.com/compute/docs/gpus/add-gpus) for ~$0.75/hr (prorated to the minute), although Google Compute Engine requires configuring the GPU drivers first.

[uberneo]

If you are just getting started then https://www.floydhub.com/ is a good place as they provide initial 100 mins on GPU for free and then later on approx 50% cheaper than AWS

[neil1023]

Correction: 100 hours

[chaosmail]

I used the ami with a p2.large for a deep learning project at university. In my opinion it was quite pricey and reading from disk storage was disappointingly slow (we had a 60gb dataset). What i found really great was using the same machine first as a t2.micro instance for setup and testing s3 connection, a m4.large for some preprocessing and finally the p2 for deep learning, all on the same AMI and SSD.

[kshnell]

www.paperspace.com/ml has much more affordable GPUs than AWS and a new Pascal card. There's a pre-configured Deep Learning template and a couple interesting features like a browser based terminal and desktop (you can still SSH if you prefer).

[dkobran]

Paperspace co-founder here -- happy to discuss our AWS alternative which is focused on affordability, simplicity and performance. Recently, we've had a huge influx of people in moocs offloading their work to our GPUs/cloud. Would love any feedback you have.

[coffeepants]

https://paperspace.com is great. Takes about 2 minutes to go from no account to running a VM with all of the ML frameworks and drivers installed. The K80 on AWS/GCP is three years old and is just not built for Deep Learning. Paperspace has newer, more modern GPUs and I heard they are adding more cards soon (forget where I saw that).

[dhruvp]

Hey OP! I've been using it extensively and it's definitely more than good enough for most tasks you'll be doing. If you're going this route, I'd highly recommend following the Keras' authors guide on how to set up the instance and connect to it with a jupyter notebook: https://blog.keras.io/running-jupyter-notebooks-on-gpu-on-aw.... This will boost your productivity a ton in the long run.

Best of luck!

[ap46]

Just get the GTX 1060 6GB variant & that should be enough RAM to run most models without suffering pagination.

[chrischen]

A pascal titan x is the best card for deep learning.

Even my gtx 1080 is faster, albeit with slightly less ram than the K80 or M60s available in the cloud.

A titan X would fix the ram issue though.

The most economical would probably be to colocate some titan X boxes. Not only cheaper, but more percormant than anything you can get from IaaS right now.

[lekker]

GPU hardware on Amazon is quite pricey. I would be careful since it can get out of hand rather quickly.

[farhanhubble]

I'm using p2.xlarge for Udacity Self Driving card nanodegree and Fast.AI assignments. It's super pricey as everyone has mentioned but I prefer using AWS to owning my own rig because I can download large datasets quickly.

[hackpert]

Yeah. I mostly end up using p2.xlarge spot instances, and they have worked fine for my jobs so far. Be sure to checkpoint often and you'll be okay with spot instances.

[Baeocystin]

No. CUDA-capable hardware is dirt cheap, and once you have your own, you can run experiments 24/7.

[benmanns]

I don't have experience with the Amazon AMIs for deep learning, but I'll give the tip that you can use spot instances with limited lifespans (x instance at $y price for at most z hours) to reduce cost and the risk you forget to shut something down.

[Fripplebubby]

Doing spot requests is a great way to keep the costs down if you just want to rent a GPU for a while. But bear in mind that GPU instance types tend to spike in price all-at-once when somebody brings a huge compute job online, which will exceed your spot request's max bid with no warning and immediately terminate your instance. So make sure whatever you're doing can be interrupted!

[gm-conspiracy]

So, what is a recommended, buildable GPU rig (bang for buck)?

i7 w/ dual 1060 GPUs?

[projproj]

I just built one partly for this purpose. I went with AMD's new Ryzen 1700. It has 8 cores and 16 threads. More threads are great if your application can use them. My understanding is that TensorFlow, for example, does[1]. i7s do better in applications that only use a single thread, but you get a really good cost/performance ratio for multi-threaded work from Ryzen [2]. The 1700 also comes with a serviceable air cooler, so you save some money there too.

The other components are a geforce 1060, Asus Prime X370-pro (lowest-priced x370 motherboard), and 16gb ram.

[1] https://stackoverflow.com/questions/39395198/configuring-ten... [2] https://www.phoronix.com/scan.php?page=article&item=amd-ryze...

edit: added other specs

[lloyd-christmas]

my year and a half old rig is 2x 980ti and a 5930k, which is also my generic home computer. I only use AWS when it's a long running job and the weather is hot. It can really heat up your home on long running jobs, so keep that in mind.