top | item 19489607 (no title) ss7pro | 7 years ago Have a look here: https://github.com/IntelAI/OpenVINO-model-server/blob/master... You can replace tf-serving with OpenVINO to get even better performance and latency when running on CPU discuss order hn newest londons_explore|7 years ago What useful models run at decent speed on a CPU these days?Even basic image classifiers tend to be 100x faster on a GPU or TPU... bitL|7 years ago Inference is not that super slow on CPU, especially for network requests that already have quite a bit of latency, so plenty of companies use CPUs on the cloud for lambda/flexible loads where GPUs aren't available.
londons_explore|7 years ago What useful models run at decent speed on a CPU these days?Even basic image classifiers tend to be 100x faster on a GPU or TPU... bitL|7 years ago Inference is not that super slow on CPU, especially for network requests that already have quite a bit of latency, so plenty of companies use CPUs on the cloud for lambda/flexible loads where GPUs aren't available.
bitL|7 years ago Inference is not that super slow on CPU, especially for network requests that already have quite a bit of latency, so plenty of companies use CPUs on the cloud for lambda/flexible loads where GPUs aren't available.
londons_explore|7 years ago
Even basic image classifiers tend to be 100x faster on a GPU or TPU...
bitL|7 years ago