Just wanted to chime in on TensorRT, it's a well supported product and it's different than gpu-rest-engine. This GitHub repo is simply an example of how to use TensorRT in a specific situation.
> The added benefit of this is that you can use different versions of the drivers side-by-side (in my understanding).
No, you can only have one driver version, the one that correspond to the loaded kernel modules. Installing the driver inside a Docker image makes it non-portable.
If using Docker is an option, the official Dockerfile works well, you just need to modify the FROM line to "nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04". Or "nvidia/cuda:8.0-cudnn5-devel-ubuntu14.04", depending on which version of Ubuntu you want.
No, this is the CUDA toolkit, it doesn't depend on the driver version. You can compile CUDA code without having a GPU (which is the case during a "docker build").
Edit: in other words, your Docker image doesn't depend on a specific driver version and can be ran on any machine with sufficient drivers. Driver files are mounted as a volume when starting the container.
You don't need to match the driver version between the host and the container. Actually, you shouldn't include any driver file inside the container.
All the user-level driver-files required for execution are mounted when the container is started using a volume. This way you can deploy the same container on any machine with NVIDIA drivers installed.
One container can use multiple GPUs on the same machine without problems.
For distributed training (which Caffe doesn't actually support, not the official version), you would have to run one container per instance, but this is more a configuration problem at the framework level, than a Docker or nvidia-docker problem.
Well yes, you do need to have the driver installed on the host OS :)
You can run multiple containers on the same GPU with nvidia-docker, it's exactly the same as running multiple processes (without Docker) on the same GPU.
Author of nvidia-docker here. You can definitely have multiple containers on each GPU if you want. If you find a bug or if you think the documentation was not great, please file a bug!
That's exactly what we do, the image is indeed driverless and we mount the host driver files as a volume (provided by our volume plugin) when the container is launched.
This way, you can launch a CUDA 7.5 container on any machine with driver version 352+
Yes, running containerized machine learning workflows is our primary use case of nvidia-docker internally.
That's why we provide pre-built images for cuDNN and DIGITS on the DockerHub. Our base cuDNN image is now used by TensorFlow, Caffe, CNTK, Theano, to cite a few.
We are only wrapping the Docker CLI, not forking the full code (that would be insane). The wrapper is provided for convenience since it should be enough for most users. If you know what you're doing, you don't need to rely on the wrapper, we explain how you can do that on our wiki:
https://github.com/NVIDIA/nvidia-docker/wiki/Internals
Yes, we have been discussing with a few people at Docker, I hope attending DockerCon and discussing with the team will help us move forward.
The CLI wrapper is provided for convenience since it should be enough for most users.
We recently added advanced documentation on our wiki, we explain how you can avoid relying on the nvidia-docker wrapper:
https://github.com/NVIDIA/nvidia-docker/wiki/Internals
This section should also have plenty of information for you if your goal is to integrate GPU support in your own container tool.