Mathematica v12.3: CUDA GPU still not working

Ideas
,
8

First of all, some remarks and comments:

  • GPU support in our system is not a single piece of functionality. CUDALink (CUDAInformation/InstallCUDA/CUDAFoldList, basically everything having CUDA in its name) is completely separated from the machine learning side (NetTrain/Classify/Predict/neural net evaluation, basically everythng using TargetDevice). One does not need to run InstallCuda nor to do a manual installation to do machine learning on a GPU, only a driver installation is needed. CUDALink and ML functionality are expected to coexist smoothly in the same session, but I would avoid loading CUDALink at all when testing for GPU problems on the ML side.
  • The issue you mentioned ( CUDA not working on Mathematica 12.2 ) is about using an RTX 3090 on WL 12.2. This GPU came out too late in the release cycle of WL 12.2 and it was unsupported when the issue was opened (January). In March we have pushed an update to make WL 12.2 support these new cards
  • You can find a list of supported GPU for machine learning in the details section of the documentation page of TargetDevice: TargetDevice—Wolfram Documentation
  • We are indeed lagging behind on integrated tools for reinforcement learning, but it’s still possible to do some RL by building your own infrastructure. See this tutorial: Train an Agent in a Reinforcement Learning Environment: New in Wolfram Language 12
  • AnomalyDetection does not support GPU. TargetDevice is not mentioned in its documentation page. It does show up in Options[AnomalyDetection] though, and it shouldn’t. Same applies for LearnDistribution. The only “classic” ML function supporting TargetDevice are Classify/Predict, and only when Method -> "NeuralNetwork" is selected

Now for the remaining issue you have reported, NetTrain not using the GPU despite TargetDevice -> "GPU" was selected. What makes you think this? There is nothing hinting at this in your report. Timing calculation should be compared with a CPU training call, and even in this case I don’t expect it to be revealing because the net you have chosen is so tiny that the CPU could easily outperform the GPU. This test should be carried out with a larger net, e.g. NetTrain[NetModel["LeNet Trained on MNIST Data", "UninitializedEvaluationNet"], ResourceData["MNIST"], TargetDevice -> "GPU"|"CPU"]. Also checking the GPU activity with an OS or NVIDIA tool during training would be useful.

6 Likes