Machine-Learning

Created by Bigge, Julian, last modified by Potthoff, Sebastian on 11. Aug 2021

Palma provides a variety of tools to support statistical and machine learning based data analysis on both CPU or accelerated by one or more GPUs.

Below you can find some commonly used frameworks for different application domains that are currently supported by Palma.

Note: For the sake of simplicity, these pages focus on the frameworks and techniques based on the Python programming language. Keep in mind that there are alternative solutions in other languages such as R, MatLab or Julia which are not mentioned here.

Available Frameworks

For documentation on how to use the module system, see here.

For a comprehensive list of available software on the different queues, see here.

Normal Queue (CPU)

For simpler (image-) classification tasks with very small images or simple numerical features, training and inference does not necessarily benefit from being executed on a GPU.

In these cases, Palma provides framework installation that can be used on a default compute node in the normal queue.

Toolchain	scikit-learn	TensorFlow	PyTorch
palma/2019b → foss	0.21.3-Python-3.7.4	2.1.0-Python-3.7.4	1.4.0-Python-3.7.4 1.6.0-Python-3.7.4
palma/2020a → foss	0.23.1-Python-3.8.2	2.3.1-Python-3.8.2	1.7.1-Python-3.8.2
palma/2020b → foss	TBA	TBA	1.7.1 1.9.0

GPU Queue

For larger deep learning tasks with larger images or huge feature vectors it is recommended to execute training with accelerator devices like a GPU.

Therefore, the TensorFlow and PyTorch mentioned above are also available with CUDA support on Palma's GPU queues:

Toolchain	TensorFlow	PyTorch	Queue
palma/2019b → fosscuda	TensorFlow 1.13.1/1.14.0/2.2.0	1.4.0-Python-3.7.4 1.6.0-Python-3.7.4	gpuv100, gpu2080, gputitanrtx
palma/2020a → fosscuda	TensorFlow 2.3.1	1.7.1-Python-3.8.2	gpuv100
palma/2020b → fosscuda	TensorFlow 2.4.1	1.9.0	gpuv100