Elements of Research Computing¶

• Elements of Research Computing
  • Preliminary remarks
  • Some terminology
  • Aspects of Research Computing
  • Project management
  • Code verification
  • Debugging
  • Scability estimates
  • Profiling
  • Optimization
  • Parallelization
    • Embarassingly Parallel (= Distributed ?)
    • Shared Memory Parallelization
    • Distributed Memory Parallelization
    • General Purpose GPUs

The goal of this document is to introduce some concepts, tools and best practices for research oriented computing. Advanced documents for specific topics can be arranged according to demand. Authors/speakers for specific topics are also welcome.

Preliminary remarks¶

The categorization "Research Computing" was chosen because, on one hand, that's the audience of this document, researchers that use computational tools; on the other hand, traditional terms like HPC, Grid, Cloud tend to separate from the onset the infrastructure that is going to be used to solve a specific problem, but in most cases one doesn't know what the best infrastructure(s) will be.

In any case, not all aspects of Research Computing will be covered - we will focus mainly on non-interactive "jobs" that have some sort of intensive requirement such as performance, memory, network, storage, etc.

Some terminology¶

• High Performance
  • perform a specific task in a short period of time (e.g. low latency)

• High Throughput
  • perform many tasks in a fixed period of time (e.g. high bandwidth)

• Concurrent
  • concurrency is a property of the algorithm (e.g. independence of tasks)

• Parallel
  • concurrent parts of an algorithm can (or not) be run in parallel

• Distributed
  • distributed generally means loosely parallel (e.g. asynchronous)

• Grid
  • Grid usally means a collection of clusters with interoperability at scheduler level

• Cloud
  • Cloud means a lot of different things (e.g. Infraestructure/Platform/Software as Services)

Aspects of Research Computing¶

• Project management
• Code verification
• Debugging
• Scability estimates
• Profiling
• Optimization
• Parallelization

Project management¶

Code verification¶

Debugging¶

Scability estimates¶

Profiling¶

Optimization¶

Parallelization¶

Embarassingly Parallel (= Distributed ?)¶

• bash subshells

• GNU Parallel

• Scheduler

• MapReduce

Shared Memory Parallelization¶

• POSIX Threads

• OpenMP

Distributed Memory Parallelization¶

• MPI

General Purpose GPUs¶

• CUDA

• OpenCL

...