Parallelism
Parallelism is the physical simultaneous execution of multiple tasks (multi-core/multi-machine), distinct from concurrency's logical simultaneity.
Purpose
Leverage multiple cores/machines to speed up compute-intensive tasks.
Mechanism
Data parallelism: SIMD/GPU apply the same operation to different data. Task parallelism: multiple cores run different tasks. MapReduce: distributed data parallelism. Mind Amdahl's law (the serial portion is the bottleneck).
Use cases
- Performance speedup
- Scientific computing
- Distributed processing