What you need for this book
To follow the instructions in this book, you will need any OS (Windows, Linux, or macOS)
and any C++ compiler installed on your systems.
Who this book is for
This book is for intermediate C++ developers who wish to extend their knowledge of
multithreading and concurrent processing. You should have basic experience with
multithreading and be comfortable using C++ development toolchains on the command
line.


Table of Contents
Preface 1
Chapter 1: Revisiting Multithreading 6
Getting started 6
The multithreaded application 7
Makefile 11
Other applications 13
Summary 14
Chapter 2: Multithreading Implementation on the Processor and OS 15
Defining processes and threads 16
Tasks in x86 (32-bit and 64-bit) 18
Process state in ARM 21
The stack 22
Defining multithreading 23
Flynn's taxonomy 25
Symmetric versus asymmetric multiprocessing 26
Loosely and tightly coupled multiprocessing 27
Combining multiprocessing with multithreading 27
Multithreading types 27
Temporal multithreading 27
Simultaneous multithreading (SMT) 28
Schedulers 28
Tracing the demo application 30
Mutual exclusion implementations 32
Hardware 33
Software 33
Summary 35
Chapter 3: C++ Multithreading APIs 36
API overview 36
POSIX threads 37
Windows support 39
PThreads thread management 40
Mutexes 42
Condition variables 43

Synchronization 45
Semaphores 46
Thread local storage (TLC) 46
Windows threads 47
Thread management 48
Advanced management 50
Synchronization 51
Condition variables 51
Thread local storage 52
Boost 52
Qt 52
QThread 53
Thread pools 54
Synchronization 54
QtConcurrent 55
Thread local storage 55
POCO 55
Thread class 56
Thread pool 56
Thread local storage (TLS) 57
Synchronization 58
C++ threads 59
Putting it together 59
Summary 60
Chapter 4: Thread Synchronization and Communication 61
Safety first 61
The scheduler 62
High-level view 62
Implementation 63
Request class 65
Worker class 67
Dispatcher 69
Makefile 73
Output 74
Sharing data 77
Using r/w-locks 78
Using shared pointers 78
Summary 78
Chapter 5: Native C++ Threads and Primitives 79

The STL threading API 79
Boost.Thread API 79
The 2011 standard 80
C++14 81
C++17 81
STL organization 82
Thread class 83
Basic use 83
Passing parameters 84
Return value 85
Moving threads 85
Thread ID 86
Sleeping 87
Yield 88
Detach 88
Swap 88
Mutex 89
Basic use 89
Non-blocking locking 91
Timed mutex 92
Lock guard 93
Unique lock 94
Scoped lock 95
Recursive mutex 95
Recursive timed mutex 96
Shared mutex 96
Shared timed mutex 97
Condition variable 97
Condition_variable_any 100
Notify all at thread exit 100
Future 101
Promise 102
Shared future 103
Packaged_task 104
Async 105
Launch policy 106
Atomics 106
Summary 106
Chapter 6: Debugging Multithreaded Code 107

When to start debugging 107
The humble debugger 108
GDB 109
Debugging multithreaded code 110
Breakpoints 111
Back traces 112
Dynamic analysis tools 114
Limitations 115
Alternatives 115
Memcheck 116
Basic use 116
Error types 119
Illegal read / illegal write errors 119
Use of uninitialized values 119
Uninitialized or unaddressable system call values 121
Illegal frees 123
Mismatched deallocation 123
Overlapping source and destination 123
Fishy argument values 124
Memory leak detection 124
Helgrind 125
Basic use 125
Misuse of the pthreads API 130
Lock order problems 131
Data races 132
DRD 132
Basic use 132
Features 134
C++11 threads support 135
Summary 136
Chapter 7: Best Practices 137
Proper multithreading 137
Wrongful expectations - deadlocks 138
Being careless - data races 142
Mutexes aren't magic 147
Locks are fancy mutexes 149
Threads versus the future 150
Static order of initialization 150
Summary 153
Chapter 8: Atomic Operations - Working with the Hardware 154

Atomic operations 154
Visual C++ 155
GCC 161
Memory order 164
Other compilers 165
C++11 atomics 165
Example 168
Non-class functions 169
Example 170
Atomic flag 172
Memory order 172
Relaxed ordering 173
Release-acquire ordering 173
Release-consume ordering 174
Sequentially-consistent ordering 174
Volatile keyword 175
Summary 175
Chapter 9: Multithreading with Distributed Computing 176
Distributed computing, in a nutshell 176
MPI 178
Implementations 179
Using MPI 180
Compiling MPI applications 181
The cluster hardware 182
Installing Open MPI 186
Linux and BSDs 186
Windows 186
Distributing jobs across nodes 188
Setting up an MPI node 189
Creating the MPI host file 189
Running the job 190
Using a cluster scheduler 190
MPI communication 191
MPI data types 192
Custom types 193
Basic communication 195
Advanced communication 196
Broadcasting 196
Scattering and gathering 197
MPI versus threads 198

Potential issues 200
Summary 200
Chapter 10: Multithreading with GPGPU 201
The GPGPU processing model 201
Implementations 202
OpenCL 203
Common OpenCL applications 203
OpenCL versions 204
OpenCL 1.0 204
OpenCL 1.1 204
OpenCL 1.2 205
OpenCL 2.0 206
OpenCL 2.1 206
OpenCL 2.2 207
Setting up a development environment 208
Linux 208
Windows 208
OS X/MacOS 209
A basic OpenCL application 209
GPU memory management 213
GPGPU and multithreading 215
Latency 216
Potential issues 216
Debugging GPGPU applications 217
Summary 218
Index 219