Concurrency and parallelism

Process is the instance of running program.
- Has its own memory space
- Isolated from each other, meaning they have their own memory space and cannot directly access the memory of other processes
Thread is the unit of CPU usage.
- Have call stacks, local storages and virtual CPUs associated with them
- Share a heap space across each other within a process
- It’s possible to communicate between the threads within one process

Process

When the code of a program loaded into memory and executed by processor it becomes a process.

Includes the resources program needs to run:

CPU
I/O
RAM Resources are managed by operating system.

Has:

Process registers
Program counters
Stack pointers
Memory pages

Thread

Thread is a unit of execution within a process. Process can have multiple threads.

Has:

Stack
Registers
Program counters

Concurrency

Concurrency is when two or more tasks can start, run, and complete in overlapping time periods. It doesn’t necessarily mean they’ll ever both be running at the same instant.

This is often achieved through techniques like time-slicing or context switching, where a single processor rapidly switches between different tasks, creating the illusion of parallelism.

Concurrency is used not only in multi core processors environments. Multitasking on a single-core computer is an example of concurrency. Even though the computer has only one processor, it can run multiple programs at the same time by quickly switching between them.

Despite the fact that concurrency wouldn’t shorter the execution time (probably it will become bigger due to scheduling overhead) it’s still useful. It prevents unresponsiveness of the system. So as some action takes place we still can interact with the system.

For example: Promise.all() won’t run promises in parallel, rather concurrently. Node.js is single threaded, meaning there are no different threads which can perform the work in parallel (although this can be achieved through child-processes). Promise.all benefits from Node.js’s asynchronous behaviour (by using the V8 Event Loop) which means that Promises will be resolved in no particular order, depending on the amount of work which needs to be done to resolve each one.

Context switching

Context switching (switching between threads) is done at frequent intervals by the kernel, creating the illusion that our programs are running in parallel, whereas in reality, they are running concurrently but sequentially in short slices.

OS stores state of process within a process, so it can be restored and resume execution at a later point.

Because threads are sharing the memory of process there is no need to switch memory pages when switching context between threads. That makes it easier and faster to switch context between threads comparing to processes.

Parallelism

Parallelism is when tasks literally run at the same time, eg. on a multicore processor. With parallelism tasks take less time to complete.

Parallelism requires multiple processing units (e.g., multiple cores in a CPU or multiple computers in a cluster). Each core can work on a different task at the same time, resulting in a real increase in speed and throughput.