Concurrency, Parallelism, and Async: Untangling the Three

I used to mix these up too

You run into "concurrency," "parallelism," and "asynchronous" constantly once you start writing code. For a long time I used all three more or less interchangeably, because they sounded like the same idea wearing different hats. Then I moved into ops and architecture work, spent enough time staring at real systems, and it clicked: they start from a similar question but they live in completely different neighborhoods.

Get this wrong and you chase performance problems in the wrong direction. You add cores to fix an I/O bottleneck. The thing that actually needed to grow stays exactly where it was. In interviews it shows immediately too — if you can't separate these three, it's obvious within a sentence. So let me lay them out properly.

Why is this so confusing?

All three grow out of the same question: how should a program handle multiple pieces of work? Same question, but each one answers at a different layer.

Cooking makes it concrete. Say one cook is putting together a three-course dinner. Starting the pasta water and chopping vegetables while it comes to a boil — that's concurrency. Hiring a second cook to build the salad at the same moment — that's parallelism. Setting a timer on the oven and going to serve another table instead of standing there watching it — that's the heart of async.

Same kitchen, completely different strategies. And this isn't just a cute analogy. It's the mental frame you use to decide which resources to spend and how, when you're actually designing a system.

Concurrency: the magic of fast switching

Concurrency means multiple tasks appear to run at the same time, when in reality they're being interleaved very quickly. On a single CPU core the processor switches between tasks fast enough that it feels simultaneous, but only one instruction executes in any given clock cycle.

The mechanism goes by time slicing, or context switching. The OS hands each task a short slot, then pauses it, saves its state, and moves to the next one.

The part that matters: two tasks never execute at the exact same instant. They take turns. Total wall-clock time doesn't shrink, but the system feels far more responsive, because no task sits frozen waiting for another one to finish.

Where concurrency shines

Concurrency earns its keep on I/O-bound work. Reading files, running database queries, waiting on network responses. In all of those, the CPU is just sitting there doing nothing while it waits, and concurrency lets you spend that idle time on something else.

If a database query takes 100ms, the CPU handles other requests during those 100ms. Throughput goes way up. In something like internet banking, where every request eventually comes down to waiting on a backend database or an external system, how well you handle this is what decides your concurrent user count.

Parallelism: actually simultaneous

Parallelism means multiple tasks executing at the exact same instant, on different CPU cores. Nothing is being reordered — the work is physically happening at once on separate processing units.

Two cores means two instructions per clock cycle. This is what people are actually picturing when they say "let's use multithreading, it'll be faster."

There are conditions, though. You need two or more physical CPU cores, and the tasks have to be independent of each other. If one has to wait on another's result, you can add cores all day and they'll still end up standing in line.

Where parallelism is powerful

Parallelism is excellent for CPU-intensive work. Image processing, video encoding, matrix multiplication, machine learning inference. Those problems split cleanly into independent chunks that run at the same time, so throughput scales linearly, or close to it.

What parallelism costs you

The bill comes as shared state. When two cores try to write to the same memory location simultaneously, the result is undefined. That's a race condition.

Preventing it means synchronization primitives: mutexes, semaphores, atomic operations. The catch is that those complicate your code and can become bottlenecks themselves — lock contention.

Which is why parallel code is harder to get right than concurrent code, and why bugs in multithreaded systems are so uniquely subtle and non-deterministic. In ops, this is the category of incident that gives me the worst headaches. You can't reproduce it. It fires once in a while, only when load arrives in a particular shape, and the logs tell you nothing about why.

Async: the art of waiting without blocking

Asynchronous programming isn't a hardware property. It's a programming model — an answer to how a single thread keeps multiple pieces of work moving without ever going idle. Worth pinning down the layers here: concurrency and parallelism are about what executes and how, while async is one specific way of pulling that off on a single thread.

The core idea is the event loop. Instead of holding a thread hostage while you wait for a response (a database query, say), you register a callback or continuation, release the thread, and resume from where you left off once the response lands.

Why async is efficient

Say you need to fetch user info and order info. Synchronously that's user info (1s) + order info (1s) = 2 seconds. Asynchronously, both queries leave the gate at roughly the same moment, so total wait is about max(user_time, order_time) ≈ 1 second.

On a single thread. That's the whole efficiency win.

How async gets implemented

Most languages give async code dedicated syntax. JavaScript, Python, and Rust use async/await; Ruby has fibers; Go has goroutines. The runtime takes code that looks linear and converts it into a state machine that suspends and resumes at specific points (await).

A fiber example in Ruby:

require 'fiber'

fetch_user = Fiber.new do
  puts "Fetching user..."
  sleep(1) # simulate database wait
  Fiber.yield "User: Alice"
end

fetch_orders = Fiber.new do
  puts "Fetching orders..."
  sleep(1) # simulate database wait
  Fiber.yield "Orders: [#1, #2, #3]"
end

# The two fibers run cooperatively and never block each other
user = fetch_user.resume
orders = fetch_orders.resume

puts user
puts orders

In a real Rails app, the Async gem or the Falcon web server implements genuine async I/O on a fiber-based model. A single Rails process can then serve many concurrent requests without spawning thousands of threads.

How the three relate

None of these are mutually exclusive. Real systems use all three together. Concurrency is about structure — how you design a program to deal with multiple tasks. Parallelism is about execution — whether those tasks physically run at the same time. Async is a specific technique for getting concurrency without using multiple threads at all.

Rob Pike, one of Go's co-creators, put it cleanly: "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once."

That one sentence is my reference point whenever I get turned around. Dealing versus doing.

Which one do you reach for?

When I hit a performance or scaling problem, I work through four questions in order.

1. Is the bottleneck CPU or I/O?

Profile first. If you start from a guess you'll usually be wrong. Most web applications are I/O bound — database, cache, and external APIs eat 80–95% of response time. Throwing parallelism at an I/O-bound problem often changes nothing. You just pay for cores.

2. How many tasks run at once?

Dozens of threads is fine. Thousands of threads is a memory problem. One Ruby or Java thread costs roughly 1–8MB of stack memory. If you're expecting thousands of concurrent connections, async is dramatically more memory-efficient.

3. Do the tasks share state?

If they do, every option gets harder. Async on a single event loop sidesteps the whole thing naturally. Parallelism needs careful locking or immutable data structures. Cut corners on the design here and the non-deterministic bugs I mentioned earlier show up in production.

4. What does your runtime actually support well?

Ruby's GVL (Global VM Lock) blocks true parallelism for Ruby threads, but Ractor — added in Ruby 3.0 — enables real parallel execution while keeping state isolated. Node.js is single-threaded with async I/O by design. Go was built from the start to run concurrent goroutines on cheap thread-like primitives. Forcing a pattern against the grain of your tools is usually a bad time.

Ruby as a worked example

Ruby is a good teaching text for these concepts. Its own evolution mirrors how the industry came to terms with them.

Classic Ruby and the GVL

Classic Ruby (MRI) uses a GVL (Global VM Lock, also called the GIL). Only one Ruby thread executes at a time, even on a multicore machine. That prevents race conditions in most cases, but it also means Ruby threads give you concurrency, not parallelism.

For an I/O-heavy Rails app this isn't a problem. The GVL is released during I/O, so threads genuinely make progress together while they wait on the database.

Ractor in Ruby 3.x

Ruby 3.x introduced Ractor for true parallelism via actor-model isolation. Each Ractor has its own heap and communicates by message passing. Shared state disappears entirely, at the price of much stricter constraints on which objects can cross a Ractor boundary.

# Ruby 3.x Ractor example — true parallel execution
ractor1 = Ractor.new { (1..10_000).reduce(:+) }
ractor2 = Ractor.new { (10_001..20_000).reduce(:+) }

result = ractor1.take + ractor2.take
puts result   # => 200_010_000
# Each Ractor runs on its own OS thread, so this is genuinely parallel

Separately, the async gem brings cooperative concurrency (the event loop approach) to Ruby, letting you write async code that reads synchronously. If you're comfortable with Rails, it's a pattern that goes down easy.

Amdahl's Law: the ceiling nobody likes

Before you go parallelize everything, one uncomfortable truth: Amdahl's Law.

When only part of a program can be parallelized, the theoretical maximum speedup from N processors is:

max_speedup = 1 / (serial_fraction + (parallel_fraction / N))

If 50% of your code is inherently sequential, then even with infinite cores your maximum speedup is 2x. Not 100x. Not 10x.

This is another reason to profile before optimizing. If you're burning 90% of execution time in a serial bottleneck, no amount of cores buys you the benefits of parallelism. When someone tries to solve performance by adding infrastructure and only ends up with a bigger bill, it's usually this law being ignored.

Clearing up common misconceptions

"Multithreading always makes things faster"

Only when the work is CPU-intensive, the tasks are fully independent, and you have spare cores. For I/O-heavy code on a properly configured async server, multithreading adds overhead and no benefit.

"Async means parallel"

It doesn't. Async servers in Node.js or Ruby use a single thread. Two requests can be in flight together (taking turns), but they never execute at the same instant. Async does nothing for CPU-intensive work.

"Concurrency is dangerous"

Depends on the implementation. Async on a single event loop is surprisingly safe. The danger comes from multithreading with shared state. Actor models — Ractor, Erlang processes — remove state sharing altogether, which makes concurrent systems far safer.

"The GVL makes Ruby threads useless"

In I/O-heavy environments like most Rails apps, threads are extremely useful. The GVL is released during I/O waits, so threads genuinely progress in parallel across database queries and HTTP calls. The constraint only bites you when the workload is CPU-heavy.

How they fit together in real systems

These three end up forming a layered mental model. Async is a programming technique for squeezing maximum I/O efficiency out of a single thread. Concurrency is the broader design approach that keeps multiple tasks progressing together, whether through async or time-sliced threads. Parallelism is hardware-level throughput: splitting CPU-intensive problems into independent pieces and solving them at once.

Systems in the field almost always use all three at the same time. A web server takes on huge numbers of concurrent connections with async I/O, keeps a thread pool for blocking work that async can't handle, and hands CPU-heavy jobs like image resizing or PDF generation to background worker pools spread across every available core. The moment you try to solve everything at one layer, something bends out of shape somewhere else.

Knowing which tool belongs at which layer, and why. That, to me, is what separates a system that stretches smoothly under load from one that just snaps at some point. The answer to a performance problem usually isn't a new technology — it starts with looking honestly at whether the work you have is CPU bound or I/O bound.