Tech Deep-Dives

The design

You message a friend whose phone is off. How can your phone agree on a secret with theirs, if they are not there to answer? The trick is that your friend's phone did some homework earlier. When WhatsApp was installed, it made a public and private key pair, kept the private half, and uploaded the public halves to WhatsApp's server. These include a batch of single-use prekeys.

Your phone downloads one of those public keys, mixes it with its own keys, and both sides can arrive at the same shared secret without it ever being sent. Your friend's phone repeats the same math when it comes back online. The server only handed over public keys, which are useless for reading anything.

After that, each message is protected by a ratchet. Every message is locked with a new key made from the previous one, and the old key is thrown away. This is the Signal Protocol, which WhatsApp adopted in 2016 with the Signal team's help.

Things this explains

1. Messages still arrive after your friend was offline for hours. The server holds the locked message until their phone connects.
2. A new phone cannot read your old chats on its own. The keys lived on the old phone, so history only returns through a backup you chose to make.
3. "Security code changed" notices appear. When a contact reinstalls the app, their keys change, and your phone tells you so you can notice a possible impostor.

The design

Say your app is a website that needs three copies running so it survives a crash. With Kubernetes you do not say "start a copy on machine 4, then another on machine 7". You write down a goal: "I want three copies of this container." That goal is the desired state, and it is saved in a small database called etcd.

Then a controller takes over. It runs a never-ending loop: look at what is actually running, compare it with the goal, and fix any difference. Two copies running? Ask for one more. Four? Remove one. It does not care how the gap appeared.

Other pieces do the actual placing. A scheduler picks which node has room for the new copy. An agent on that node then starts the container. Each piece does one narrow job, and they coordinate by reading and writing the shared goal, not by calling each other directly.

Things this explains

1. Kill a container by hand and a new one appears. The loop sees two copies where the goal says three and starts another.
2. A machine can die and the app barely notices. Its missing copies are just another gap between goal and reality, so they get recreated elsewhere.
3. Changing a number in a file can scale an app. Edit "3" to "10", and the loop treats the difference as work to do.

The design

Imagine a program that asks for 8 GB of memory on a laptop with 4 GB. You might expect an error. On Linux, the request usually succeeds. Nothing has gone wrong yet, because the program has not been given real memory. It has been given virtual memory: a range of addresses that exist only on paper.

Real memory is handed out later, one small page at a time, the first time the program actually touches it. That touch causes a page fault, the kernel quietly grabs a free page, and the program carries on without noticing.

So the kernel is practising overcommit. By default it refuses only the most absurd requests, such as one far larger than the whole machine could ever hold, and accepts the rest.

The bill arrives when too many programs really do use their promises. Free pages run out, the kernel first tries to reclaim memory by dropping cached files, and if that is not enough it calls the OOM killer. It scores the processes, mainly by how much memory each one is holding, and stops the worst offender with a hard kill signal. Nobody gets to clean up.

Things this explains

1. A program can "allocate" more memory than you own and run fine. It never touched most of it, so no real memory was used.
2. A container dies with exit code 137 and no error message. That code means it was force-stopped by a kill signal, often the OOM killer after it hit its memory cap.
3. A server gets slow and stuttery before something vanishes. The kernel first scrambles to reclaim memory, and only then kills a process.

The design

Open the Uber app and you see cars drifting on a map. Behind that, Uber needs to ask area-level questions: how many people are requesting rides around the stadium right now, and how many free drivers are close enough to help? Asking this with raw latitude and longitude is clumsy. There is no quick way to say "everything within this neighborhood" from two floating-point numbers.

So Uber chops the planet into grid cells. Every GPS ping from a phone or car is turned into the cell it falls in, and that cell has a single cell ID. Counting riders in a neighborhood becomes "count pings with this cell ID", a simple lookup any database does fast.

The grid Uber built is called H3, and the cells are hexagons. A hexagon has a nice property that squares lack: every neighbor is the same distance from the center. With squares, the diagonal neighbors are farther away than the side ones, which makes "spread out one step" uneven. With hexagons, the six neighbors are all equally close, so questions like "what's around this cell?" give round, fair-looking answers.

Cells also come in sizes. Each resolution level splits every cell into about seven smaller ones that sit roughly inside it. So one cell ID can be zoomed out to its parent (the whole district) or zoomed in to its children (individual blocks), and you pick whichever size suits the question.

Things this explains

1. Surge prices apply to an area, not to you personally. Your location maps to a cell, and the price comes from that cell's supply and demand.
2. Walking a block or two can change your price. You may have crossed into a neighboring cell with a different balance of riders and drivers.
3. Heatmaps of busy areas look like honeycombs. Each hexagon is just one cell's count, colored by how high it is.

The design

Imagine a new game update that millions of people want today. If it sits on one company's server, that server's internet connection is the bottleneck. BitTorrent avoids this by cutting the file into many pieces and letting everyone who has some pieces hand them to everyone who needs them.

The group sharing one file is a swarm, and each member is a peer. To join, you first need a list of other peers. A small server called a tracker can give you one, and newer setups can find peers through a shared lookup system spread across the peers themselves. Then you connect to a few dozen of them and trade: "which pieces do you have?" "here is piece 7, send me piece 12."

You may be wary of downloading from strangers. The answer is that the small .torrent file (or a magnet link) contains a hash for every piece. After a piece arrives, you check it against its hash. A wrong or tampered piece is thrown away and fetched again from someone else. You never have to trust anyone.

Peers also try to grab the rarest pieces first, the ones few others have. That spreads every piece through the swarm quickly, so the original source can leave without the file becoming incomplete.

Things this explains

1. Popular downloads are fast, obscure ones crawl. Speed depends on how many peers have the pieces you need.
2. Your download slows when you pause uploading. Peers choke those who give nothing back.
3. Linux distributions and game patchers offer torrents. It lets them release big files without paying for a giant server.

The design

Suppose you type user. in a Python file and the editor pops up a list of things a user can do. Someone has to know what user is, where it came from and what it can do. That means parsing the code, tracking imports and understanding types. It is a lot of work.

Before 2016, each editor did this itself, for each language. Ten editors and twenty languages meant up to two hundred separate pieces of work. Microsoft's answer, created for VS Code, was the LSP. The language knowledge moves into a separate program, a language server. The editor only has to show things and send questions.

The two run as separate processes and talk using JSON-RPC messages, often over plain standard input and output. When you type, the editor tells the server what changed. The server keeps its own picture of your project. When you ask for completions, the editor sends "what fits at line 12, column 5?" and gets a list back. The server can also speak up on its own. After your edit, it sends diagnostics so the editor can draw the squiggles.

Things this explains

1. A new language often gets good editor support quickly. One language server covers many editors at once.
2. Squiggles sometimes show up a moment after you stop typing. The server is a separate process that has to catch up and send its findings back.
3. Autocomplete can feel different from one editor to another. The server gives the options, but each editor decides how to rank and display them.

The design

Say you are checking your bank balance in an app while a transfer is changing that same balance. The simple way to stay safe is a lock: the transfer locks the row, and your read waits. On a busy system, that means lots of people standing in line.

Postgres does something different, called MVCC. An UPDATE does not overwrite the row. It writes a brand-new version next to the old one and marks the old one as "replaced by transaction 105". Every row version carries the id of the transaction that created it and, if there is one, the id of the transaction that replaced it.

When your transaction starts, it gets a snapshot: roughly, a note of which transactions had finished at that moment. Then, for each row, Postgres shows you the newest version that your snapshot is allowed to see. The transfer's new version is invisible to you until it commits and you start a new query. You read the old balance, the transfer writes the new one, and neither blocks the other.

Things this explains

1. A slow report doesn't freeze your app. It reads its own snapshot and does not hold locks that block writers.
2. A table can grow even when you only update rows. Each update adds a new version, and the old ones stay until vacuum cleans them up.
3. A transaction left open for hours is a problem. Its snapshot may still need old versions, so vacuum cannot remove them.

The design

Picture a rectangle on the canvas. You change its colour to red. At the same moment, a teammate changes its colour to blue. Somebody has to decide what the rectangle ends up as, and everyone must see the same answer.

Figma keeps this simple with a server as authority. Every change goes to the server, and the server decides the order. A change is tiny: "object 42, property fill, new value red". Each property of each object is its own little slot. If two people change the same slot, last-writer-wins: whichever arrives at the server second is kept. If you change different slots, such as one person moving a shape and another recolouring it, both changes survive and nothing is lost.

Your screen does not wait for the server. Your edit shows up immediately as an optimistic update. If the server's news about someone else's edit touches a slot you have changed and not yet had confirmed, your app ignores that news for now, so your shape does not flicker back and forth. Once the server confirms your change, everything lines up.

Things this explains

1. Your own edits feel instant even on slow Wi-Fi. Your screen updates first, and the server's confirmation arrives later.
2. Sometimes a colour you set quietly changes to someone else's. You both hit the same slot, and the server's order picked the winner.
3. You can move a shape while a teammate recolours it and both changes stick. They live in different slots, so there is nothing to fight over.

The design

Scrambling data is the easy part. Once both sides share one encryption key, they can use fast, simple encryption for the whole conversation. The hard part is getting that key onto both sides when you cannot meet in private first.

That is what TLS does in its first moments, called the handshake. Your browser and the server run a key exchange. Each side picks a throwaway secret number and sends only a number derived from it. Mixing your own secret with the other side's public number gives both of you the same result. A listener sees both public numbers but cannot work out the result. Because the secrets are thrown away afterwards, recording the traffic today does not help if the server is broken into next year.

That alone does not prove who you are talking to. An attacker in the middle could run a key exchange with you and another one with the real server. So the server also sends a certificate and proves it owns it by signing the handshake with its private key, a digital signature. Your device checks that the certificate names the site you asked for, and that it was signed by a certificate authority your device already trusts. Your operating system and browser ship with a list of these.

Things this explains

1. The browser warns you when a certificate has expired or names a different site. Those are exactly the two things it checks: the name, and that a trusted authority signed it.
2. Padlock sites can still be scams. The certificate proves you reached the owner of that name, not that the owner is honest.
3. Reconnecting to a site you just visited feels faster. Both sides can reuse what they already agreed, so less of the handshake is repeated.

The design

Redis is a key-value store that keeps everything in RAM. Your app asks "give me the value for cart:42" and gets it straight from memory, with no disk read in the way. That is why it is so often used as a cache in front of a slower database.

The values are not just text. Redis can hold lists, sets, counters and sorted lists. A game leaderboard, for example, is one sorted list where Redis keeps players ordered by score for you.

The second surprising choice: Redis runs your commands one at a time on a single main thread. No two commands ever run at the same moment, so there are no locks and no half-finished updates to worry about. Adding 1 to a counter is always safe, even with thousands of apps hitting it.

Memory is wiped on restart, so Redis offers two ways to save to disk. A snapshot writes everything out every so often. An append-only file adds each change to the end of a file as it happens. Snapshots are compact but can lose the last few minutes; the append-only file loses far less but grows bigger. Many setups use both.

To take a snapshot without freezing the app, Redis asks the operating system to make a copy-on-write clone of its own process. The clone sees the data as it was at that instant and writes it to disk, while the original keeps serving requests.

Things this explains

1. Apps sometimes show stale data for a moment. A cache holds a copy, and until it expires or is updated, you may see the old one.
2. Rate limits and like counters rarely go wrong under heavy load. Each increment is one uninterrupted command, so two requests cannot overwrite each other.
3. A cache restart can make a site briefly slow. If memory is empty, every request falls through to the slower database until the cache fills again.

The design

Picture a billing program that is down for ten minutes while orders keep arriving. If the order system sent messages straight to billing, they would be lost, or the order system would have to wait. Kafka sits in the middle instead and simply writes each event down, in order.

The heart of it is a log. Events are added to the end and are not edited or removed when someone reads them. A named stream is called a topic, and each topic is cut into partitions so that many machines can share the load. Events with the same key, say the same customer, go to the same partition, so their order is kept.

Each consumer keeps its own bookmark, called an offset. Billing is at event 900, analytics is at event 1,200, and neither affects the other. A slow or crashed reader just resumes from its bookmark later.

To survive machine failures, each partition is copied to other machines as replicas. One copy is the leader that takes new writes, and the others follow it. If the leader's machine dies, a follower that is fully caught up takes over.

Things this explains

1. A new feature can read the whole history of events. Events stay in the log for a set retention time, so a new reader just starts from the beginning.
2. A downstream outage does not lose data or slow the sender. The log keeps filling, and the reader catches up from its bookmark when it returns.
3. Events about one customer arrive in order, but across customers they may not. Order is only promised inside a single partition.

The design

A container is not a small computer. It is a normal program running on your normal kernel, with two kinds of fences put around it.

The first fence controls what it can see. Linux namespaces let the kernel show a program a filtered version of the machine. Inside the container, the app thinks it is the only program running, has its own network address, and sees its own file system as the whole disk. The same machine can show a different view to every container.

The second fence controls what it can use. Cgroups set limits, so one container that goes wild cannot eat all the memory and starve its neighbours.

The other half of Docker is how the files get there. A container starts from an image, and an image is built from stacked layers. One layer might be a base set of system files, the next adds a language runtime, the next adds your app. Each layer stores only what changed.

Because layers are read-only and shared, ten containers from the same image do not copy the files ten times. They all read the same layers, and each gets a thin writable layer on top for its own changes. Edit a file and only that one file is copied up into your private layer.

Things this explains

1. Containers start in about a blink, while virtual machines take a while. There is no operating system to boot, just a program to launch inside its fences.
2. "Works on my machine" mostly goes away. The image carries the exact libraries and tools the app was tested with, so the container sees the same files everywhere.
3. Docker on a Mac or Windows laptop quietly runs a small Linux virtual machine. Containers need a Linux kernel to share, so those systems bring one along.

The design

Imagine asking a friend the quickest way to the airport. They would not think about every street in the country. They would think: get to the highway, stay on it, then take the airport exit. Maps software does the same, in two steps.

First, the road network is turned into a graph. Each road segment gets a cost, usually its predicted travel time. Finding a route then becomes a shortest-path search.

Second, the search is helped by work done ahead of time. Big roads are the ones almost every long trip uses, so the system can prepare shortcuts between important junctions in advance. When you ask for a route, the search only crawls through small streets near your start and end, and leaps across the middle using the shortcuts.

The arrival time is its own problem. A road that is jammed now may be clear by the time you reach it. So Google's research team has described using a machine-learning model that looks at current speeds and at how traffic on neighbouring roads usually behaves, to predict what each segment will be like when you get there, not just what it is like now.

Things this explains

1. Directions come back almost instantly, even for a trip across the country. The search only works hard near the start and end; the middle is covered by precomputed shortcuts.
2. Maps can reroute you mid-drive when a crash happens ahead. New speed reports change the cost of those segments, and the search runs again with the updated costs.
3. The arrival time sometimes changes even though you are on the same road. The estimate is a prediction about roads still ahead, and it gets revised as conditions shift.

The design

When you type example.com, your device needs its IP address. Your device does not go hunting for it. It asks a resolver, and the resolver does the work.

The names in DNS are organised like a tree, read right to left. The resolver starts at the top, at the root servers, and asks who handles .com. A .com server then says who handles example.com. Finally that server, the one the site owner chose, gives the actual address.

Each step only points to the next one. No server knows everything, and each one knows only its own slice. That is what lets the system have no central owner: whoever owns a name controls the answers for it.

That walk would be slow if it happened every time, so the second half of the design is caching. The resolver remembers answers. If thousands of people on the same provider look up the same popular site, only the first one triggers the walk. Everyone else gets the saved copy straight away.

Each answer comes with a TTL, chosen by the site owner, that says how long the saved copy is allowed to live. Your own device and browser keep a cache too.

Things this explains

1. A site you just moved can look broken for some people but not others. Different resolvers hold old saved answers until their TTL runs out.
2. Switching your DNS provider can make browsing feel snappier. A busy resolver is more likely to already have the answer cached, so it skips the walk.
3. When DNS has an outage, it feels like "the internet is down" even though servers are fine. Without the lookup you have no address to connect to, so nothing starts.

The design

Think about what a web server actually does for one request. It reads a little data, asks a database a question, waits, gets an answer, and sends a reply. Nearly all of the time is waiting. The server's own work takes a tiny slice.

The classic approach gives every connection its own thread. When that thread hits a slow step it is blocked, and the other threads carry on. It works, but ten thousand mostly-asleep threads eat a lot of memory.

Node.js does something different. It runs your JavaScript on one thread, and that thread never waits. When your code says "read this file" or "query this database", Node hands the job to the operating system, along with a callback, and moves on to the next piece of work.

Running underneath is the event loop, built into a library called libuv. It keeps asking the OS "did anything finish?". When a network reply has arrived, it takes the matching callback and runs it on the main thread.

One detail worth knowing: not everything is handed to the OS in the same way. Network connections can be watched efficiently by the OS itself. Some other jobs, like many file operations, are run on a small pool of helper threads that Node keeps behind the scenes. Your own JavaScript still stays on the single main thread either way.

Things this explains

1. A tiny Node server can hold a huge number of open connections. Idle connections cost almost nothing because no thread is sitting on each one.
2. One heavy loop can make a whole server unresponsive. All callbacks share one thread, so a long calculation delays every other request in line.
3. You see callbacks, promises and async everywhere in Node code. They are how you say "carry on with this when the slow thing is done" instead of stopping to wait.

The design

The obvious idea is to compare your recording against every song, wave against wave. That fails right away. Your clip has background noise, the phone microphone squashes the sound, and you started recording mid-song.

Shazam's approach, described in a 2003 paper by one of its founders, is to throw most of the sound away. First the app turns the clip into a spectrogram. Then it keeps only the peaks, the loudest spots in each small patch of time and frequency. What remains looks like a scatter of dots, like stars on a night sky.

Loud spots are a good thing to keep because they tend to survive noise. A chatty cafe can drown out quiet details, but the strongest notes of the song still poke through.

Single dots are not distinctive enough, since many songs share a loud note. So the app links dots in pairs: one dot, plus a few nearby dots that come just after it. Each pair becomes a small code made from the two frequencies and the time gap between them. That code is the audio fingerprint, or really a bag of hundreds of them.

On the server side, every known song went through the same process ahead of time, and all its codes were stored in a giant index, keyed by the code. Each stored entry also remembers which song it came from and when in the song it happened.

When your codes arrive, each one is looked up and returns a list of songs that contain it. Many are coincidences. The trick is timing. If your clip really is from song X, then the codes line up: every match sits at the same offset, say "your clip starts 73 seconds into the song". Coincidences scatter all over the place. The song with the most matches agreeing on one offset wins.

Things this explains

1. It works when you start mid-song. Codes are matched by their gap to each other, not by where your clip began.
2. It can work in a loud room but not for a cover. Loud peaks survive noise, yet a different performance has different peaks.
3. Results come back fast. Only compact codes are sent, and each is a direct lookup instead of a scan over the catalogue.

The design

Picture a pizza chain with one phone number. You dial it and the phone network connects you to the shop nearest your house. You never pick a shop, and the chain never needs to know where you live.

That is anycast. Normally an IP address belongs to one place. With anycast, Cloudflare's data centers all around the world announce the same address at the same time. They do it using BGP, the system networks use to tell each other which addresses they can deliver to.

Every router on the way picks the best path it knows. So a visitor in Mumbai tends to end up at a Mumbai building and one in Paris at a Paris building, with no special logic and no lookup table of users.

Failure handling comes almost for free. If a data center goes dark, it stops announcing the address. Routers elsewhere forget that path, and traffic that used to land there quietly slides to the next-nearest building.

Things this explains

1. Public DNS like 1.1.1.1 is easy to remember. One short address works worldwide, because every location answers to it.
2. Big sites survive outages you never hear about. A failed building just stops announcing itself and traffic flows around it.
3. Attacks rarely take a whole network down. The flood is chopped up by geography before it reaches any single machine.

The design

Imagine a filing cabinet where you never choose a label. Instead, a clerk reads each document, works out its hash, and files it under that fingerprint. Ask for the fingerprint, get the exact document back. That is Git's core: a content-addressable store.

Everything is built from just a few kinds of items filed this way. A file's contents. A folder, which is a list of names pointing at file fingerprints. And a commit, which points at the top folder and at the previous commit.

Notice the trick: a commit's fingerprint depends on the fingerprints inside it, including its parent's. So the history is a chain where each link vouches for everything before it. Change one old file and every fingerprint after it changes too.

Does saving a full snapshot every time eat your disk? No, because of the filing rule. If a file didn't change, its fingerprint didn't change, so the new snapshot simply points at the copy already in the cabinet. Only edited files add anything new.

A branch is then almost free. It is a sticky note with a commit's fingerprint on it. Creating one writes a tiny note; no files are copied. Committing just moves the note forward.

Things this explains

1. Creating a branch feels instant. It is one tiny label, not a copy of your project.
2. Those long random-looking commit IDs. They are the fingerprints, and they are the commit's real name.
3. "Rewriting history" breaks teammates' copies. A rewritten commit has a new fingerprint, so everyone else's copy suddenly disagrees about what the past was.

The design

Most databases work like a restaurant. You sit at a table (your app), and a kitchen (the database server) does the cooking. SQLite is more like a cookbook on your own shelf. You read it and cook right there in your own kitchen.

Technically, SQLite is a library. When your app asks a question in SQL, the code runs inside your app's own process and reads one ordinary file on disk. No network, no setup, no password. Copying the database means copying the file.

Inside that file, the data is cut into equal-sized pages, and each table is organised as a tree of pages, so looking something up means hopping down a few levels instead of reading everything.

That raises the hard question. If your app writes straight into the file, what happens when the phone dies halfway through saving? You could be left with half-old, half-new data.

SQLite's answer is the transaction. Before changing a page, it first saves a safety copy in a journal. If the power dies, the next program to open the file sees the leftover journal and rolls the file back to how it was. A save either fully happened or never did.

Modern setups often use a different trick, called WAL mode. Instead of copying old pages aside, new changes are simply added to the end of a log file. Readers look at the main file plus the log, so a writer never blocks them. Now and then a checkpoint tidies the log into the main file.

Things this explains

1. Apps recover fine after a crash or dead battery. The leftover journal or log tells SQLite how to roll back the half-finished save the next time the file opens.
2. Backing up an app's data can be as easy as copying one file. The whole database lives in it. (Copy it while nobody is writing, or use SQLite's backup tool, to avoid grabbing a half-finished state.)
3. It's a poor fit for a busy multi-user website. Writers queue behind one another, and sharing a database file across a network is risky. That's what database servers are for.

The design

First, the live part. When you open Discord, your app keeps a WebSocket open to Discord's computers. Every connection gets its own tiny worker (a process), and every server gets one too. Think of it as a switchboard operator for that community. Send a message and it goes to the operator, who passes it on to everyone who is online. All of this runs on Elixir, a language made for exactly this kind of job.

The obvious approach is for the operator to send copies one at a time. That's fine for ten friends. For a giant server it's a traffic jam, because the operator can only do one hand-off at a time. Discord's answer: the operator sends just one copy to each machine (a node), and a helper on that machine passes it to the people connected there. It's still fan-out, but the work is shared between many workers.

Now the remembering part. Messages are stored like papers in a filing cabinet: one folder for each channel and stretch of time, with the messages inside kept in order. Every message ID includes the time it was sent, so "show me the latest messages" just means opening the right folder and reading the end.

In front of the database sits a helper layer called the data services. Picture a librarian who gets asked the same question by a big crowd. They look it up once and read the answer out to everyone. That's request coalescing. It works because all questions about one channel go to the same librarian.

Things this explains

1. Big servers only show part of the member list. Updating all 200,000 names for everyone would cause the same traffic jam, so you only get updates for the names on your screen.
2. Jumping to an old message is fast. Message IDs include the time, so Discord knows which folder to open. (This is my own reasoning from how the storage works, not something Discord says directly.)
3. An @everyone ping doesn't break things. Thousands of people open the channel at once and ask for the same recent messages. The librarian answers once for all of them.