Why some problems feel impossible

Some problems resist us for decades. They are hard because they are multi-scale, tightly coupled, and laden with uncertainty. We can’t see the whole dragon at once, and each swing of the sword vibrates through the system.

"All life is problem solving." — Karl Popper

Philosophers have long offered lenses for this difficulty: Francis Bacon urged disciplined empiricism (knowledge is power). Alfred Korzybski warned that the map is not the territory. Marcus Aurelius reminded us that the impediment to action becomes the way. Together, these ideas hint at a method: form models, test them, correct them, and turn obstacles into instruments.

What we actually solved

It’s easy to forget how many once-impossible problems now feel mundane. Below are a few that didn’t just work technically; they reshaped expectations and rewired society’s default settings.

  • The Internet (packet switching, TCP/IP): From ARPANET to a planetary network, we learned to move information by chopping it into packets and routing around failure. Vint Cerf and Bob Kahn’s layered protocol design embraced heterogeneity and change.
  • Mobile phones at scale: Shrinking radios, multiplexing spectrum, and building global standards turned mobility from a novelty into an expectation. The win wasn’t one invention; it was the patient integration of many.
  • Video calls for billions: Compression (MPEG, H.264/AVC, AV1), congestion control, and cheap cameras made presence routine. The miracle is mostly invisible math—perceptual coding and fast transforms.
  • AI systems that generalize: From perceptrons to deep learning to transformer-based LLMs, progress came from data, compute, and architecture—but also from the unglamorous work of tooling, evaluation, and safety scaffolding.
  • Public-key cryptography: Diffie–Hellman and RSA turned secrecy into math, enabling secure web commerce and private messaging without a shared secret.
  • Relational databases: Edgar F. Codd’s relational model separated logical data from physical layout, letting us ask powerful questions without knowing storage details.

The “hidden” breakthroughs behind the scenes

Behind the headline moments live quiet enablers—mathematical shortcuts, material science, and signal tricks—that change the slope of progress. They rarely get a launch event, but without them, the “big” breakthroughs wouldn’t scale or survive contact with reality.

  • Error-correcting codes (Hamming, Reed–Solomon): Let us store, stream, and talk to spacecraft reliably by turning noise into something we can detect and fix.
  • Fast Fourier Transform (Cooley–Tukey): Made digital signal processing practical; much of modern audio, video, and wireless rides on this acceleration.
  • Erbium-doped fiber amplifiers (EDFA): Unlocked transoceanic optical links without constant regeneration, quietly enabling the web’s global backbone.

Patterns in how hard problems yield

Across these stories, certain moves repeat. They aren’t slogans; they’re operational habits that reduce risk, buy optionality, and compound learning.

  1. Abstraction layers tame complexity. We don’t solve the whole thing—we define clean boundaries so teams can move independently.
  2. Standards create compounding returns. Shared protocols (TCP/IP, SQL, codecs) turn isolated wins into ecosystems.
  3. Compression and error correction buy reality. The world is noisy and scarce; we trade compute for robustness and reach.
  4. Feedback beats foresight. Iteration, measurement, and rapid prototypes discover truth faster than planning alone.
  5. Simplicity at the core, complexity at the edges. Keep the kernel small, push variation outward.
  6. Scale after fit. First make it work, then make it fast, then make it everywhere.
  7. Cross-pollination matters. Breakthroughs arrive where fields meet: math with hardware, theory with operations.

"Civilization advances by extending the number of important operations we can perform without thinking about them." — Alfred North Whitehead

The character of the people who did it

Tools change, temperaments rhyme. The people who bend reality usually share a stance that blends clarity with humility and an almost stubborn kindness toward the constraints of the world.

  • Clarity of problem framing: They rewrite the question until it becomes solvable.
  • Taste for simplicity: They prefer one idea that scales to ten clever hacks.
  • Stubborn optimism, flexible methods: The goal is non-negotiable; the path is negotiable.
  • Humility before reality: They measure, admit when they’re wrong, and change their minds quickly.
  • Bias for experiments: They run cheap tests and let evidence, not ego, decide.
  • Collaborative ambition: They build standards, not empires; they publish and interoperate.

"The impediment to action advances action. What stands in the way becomes the way." — Marcus Aurelius

A small playbook for big problems

If you want a portable checklist to keep at arm’s length, start here—and then adapt it ruthlessly to your domain.

  1. Name the constraints; turn them into design features.
  2. Draw the map, then remind yourself it is not the territory.
  3. Pick the simplest viable architecture; protect its interfaces.
  4. Instrument everything; shorten feedback loops.
  5. Codify wins as standards so others can compound them.
  6. Teach the system to forgive reality with compression and error correction.
  7. Keep going. Progress compounds for those who persist.

Visual summary: How hard problems get solved

flowchart TD
  P[Problem] --> A[Abstraction layers]
  A --> S[Shared protocols / standards]
  S --> F[Rapid feedback loops]
  F --> R[Robustness: compression + error correction]
  R --> E[Scale after fit -> Ecosystem growth]

  class A abstr;
  class S std;
  class F feed;
  class R robust;

  classDef abstr fill:#3b82f6,stroke:#1e40af,color:#ffffff;
  classDef std fill:#22c55e,stroke:#166534,color:#ffffff;
  classDef feed fill:#f59e0b,stroke:#92400e,color:#111111;
  classDef robust fill:#a855f7,stroke:#6b21a8,color:#ffffff;

Hard problems aren’t walls; they’re terrains. Once we learn how to walk them—layer by layer, standard by standard, test by test—the path appears underfoot.