Few people doubt that we need a new energy system: new power sources and new kinds of engines to go with them. That still leaves plenty to disagree about: What should the new power sources be? How fast do we need to switch over? How do we get there—carbon tax? research subsidies? markets alone? Do we also need to reduce energy use? But the basic need is widely recognized.

A look at earlier energy transitions is therefore sobering. Even when a new system seems clearly superior in retrospect, all sorts of people have reasons not to switch. They may own reserves of the old fuel, work at jobs that depend on it, or have machines and skills tailored to it. Or they may see real and imaginary problems with the new system that they want to have solved before they switch over. In 1876—probably 200 years after coal became a bigger fuel source than wood in Britain—the United States, with plenty of both commodities, still got more than twice as much energy from wood as from coal. But once a tipping point is reached, change can come rapidly: By 1900 coal energy led wood energy in the U.S. by more than three to one.

The surprising paths to such tipping points stand out when we look at the next, still incomplete transition: from coal to oil. In purely technical terms, oil has huge advantages over coal. Each ton provides twice as much energy, so less refueling and less storage space are needed (particularly important on ships). Because oil is liquid, it can be piped in, and the hard, hot labor of shoveling fuel into an engine becomes unnecessary. Plus, liquid fuels, unlike solids, can be used in internal combustion engines, which first hit the market around 1860—just a year after the first commercial oil well in Pennsylvania. Not only were internal combustion engines much more efficient than steam engines, but they could be so much smaller that they opened up a host of new uses—from cars and motorcycles to chainsaws—chainsaws—that were unlikely or just plain impossible with steam. Yet as late as 1925, only two countries—Mexico and the Soviet Union—got even 20 percent of their commercial energy from oil; the oil-rich and automobile-loving United States was at 11 percent, and industrial Western Europe was well under 5 percent. Oil did not reach even 10 percent of Britain’s fuel consumption until 1953—though it then jumped to half by 1973. Why so much inertia? And how was it overcome? Local quirks mattered a lot. What people really wanted from that first well in Pennsylvania was kerosene: oil for illumination was rapidly replacing wax and tallow candles, and demand was surging all over the world.

At first, fuel oil remained a byproduct of the more profitable kerosene. When huge amounts of heavy crude were found in California, Oklahoma, Texas and Mexico around the turn of the century, those producers joined the effort to make oil a competitive fuel. Again, regional markets were easily conquered: huge gushers meant cheap prices, and these fields faced little competition from nearby coal. In wider markets, the going was tougher: calorie for calorie, energy from oil did not become consistently cheaper than coal in the United States until the late 1920s.

Given such prices, and persistent fears that oil supplies would run out, few people would invest in converting to internal combustion engines that required liquid fuel—no matter how superior they were. Instead, oil made inroads in existing fuel markets as people took the cheaper, reversible step of converting steam engines into hybrids that could use either coal or oil. Thus hybrid ships, for instance, could run mostly on oil, saving weight and space, and using smaller engine crews; navies also liked the higher speeds that oil allowed and the fact that it put less smoke in the air (making stealth easier). But if oil ran out, a hybrid ship could revert to coal; this particularly reassured the British, German and Japanese navies (which had no domestic oil supplies), but influenced U.S. planners as well. Only gradually did the enhanced capabilities oil allowed—and the need of rival navies to match them—lead to fully oil-burning fleets. Merchant ships, railroads, electric utilities, and other users of big steam engines were slower still to convert, as the figures mentioned above demonstrate. Thus, the hybrid steam engine played a crucial historical role: by ensuring a growing market for fuel oil, it stimulated progress in extraction and refining, research on applications, and the growth of a cadre of technicians accustomed to oil-burning engines.

Meanwhile, people building new plants from scratch—especially in oil-rich countries—or pursuing new activities (such as building cars) that required internal combustion engines also moved the transition slowly forward. Still, it is striking how slow change was, even after oil had become cheaper than coal, and how large a role nonmarket forces had in the transition. The navies that played a pioneering role were not thinking much about fuel costs; then as now, militaries focused on performance and were not particularly price-conscious shoppers. Beginning in the 1920s, the Soviet Union shifted sharply away from hybrid engines using fuel oil and toward making gasoline for internal combustion engines. In part this shift went with a massive push to produce tractors and trucks, but also it reflected a top-down decision by central planners that gasoline was simply a superior product.

Oddly enough, something similar happened in the strongly market-oriented United States: The Federal Oil Conservation Board, established in 1924, designated gasoline as the best, most efficient use of petroleum and pushed hard (with the aid of new technology) to replace hybrid engines with engines that used only oil products. When Western Europe and Japan began converting in earnest, politics was again crucial. The huge Arabian oil strikes of the 1930s, plus Cold War alliances with the United States (and its navy), meant that having no domestic oil no longer raised security fears. During the Marshall Plan years, the United States actively nudged Europe toward greater oil use, and more than 10 percent of Marshall Plan aid was spent on oil imports: this brought in fuel faster than reopening damaged coal mines, prevented U.S. allies from becoming dependent on Russian oil (as they had once been and would become again once the Cold War thawed in the 1970s), and reduced the clout of often militant, left-leaning miners’ unions.

Time brought wider use of cars, planes, and other technologies for which coal simply would not work. But it took increased concern about pollution to turn many big European users, especially utilities, away from coal (in some cases, toward nuclear power rather than oil). In short, all sorts of factors played a role in oil surpassing coal: brand-new technologies, new/old hybrids, geopolitical pressures, new industries incubated under peculiar local conditions, innovative navies with peculiar needs and big budgets, environmental concerns, and government regulators, among others. Simply being a better fuel, or a more efficient engine, was not enough, by itself, to power rapid conversion. It still is not: many U.S. utilities still burn coal, and with oil prices rising, some European ones are reverting to it.

From The World That Trade Created: Society, Culture and the World Economy, 1400 to the Present by Steven Topik and Kenneth Pomeranz.