At 5 feet 4 inches, Madison was America’s shortest Commander-in-Chief—but he left behind a towering legacy. Here are some lesser-known details about the “Father of the U.S. Constitution” and the colorful life he led. 

1. James Madison was prone to sickness.

Madison wasn’t the healthiest kid—or adult for that matter. A frequent victim of nasty stomach aches, he’d also weather attacks “somewhat resembling epilepsy” throughout his life.

2. James Madison fought—and won—an oratory duel with Patrick Henry.

This was no small feat. Throughout their home state of Virginia, Patrick Henry was renowned as a public speaking heavyweight who wooed crowds wherever he went. Thomas Jefferson declared that the booming debater spoke “as Homer wrote.” Madison, in contrast, had a quiet voice and little talent for theatrics.

In 1788, a ratifying convention was held in Richmond to determine if Virginia would grant her approval to the Constitution Madison had helped engineer. Henry staunchly opposed this document, and loudly decried it for leaning “towards monarchy.” But though several people in attendance complained of Madison’s mumbling, he offered concise, well-articulated rebuttals to every argument. When the dust settled, his slow and steady approach paid off big-time: Virginia’s representatives voted 89 to 79 in favor of ratification.

3. James Madison competed with James Monroe for a seat in the newly minted House of Representatives.

In 1789, both James Monroe and Madison sought the job, with Madison decisively winning the election. Throughout their campaign, the two got along amicably and, every so often, would accompany each other en route to debates.

4. James Madison once thought America should rent Portugal’s navy.

Congressman Madison suggested that the American government ought to guard its oceanic interests by hiring the Portuguese navy for anti-pirate protection instead of constructing one of her own.

5. Aaron Burr introduced James Madison to his wife, Dolley.

Alexander Hamilton’s future dueling opponent knew Madison through Congress (where he also worked at the time) and had gotten acquainted with Dolley by virtue of her mother’s boarding house, which Aaron Burr frequently visited.

6. James Madison fathered no biological children.

Madison didn't father any children, but he still helped raise one. Dolley’s first husband and oldest child had both died the year before she married Madison. The statesman immediately took an active role in caring for her only surviving son, 2-year-old John Payne Todd.

7. James Madison loved ice cream.

Before freezers came along, ice cream was one labor-intensive treat. But that didn't stop the Madisons from serving several varieties of this chilly dessert during their tenure in the White House. Apparently, the First Lady’s favorite flavor was oyster (don’t knock it ‘till you’ve tried it).

8. James Madison employed a number of secret codes.

Madison—like any good politician—was terrified by the idea that someone might intercept one of his private letters. Along with Jefferson and many mutual allies, Madison used complicated encryptions when relaying delicate info. “Having now the use of my cypher,” he informed Jefferson in 1784 after mastering a new system, “I can write without restraint.”

9. When the British burned part of James Madison's White House, they also ate his dinner.

On August 24, 1814, an eerie emptiness fell upon the presidential mansion. As British forces advanced toward Washington, D.C., Madison’s home and office was completely abandoned mere hours before it was besieged. In addition to breaking in and setting the place ablaze, the Brits gobbled up a meal that had been intended for the president. Apple, cider, and wine complemented the menu.

10. Both of James Madison's vice presidents died in office.

George Clinton kicked the bucket in 1812. His short-lived replacement was former Massachusetts governor Elbridge Gerry, who’d redistricted that state to tip the political scales in his favor—a process we now call “Gerrymandering.” Gerry also died in office, and is the only signer of the Declaration of Independence to be buried in Washington, D.C.

11. James Madison had a sharp sense of humor.

Politics is a pretty serious career, but that didn't take away Madison's sense of humor. Among friends, he was known for his quick wit, quippy banter, and well-timed personal anecdotes.

12. James Madison once proposed a brutal hangover experiment.

At an 1804 party thrown in Jefferson’s honor, journalist Samuel Harrison Smith spent part of his evening drinking with Secretary Madison, who voiced his desire to find out how many glasses of champagne would be necessary to trigger “a headache the next day.” It’s unknown if anyone present ever actually tried this experiment.

13. James Madison helped amend Virginia’s state constitution at the ripe old age of 79.

This event—which consisted of a 96-person assemblage—became his last public appearance as a politician.

14. James Madison played a big role in establishing the University of Virginia.

Among the University of Virginia’s original trustees, Madison later served as its second rector (“chairman”) from 1826 to 1836 and bequeathed most of his personal library to the school.

15. James Madison was the last surviving signer of the U.S. Constitution.

For months, the dying ex-President remained bed-ridden. On June 28, 1836, Madison found he couldn’t swallow his breakfast. “What is the matter, Uncle James?” asked his beloved niece, Nelly Willis. “Nothing but a change of mind, my dear,” he replied before died.

16. James Madison's face appeared on a $5000 bill.

Good luck getting your hands on one of these bills—the government stopped printing them in 1945.