Although today the pennies in your pocket bear Abraham Lincoln's likeness, the first pennies made by the U.S. Mint bore a startling etching of a woman, shown above (here's a huge version). This woman is supposed to be a personification of "liberty," and she unfortunately looks a bit freaked out. I mean, seriously, I could probably draw a better face than this:
National Numismatic Collection, National Museum of American History
Called the chain cent, this penny was only minted in 1793, and it was much larger and heavier than pennies today. It was also made from nearly pure copper, unlike modern pennies, which are made mostly of zinc (pure copper would be worth vastly more than $0.01 to make now, and even the zinc version costs too much to make).
Walter Breen wrote a history of early U.S. coinage, including this notable snippet about the chain cent (emphasis added):
Use of a Liberty head design was inevitable because of the terms of the Mint Act of April 2, 1792, mandating "a device emblematic of liberty." Her unbound hair was meant to symbolize freedom; instead, what its disheveled look then suggested was failure of respectability, either savagery or, more often, madness. This explains such criticisms as Carlile Pollock's comment in a letter to General Williams, January 25, 1796: "A plough and a sheaf of wheat would be better than an Idiot's head with flowing hair, which was meant to denote Liberty, but which the world will suppose was intended to designate the head of an Indian squaw." Sheldon quotes others, notably an anonymous gibe at the "wild squaw with the heebie jeebies," supposedly antedating by over a century Billy DeBeck's coinage of the phrase in Barney Google.
Within a year the design was revised to clean up the wild hair issues, and Liberty's visage grew more respectable still in a series of redesigns over the following decades. It wasn't until 1909 that the Lincoln cent (with wheat on the reverse) became standard.
(Trivia note: Prior to the chain cent, there was the 1787 Fugio cent, designed by Benjamin Franklin and featuring the very Franklin motto: "Mind Your Business.")
If you found this interesting, you'll love the story behind the original $1 bill.