Before photoshop, there was painting. If you were commissioned to do art, you weren't about to paint an ugly portrait of the guy. So chins were lifted, skin was cleared, etc. and it gets sorta hard to tell what people really looked like. Until now.
Eric Altschuler, a history buff, studied the early 19th century painter Gilbert Stuart, who painted portraits of the founding fathers, the first six American president and other early famous individuals. He compared those portraits with early photographs and determined that Stuart's artistic style differed from reality (the pictures). Supposedly he added fuller cheeks and higher eyebrows.
[Atschuler] then created a computer algorithm that took an average of the portrait and the painting. They applied the method to portraits of the presidents who lived before photography, effectively subtracting Stuart's signature changes.