AI Helps Restore the Missing Pieces of a Chopped-Up Rembrandt

The artist's largest work, "The Night Watch," has been missing about 36 square feet for hundreds of years.

Image for article titled AI Helps Restore the Missing Pieces of a Chopped-Up Rembrandt
Photo: Peter Dejong (AP)

A mammoth 1642 Rembrandt is now complete after centuries of disfigurement, thanks in part to artificial intelligence.


Seventy years after Rembrandt painted “The Night Watch,” edges of the 16-foot-wide piece were chopped off in order to fit Amsterdam’s Town Hall; the hack job cost the painting two feet on the sides and about a foot on the top and bottom. The digital border resets the composition, restores partially-cropped characters, and adds a few missing faces. (If you’re wondering where they are above, we, too, have maimed this due to Kinja’s barbaric aspect ratio rules for lead images, so scroll a little for the full piece.)

Using a 17th-century reproduction of the original for reference, a team of researchers, conservators, scientists, and photographers used a neural network to simulate the artist’s palette and brushstrokes. The four-month project involved scans, X-rays, and 12,500 infinitesimally granular high-resolution photos to train the network. It achieves a greater level of detail than possible from the reproduction by Rembrandt contemporary Gerrit Lundens, which only measures about two feet wide.

Per the Rijksmuseum, where “The Night Watch” has been part of the collection since 1808, the piece is Rembrandt’s largest and best-known work, as well as the first-ever action portrait of a civic guard. Now the center of attention, a captain in a pool of light, shifts slightly to the right. Two new softer figures watch in shadows on the left. Additional darkness gives the eye space to rest, making the previous iteration look crammed by comparison. See the original crop in the dotted box:

Image for article titled AI Helps Restore the Missing Pieces of a Chopped-Up Rembrandt
Photo: Peter Dejong (AP)

Rijksmuseum director Taco Dibbits told Gizmodo via email that the additional space reorients the piece and restores Rembrandt’s carefully constructed composition. “Rembrandt was asked to paint the portraits of the militiamen,” Dibbits said. “Instead he painted a story. This story gets stronger when you see this reconstruction. His atmospheric perspective gets stronger as the two extra militiamen on the left are painted less detailed. The painting gets even more dynamic now the two central officers are no longer in the middle of the painting. This increases the movement as the company is now coming from around the corner.”

“The Night Watch” belongs to generations of cropped and aged and altered paintings. Thieves cut another Rembrant, “The Storm on the Sea of Galilee” (1633), from its frame which remains empty at the Isabella Stewart Gardner Museum. Vermeer’s “Diana and Her Companions” (1655-1656) is believed to be missing its right edge. The “Mona Lisa” (1503) debatably lost some edges itself, as well as a range of blues and pinks to yellowing varnish.


Wisely, the museum has encased the piece, which has miraculously survived two slashings and a spray of acid as well as being rolled up and transported throughout World War Two, according to the Art Newspaper. The museum unveiled the painting this morning and will display it for three months. You can see it digitally stitched together on the museum’s site.



and transported throughout World War Two

One of those places was in the caves near Maastricht where it and hundreds of other works of art were hidden from the Nazis. There are nearly 200 miles of passages and during the war, housed a bakery, church, and other amenities for the nearly 6,000 people who took refuge, there: