AI Restores Mutilated Rembrandt Painting ‘The Night Watch’

One of Rembrandt’s finest works, The Militia Company of District II, commanded by Captain Frans Banninck Kock (better known as The Night Watch) from 1642, is a vivid representation of Dutch Golden Age painting.

But the painting was severely disfigured after the artist’s death when in 1715 it was moved from its original location in the Hall of the Guild of Arquebusiers to the city hall of Amsterdam. The city officials wanted to place it in a gallery between two doors, but the painting was too large to fit.

Instead of looking elsewhere, they cut out large panels on the sides and some sections at the top and bottom. After deletion, the fragments are lost. Now, centuries later, the painting has been created using artificial intelligence.

The Rijksmuseum in the Netherlands has owned The Night Watch since it opened in 1885 and has considered it one of the most famous paintings in its collection. In 2019, the museum embarked on a multi-million dollar restoration project, known as Operation Night Watch, to restore the painting. This is the 26th restoration of the work in its entire history.

Initially, the question of restoring the “Night Watch” to its original size was not considered until the prominent Rembrandt researcher Erst van der Wetering suggested this in a letter to the museum, noting that the composition would change dramatically.

The museum invited its senior researcher, Rob Erdmann, to lead the work, using three main tools: the remainder of the original painting, a 17th-century copy of the original painting attributed to Gerrit Lundens that was made before the cuts, and Artificial Intelligence Art Technology.

On the decision to use AI to restore missing pieces of Rembrandt’s The Night Watch rather than instructing an artist to repaint the work, Erdmann told that there is nothing wrong with an artist recreating the missing pieces while looking at a smaller copy. Instead, they wanted to see if it was possible to do it without the hand of an artist. This meant turning to artificial intelligence.

Artificial Intelligence has been used to solve a number of specific problems, the first of which was that Lundens’ copy is one-fifth the size of the original and is nearly 12 feet long. Another problem was that Lundens painted in a different style than Rembrandt, which raised the question of how the missing pieces could be reconstructed, much like the way Rembrandt drew them. Erdmann created three separate neural networks, a type of machine learning technology that trains computers on how to perform specific tasks to solve problems.

The first neural network was responsible for identifying common details. More than 10,000 common features have been found between The Night’s Watch and the Lundens replica. Regarding the second, Erdmann said that once you had all these parts, everything had to be deformed into place, essentially reworking the parts, sliding one part slightly to the left and creating another section of the painting 2 percent larger, and rotating the other four degrees.

Erdmann performed a card-like neural network test by dividing Rembrandt`s painting The Night Watch into thousands of tiles and placing the corresponding tiles from the original and the copy side by side. Erdmann performed a card-like neural network test. He divided the painting into thousands of tiles and placed the corresponding tiles from the original and the copy side by side.

Artificial Intelligence ​​then had to create a Rembrandt-style approximation of these tiles. After