I disagree with the criticism. Experiencing the original artifacts of history, like the original unenhanced film, is still possible. And totally understand that it's vital that that is still possible. But experiencing history is not just about experiencing its old, degraded artifacts. It's also useful and enlightening to experience history as it might actually have been back then.
Of course that's partially an illusion, and it's important to be aware of that. But it's still a useful illusion. It's not for nothing that many ancient ruins have been restored so people can get a better impression of what they were actually like. With ruins this is actually a lot more questionable than with film, because you actually change the original historic artifact (which is probably why nobody repaints those ancient statues that probably used to have colour). But we do restore even ancient paintings that have been darkened by the passing of time.
With that in mind, why would we not take this opportunity to create a much clearer window to the past?
It's an illusion, but so is the original footage. People back then didn't experience reality in BW either.
I know it's silly, but I was kinda surprised the first time I saw early color photographies. I was so used to the BW or sepia old images that it had never occurred to me that, you know, people of that time did live in color too just like we do.
> it had never occurred to me that, you know, people of that time did live in color too just like we do.
I think this is an important (and common) note. Especially when you look at the time-frame of most of the images and video we have being only ~100 years old (20th century). That sounds like a long time ago. B&W photos make that feel like a long time ago. But in reality, that's when your great grandma was born. That's not many people ago. In one respect it is accurate to say 100 years was a long time ago but saying it wasn't is also equally valid. While tech has changed a lot in that time, it is hard to say people have. By upscaling and colorizing these photos it makes it feel much more recent. If we accept that long and not long ago are equally valid takes, then we'd have to accept that both these ways of presenting historical information (a picture is information) are valid too. But they do tell different stories, and it is important to remember that.
That is so true, I've also gotten similar kind of realization with being happy - so serious in most pictures, as was the style then, but then a picture of a farm couple laughing reminds me that they infact were living their lives just like we are
The other aspect is the early photos required long exposures. It is hard to maintain a smile or stand still long enough, so a neutral face was requested by the photographer. Daguerreotype's required upwards of 20 minutes!
Cameras tended to have long exposure times. If you have to hold a position for 30 seconds, it is easier to do so with a serious face than a laughing one.
The colorized Peter Jackson WW1 documentary is the most dramatic example of how useful this is, of any that I've seen.
I think making the past seem more real and relevant is useful in helping people to learn from and understand our history by making it more relatable. Those were real, warm blooded people with hopes and dreams, dying by the tens of thousands in 1916. The more you humanize history (rather than being a rote, soulless text or object in a book), the better. Which takes nothing away from how beautiful B&W can be, there's no reason we can't preserve that history simultaneously.
They way they held back on the transition from black and white to color until a good 20-30 minutes into the movie made it all the more breathtaking when it happened. At that point you were already absorbed in the black and white footage, it was magical.
One thing I didn't know is that statues were not white, they had colors (very strong ones), but time deleted the pigments of color leaving only the white marble.
Rather than thinking of colorization (or upscaling) as a thing we do "to" a photo to alter it further from a primary-source ground-truth; another perspective (the one shared by art conservators, I think) would be to think of a photo that was taken in a too-low resolution, or in B&W (when that wasn't intentionally done for artistic effect) as a kind of "damage" done to the photo right at the time of its creation, where that "damage" moved the photo further away from achieving the photo-taker's intent (which usually, with most photos, is to losslessly convey the experience of looking at the subject of the photo, as it looked at the place and time the photo was taken.)
Colorization, upscaling, etc. could then be thought of as repairing this damage, bringing the photo closer to being a lossless communication of the information it was created to convey.
(By analogy: imagine a technology that could analyze an illustration drawn by someone whose hand was shaking due to Parkinsonism, and then compute what the illustration would have looked like if the artist's hand hadn't been shaking. That would also be a kind of "repair.")
> which usually, with most photos, is to losslessly convey the experience of looking at the subject of the photo, as it looked at the place and time the photo was taken.
That’s quite a strong statement, and one that I think many photographers would disagree with.
Snapshots aside, this severely understates the artistic choices a photographer makes. For one thing, black & white and color read very differently to the eye, and offer different palettes for composition (although it’s not about photography, Scott McCloud’s “Understanding Comics” has a great exploration of the effects of b&w vs color and is a fun read to boot). Basically all b&w cinematography also gives the lie to this statement: it’s impossible to honestly argue that a colorization of, say, the stark expressionism of Night of the Hunter would be a repaired version.
The extent to which a photo is, or is even intended to be, a representation of “primary-source ground truth” is something we’ve been arguing since, like, Edward Said. I’m not saying images should never be colorized, or upscaled, or whatever. Obviously that’s not the case. Peter Jackson’s WWI documentary, as another poster mentioned, is a triumph. But it’s too complicated a conversation to dismiss by declaring, in one sentence like Moses coming down from the mountain, that the purpose of photography is the “lossless communication of information”.
I'm not trying to argue that one should look at the colorization of "art photos" as a repair/restoration. Artists of all kinds indeed choose their medium carefully, and their work is, well, their work. It's an expression of their choices made in response to the constraints of the time. The best way to conserve art, is to try to restore it to exactly the way it was when the artist created it; not to try to second-guess the artist.
But most photos are not intended as art. Most photos are candids (or portraits, or photojournalism.)
Or, to say that another way: most people who take "photos"—both historically, and today—are not "photographers" by profession, trying to paint with a lens and film. They're just people who want to preserve a view of something, and have a convenient technology for doing just that.
If you want a general policy on how to conserve photos, it should be based around the idea that unless you know better, you should assume that the taker of a photo very likely wasn't an artist choosing their medium for effect; but rather a person constrained by the media available at the time of the photo's creation. Most photos that were taken in the period when B&W photos were the only photos you could take, were not intentionally B&W. If given the option, a color photo would have been taken instead. These photos are, in est, "damaged" color photos. As if the taker expected a color photo, but due to bad camera settings, a B&W photo was taken instead. The equivalent of a photo taken when there was a hair on the lens—an incidental aberration.
Just like, for example, most photos taken at 320x240, when that was the best quality a digital camera could give you, were not intentionally taken at 320x240. It was just the fact of the photo-taker being unable to afford a fancy film camera and rolls of film, but indeed being able to afford an early-2000s webcam and no rolls of film.
Or, for another example, most people who recorded mono audio of themselves onto wax cylinders, early reel-to-reel tape, etc., weren't going for the "mono audio mix-down sound." They just were in possession of only a single-track audio storage or distribution technology. Remastering the sound with source-separation more faithfully achieves the sound they wanted to record, but didn't have the technology for. (Do the surviving Beatles disapprove of the stereo remasterings of their mono albums? No; they think they're grand!)
I think we actually broadly agree. I was thrown by the mention of art conservationists, but probably took it too literally.
When it comes to archival conservation and restoration, it seems to me that there are two opposed goals which both have value: restoring to the highest-quality version we possibly can, and preserving the material as it would have appeared to its original audience. So for the wax cylinder example, we definitely want to produce the best, truest to intent restoration we can, but at the same time the experience of listening to a wax cylinder recording is also worth preserving.
As long as the restoration is non-destructive (which it always should be, with digital tools), that’s great! We can have it both ways.
There’s an interesting modern twist when it comes to vernacular photography, too. The popularity of instagram filters and the like suggests that there’s something more going on now even with snapshots.
When an image is converted to black and white, information is destroyed. This information can never be recovered without access to the true colors in the scene when the image was taken. You can tell that information is destroyed because there are multiple possible re-colorizations of a black and white image.
When you convert a black and white image back to color, you are not restoring the original true colors, rather you are filling them in based on your priors. It is logically impossible to "restore" the original colors by doing data processing on only the image itself. It is only possible to give the appearance of a restoration.
Yes, and? Art conservation isn't about recovering the original information. It's about making the work do the job it was intended to do.
If there's a big hole in the middle of a painting, the painting is currently doing a bad job of being a painting, because the hole is distracting people from looking at and experiencing the rest of the painting.
If you, as a conservator, fill in the hole (in a reversible way) with, well, anything, it'll likely stand out less than the hole itself did. And, therefore, the painting will now have been "repaired", in the sense that it's now doing a better job of communicating the art that the painter intended to communicate by painting it, than it did when there was a big hole in it.
Same thought process with colorization. A B&W's medium distracts the modern viewer, who is used to color photos, from the photo's intended message. For modern B&W photos, the B&W look is part of the message; but for B&W photos taken when B&W photos were the only kind of photos (especially candid or portrait photos, rather than "art" photos), this choice of medium is mostly unintentional.
Colorizing the photo—even somewhat poorly, like with old Technicolor movies—will result in the viewer of the photo being less distracted by the photo's medium than they would by a B&W photo; and therefore paying more attention to the content of the photo itself. Which is, in most cases, what the photo's taker would want.
It's funny. Conversely, I've seen instances where people purposely post things in B&W to make it feel old.
I'm 49 and my old school has an alumni facebook group where they occasionally post old photos. Sometimes they're from my era (ie high school in the 80's) and they post them in black and white. When I first see them, I think they're from the 40's or 50's. Then I realise they're from the 80's and I feel kind of annoyed!
Depending on where the photos came from the source material may have been B&W. For example many year books in the 80's were still printed in B&W because it was significantly cheaper.
Colorization can also help humanize the past and make it seem less distant. I recall watching a documentary on WWI (World War I in Color) that was all colorized footage. It just makes everything seem more real.
Every time we apply an interpretation technique, we're losing information. It is strictly an increase in entropy. That's not good from a historical perspective
That would be true, if people were perfect absorbers of information. But they aren't.
So, leaving aside that the original is still, an interpretation technique might strictly lose a bit here or there, but it might actually result in many more bits delivered to the end user.
As an African-American it was actually useful to watch some of these colorized videos and get a better since for how African-Americans integrated into the society at that time.
Having it in color with sound made the video seem much more real and relatable.
For me this just ads another facet to develop a better-rounded grasp of the era.
Inserting the extra frames seems to slow the footage down and allow more time for digestion.
I find it often difficult to track what's going on in old videos because of the off-speed of the recordings.
But for the historians who spend a lot of time with these artifacts, I'm sure this feels like a bastardization of history.
But as a lay person, it makes me want to devour more of these videos to get a fascinating and easier-to-digest look at life back in those times.
The dance scene from Hellzapoppin' is my favorite use of the colorization algorithms so far- it's so much easier to follow the dancers when the color pops them out of the background. Color contrast helps a lot. In that case the whole setup- the costumes, the lighting, etc, are all artificial in the first place, adding some fake color on top doesn't seem like a historical sin.
Personally, I don't find frame interpolation to be as effective as colorization; the artifacts it causes are often more distracting than just leaving the framerate alone. It can create a sort of artificial floating feeling to the motion- a problem that people often complain about with native 60fps, and is much worse with interpolated 60fps. Algorithms keep improving though, eventually I might find them more satisfying.
I love the hellzapoppin' colorization but it does show that we have a long way to go -- you can see the colorization struggle with keeping the skin tone of the dancers consistent. obviously, this has a lot to do with training data bias.
> I love the hellzapoppin' colorization but it does show that we have a long way to go -- you can see the colorization struggle with keeping the skin tone of the dancers consistent. obviously, this has a lot to do with training data bias.
The colorization is struggling to keep colors consistent on the costumes, walls, curtains... basically everywhere the lighting keeps changing a bit.
Many things in the scene are constantly shifting minutely between pinkish-beige and bluish-grey, often making them look somewhat iridescent.
I've noticed this sort of effect in many colorization attempts. This one does better than most.
Training data bias is obviously part of the problem, but not the whole story. It might not even be the biggest cause. Image segmentation is one way to help the colorization stabilize.
> the average person will get most of their understanding of the past from time period movies and shows
actually, that's how lots people get their understanding of the present, too.
For example, I'd wager most people on HN have seen more horses in movies than in real life. I've never actually seen what the ocean looks like underwater with my own eyes.
It's actually mind boggling how much of our worldview is shaped by content we consume. Think about how different your worldview would be if it were based solely on things you've experienced first-hand.
And that humans can do that is one of our big evolutionary breakthroughs.
Apes can only really learn from what they experience themselves. Teaching involves some other ape guiding them through the act, at best. Humans can learn from other humans (or their artefacts like books and films etc), even when the activity in question is not performed.
(There might be some of that learning with some apes. There's some with crows. But it's not nearly as prevalent as with humans.)
And to the training set. It wouldn’t surprise me if machine learning resolution enhancement methods would add a smartphone or some out-of time branding such as Adidas stripes or a Nike swash to some movie scenes.
It uses CG, but in an interview the head of production talked about how he kept trying to get his team to push the audio and visuals as much as possible since he really wanted to get the "viewer" to understand even a fraction of what it would have been like to be there.
He wanted to make sure the viewer felt something instead of just being a fly on the wall like a lot of VR experiences.
These people are just doing for photos and video what historians have always done for history in general. They are filling in the blanks with their best guesses according to the information present.
As long as they aren't presenting their results as more real than the original images, I don't see it as a bad thing.
In fact, I think that the field in general needs to be more open about just how much this happens with history in general. Our understanding of the middle ages comes from evidence far less clear than these photos and yet its rare to see professionals admit how much of the field involves countless evolving, educated guesses.
Others may feel differently, but I find the field even more impressive with the understanding that historians are not seeing a clear picture.
> why would we not take this opportunity to create a much clearer window to the past.
My only concern would be that by the very nature of deep learning we are seeding it with simulated information from the current era so we could be injecting temporal biases. Imagine the process of colorizing -- imagine there was a dye color that we couldn't manufacture prior to year X. And we train colorization on a dataset that whether known to the curator or not includes this dye. Are we creating images that are a misrepresentation of the past?
On the other hand, "science" often does some pretty (I think) extreme extrapolation of things like dinosaur fossils, so maybe it shouldn't be such a big deal and it's just something we have to live with.
> experience history as it might actually have been
I think the might in the above is key. If you add new information to take the place of lost information, you're taking a risk that the new information does not accurately mimic the information that was lost. The more bits of new information you add, the more this risk increases.
I would like to see AI upscaled and colorized versions of the past when there are at least 15 9s assuring me that I won’t see Ryan Gosling on the front lines.
I think the problem boils down to the difference between a "pattern-matching" AI filling in details by "inventing" them, or a human restorator doing essentially the same.
A human restorator most likely has more background knowledge from other sources, while the AI might fill in completely unrelated details it's seen in completely unrelated training material.
As long as there's a disclaimer about the type of restoration that's been used, both approaches are fine IMHO. The audience needs to understand that both are just "projections". Similar to how we thought for a long time that ancient greece was "marble white", when in reality it was full of colors.
Definitely have to agree with you here, as well as other posters saying it's more relatable seeing enhanced and colorized footage. To me, history becomes much more engaging and thought-provoking when presented that way. Not strictly related, but it brings to mind the 3D mapping of Pharaoh Ramesses VI Tomb [1]. Walking around the tomb in VR was very eye-opening and really made me appreciate the magnitude of people having built that so long ago.
Agreed. Laypersons and experts alike understand that fact intellectually, we believe that the past was not black-and-white and silent.
But our experience in school tells us that a scratchy, low-framerate video is probably from the past. We grow familiar with Hollywood representations of history. Not having any evidence to the contrary, we develop an alief [1] about the past that it was sepia and grainy and distant, when in truth, it was just as vibrant and real as today. No one except Calvin [2] would say that they believe that Gettysburg was black and white, but after looking at a lot of historical photos in a textbook that look like [3], if I go to Gettysburg, see [4], kneel behind the wall, and touch the stones I have a completely different level of comprehension and connection to the people who fought there that I, at least, am incapable of generating for myself from a black and white grainy photo.
I completely understand that upscaling and AI can invent things that aren't there, like Gigapixel inserting the face of Ryan Gosling in this photo [5] or Xerox copiers replacing a 6 with an 8 [6], as well as more subtle changes like assuming greyscale pants that might have used no dye at all or natural, local dyes were always indigo blue jeans.
The important question is whether or not the AI-invented, upscaled, colorized photos are closer or further from reality than what your brain invents without the aid of that tool. If you subconsciously invent a muted reality, or worse, your brain gets lazy and assumes it's only from a book and wasn't real at all, I think your ability to empathize with and comprehend the past would be better served by a 4k 60fps artist's impression.
I've read all the strips at least half a dozen times, I know about that gag ;) I'm just suggesting that Calvin might have internalized some of those since he's exposed to it so much.
It's totally an illusion. Up-scaling is a lie. Colorization is a lie. It's literally missing information that can never be verified.
It's like if I used machine learning to double the length of the federalist papers. It would just be absurd to treat that like it was a real piece of history.
Translation of historic texts and glyphs is also "a lie" since information (meaning) is lost or distorted during the translation process.
Sometimes that distortion is intentional, sometimes not.
The point being, as long as we're aware of the source of the (image or text) translation we can decide for ourselves whether that source is reputable enough to trust.
We're not that evolved yet. This technology is outpacing our own evolution as is evidenced by all the people in the comments who claim they actually learned more about history by watching doctored videos.
> It's like if I used machine learning to double the length of the federalist papers.
I don't think that's an appropriate analogy at all. When colorizing or up-scaling, you're making informed guesses to fill in gaps, and even if it's wrong occasionally, it doesn't matter.
The process for forming those guesses is the same whether you're generating text or generating pixels. Data is data. Another risk this poses is that video is not as simple as many assume. A naive approach of simply desaturating or downscaling an EiB of video data is not enough to achieve even a 50% level of historical accuracy because that doesn't address the problem of normalizing those processes across different camera models, lenses, film processes, digital scanning processes, codecs etc.
It's less like OCR'ing the federalist papers and more like increasing the length by 2400%. That's what blowing up a video from 480p to 4k entails: fabricating 96% of the pixels based on the 4% which are actually available in the original piece. It really is as useless as I make it sound.
There's a bit of a difference between interpolating to fill in spaces between pixels with hints from what's next to them and adding whole new words and sentences.
Right. A better example might be removing every 5th word from a famous work, say one of shakespeare's sonnets. Then run GPT3 on it to interpolate the missing words. I bet a reasonable fraction of would match the correct words. Some would be acceptable substitutions, maybe you lose some fidelity or wit from the original, but it doesn't impact the overall enjoyment or understanding of the work. Then will be some minority of cases where we get something weird or wrong. Maybe a word that matches the general meaning, but is modern slang for example. Or maybe it guessed something that is plausible in context, but is opposite in meaning that changes something substantial in the work. I would expect that last scenario to be very rare as a fraction of total word substitutions, but also quite likely over the course of a whole piece of work. On the other hand, if we were able to render have a new undiscovered play available constructed from some sort of damaged fragment, rather than my hypothetical test ... having something we could read or see performed rather than just analyze academically seems worth the chance of getting some (even important) nuance in the work wrong.
That's actually a worse analogy. You're obscuring what's actually going on here. I was being nice when I said it was like "doubling the length of the federalist papers." In a proper analogy, our fake document would be 24 times the length of the federalist papers. 96% of our document would be completely fabricated. FAKE. At that point you're learning more about the computer than whatever document you're viewing and it's the same for video. Scaling up from 480p to 4k requires the computer to fill in 96% of the information. That's really only good for entertainment value, not serious academic research into a historical period.
Computers aren't magic time-traveling machines. If you want 4k reproductions of old films, just scan the actual film with better equipment. If you want colorization, that requires actual investigation and to figure out what colors were in use, and how to map them onto the video data. If no such investigation is possible then we're SHIT OUT OF LUCK. It happens. Move on. Stop revising history for entertainment.
I think this could easily be taken a step further. Who cares what historians think when they're so obviously vulnerable to contemporary politics? When a random group like the NYT can kickstart a politically motivated effort to rewrite history and the only meaningful objectors are the economists it's pretty obvious that American historians have lost the integrity needed to engage with their subject in an objective fashion. It's beyond laymen to understand which parts of historical study have been politically coopted and so the only solution is to assume all histories are now compromised. From that position, things like upscaling don't matter in this context because the integrity of the historical record was already sacrificed.
As an avid reader of history, I tried hard to understand the points the points these academics made, however, it's difficult for me to come to any other conclusion that their position is rooted in elitism—that "true history" can only be discovered by straining and toiling, as they do.
A book is a tool to help us connect with others' experiences. Colorization and other techniques—done sincerely and as accurately as possible—are additional tools that can accomplish this in other ways.
Academic historians are concerned with not just what happened in history, but how we know what happened in history.
Altering primary documents to make them more palatable to modern audiences is great for emotional engagement and attracting attention, but it risks creating false impressions of what history actually was like and how we know what it was like. "Hamilton" is a powerful piece of art--today--but it's not accurate history in an academic sense.
(Well, I should say it's not history of Hamilton's time. It will be studied as an important part of the history of our time.)
The folks running these upscaling operations understand the concerns:
> Antic and Kelley aren’t under any illusions that images treated by DeOldify will come out historically accurate, though their reservations are with the practicalities of training a neural network. Making sure colourised films are accurate is “a literally impossible problem,” Antic says. DeOldify uses modern images to train its AI on, he explains, “and we know that's a big weakness, because, amongst other things, it biases people to wearing blue jeans.”
The challenge is, once they create one of these films and post it for the public, the implications and effects are out of their hands.
You might have millions of people watching an AI-changed film and think that what they are seeing is more accurate than the original, since it transmits more information (detail, color, etc). But if that additional information is made up, it's actually not more accurate, and maybe actually less accurate (e.g. wrong color instead of no color). That nuance is going to be hard for a lot of people to understand.
I doubt those youtube historians could possibly do more damage to false history lessons, than all those cheaply made hollywood "historical" movies.
And the youtube historians I have seen, were very good. So mostly on the right side in battle to truth (dramatically speaking) and mostly not on the side of cheap effects for drama, or even intentional misleading for political reasons.
(seriously, there is nothing wrong with historic fiction - but it disturbs me deeply that a) most movies/books are not labeld that way, even if they should and b) even if it is labeled, most people probably do not notice)
Most expensive "historical" movies are wrong too. Otherwise I totally agree with you. I just don't think how much a movie cost to make matters much with regards to historical accuracy.
>all those cheaply made hollywood "historical" movies
There's a British cartoon from 2003 called Monkey Dust, where a recurring feature is completely historically inaccurate Hollywood movies ("Dedicated to all the Americans who died in the early Middle Ages").
I keep thinking about dinosaurs in this context. How do we know how dinosaurs looked like? All we have are some scattered fossilized bones (and footprints) found in a layer of sediment, which experts have carefully (and perhaps wrongly) puzzled together.
Is it wrong to extrapolate the missing pieces to make a complete skeleton? Some experts are able to make really good academic guesses on if the dinosaur species in question had feathers or scales, the size of its lips and toungue, and sometimes even its color. Is it wrong to create a drawing or a 3D rendering of these guesses?
I suppose, to an extent, it is really important to understand how we can know these things from only a few fossilized bone fragments. But to a layperson it is certainly more interesting to see our best guess of how it looked like in reality. Just a picture or description of the original pieces of fossils in that layer of sediment certainly does not do justice to an amazing animal that once walked the earth.
I suppose you could also make the case that we can get it wrong and a false picture of dinosaurs would enter the popular discourse. But I would argue that it would probably be even worse if experts wouldn’t paint their best guess at the time, and left it to the imagination of fiction media to recreate it for the population.
Nit: We have some very good fossils of dinosaurs that have preserved skin, coloration, feathers, and even gut contents. It's not just bones anymore. PBS Eons had a good one on a recent-ish find:
I was hoping someone would bring up PBS Eons, such a lovely series. In fact I was thinking about An Illustrated History of Dinosaurs when writing this comment.
I feel like even a child knows that a dinosaur is a reconstruction. How could it be anything else? They don't exist today.
But a high definition video is what we've naturally been used to "believing in" as being real.
Of course the next generation of people will grow up with deep fakes, so they will not trust anything they watch unless it's digitally signed or something.u
I remember as a kid that dinosaurs felt just as alien to me as lions. Neither existed in the country where I grew up. They had real footage of lions, but they also had what felt like real footage of dinosaurs. Off course I knew lions existed today in Africa, but dinosaurs existed in the past all over the planet (except on the island where I grew up). To me it didn’t feel like neither the lion nor the dinosaur were reconstructed.
I’m not saying that’s a bad thing. They were able to capture the fascination of a kid by making the dinosaur look as real as a lion, and that’s really cool. It turned out their recreations were wrong, e.g. the T-Rex didn’t have any lips and no feathers in these pictures, but now we think they had both. That’s OK. Growing up I learned that they don’t know everything, and there are still things left to discover.
I bet a future teenager will be astonished to find out they didn’t have high quality color film in the early 20th century, that the films they’ve seen from that period have been colored in using modern technology. Just as I was astonished to find out that they didn’t actually know how dinosaurs looked like, but were able to guess using varying techniques.
> I feel like even a child knows that a dinosaur is a reconstruction. How could it be anything else? They don't exist today.
It's possible to see recreations of extinct animals that humans have overlapped with enough to write and draw how they looked. It's even possible to see real specimens of extinct creations, such as the Oxford Dodo
https://www.oumnh.ox.ac.uk/the-oxford-dodo
You don't need much more knowledge, but definitely more than "they don't exist today". But equally with colourised photos/video: you don't need more knowledge than "this was filmed before it was possible to film in colour, so all colours are guesses".
> You might have millions of people watching an AI-changed film and think that what they are seeing is more accurate than the original, since it transmits more information (detail, color, etc). But if that additional information is made up, it's actually not more accurate, and maybe actually less accurate (i.e. wrong color instead of no color). That nuance is going to be hard for a lot of people to understand.
I can confidently say there is a 100% chance that ML/AI-invented content (colours, fill-ins during upscaling, textures, etc.) already outperforms genuine historical content in every possible way in terms of outreach. Just look at any "today in history" youtube or twitter channel and compare it to any actual museum or historical society's website/channel/social media. Orders of magnitude difference.
On the tweets you see replies commenting about how beautiful and vibrant a [false colour, invented content] image is. Hell, those places will post blatantly totally false content (like.. a comedian) and say it's a historical image: https://time.com/5028121/history-twitter-photo/
They are not there for historical accuracy, or accuracy of any kind, those accounts are 100% purely to farm clicks and likes to spam with later.
Better (colours, fill-ins during upscaling, textures, etc.) is not more accurate. Any change is worse from the original because now its modified (however ugly the original is). It suddenly becomes a fake.
This would be usefully true were it not for the fact that laypersons understand that the past was not black-and-white and silent. Nobody other than Calvin is misled into thinking that those records are faithful recreations of color and sound. But without those obvious markers, people lose their incredulity. What you're seeing in these modified records pretends to be real in a way that the originals do not.
I would expect tech people to understand the risks here by comparison to using AI to perform image upscaling: you are inventing detail based on what the AI expects to see. For artificial works, this is often great. For, say, a police department trying to upscale grainy CCTV footage, it produces a work of fiction that has the potential to become outright dangerous.
It's fine to present these as works of art, and with the proper context, even as records of history. But pop culture rapidly divorces the art from the context, and I would expect any responsible historian to be extremely clear about the processes use here, even to the point of protest (it is analogous to how astronomers have a hard time expressing to people that false-color photography and artistic illustrations of far-off bodies do not represent reality).
The conscious mind knows, and that only if you think to ask it. The subconscious does not. I wrote about this in reaction to one of the videos earlier: https://news.ycombinator.com/item?id=22256369
I would assert that even with the errors, people get a more realistic view of the past through these reconstructions than by watching the original footage. The reconstructions drop the barrier of otherness, which greatly diminishes the value of the old footage as a historical lesson for the masses. The reconstructions make it seem like a real place that had real people in it.
Moreover... what terrible, horrible thing are we worried about with these reconstructions anyhow? People will have slightly wrong ideas about what color roofs were in the past? People might have wrong ideas about exactly how slidey people's gaits were in the past? It's not like you feed in footage about a train ride or people riding horses down the street and the AI is adding in cell phones, translating people's speech to modern slang, and changing political posters to features modern politicians. We're fighting videos that increase people's connections to the past because we're afraid they... what, exactly?
By all means keep the original and label the reconstructions, but I see no reason to start complaining about them or running around telling people "STOP! Watching these videos might give you WRONG IDEAS about COLORS! What if that BLUE WINDOW was in actuality MAUVE?" This seems more like Victorians having conspicuous cases of the vapors than a real concern.
The reconstructions drop the barrier of otherness, which greatly diminishes the value of the old footage as a historical lesson for the masses.
I think you make a very important point. I watched lots of silent movies as well as historical footage growing up, and the jerky motion and slow frame rates gave them a cartoonish quality that I still find artistically endearing but which makes history less real to the viewer. For historical figures one admires, it confers a sort of mythic aura, and for ones you don't it makes them seem clownish or misshapen. This impacts written history too, where it's already hard to separate our posterior knowledge of how things turned out from the historical figures of whom we have a visually distorted mental image.
I watched a historical series on WW2 where the imagery was colorized and somewhat stabilized/cleaned up a year or so ago and was struck by the different perspective it offered without all the 'emotional blurring' that occurs due to the technical limitations of the time. It was much easier to relate to events, both good and bad, through a 'happening now' frame of reference that would have been closer to what people experienced at the time, when newsreel footage offered immediacy and accuracy that was new and modern.
No doubt in the future there will be debates over whether HD and 4k footage of today and the recent past should be given the full VR treatment, allowing people to experience current events from the point of view of the participants and so forth.
Perhaps the Academic Historians are critical because enhanced and colorized documents make the past more understandable to lay people and reduce the need for the Academic Historian to define the historical context compared to looking at the world through a dirty, scratched up, black and white windshield.
>Altering primary documents to make them more palatable to modern audiences is great for emotional engagement and attracting attention, but it risks creating false impressions of what history actually was like and how we know what it was like. "Hamilton" is a powerful piece of art--today--but it's not accurate history in an academic sense.
That's such an arrière-garde cause. It doesn't take too much time nor too much effort for any self-examining historian to understand that you can't create a "right impression" of history at the level of the general public. It will always be partial, truncated, falsified. Your cherry-picked period is misunderstood by the general public ? Too bad, that's part of the job. The best you can hope is making somebody interested enough that someday they'll have to reflex on historiography, philology and epistemology.
But they're not altering primary documents, any more than someone who writes a new history of some well-covered topic is trying to destroy all other commentary on the period, or artist's renditions of what ancient societies looked like are an attack on archaeologists, or new translations of works in other languages are meant to prevent people from learning those languages.
If that person is wearing an air force uniform and you colourise it to look more like an army uniform, or a different country's uniform, you've made it less accurate as it looks like they're in a different branch of the military.
If you've got a photo of someone whose hair was white from age, and you colourise it as blonde, you've made them look younger and maybe given them a hair colour they never had.
If you've got a photo of a famous scientist or war hero who had a Mediterranean complexion and you colourise them with more of a pale complexion, you've made it less accurate as it looks like they're a different race.
(Of course, these sorts of accessibility-accuracy trade-offs aren't unique to colourising photos - they also apply to everything from translations of historical documents, through illustrations and selecting what content makes it into books, to questionably accurate history facts shared on social media)
> If you've got a photo of someone whose hair was white from age, and you colourise it as blonde, you've made them look younger and maybe given them a hair colour they never had.
But what about the opposite? What if a B&W photo shows someone with white hair, but his hair was blonde. Now the colorized photo is more accurate than the B&W photo. I just don't think you can claim B&W is more accurate absolutely.
> If that person is wearing an air force uniform and you colourise it to look more like an army uniform
Do you understand that the typical consumer of this media cannot tell the difference between air force, and army uniforms from 100 years ago? Get people interested in history, then worry about the little details like the color of someone's hair or uniform.
Get out of your ivory tower and into the real world. Most people's knowledge of history is abysmal. History is boring, as it is taught in schools today.
Normally I think you're right about these things, but between the ivory tower comment, and the sentence after, I think there's an implied argument that's pretty relevant to the situation.
That argument being that the academic approach hasn't lead to people having a more accurate image of history. It has scared them off and resulted in most of them having almost no knowledge of history. The approach doesn't seem to lead to it's intended goal.
The comment was perhaps made a bit crassly, but I think it's something worth considering.
You may have missed one more part. When you see a black and white picture, you assume that the black, gray and white are not real. When you see color, it biases your brain to think the color may be close to correct.
The original films were not altered in any way. They still exist and were not destroyed or compromised by the process. They can still be studied by historians while others are free to enjoy the colorized versions.
I've always had the thought that religious texts should be updated with modern prose and sensibilities. Not for the believers of the religion, but for the non-believers. People are not incentivized to slog through some text that was written centuries ago, using phrasings that they've hardly ever encountered. But if a modern twist was put on it, perhaps it would be better accepted as a good yarn by the general population, and widen the readership.
I think the purpose of colarization of a video is same as mimicking a drawing in ruin that lost its paint. It is an interpretation of history from current time. If historian find the color is not represent the event accurately, it can be changed anyway, just like a painting.
> Academic historians are concerned with not just what happened in history, but how we know what happened in history.
No. Academic historians are concerned with maintaining the dominant historical narrative that they themselves have created.
> "Hamilton" is a powerful piece of art--today--but it's not accurate history in an academic sense.
Sure, but academic history is no more "true" or "accurate" than the Hamilton play. The historical concept of Hamilton is fiction created by historians affected by their own biases. History is interpretation. It isn't fact. It isn't science.
> That nuance is going to be hard for a lot of people to understand.
That's true for any "history". Nevermind that almost no historical document/artefact/etc is the "original", but the "history" we know is ultimately manufactured fiction. There are facts and then there is history ( which is what is colored in between the facts by historians ).
There is Hamilton the actual person, then there is the historical adaptation of Hamilton, the play adaptation, movie adaptation, documentary adaptation, etc which are all fiction.
As a note, if you are going to go all post-modern, you should go all the way. Inconsistency is bad.
"History is interpretation. It isn't fact. It isn't science."
Science is interpretation. Scientists are concerned about maintaining their dominant narrative. "Facts" are very scarce on the ground. (I'm willing to act as if Mumbai exists, but I have no personal experience of it and therefore no positive reason to believe it does.)
In more serious terms, a good academic historian will point to evidence as a way to explain why they want you to believe some statement, and---because they recognize the perils of what they're doing---many of them will complain about potentially misleading modifications to that evidence.
Note: My opinion is that I am neutral/do not like or dislike historical media, I am not a historian, but I do dislike those awful Twitter spammers that steal random "historical image of the day", often with a completely wrong caption, stolen content, no sourcing, and then post affiliate links all over them. I have previously done professional work with ML/AI.
--
I can kind of understand this if it was made very obvious that it was "AI generated"; that is, the colours are totally invented and made out of thin air, that spaces might be filled in with random computer generated content to support upscaling and make it look better.
I don't want a machine-generated random filler content to be repurposed for "oh, here is a source video for the skin colour of specific people back in the day" or "you see those marks in the corner (that didn't exist in the original source), those are hieroglyphs"
AI does not mean you magically get colour restored to an image that did not have any in the first place, it is invented, machine created content that potentially has absolutely zero bearing on the real world.
Additionally, stuff like this ends up ranking higher than the original source content; preference is given to HD+ videos in search, "viral cool colourised version of the past" on instagram will rank higher than "actual history"; it is potentially a large problem when you have people that don't actually know how to separate genuine primary source vs computer generated content.
Think of it as translation. Should you read Dante in his original medieval Italian or in a modern English translation? Yes. But if you want to understand and make strong arguments about Dante, you'll probably have to dig out the Italian.
Or, alternatively, back in the day there were these shows called "situation comedies" or sitcoms. One of them was "Three's Company." Many of the stories of the episodes of Three's Company were taken more or less directly from Shakespeare's comedies. Is it elitism to suggest that seeing a production of Shakespeare's plays is a better way of connecting with what he was trying to say than watching a '70s sitcom?
Fair enough, but then the argument of these historians is basically equivalent to "There shouldn't be an english translation of Dante". Which I think is still a bit silly.
The play is pretending to be a Shakespeare production when it isn't, and will give people unrealistic views on the history, whereas the sitcom won't do that
A very old rule of thumb that I learned at one of my first jobs: "Never give anyone more than two significant digits. Never give a manager more than one."
In fact I think we might even need more of these. How many people imagine ancient Roman architecture with white columns and white statues, when in fact there is evidence those were painted in varied colors. Without tools to aid our imagination we have effectively bleached ancient Rome.
I visited the Vasa museum in Stockholm a few years back. The Vasa is an amazingly preserved shipwreck from the 1600s. There ship is mostly original and looks like bare brown wood like everyone imagines a shipwreck to look like.
However, they have a model and reproductions of parts of the ship showing how they were brilliantly painted. The tour guide made clear that the aesthetic beauty and conspicuous consumption on display was as much a part of the kings arsenal as the cannons. It really changed the way i looked on the past.
By reading historical accounts of what colors different things were.
If you give people black-and-white film to watch, people are guess the colors that are there based on what they are familiar with anyway, so this has a chance to be more historically accurate than that.
Consider the film "They Shall Not Grow Old" in which WW1 footage was painstakingly restored and colorized. They went and found old uniforms and weapons to check the colors. They went to the present-day locations to check the colors of the grass and trees. Etc.
it's difficult for me to come to any other conclusion that their position is rooted in elitism—that "true history" can only be discovered by straining and toiling, as they do
I think there's a very similar pattern in tech with developers who hack on apps in 'proper languages' like Rust and C++ dunking on developers who write JavaScript or PHP because those aren't seen as 'true development' because they make things 'too easy'.
Perhaps it's because of loss of control? Previously one may have relied on an academic's interpretation via interview and now the upscaled video becomes the more compelling medium.
You can't really know history without paying tens of thousands of dollars to a university -- and it has to be a university that pays big salaries to professors too. Not one of those cheap ones. Then the professors will say that you've really learned something.
<...> I tried hard to understand the points the points these academics made, however, it's difficult for me to come to any other conclusion that their position is rooted in elitism—that "true history" can only be discovered by straining and toiling, as they do.'
With respect, historians who are involved with archiving aren't grinding these historical points fine because they're 'rooted in elitism'. Even if some are elitist (some people are), that's not what the profession of archiving is all about. Snowwrestler before me aptly sums the matter up. I'd also offer the following comments but I tackle some of the reasons why archivists do what they do from a more technical perspective.
Archiving is a very complex business, especially so if historical context is to be preserved with fidelity. Moreover, as we've seen from both the article and these posts, that historical 'fidelity' is defined or set by the different contexts in which the archive material is used. Simply, people will use archived material in many different ways according to their individual requirements; furthermore, one's requirement will also likely vary depending on one's circumstances or surroundings. (For instance, a curator will likely require the highest quality of reproduction for a museum exhibit and a lower one on his/her smartphone.)
Here, I'll mainly confine myself to archived images as per the article, however these issues could also involve paintings and other works of art, and even objects (have you noticed how the value of a rare coin dramatically decreases in value if you are silly enough to remove its patina?)
Back to images: Ideally, for preservation, we'd like to replicate an image perfectly and end up with an identical clone—and the only way to do that is copy the whole physical object molecule-for-molecule. Clearly, this is impossible, so what’s next?
The limits of current technology means that essentially we are limited to capturing a planar reproduction of the image by photographing or scanning it from directly above or by projecting light through it onto some recording medium. So what are the criteria for doing this, that is if we wish to copy every trace or single bit of information from said image?
1. First we have to acknowledge that even if we were able to record every bit of information that's available from this planar view/projection (which is very unlikely), that it is not all the information that comes with the image. Being a physical object, it has thickness; it has a photographic emulsion; its emulsion is chemical and we may wish to know how it was made. Is the emulsion only blue sensitive, say a collodion emulsion, or is it new enough to be either orthochromatic or even panchromatic? What is its physical condition, has it been damaged, or scratched, or has it been repaired, and how was it stored, .etc (as that extra (metadata) information tells us a great deal more about the image and its history than just the image itself)? Clearly, those who require this ancillary physical information are likely to be different people to say casual YouTube viewers—this is an important difference.
2. Incidentally, archivists are also concerned not just the intrinsic nature of the image itself but also aspects about the way it was viewed, its environment, etc. For example, did the original film stock have a tint or was it completely neutral? Has the film changed in color since the print was made, does it now have sepia tint? When film was first viewed, what was the brightness and color temperature of typical projector lamps of the period? This information is important as it will influence how we end up viewing the film today,
3. To capture every detail of the image requires some very stringent requirements and often it's not possible to meet them. If we attempt to capture every detail to the point that the law of diminishing returns has determined it's not worthwhile going further, we'll almost certainly never get to that point. Even today, it's almost impossible to scan even 'ancient' 35mm film to the limit of its resolution and density because scanners just don't exist that are capable of doing it in any practical way. Sure, for still images, you could use a $100k+ drum scanner that uses photomultipliers but it's a very expensive and extremely tedious and messy process. For moving images, you could use a telecine that also uses photomultipliers but I've never come across one that's capable of exceeding the limiting spatial resolution of 35mm film made circa 1900 (some of this film was remarkably sharp)! (And these days, every Telecine I've seen uses lossy compression to store images.) Sure, one with suitable specifications could be made to order I suppose but I hate to think of the cost (the fact is most people just don't need to reproduce every bit of info that's on film)! That said, the film may not even be the limiting factor, the image itself could be out of focus or otherwise deteriorated (but you have to allow for best case).
4. With still images, when scanning, you need to consider the limiting spatial resolution of the film as well as consider the Nyquist limit of the film/scanning combination. For images, this can be a complex matter (especially movies) but theoretically an ideal scanner should have around double the resolution of the image (it's essentially the same reason why 44.1kHz is used in CDs to record audio that only goes to 20kHz). Again, no scanner that I know of outside drum scanners comes even close to these figures).
5. Therefore, from the outset we have to compromise. Whether current-day best performance is satisfactory for most archivists is moot. I'd suggest, given the chance, that almost every archivist would keep the original film after scanning so that at a later time a better facsimile can be made. Of course, with disintegrating nitrate film this is not likely.
6. OK, we have the best photographic reproduction we can muster, so what comes next? First this, true-as-is-possible image is the master copy and it must be distinguished as such from any later reproduction that 'enhances' the image in any way—irrespective of reason. There should be no argument whatsoever about this (I doubt you'd ever find a respectable archivist who would disagree with this logic.)
6. I see no objection to image enhancement so long as it is made clear that the image is enhanced and that it not the original or master copy. It's a big subject so I'll try to keep matters short here. Essentially, the future of image enhancement has hardly begun. In future expect to see images such as those horrible 16mm grainy WWI battlefield scenes come to life with enormously enhanced resolution as well as having accurate 'true' color—color that's calibrated to real-world object.
7. AI will make this possible by 'learning' the properties of the objects in these old movies then applying new data to the existing images. Moreover, it's already beginning to do so now. What about the accuracy of the color you ask when we had none to start with. In short, AI will scan millions upon millions of objects including thousands of items from WWI and it will do so in color and in high resolution and even in 3D. To reduce noise and film grain, it will also scan multiple instances of the same footage and compare side-by-side frames both within and across films (and also from other source material such as related hi-res B&W photos—for example, Frank Hurley's famous, wonderfully-sharp plate camera photos of WWI). It will then match and analyze all this information to provide both stunning and realistic images. But as I said, these are not the originals, and they serve a completely different purpose altogether!
There's much more to this story (such as making almost flawless copies of famous paintings including using identical paint pigments, paint textures and brushstrokes as well as keeping existing damage and crazing marks for reproduction authenticity). This technology will act as a means of insurance against damage, loss etc. but I've not time to cover more about it here.
"The colours that suddenly flood into the streets of 1910s New York aren’t drawn from the celluloid itself; that information was never captured there."
I've tried to make that point, but I failed many times. Let's try this crowd: We know what colors human faces have, so we can nail those, but coloring a film from, say, the 1950s the way photos from that era looked, is not what the colors back then actually looked like. That's just how camera technology was able to capture them at the time.
So even if you captured color back then, it probably wasn't very realistic.
I don't think people should stop experimenting, I find these videos fascinating and loved Peter Jacksons film, but the past didn't look like you think it did. You're just used to it because all the photographs from the era look a certain way but they were limited.
People didn't look like books either, and didn't look like pottery/statue fragments with the paint washed off, but historians aren't trying to ban that.
There are whole pseudo-intellectual cultures like Objectivists who think that peak of culture are statues with the paint worn off and reading scripts of plays meant to be experienced in live performance.
I can empathasize with the argument to keep the film unmolested and not for snobby reasons like that.
I watched Peter Jackson's "They Shall not Grow Old" which is a collection of restored WW1 footage. Was super hyped going in, but when I saw the footage itself, it just felt off. Like the colors maybe weren't quite right or the motion was super messed up and the whole thing just felt weird and made watching it unenjoyable. I would have prefered to see it in black and white.
My other gripe with the film was that he also loaded it up with sound effects and even added not well mixed VFX like extra explosions and shingles falling off roofs that appear from nowhere (hence why I'm pretty sure they were added in post). Dumping that kind of cheap blockbuster action elements actually felt pretty disrepectful of the soldiers that got filmed. Kind of like they were using them as action figures to tell a story that wasn't quite what happened in reality.
Remember, this footage was created using silent-era film equipment over a hundred years ago. Much of the footage used in the film is borderline unwatchable without the post-processing used: hand-crank cameras meant wildly variable framerate, volatile & unstable early film stock on the frontlines of a war exposed to smoke, dust, heat, cold and gas attacks, over/under-exposed footage caught in the heat of the moment, the list goes on. What I'm saying is, you'd probably complain a lot more if you saw the original footage, or you'd skip most of it as unwatchable in its original form.
Finally, there's an education factor at play. Young people deserve the best possible chance they can get to engage with their history, especially when it offers such a tremendously impactful lesson. And like it or not, I think 90% of people are going to be more engaged, not less, by well-curated, well-produced and respectful recreations of the past. Jackson recorded the SFX using original historical weapons, and colored uniforms using historical examples.
"However, it’s a bizarre sight that greets us, and not just because we lose the top and bottom of each frame. The effect is initially impressive, but that soon wanes. Almost every man has a creamy, peachy skin tone, and the grass in each shot is a warm, yellowy green. The sky is blue. If it weren’t for the daubs of bright red blood, and the bomb craters, this would risk being a unnaturally prettified image of war, with remarkably consistent scenery. These are really incredible images – so homogenising them in this way does them a disservice. It’s not easy either to dispel the thought that all these colours (as well as many of the sounds) are simply guesswork: the colour of hair, blankets, signs and wildflowers having been plucked out of the air." (https://silentlondon.co.uk/2018/10/16/lff-review-they-shall-...)
> Much of the footage used in the film is borderline unwatchable without the post-processing used
That's an interesting commentary on how much technology has advanced, and how we have become so desensitized to it. Consider that when silent films first started, the movie of a train coming right at the camera scared the audience so much that they ran out of the theatre to get out of the way of the train, IIRC.
I agree on your point regarding engaging audiences. Saving Private Ryan and Dunkirk probably taught more people about WW2 than many classes or books.
The difference with moving pictures is that it comes with the implication of photo-realism, while other art forms are self-aware about being representative or expressive rather than being high fidelity simulacra.
I don't mind the colorizations, but to give you some additional ammo for your argument, check out the TV show "The Munsters" and "The Addams Family". If you're not familiar, they're supposed to live in creepy dark houses, and in black and white, the houses do indeed look dark and creepy.
But in reality, those sets were bright pastels. Actual dark walls and floors didn't show up well on film. Which makes me wonder how right we are with some of our other colorizations.
You can google [what did the munsters set actually look like] to get some great examples.
For additional additional ammo look at Star Trek (TOS). A lot of the campy bright colors weren't used to make it look campy but to show up equally well on black and white and color TVs. By using brightly colored uniforms it was obvious even on a B&W TV that Kirk, Spock, and ensign C. Fodder were different and one wasn't coming back from the away mission.
Yes, and we have artifacts (I believe they're called antiques :-)) from that time too that tells us what color clothes were, signs, etc. So we should be able to do alright.
And being such artifacts the means used by historians to reconstruct the past, while they deny others the same rights to interpret it, is quite ironic.
> And being such artifacts the means used by historians to reconstruct the past, while they deny others the same rights to interpret it, is quite ironic.
Maybe that's because those others are more likely to do a bad job, because they lack the training to do a good one. Sort of like how sci-fi filmmakers have taught millions of people that you can hear explosions in space and that lasers make a "zap" sound.
But those artifacts have usually faded with time and exposure to the light, so even those don't show you what the colors actually looked like, unless they have been sealed away from light exposure all this time.
Often you can find a bit of the paint in an overspray area that was covered by some other piece that is well preserved, even if the original paint is totally lost. Start taking off bolts and look at the flecks of paint stuck to the underside of the bolt.
Also, we still have the techniques and materials used to make old paints. New batches can be whipped up (assuming they aren't too toxic) for comparison. Lead based paints obviously are more problematic, but modern technology is really good at replicating paint colors.
You've seen though where a fabric covered couch, color faded over time, will reveal its original color when you roll back the cording covering the zipper or whatever. But I get your point.
Well, technically you're right, but I don't think the pigments in our skin made huge evolutionary advancements in ~100 years. I'd guess people were working more outdoors, but that's it.
So if you were to recolor a film you can take skin colors from today. But an advert, car color or fabric is incredibly difficult, even if you have the original, since the material will have degraded. Even if you repaint it with the original paint it will look different, since the paint will have changed, or if you use new paint the process to manufacture the paint (->led).
which skin colour is a question here as well. There are a lot of variations and I'm not sure that a black and white film is going to contain enough information for you to know. There's a danger of white-washing (or the reverse!) history by mistake.
I think the point is that you simply don’t know in a lot of these videos. They’re random bystanders, not known figures with photographic evidence. So you’re not going to know what to recolour it to, or if it’s even wrong in the first place.
A couple years ago I used deep learning to recolor some old family photos. The net I was using would do it's best guess first, then present a limited set of options for me to choose from when it didn't know what to do. In general, its first pass would immediately make human faces look correct, but leave everything else really desaturated.
I think some of it comes down to the fact that convnets are really good at recognizing human "shapes" (faces, silhouettes, etc) and also the valid color space for human skin tone is pretty constrained if the photo was well lit and you have the luminance value for a grayscale image.
The network was particularly bad at getting the colors of balloons in a picture. I don't know what the real colors were, but the network was trying to tell me 4 of the balloons in a picture were pale yellow and beige while one was a super saturated red.
Another point is that displays can never provide real color representation anyway.
The process flow goes from converting the full spectrum lights in nature into color filtered photons captured on a camera sensor with an artificial color profile, it's own resolution and pixel arrangement, and its own lens specifics, then converted to another limited color palette with another artificial color profile and it's own resolution and pixel arrangement that is the display, in order to simulate the colors of nature as detected by yet another set of filtered detectors with it's own color profile, resolution, neuron arrangement, and lens specifics that is the human eye. There is already no such thing as objective truth when it comes to photography, every step involves heavy interpretation.
I just tried watching one of the upscaled and interpolated videos, and then comparing it to the source. Please consider doing this before commenting, because in my opinion it really illuminates how silly this all is.
The colorization, of course, is lies. The upscaling and interpolation, however, is harder to argue about. Sure, if you were to freeze a frame and zoom in, you are looking at data that may not directly correspond to what was recorded. But it didn’t come out of thin air either - its upscaled and interpolated. I have read the arguments and some of the comments on this page and I still cannot figure out how this isn’t just rationalizing elitism.
Of course we need to keep the source material, but nobody is suggesting to not do that.
Wow, I played those in sync and for me the 4K version is significantly more relatable. Even though the colours are obviously fake, the smoothness of the movements of the people in the video is what makes it great because I can actually relate and feel the emotions through the body language. It also feels more immersive because the nature's physical behaviour looks more accurate.
After reading your comments, I was curious so watched the two clips as well. Although the 4K version was a bit nicer to watch (which I watched first), I didn't feel a significant difference compared with the original. Perhaps I am biased, as I do enjoy watching old (or new) B&W movies.
The thing that struck me most about the colorized video was how every single person in it was wearing a hat. I did not notice this in the B&W video for some reason.
The fake colors are completely made up by the computer. The people could have been wearing drab green or bright red or hot pink suits and it would show up as the same sepia/navy blue that the computer makes all the clothes look like in the colorized video.
I don't know about that particular video, but they're not always made up. There are videos and photos of objects that we know were colored in a certain way (from contemporary descriptions) - flags are very handy in that regard, for example. These can be used to calibrate the algorithm. Of course, a B&W image will still be ambiguous, and the algorithm has to make guesses. But those guesses aren't completely random.
It's the computer trying to copy the color from the training image set onto the video. My point about a guy in a bright red or hot pink suit in the B&W original being represented as sepia/navy in the colorization is still the same. It's impossible to turn B&W image into a color image without guessing what the colors are. The average color of a guy's suit in the colorized copy might be right if it's trained on a realistic dataset, but ultimately it's still guessing and making up information for each individual suit.
The past was also not black and white and grey, or sepia toned. The original photographs and film are imperfect captures of the environment at the time. Given this it feels there's a fallacy in holding the source material as somehow more accurate because it was produced by then contemporary technology - it's all just approximation
It's not that the source material is accurate, it's that our knowledge is based on the source material, so the limits of the source material make clear the limits of our knowledge.
It's kind of like the concept of "significant digits" in science. If your experimental apparatus can only measure a value with precision plus or minus one tenth, it's inappropriate to report your results with digits out the thousandths. Doing so would make it seem more precise to the reader, but the additional precision is an illusion.
In much the same way, using software in 2020 to guess what things "really looked like" can give viewers a false impression of how much we actually know about what things really looked like.
Are you saying that we should only display dinosaur bones and never draw illustrations of what the dinosaurs actually looked like, because it might fool someone into thinking we know more than we actually do?
Wouldn't the same logic apply to someone drawing or painting a historical event based on written source material?
I find it hard to believe these academics would criticize Paul Philippoteaux for the Gettysburg Cyclorama despite it going far beyond the limits of the source material.
The source material doesn't lie to you, though. The black and white you see is the true brightness of the scene, without any fake colours dreamed up by a neural network. Your mind is free to fill in the details itself without having someone else's guess at them imposed on it.
It is like the difference between a film adaptation of a text and the original text. If you see the film first, it will permanently colour your reading of the text, taking away from you the opportunity to draw your own meanings from a blank slate.
Hold on -- every capture of reality, whether it's a 100-year-old black and white video camera or a modern day 3d video -- "lies" to you. There is no such thing as an objective capture of reality. Media is inherently representative, from the limitations of the medium itself to the biases of the recorder in what they choose to focus on and what they choose to not film. Hell, even our own eyes lie to us frequently.
Photographers and filmmakers also make art in their own style. There is no such thing as a perfect representation of the world on film: they are compressing the dynamic range and adjusting the color/brightness profiles through the film they use, the exposure settings and any added filters on the camera, the development process for the film (chemicals, timing, pushing/pulling), the paper for the print (photography) or the film for the positive print (movies), the dodging/burning of the print, and others.
It's not about correcting the image to match what their eyes see, it's an artistic and creative act because a photograph/movie can never represent what the eyes see, and photographers/filmmakers have to interpret the image to get their message across.
I totally in no way meant to imply that camerawork has somehow less artistic value than any other medium, but just as pencil and paper can be used to produce technical drawings of architecture and field sketches to outline geography, documentary footage can be recorded with the primary intent of capturing a scene and setting before rendering a creative work.
Intent isn't necessarily the significant part of a historical artifact, but even then: The way these photos look is part of the state of the world at the time. The way they aged is part of history.
What does it mean for something to be "old"? What does it mean for something to be "well-preserved through time"? All of this is information that may not be significant to you, but it may be significant to a historian and their audience.
I believe gridlockd's point was that the colorization and interpolation applied to these films and photos is a modification to and a stylistic choice about a representation of the state of the world, in the same way Van Gogh's skies are modifications and stylistic choices about the skies he saw.
It's not even so much about stylistic choice but about the historicity of it. Van Gogh belongs to an era in art history. If you chose to paint in this style today it would have a different meaning, even if it looked the same.
The way these photographs look is as much part of history as what they portray - whether intentional or not. These enhancement processes will remove that part.
An artist is executing a vision with the tools available. The final product is what they intended. And sometimes not (see how George Lucas kept updating Star Wars as new tech came out to match his vision).
But for historical documents, the creators are usually hamstrung by technology, not embracing it. If they could have had 4K HDR, I'm sure they would have.
Van Gogh would still use oil on canvas to execute his vision.
The historical films also represent the directors' vision, as they had to choose the lens/film/exposure/processing to interpret the image and get their message across. 4K HDR would just gives them more knobs to tweak. A documentary has less control over what happens in front of the camera, but the whole workflow behind the camera is still completely under the directors' control.
With regards to Van Gogh, seeing how he likes to make his paintings three dimensional (if you ever had a chance to see his paintings in person) in additional to his sculptural works, I wouldn't be surprised if he became a 3D graphics or VR artist had he lived in modern times.
This is BS. I enjoy these videos and encourage people to keep making them.
It’s a great way to get a better idea of what things really looked like which is hard to do with choppy grainy footage. Historians should be doing everything they can to help us be more connected with the past, so being all curmudgeonly on this doesn’t make sense.
>Historians should be doing everything they can to help us be more connected with the past
No, Historians have the job of creating an accurate account of the past, their job isn't to connect you to anything, they're not Facebook for the past or the history channel.
Yes but that super accurate account is worthless if the accounts never actually get told to someone who isn't a historian. It's somebody's job to take those accounts and present them to non-historians/muggles and they care very much that you feel connected to the past.
No, that's a cultural issue about the confusion between education and entertainment. If people aren't interested enough in actual history the solution is not to turn science into entertainment, it's to foster a genuine interest in actual science in the population.
Trying to turn everything into entertainment is a race to the bottom, because people will more and more demand that their scientific education is 'engaging'. That's actually how the history channel went from well, actual history to ancient aliens
This kind of purist attitude is so self-defeating in my opinion. The choices are "make history interesting/easily digestible/accessible" or "not have people care", unfortunately "make people care more about history" doesn't just happen. I'm sorry but I'm not motivated at all to pick through old/inaccessible texts trying to pull out some historical significance but I'll gladly watch a documentary/YT series/TV docuseries that does a good job of explaining events, painting a full picture, and puts events in context.
The type of approach you suggest feels like the arguments about how people should just learn the CLI or low-level computer operations. The "We shouldn't bother with GUI's", "it's not important that we expose functionality with pretty UI", or "if people cared they would learn the CLI and be happy about it".
Making history/art/computers/etc more accessible is not a bad thing.
We don't have to forbid AI-upscaling videos, we just have to maintain the proper awareness of what those actually are (entertainment) and are not (accurate history). That's what academic historians are speaking up about.
"Alter things to make them more popular" is fine in some contexts, but if everyone starts confusing that with the original study of factual knowledge... that's a huge problem.
Good point about ancient aliens. I feel like that is the Walmart of science shows. But the same trend towards "edutainment" is also what brought us Neil deGrasse Tyson, someone who can be entertaining and didactic at the same time.
It's also how we got Titanic, Hamilton, Saving Private Ryan, and the entire genre of Westerns. Ancient Aliens is very dumb, granted, is it so bad that it's worth discarding the entire idea of historical entertainment?
They can do whatever they want. It's a free country. But on day zero, they're on the same ground as the recolourizers. On day 365, they'll find that I'm willing to pay the recolourizer more. Because the utility of a historian to me is making me understand the past.
They may not care that I redirect more funding towards more understanding of what happened. Or they may. Either way they are not in control. I am.
Interpreting the past is literally a subjective process. Previously we used a human neural net - famous for assuming relationships between individuals that do not exist and for denying ones that do.
In time I will use an artificial neural net and I won't need a gating human. It will paint me a picture of a place and time and its best possible interpretation of the evidence.
After all, knowledge that does not adjust my actions is worthless. So if there is truth somewhere that I cannot access and some less accurate information that is well accessible and which will move me closer to the truth then the latter is superior every time. I'll just multiply it appropriately with my prior probability on how accurate an ANN reconstruction can be.
"They may not care that I redirect more funding towards more understanding of what happened."
You may have more understanding of something, but you have to be very careful to remember that it's not "what happened". Adding another layer of inaccuracies does not make something more accurate.
"In time I will use an artificial neural net and I won't need a gating human. It will paint me a picture of a place and time and its best possible interpretation of the evidence."
I wouldn't bet on it. In fact, if you do get an "artificial neural net" to do that, you might want to start worrying about what impressions it wants you to take away.
It’s just a machine. People have ulterior motives. Machines just do what you tell them to. People need to worry about their jobs, their ethnicities, their status. Machines don’t even care about replication. There is no silicon gene that drives them.
And as for what’s happened, it doesn’t matter that it’s less precise. Every piece of information only moves my belief tensor in some direction. It only has to move me in the direction of what’s happened and it’s already superior to an exact description of what’s happened that doesn’t move me in that direction at all.
You overvalue inaccessible truth. And you overestimate historians’ propensity to tell the truth. They have every incentive to lie. And they do. And it works because I suspect you value precision over accuracy. The net result is you have lies told to you by people with a vested interest in you believing a thing.
And I, I have a machine that I can make that makes mistakes. But you know what, you take your approach. I’ll take mine, we’ll see who wins in the marketplace of ideas.
If anything, I an overvaluing the error bars on what is knowable. And I know I will lose because the "market place of ideas" strongly prefers the certainty of a comfortable lie.
No, the Historian's job is to preserve and transmit. Then, it becomes the job of the writers and artists to connect us to that knowledge in a cultural and emotional way. Two very different jobs. Some individuals have the talents to do both, but they are discrete disciplines/jobs/roles.
> It's somebody's job to take those accounts and present them to non-historians/muggles and they care very much that you feel connected to the past.
Yep. That's the job of writers of popular history books and school history textbooks, which historians themselves sometimes write, and creators of historical entertainment such as novels, plays, movies, and TV shows.
Every field of study contributes to building a bridge: math and science the engineering, language the coordination of labor. History alone tells you why to build the bridge.
This was my first thought as well. However, I can think of at least one possible disadvantage --
If these "enhanced" videos proliferate, then as a side effect there will be less desire for the original "source of truth" videos in grainy B&W and low framerates. I.e., this scenario: the original, primary-source video, based 100% on reality, gets 100 views on Youtube while the same video artificially manipulated to look better gets 100k.
Eventually the enhanced one has replaced the original, and possibly even the original gets lost, and in the future historians will only have our grossly-artificial reinterpretations of the video instead of the source of truth.
(but then again, YT is not exactly a historical archive, I'd imagine the originals should be archived properly to prevent this)
If the "improved" version didn't exist, how many Youtube views would the original get? 110? 200? 1100? It sure wouldn't be 100,100. On what basis would you assert that 10 or 100 or 1000 people viewing the original was better than 100,000 viewing the enanced version?
And eventually the original gets lost? What it if eventually gets lost anyway, and we don't even have the enhanced version?
I don't care for them. If people enjoy that good for them I guess, but nothing changes the fact that they're fantasy. The colors are simply made up, that information isn't in the film, you can't tell from luminance which colors were there.
I cringe at colorized films for the same reasons. If you can't take B&W you should consider that make-ups, costumes and all settings were chosen according to how they would look in B&W. In the same way that Technicolor forced you to have one of their color advisors supervising everything. At some point some audiences decided that B&W was boring and the aesthetical appreciation of that whole graphical universe was lost, as a result the expectations are that everything has to look in a very definite way or else it's unwatchable. I find such way of thinking the most narrow-minded by far.
There's a big difference between a colorized movie and historical film that's been colorized.
The movie represents a directors vision, and how it looked in B&W was part of that vision. I agree, they should not colorize old movies.
But historical films are in B&W because that's all they had, not because they wanted to depict it that way. I'm sure if the creators of those films had the option of 4K HDR, they would have used it.
The historical films also represent the directors' vision, as they had to choose the lens/film/exposure/processing to interpret the image and get their message across. 4K HDR would just gives them more knobs to tweak. A documentary has less control over what happens in front of the camera, but the whole workflow behind the camera is still completely under the directors' control.
> If people enjoy that good for them I guess, but nothing changes the fact that they're fantasy. (...) If you can't take B&W you should consider that make-ups, costumes and all settings were chosen according to how they would look in B&W.
So the colorized versions are a Fantasy, but the original b/w versions were already a Fantasy.
If we're looking at colorized feature films, everything is a fantasy of course. The cinematography used various colors to get the desired look on black and white film, in some cases to evoke various percieved colors with totally different colors.
When colorizing, sometimes they have color photos from the filming. You can try to match those, but that may not match the intent of the film. But you might not know the intent either, so you have to guess at that too. It might be easier to convince people to watch colorized films, but it's important to realize it's not really the same film.
Likewise, I've personally been in awe at some of the upscaled Nuclear testing footage that's cropped up on YouTube recently. It's literally awe-inspiring. So what if it's not 100% accurate?
Yeah this article presents a really tiring argument. Learning should be boring! How dare you make it more exciting. These YouTubers don't claim to be some archivists for a national museum..
That video of NYC is breathtaking. I don't care in the slightest that the colors might technically be wrong. The emotional impact of feeling like I could actually be standing on that street is genuinely amazing.
Maybe other people are different but I can't cross the imagination gap with b/w footage but this did it instantly.
"...a better idea of what things really looked like..."
But that's the problem. Is it what things really looked like?
I am a curmudgeon, so presumably I don't count, but I think you should feel free to enjoy the modified photos and videos. But don't forget that that is not what it really looked like.
Historians fill in gaps from partial information about what happened in the past all the time. I see this as no different then recreating a scene based on an old video.
One of the most eye opening videos for me was that sky train that ran through a city. Germany I think? Such high quality I could really begin to feel what that era was like in some ways.
This feels like what happens when someone with technology tries to merge it with another discipline that has failed to utilize the technology: gate keeping and stubbornness about the right way to do things.
I think we're also jaded now, in that all expectation that a photo is a totally truthful artifact produced by a physical process has long since been eroded. Phone camera software tries to create the 'best' image. When the Bay Area had orange skies, everyone noted how their phone cameras were auto-balancing out the orange. Zoom touches up your face. Yesterday's Pixel announcements emphasized changing lighting after a photo is taken. You kind of have to go out of your way to see a recent photo which _isn't_ manipulated in some content-informed way.
So when we see an upscaled, colorized image ... maybe it's not an accurate reflection of the past. But we're no longer used to seeing an accurate reflection of the present, so of course we don't balk at the plausible-but-perhaps-mistaken color choices or interpolations.
Wow, I never would've thought this would be an issue -
If more people knew of how the past looked like, and could relate to it, and understand - they would be shocked at the world we have today.
Especially since most people can not relate to black and white videos/photos because they don't have the context required to expand on that thought. They just see an exposure, thats a photo, enlarged and think "that's it."
Even, ironically, there's the expression/meme that kids/younger generation thought "the past wasn't in color." I remember hearing this when I was a kid in school, during the 90's and 00's.
From the article...:
"That’s not a view many academics hold, however. Luke McKernan, lead curator of news and moving images at the British Library, was particularly scathing about Peter Jackson’s 2018 World War One documentary They Shall Not Grow Old, which upscaled and colourised footage from the Western Front. Making the footage look more modern, he argued, undermined it. “It is a nonsense,” he wrote. “Colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference.”
But on the flipside, this argument does bring to light why it may be bad, but still - I don't think it's good enough to be up in arms about someone upscaling archive footage.
"For Mark-FitzGerald and other historians of photography tools like DeOldify and Neural Love might make pictures look amazing, but they risk obscuring the past rather than illuminating it. “Even as a photo historian, I look at them and think, oh, wow, that's quite an arresting image,” she says. “But always then my next impulse is to say, 'Well, why am I having that response? And what is the person who's made this intervention on the restoration actually doing? What information has this person added? What have they taken away?”
> Colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference.
This really makes me wonder. How does it increase the gap, in comparison to B&W pictures? How is the difference not there already? And why is that bad?
So I checked the source of that quote, and there is no argumentation. None.
And (IMO), the whole point of "They shall not Grow Old" is that something like WW1 can happen again, and you, yes, you personally, will suffer if it does. Everyone has to realize that the people in that old footage are just like you and me, and Peter Jackson's film manages to effectively bridge the gap that blurry, jittery, B&W footage has, precisely by making it look recent.
>> Colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference.
> This really makes me wonder. How does it increase the gap, in comparison to B&W pictures? How is the difference not there already? And why is that bad?
Probably because the colorization can be misleading. With a B&W images, it's clear the color data is missing. With a colorized photo, a red building could be shown as blue or gray, and someone could leave with the false understanding the the building was really a different color than it actually was.
> So I checked the source of that quote, and there is no argumentation. None.
Yeah, these quotes all seem like ad lapidem, just unsupported vague claims. I'm not even sure I agree people can be "closer" or "farther" from the past when looking at an image, it's a poetic way to look at the difference but not precise, and seems like it would evaporate if we tried to find any quantifiable measurable effects.
I can't believe they're shitting on "They shall not Grow Old". The whole point of it was to give an emotional and immersive experience of one of the worst wars humanity has seen to modern audiences. There can be other goals than absolute historial accuracy. All Quiet on the Western Front is one of my favorite books and not because it's the most exhaustive and accurate history of WW I.
Because a photograph is already a lossy simulation of some real situation. But then, even though we don't know what WW1 or Marilyn Monroe really looked like, we know what the artifacts look like. Upscaling is yet another level of simulation. What it says beyond the immediate impact is more about the early era of deep learning than it says about WW1. But it also entices us to mistake this Disneyland WW1 for something that's supposedly behind the original artifact (which did have some reference to the real battle). This is what Baudrillard calls the hyperreal.
> This really makes me wonder. How does it increase the gap, in comparison to B&W pictures? How is the difference not there already? And why is that bad?
It increases factual distance and reduces emotional distance. When just "distance" is used, some people assume one, some people assume the other, and everyone values each differently.
I don't know what you mean by factual distance. Those buildings and people were definitely not black, white or gray. Now they've got a color which is (in most cases) much closer to the original. The original recording was probably also distorted in other ways as well.
You might say you're not looking at the (probably) only reliable source of information for that particular scene, but that's an entirely different proposition.
> What information has this person added? What have they taken away?
That bit is quite relevant. These upscaled videos are interesting, but most people watching them won't know what was there originally and what was added through interpretation and extrapolation. The colours in particular are tricky and not likely to always reflect reality very well, and any sounds added seem to be an amateur's best guesses using what's available in audio libraries mostly.
But it is not just the present-day processing of the material; as Mark-FitzGerald notes, photographs and videos from that age were taken with an objective in mind which may not be as neutral as one might assume. It's not always straight-up propaganda, but whoever took the pictures (or paid for them) had their motives as well. That is part of the context that you need to fully understand what you are seeing (and what you are not seeing), and which is understandably missing from Youtube.
It is unclear to me how viewing the originals, as opposed to these derived works, gives you any greater insight into the motives of those who made them.
The only corner-case I can think of is the erasure of obvious signs of faking, but that does not seem to be the issue that people are getting worked up over. Is anyone even wasting their effort on upscaling faked film?
I think it’s so strange to see all of the arguments saying “if this building is colored as gray instead of blue, then it is making it harder to understand the past”.
A layperson does not care about the exact color of a building or a jacket or a hat, and if it is really that important then it’s probably documented somewhere.
It might be important to historians and that is ok. But they are not the intended audience of these type of upscaled/colorized clips.
> Especially since most people can not relate to black and white videos/photos because they don't have the context required to expand on that thought.
Can anyone expand on this sentiment? I’ve heard it expressed, especially around movies, but it’s always been incomprehensible to me. What is it about being black and white that makes it unrelatable?
But this is because of strange racial issues. Americans expect "Muslims" to all be uniformly brown (I remember circa 9/11 the slur "sand n#$%r" was common) , but Afghans and Iranians have whiter skin and often eyes that are not black. And then we decode that as "this is beauty and it's surprising that there is beauty among these people".
I thought it was silly as well. But then I bought a copy of The Colour of Time [1]. I don't know why it makes a difference, but it definitely does (at least for me). Obviously, your mileage may vary.
When all the images you see from a certain time period are captured in a certain way (black and white, in this case) then it's easy to imagine that this is also how the people at the time perceived it, which of course isn't the case at all.
I wouldn't call it "unrelatable" as such, but adding colour – even if not completely accurate – certainly gives me a better impression of how people at the time saw things, and it becomes more relatable.
Black and white can certainly be used for fantastic artistic intent in e.g. movies, by the way. But for these kind of historical records that's not what was going on; it was just a technological limitation at the time.
> Even, ironically, there's the expression/meme that kids/younger generation thought "the past wasn't in color." I remember hearing this when I was a kid in school, during the 90's and 00's.
This is the fault of the journalist, not the historians. My wife is a history professor. Trust me when I say that historians don't hold opinions just for the sake of it.
She thinks that this is an unusual opinion among most historians but perhaps more common among art historians. She doesn't see much problem with this sort of work, but it is possible for this sort of thing to be done badly. She doesn't doubt that the cited historians have formed a detailed opinion here (rather than just being emotionally upset at somebody touching their things) but doesn't agree with them.
She rolled her eyes hard at the name "deOldify" and mentioned that there are a lot of dumb attempts at doing this that don't have any empathy for people (ex. redoing Shakespeare as rap to connect with young people without actually trying to understand young people, ironically this sort of thing was actually mentioned by Shiryaev).
There is a cost to careless distribution of primary sources (a good example I never considered was the loss of marginalia when digitizing books badly) but it is the job of historians to do this well and it can be done well.
This is a manufactured controversy. There's always some people opposed to anything. It's a non-story to moderate something that is just far too awesome: a portal into the past.
> “The problem with colourisation is it leads people to just think about photographs as a kind of uncomplicated window onto the past, and that's not what photographs are,”
> “Colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference.”
I don't find either of these stances intuitive, but the article doesn't help. A simple follow up question of "why?" would have gone a long way. The technology is clearly explained -- there is interpolation happening -- but clearly the opinions expressed by these historians are philosophical in nature. It's worth having the discussion but the article doesn't go far enough, IMHO.
And even those 2 don't have any real arguments, just opinions. They really sound like they're trying to say something profound while they stroke their chin.
"Colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference."
It's just an opinion. I can say the complete opposite and the opinion sounds just as valid:
"Colourisation brings us closer to the past; it decreases the gap between now and then. It enables immediacy; it removes differences."
These academics seem to think they have some ownership of history because they study it. And it seems they’d rather historical content rot in an irrelevant pile than be accessible. All of this with an argument that “something” is lost by upscaling, without any real elaboration as to what is actually lost.
I worked at the Library of Congress on their Digital Preservation Project digitizing and archiving their collections.
When an archivist analyzes a piece of media, everything ends up being important. For something like an album, you don't just have the audio but the physical media itself, the sleeve, the notes, and the settings of every piece of equipment used in capturing each of those things. Imagine for example, you realize the Photoshop filters were off/wrong in some way?
When you modify the media and treat it as definitive in some way, you're introducing your own analysis (good or bad), just like a Photoshop filter. Even if you're 100% "correct" in the result, it's no longer the same and the next person is applying their analysis on top of yours. It's effectively a game of telephone.
It's not a matter of "most accurate" representation of a moment in time but an "unmodified" representation of a moment in time.
The difference between that and editing books, movies, music, etc applying current norms and expectations becomes semantic.
AI sometimes adds things which were not present in the original scene. For example some upscaling algorithms love to draw tiny glasses. While I don’t see much harm in making some scenes more fun for people, it’s important to keep original sources, because they are ultimate truth, not algorithm outputs. And there’s danger that some people will destroy originals thinking that they are inferior.
Take the original Star Trek. Its color scheme is 'iconic' now but at the time most people watched it in black and white. They specifically picked those colors to look decent on the semi new color TV's but just as importantly to not smear out into the background on black and white TV's. Gilligans island was similar.
I don't think so. I think eventually we would have an ensemble of restorations with different flairs, like how people see movies based on history almost
It's the artifice of presentation that gets me. The color's not real, the filled in scratches aren't real, and once upon a time a photographer sat and composed the shot in the first place. It's an artist illustration of history. Which is fine, those are important to have. But it's too easy to forget that it's not reality.
As the production value of video gets better (in general, not just historical film), people forget the little bit of McLuhan they once half-understood. And the more time we spending looking at screens, the more we WANT to forget that the pictures on the screen don't reflect reality. For example: go tweet what you really think about the world. About politics, about sex, about all the white lies we tell eachother to keep society functioning. Don't want to? OK, now ask yourself why you treat anything you read as less artificially constrained than what you, a relative nobody, are willing to tweet.
"He was talking about how all his life these movies of history had been getting better and better looking. How they'd started out jumpy and black and white, with the soldiers running around like they had ants in their pants, and this terrible grain to them, and the sky all full of scratches. How gradually they'd slowed down to how people really moved, and then they'd been colorized, the grain getting finer and finer, and even the scratches went away. And it was bullshit, he said, because every other bit of it was an approximation, somebody's idea of how it might have looked, the result of a particular decision, a particular button being pushed. But it was still a hit, he said, like the first time you heard Billie Holiday without all that crackle and tin." - William Gibson, Virtual Light (1993)
My cynical take is that these historians’ objections stem from jealousy: they lack the ability or means to create these restorations, and are thus envious that the upscaled versions are receiving more attention than the originals ever did.
My cynical take is that there are no historians saying that we shouldn't upscale images. There are historians who are warning that lots of the information in the upscaled images are ahistorical (blue jeans in the 1910s) so people should view this images as interesting but not accurate views of history. The journalist is the only one saying anyone should stop anything. None of the historians in the article ever say the youtubers should stop. They're just saying pretty much: "Hey, history cares about the context and accuracy of sources and these images are worse sources than the originals, they're good entertainment but bad historical sources". Pump it through a journalist and this is what you get. It's at this point I make some ironic point about how the journalist is using his neural net to upscale the story, inserting innaccurate information to the picture to create a more entertaining piece than the underlying facts.
I really enjoy video from that time period for one particular reason: no cell phones. Seeing people walk not hunched over anything.
One of my favorite music videos (of sorts) is just a song over what appears to be a Mountain Dew street team passing out free soda on a random night outside bars, while filming their efforts.
Apologies in advance, as they definitely focus on a particular gender, but it's still an interesting slice of life in the mid 90s: https://www.youtube.com/watch?v=c7sYupd5dng
I dunno, I saw the title and scanned through...it just feels like click bait to me.
I would love my peers to be more interested in history even if it's as trivial as being amazed by some fleeting vids of altered old footage, even if colors are incorrectly assigned.
Maybe I'm just ignorant, but it sortta seems to me like an interesting view point that's more akin to something you'll see in the comment section of these vids than a whole drawn out article on the matter.
I wonder if it is possible - if not practical - to extract color information from processed black and white film negatives.
It seems plausible that there are trace chemical imperfections in the film medium, and these might react differently to different wavelengths, so that if you could inspect the image at a molecular level you could make pretty good guesses about colors.
“Colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference.”
I don't buy this argument. I feel like the reason these upscaled videos are going viral is because it makes the events in the footage feel like something that actually happened, rather than an old-timey silent movie.
Having said that, I'm not always a fan of upscaled historical footage. I think the best argument against it is that it's a distortion of reality. Regular people don't have the technological sophistication to understand that what they're seeing is the result of a machine filling in the gaps and making a "best guess". They assume all the new information they're seeing was already hidden somewhere in the original image and just had to be pulled out. I think if everyone was on the same page as to what these videos were, there wouldn't be a problem.
I think these videos are analogous to historical fiction novels. Those books are a niche pretty far removed from academic research and serve a very different purpose.
If you want to learn the most detailed academic knowledge of a time period, reading stories written by James Clavell or Colleen McCullough wouldn't make any sense. On the other hand, for a person who's casually interested in a historical time period and wants a fun immersive way to learn and think about it, reading well written novels about that period is an excellent way to go about it. For that same person, reading primary documents and research papers would probably be quite tedious.
I think the videos from this article are a fun and unique way to explore past time periods, while also being of little value to academic research. There's nothing wrong with different niches serving different purposes, even if they cover the same topic.
I support the upscale for the exact same reason as the criticism. Old photos and film almost seem fake or fictional to me. Seeing a more realistic portrayal humanizes the pictures for me. I guess the other question I have is why is this mutually exclusive? We can’t both preserve and digitize as it was and also enhance?
Fixing film speed problems is fine. Increasing resolution is OK if not overdone. 2x, maybe; 4x, no, as Photoshop users know. Remove dust, scratches, camera sprocket drive jitter, no problem - those are all mechanical problems with the original camera.
Colorization is bad, though, because it's often a total guess. The article linked to the Wuppertall monorail shows this. Look at the monorail supports. They keep changing color. The system doing the colorizing has no baseline data for "color of late 19th century monorail supports". It might match them to elevated railway supports, but there's no official color for those. The Wuppertall monorail was a different color in the 1960s than it is now, a much darker green. Parts were brown.
Colorization can probably be stopped by claiming it's racist, when it guesses wrong and makes a Black face white.
Colorizing and upscaling is marvelous technology, and greatly improves my enjoyment of old photos and films. (I saved many colorized photos as wallpaper.)
I know it isn't historically accurate, but the originals are not destroyed, and the historian has those available.
I suspect this article is trying to problematize the issue more than describing an existing, widespread controversy among historians.
It lumps digital restoration, upscaling, frame interpolation, and colorization together, but all of these techniques are different, and some are more controversial than others.
For example, it's hard to think of an argument against using AI to remove imperfections from an image, if you are not troubled by having a human do the same thing. In both cases, information is being added where it had been lost. There's no way around that.
Colorization is a different thing altogether. But, pretending that historians view noise removal and colorization as equally problematic is misleading.
This is the first time I am seeing these remarkable videos. I cannot tell you why, but I am feeling awfully emotional while watching them. It cannot be nostalgia, as I wasn't there during the recordings of these videos, so I'm not sure what it could be.
The academic 'controversy' aside -which I think is BS, at least as it is presented in the article-, two things truly amaze me:
1) How beautiful of a city Wuppertal was in 1902. I have been to Wuppertal about 5 years ago, and it's ... well, not beautiful. Great people, though!
2) The appetite for engineering, for creating things, and the audacity of the late 19th century, early 20th century. A freaking flying train for a city of less than half a million of people. Contrast this to the changes over the past two decades in my hometown, which is about the same size as Wuppertal. The changes were marginal; a cancelled train route is what pops to mind.
This article is nonsense. It absolutely fails to make a point.
It's not like up-scaling stuff with neural networks is going to introduce a random mobile phone into the medium being upscaled or introduce any other glaring misrepresentations. The upscaling is purely for improving quality.
Accuracy of the colors is rather secondary. And if anything, historians should just help out if they can.
Personally I've spent a lot of time watching upscaled&colorized videos on YT, and I'm just mesmerized. It's sooo much more vivid and gripping to look at the past like we can look at the present.
In studying history I find some of the most useful documents are older histories. For instance, reading about World War 1, finding the account written in the 20's or 30's can be very informative. Of course those histories have bias but so do modern histories. Reading both and comparing the two provides even more benefit. Newer histories, in spite of their bias, may have more accurate facts or facts from records that were not available to the historian of 1920. But the mores and culture of the 1920's also plays a role and no modern historian can reproduce that.
It can be comparable to old manuscript, hand-copied many times and each copy modified the original. We certainly wish the original text be survived. But it won't happen anyway. It was copied, modified thus increasing the value along the way.
It's either valuable enough to be copied or not to be copied and lost. We have certain old text because it has been copied and modified.
If they have time and resource to complain it, they should rather start copying the original photo and films in order to preserve the historically important materials. The more copied, more change to survive.
So what is the state of the art in auto upscaling and detail re- imagining? There's a lot of 240p VHS transfer music videos on youtube I'd like to see less bad versions of without regards to accuracy.
I can't believe people actually find those results to be amazing. None of these algorithms, or as they call them - "AIs", are yet capable of adding detail (be it spatial or temporal) in a way that actually makes it look believable. It's all approximation, a very, very poor approxiation. I'm sure we'll reach that level one day, but for now, we are far away from it.
Add to that the fact that those film scans are rarely shared in their original framerate and you're getting an absolutely terrible result.
It seems other fields have already been doing this sort of alteration for some time. For example, space-related research that often relies upon computer-assisted rendering to help the average consumer better understand context and "what things would look like."
Alternatively, many statues from the classical era were actually painted, not bare marble. Is it frowned upon to create an image that renders those statues in color based on trace pigments still remaining on those statues?
I though there might be some reasonable arguments in here, like restorations might be misleading and badly labeled, it might make it harder to find originals because all you find are restored versions, etc. But no, it's just "I'm a proper historian and let me tell you I don't like the way other people are doing history. Making it all accessible and whatnot. History should be hard, or you're cheating."
So people are 'enhancing' videos and photos. Just like we've been doing for decades. Only the process was manual and labor-intensive. And usually NOT performed by historians.
As long as we don't destroy the originals, I can't see why there's even a debate. Every medium captures an interpretation of reality, they are not reality itself.
It might not be to everyone’s taste but it’s impressive work.
Why do these particular academic historians not trust people to understand the limitations of this? To me this is not dissimilar from translating a book from an ancient language - there are problems of bias and so on, but why not make good things a little more accessible? Nothing wrong with something that sparks the imagination.
There are shades of green, brown, red, but not exactly and it doesn't stick or stay, just the way I remember colors in dreams. They are colorful, but I can not exactly recall vividly.
This along with the ‘wandering’ videos are seemingly getting popular on youtube. Eg nippon wandering tv, rambalac. These colorized + hd historical videos are super fascinating. Love how AI has been the driving force for towards creating these
I suppose these historians would also object to me watching star trek tng on streaming in HD since it was clearly meant to be only ever be seen transmitted over radio waves to a cathode ray tube.
ugh, historians have done much bigger damage to actual original artifacts, at least these deoldifying methods are not as biased, or, at least, their bias is quantifiable.
The most interesting thing about these films is watching the mannerisms of the people, something which is generally not observable on the orignal, due to choppy motion/artifacts/cartoonish quality. I don't know how accurate the restoration method for the motion is, but I can never trust historic movies or documentaries to accurately recreate the way people walk, talk or just comport themselves.
As someone with a historian's training, I don't mind it at all. Every narrative and theory of history is based on some element of fiction. Some in the field don't like being reminded of that.
Some Youtube accounts post recordings of video games, some of which are set in a historical context. Nobody takes those for a view of reality.
By the same argument art restoration should not be attempted because cleaning paintings discolored by centuries of candle soot may alter the historical context of the painting and confuse the lay person about what the art really looked like. Or not.
Mostly the argument by the Academic Historians seems to be an attempt to retain control of the historical narrative and colorizing and sharpening old movies means the historian has less room to maneuver.
You can find this phenomenon of self-appointed protectors of values everywhere: the clergy, rap purists, rustaceans, Apple users... in their echo chambers they are always right and only they know how things should be done. If you somehow have the misfortune of treading upon their proverbial holy turf and doing something they consider against their teachings... boy will they get pissed. Especially if you make their world more understandable to those not in the know, then you're immediately a problem.
This sounds like just another form of gatekeeping.
If historians want this to stop, they should first reduce history to a chronologically sorted collection of bare facts, free of opinion and interpretation.
Fact is that if you watch the original footage, you mind "upscales" it anyway. You recognize objects in the scene and forms a description. That description consists of idealized objects. If you see someone's face, you remember that you saw a face, not that you saw a blurry 64x80 grid of pixels with specific defects in resolution and noise.
Is it only me that finds the article quite poorly written? There is no lead throughout the different section, the exact same ideas are repeated over and over, etc.
I think che content could have been reduced to something like 50% of its original length.
There are romantic arguments to be made on both sides here - historians saying that something is lost when this transformation happens, and that this is simply progress that people react to positively on the side of the doers.
Both of those seem misguided and rooted in one paradigm or the other. When I think about the tangible effects of something like this, I don’t see many downsides.
One thing that sticks in my mind - how easy it is for people to look at photos and videos from, for example, the Holocaust, and say “wow, the world was so very messed up way, way back then.”
The technology of the time isn’t telling the story in the full fidelity of those who lived it. Instead, it looks nothing like the reality that it was, and makes it feel further removed. Fine as an art form, but not a service to reality.
Learning from, teaching about, and documenting history is extremely important. The fact that you or I disagree with some historians about the proper balance between accuracy and relatability does not mean we should denigrate the study of history. I tend to agree with you that exposing more people to history is a higher priority than painstaking color accuracy, but I also believe that there is plenty of room for the kinds of historians that care more about accurate documentation than about public outreach.
This feels like clickbaity story generation - cool new tech doing something many enjoy.... not enough tension... go and find a handful of grumpy people who don't concur
'“The problem with colourisation is it leads people to just think about photographs as a kind of uncomplicated window onto the past, and that's not what photographs are,” says Emily Mark-FitzGerald, Associate Professor at University College Dublin’s School of Art History and Cultural Policy.'
Why do these academics always want to complicate things and dictate how we feel? Nobody put them in charge!
'“There's something that's gained, but there's also something that's lost,” says Mark-FitzGerald. “And I think we need to have a conversation about what both of those things are.”'
I'm guessing that would be a pretty one-way 'conversation'.
Edit: looking at the Leeds video it's clear to me the processed version adds a great deal - you can even see clearly how much fun they are having by the smiles on their faces, which is pretty hard to spot on the raw footage. It's charming to think of them fooling around in the garden with new technology, 132 years ago this month.
They're not dictating anything. They're giving their opinion. Just as you are.
> Edit: looking at the Leeds video it's clear to me the processed version adds a great deal - you can even see clearly how much fun they are having by the smiles on their faces, which is pretty hard to spot on the raw footage.
I mean, isn't that exactly the issue that the critics are pointing out? The algorithm is inserting something - the behaviour of people - into the source material that may not have been there. It may be biased towards falsely making the past look like the present because that's what the algorithm has been trained on.
I strongly disagree with these historians, for historical reasons. Our limited recordings of the recent past make it hard for us to understand that it wasn’t that long ago and things weren’t that and people weren’t that different. By 1911, the railroad I used to get to from Westchester NY to Manhattan was already electrified, thanks to an investment from JP Morgan, where my brother works today. By 1910 the subway line running down Lexington Ave I’d take all the time had already been built. So had the Grand Central Post Office, on top of which the office building I used to work at was later built. By 1910, all of Manhattan had been electrified for decades, and you had electric refrigerators and curling irons and stoves. By 1910, vaccines had been developed and the first antibiotics were coming out.
It’s important not to forget how close in time we are to history. Imagine if, instead of always seeing black and white pictures of Martin Luther King Jr., people saw him in 4K color. Maybe they’d remember he was shot for his civil rights activism when Bill Gates was already a teenager, just a couple of heads before Unix and C.
Of course that's partially an illusion, and it's important to be aware of that. But it's still a useful illusion. It's not for nothing that many ancient ruins have been restored so people can get a better impression of what they were actually like. With ruins this is actually a lot more questionable than with film, because you actually change the original historic artifact (which is probably why nobody repaints those ancient statues that probably used to have colour). But we do restore even ancient paintings that have been darkened by the passing of time.
With that in mind, why would we not take this opportunity to create a much clearer window to the past?