Artificial intelligence, imaging and genealogy

One of my favorite family photos is well over a century old. My great-grandfather, Dr. Thomas Joshua Frost, and his family are gathered on the porch of his home in Erin Springs, Oklahoma Indian Territory, about 1897. The version you see here, though, has little resemblance to the original due to some interesting photo processing available to those of us fortunate enough to have a trove of old family photos passed down from our ancestors.

First of all, the original photo was scanned and tightly cropped to show the family on the porch. It was cleaned up and sharpened a bit in my ancient version of Adobe Photoshop. Then, MyHeritage.com came out with its online app that enhanced and colorized old photos. (The genealogy website also features an algorithm that animates portraits, but more on that later.)

I had been playing around with colorization in Photoshop for years, but it was a slow and methodical process that required many hours per photograph. Now, MyHeritage.com was making the process instantaneous. (I have since learned that newer versions of Photoshop feature a colorization filter, and Ancestry.com now can colorize your family tree photos.)

I am aware of the longstanding debate over colorization, and I get the concerns over artistic integrity when it comes to cinematography and photography. But I’m less concerned about artistic integrity than making a connection with members of my family whose lives unfolded long before I was born. Colorizing old photos like my great-grandfather’s cabin shifts moments in time closer to my present.

AI facial restoration

My next imaging epiphany came with the discovery of Topaz Labs’ suite of image-editing products. At first their software, which uses artificial intelligence to enlarge, denoise and sharpen images, seemed to produce much the same results as Photoshop through a process that is largely automated. But frequent updates improved its performance until a couple of months ago when Topaz released a new version called Topaz Photo AI.

While the new version does a pretty remarkable job at restoring old photos, the most amazing function of the software is facial restoration. AI is far from perfect, particularly when dealing with low resolution photos with scratches, dust and other artifacts. An AI algorithm can transform a smudge on a person’s nose into a face right out of a horror movie. But other times the results can be amazing.

For instance, the only photo I have of my great-grandmother, Mary Elizabeth Morris, is a wide shot showing her sitting on the porch of her home with two of her sons. Her face in the photo is not much larger than the head of a Q-tip cotton swab. I cropped the photo to a portrait, cleaned it up the best I could in PhotoShop, then ran it through Photo AI.

A distortion on her chin in the original photo seems to have distorted her mouth in the restored photo, but the results were still pretty amazing. I now have a better idea of what my great-grandmother looked like. Or do I? Does the image depict my great-grandmother, or does it depict images that were used in training the AI software? I really won’t know unless another image of Mary Elizabeth turns up, but for now I am happy to have this image.

Similarly, I can easily recognize the members of my great-grandfather’s family in the top photo — my grandmother’s twin sisters in matching dresses, my grandmother as the 5-year-old at her father’s knee, etc.

It can get a little creepy

MyHeritage.com didn’t stop with enhancement and colorization. The next thing it introduced online was animation of portraits. Its algorithm recognizes and extracts faces from uploaded photos, then uses your choice of 20 algorithm variations to animate the image.

The animations are interesting, a bit gimmicky and often a little creepy. If the subject is not looking straight forward, the results can be a little distorted. In a photo animation of one of my wife’s ancestors, Nellie Mae Smith, she is wearing eyeglasses and her glasses blink when she blinks her eyes.

Admittedly, much of the fascination is the novelty of these animations rather than any historical context. But there is one MyHeritage.com online tool called DeepStory that I believe has value in documenting our families’ stories. It combines AI animation, text-to-speech and a slide-show editor to compose a story about the life of an ancestor.

Yes, it’s still a little gimmicky and you are unlikely to find a voice like that of a family member whose voice you remember from childhood, but it does capture a slice of family history that might just spark an interest in genealogy for YOUR grandchildren.

I can envision a day in which artificial intelligence might have immeasurable benefits in rapidly retrieving and organizing a family’s genealogical records, a task that now takes decades of gathering, evaluating and sorting data into physical or digital collections. I also can imagine AI software capable of writing family history books — though as a retired newspaper editor, I’m pretty sure there always will be a need for human intervention.

But for now, I benefit most from — and get the most satisfaction from — AI enhanced photo restoration. While it doesn’t match the pixel-by-pixel restoration for which I’ve paid hundreds of dollars for a handful of our most cherished photos, it has had a huge impact on our collection of 1,200-plus ancestral photos.