Adventures in AI, Part MMXIX: Google Photos’ wardrobe woes

It may surprise you to know that I am not a fashionista. I do wear “fast fashion” — clothes that I can put on quickly, although perhaps not as quickly as Mark Zuckerberg and his famous supply of hooded sweatshirts.

Despite my laissez-faire approach to my wardrobe, I was intrigued by Google Photos’ recent feature that catalogs your clothing and then allows you to try on outfits in a virtual dressing room.

I gave it a shot, but … stop me if you’ve heard this before about what we’re calling artificial intelligence … the feature had some pitfalls in initial setup and then assembled a virtual outfit that gave me a mobility aid that I don’t need.

Alas, it wasn’t a Hoveround.

Digging through digital drawers

It was easy to turn on the wardrobe feature, but it took about five days for the app to provide any results. The requirements for using this feature seemed reasonable. According to a Google Photos help page, the feature currently requires a user to have Face Groups (facial recognition) turned on and have more than 1,000 photos of themself.

There are some other requirements, including being located in either Brazil, India or the United States. It also appears that initial rollout went to Google AI Pro and Ultra subscribers and then to “other select users.”

I guess I was among the “other select users” when I found the feature on my recently acquired Pixel 10 Pro. Although I try to use AI elements sparingly because I appreciate human creativity (in addition to other moral and ethical concerns), I was tempted to see what Google Photos could come up with.

I turned on the wardrobe feature.

And waited.

After the five or so days, Google Photos was able to display all the items it was able to identify me wearing. It was a bit underwhelming.

A Google Photos screenshot shows some of the 18 articles of clothing identified by the Google Photos AI, including glasses, two hats, pants, a hoodie and a jersey.
A Google Photos screenshot shows some of the 18 articles of clothing identified by the Google Photos AI, including glasses, two hats, pants, a hoodie and a jersey.

Out of the 2,001 tagged photos I initially thought Google Photos scanned (more on that later), the app identified me wearing just 18 items —

  • a pair of glasses
  • a pair of curling shoes
  • a pair of pants
  • a T-shirt
  • one button-up shirt
  • two sports uniforms
  • three baseball caps
  • three jackets/sweatshirts
  • five polo shirts

The fact that Google Photos’ AI was able to extract these items from my photos is fascinating. The generated replicas range from spot on to only slightly off. It gave me some glee to see my clothing rendered as standalone items like they would be in an online store or catalog. All that was missing was a “Buy now” button.

The app actually handled most logos well — usually a massive and glaring shortcoming for AI. Most logos for Oval Curling Club, Real Salt Lake and the Utah Royals appeared to be accurate, although zooming in on some text displayed some of that ol’ fashioned AI greeking. It also gave up trying to render the USA Curling logo on a ball cap, omitted the Michelob red ribbon on another hat and goofed on the rendering of the wording on the Southern California Curling Center logo.

Additionally, Google Photos’ AI screwed up a couple pieces of clothing. In real life, the Real Salt Lake jersey featured a checkerboard pattern across the front, but Google Photos only caught a portion of it.

Likewise, an Oval Curling jersey was designed to feature a mountain motif with colors lightening as one moves up the shirt. For some reason, Google Photos reversed that on the short-sleeve jersey but got it mostly right on the jacket.

What’s missing

I have more than 18 pieces of clothing, so why didn’t Google Photos find other items (likely shirts) among the 2,001 tagged photos of me?

It is worth noting that Google Photos _didn’t_ analyze all 2,001 photos of me. The support page notes that the app will only scan photos taken within the last four years. Suddenly, that 2,001 becomes a much smaller 417 photos.

My working theory is that the vast majority of those photos are candid photos, geared to capture me and others in the moment. I tend not to take a lot of selfies or overly posed photos, especially not those that would clearly show my whole body and what I’m wearing.

As an example, here are some photos I took of myself at some events in March and April. Most of the shots are close-up shots from the shoulders up. I do have some wide shots, but those are usually in groups or in front of a unique backdrop.

A Google Photos screenshot shows recent photos featuring the author. Nearly all of the shots are close-ups, although two are wide shots in front of a unique curling backdrop.
A Google Photos screenshot shows recent photos featuring the author. Nearly all of the shots are close-ups, although two are wide shots in front of a unique curling backdrop.

Perhaps the feature would be more useful for those people who post their looks to Instagram on a near-daily basis. I can envision Google Photos having an easier time picking out possible articles of clothing from those sorts of photos.

Fashion show! Fashion show! Fashion show online!

Despite the lackluster crop of clothing, I was interested to see what it would be like to virtually try on an outfit.

I was a little disappointed to find that Google Photos wouldn’t recommend any specific outfits. I was hoping that the app would make some suggestions to help me rock the runway, like a personal edition of the old TV show “What Not to Wear.”

Alas, the “Outfits” feature allows you to pick items from the wardrobe but there’s no guidance. Even if Google Photos wouldn’t automatically generate a specific “look,” it might be nice to have the app make suggestions while the user is picking out items, like “don’t wear Crocs with a tuxedo” or “orange camo is not necessarily a good fit for a beach party.”

I could picture a “outfit advisor” similar to Google’s “Camera Coach” feature for the Pixel 10 Pro that uses its Gemini AI to analyze a photo preview to offer recommendations on the type of shot to take.

For my first outfit attempt, I picked glasses, pants and the RSL jersey. Once you save a specific outfit, Google Photos gives you the option to “try it on,” but warns that “results may be unexpected.”

Boy howdy, they weren’t kidding about the unexpected results.

Google Photos' first attempt to use generative AI to depict an avatar of the author wearing a virtual outfit, including a forearm crutch that the author doesn't use.
Google Photos’ first attempt to use generative AI to depict an avatar of the author wearing a virtual outfit, including a forearm crutch that the author doesn’t use.

The virtual dressing room assembled the outfit competently on a virtual representation of me. Everything looked pretty accurate, including my freckles on my arms, awkward facial expressions and being just a bit overweight. There was one glaring exception …

The AI outfitted my left arm with a forearm crutch — something I don’t need and have never been photographed using.

I found the situation amusing and puzzling, even while I paused to consider the ramifications of the app showing people using mobility aids. It’s far too common for popular culture to try to make disabled people and their situations invisible. It’s interesting that the Google Photos programming chose to render an assistive device (one that is typically used for longer-term care in the United States).

When I initially shared the rendering with Facebook friends, I suggested that the Google Photos AI may have analyzed photos of me holding a curling broom but incorrectly identified it as a crutch.

I tried a couple of different outfits to see what the AI would generate. One didn’t feature me holding anything at all, but a final rendering depicted me holding a curling broom!

It took three tries, but Google Photos finally got it right.

(As an aside, if you don’t select enough apparel, Google Photos will render clothing so your virtual self isn’t “Donald Ducking It” without pants.)

Will a virtual wardrobe contribute to the end of the world?

A later attempt by Google Photos to use generative AI to depict an avatar of the author wearing a virtual outfit more accurately shows the author holding a curling broom instead of an assistive device.
A later attempt by Google Photos to use generative AI to depict an avatar of the author wearing a virtual outfit more accurately shows the author holding a curling broom instead of an assistive device.

Ultimately, this wardrobe feature may be Google trying to find a useful application of so-called artificial intelligence. Part of the debate on AI adaption is developing beneficial features that the general public would want to use.

The other side of the debate are the ethical and moral ramifications of using the technology. I typically tend to favor the belief that technology is inherently neutral, but its applications can be good, evil or somewhere in between.

I don’t personally mind using large-language models to analyze my writing (going all the way back to IBM’s Watson), but I wouldn’t use it to generate stories or images based on my works. As someone who worked to report and present factual information, I’m also concerned about AI uses that generate falsehoods or harass people.

Also, companies’ push to encourage AI adoption en route to possible profitability appears to be leading to some questionable uses at considerable economic, environmental and social costs.

I think it’s important to highlight these dubious developments, which is why I started sharing “Adventures in AI” with links to stories on my former employer’s Slack workspace.

At the end of the day, being able to virtually don clothes in Google Photos is a mildly interesting use of generative AI. I don’t know what the economic and environmental costs of the technology are, but I don’t see it having a steep social cost given that feature is limited to the specific user.

I don’t necessarily think it’s something I would use a lot although it might be interesting to try on outfits that I don’t already own.

When it comes to dressing to the nines, I would give the Google Photos wardrobe feature a two out of four.

(Featured image: A screenshot from the Google Photos app on a Chromebook shows the option for the user to see their virtual wardrobe.)

Mapping my journeys from sea to shining sea

An iPad screenshot shows a heat map from Google Photos indicating where photos had been taken based on geotagging or Google determining identifiable landmarks.
An iPad screenshot shows a heat map from Google Photos indicating where photos had been taken based on geotagging or Google determining identifiable landmarks.

With this year’s Independence Day celebrations fading into memory, I was thinking about the fact that I’ve had the opportunity to travel across great swaths of this wonderful and beautiful nation. In one sense, I’ve been everywhere (man) — from coast to coast to coast to coast. In another, I haven’t been to nearly enough destinations.

I’ve wanted to write about the map feature in the Google Photos app (Android and iOS) for quite some time, but it was hard for me to summarize why I feel this feature is so amazing. The Fourth of July celebrations brought this picture into focus as I realized that I’ve been able to visit so many different destinations, as specified in many of the songs about our nation.

The first song that came to mind was “This Land is Your Land” by Woody Guthrie. This song stands out strongly in my memory in part because we sang it every morning in kindergarten when I was living in Southern California.

A photo of the author in front of a pair of redwood trees at the Muir Woods National Monument in Mill Valley, Calif, on May 8, 2016.
A photo of the author in front of a pair of redwood trees at the Muir Woods National Monument in Mill Valley, Calif., on May 8, 2016.

For as encompassing as the song is, “This Land is Your Land” mentions only four specific destinations and they’re in the first verse — “From California to the New York island/From the redwood forest to the Gulf Stream waters.”

Of those four, admittedly broad locales, I realized that I’ve been able to either visit or live in all four over the course of my life. Unfortunately, the map feature in Google Photos doesn’t reflect this — my trips to New York and the coasts of Texas, Florida, Georgia and South Carolina all predate when we all carried cameras with us on a daily basis and where we could take endless numbers of photos without worrying about loading or developing film (so before the iPhone, basically).

Using the photos that are uploaded to the site/app, Google Photos generates a heat map based on the location data in the user’s photos plus whatever identifiable landmarks Google can identify in the images itself. The features is accessible under the search menu. It shows up as “Your map” (but it’s not available on the desktop version of the app).

A screenshot shows the Places map in Apple Photos.
A screenshot shows the Places map in Apple Photos.

I will note that Apple Photos also has the option of showing photos based on where they were taken, but I don’t find Apple’s solution as engaging or appealing as Google’s. Apple Photos shows thumbnails based on location. Zooming in will increase the number of thumbnails, but it seems clunky and inelegant compared to Google’s heat map.

The heat map evokes memories of past travels, family reunions and other adventures. I’m agog seeing some of the paths that I’ve taken over the past 39 years.

It evokes the spirit of roaming and rambling from Woody Guthrie’s song. There’s also some of the energy from the burgeoning lists of destinations called out in songs like “I’ve Been Everywhere” or “Living in America.”

The map displayed by Google Photos also reminds me of the map from “The Voyage of the Dawn Treader,” by C.S. Lewis. In that book from “The Chronicles of Narnia,” a magician gives the captain of the titular ship a map of the ship’s voyage to that point. The map was so detailed that when one looked closely with a magnifying lens, they could see accurate drawings of the actual place albeit at some distance. On the other hand, the map was incomplete in some areas because it could only depict where the ship had traveled.

The Google Photos map can feel like magic at times, even though it is simply a matter of technology and information (although Arthur C. Clarke did say something about technology and magic…). Zooming in on the map offers some joy as I can see the areas where I’ve been able to visit. Like the magician’s map from “Dawn Treader,” it can show documented locations in remarkable detail but be incomplete in areas lacking photos.

A screenshot from Google Photos shows a map indicating where photos were taken during a flight from Salt Lake City to San Diego in October 2016. Thumbnails of the photos from the trip are displayed on the right side of the iPad application.
A screenshot from Google Photos shows a map indicating where photos were taken during a flight from Salt Lake City to San Diego in October 2016. Thumbnails of the photos from the trip are displayed on the right side of the iPad application.

When one zooms in far enough, one can sometimes see individual points along a trip, which can be an interesting way to relive a past journey. At other times, the data can be a bit of a muddled mess — good luck sorting through the 2,800 photos I’ve taken over the course of six years around the venue where I curl, the Utah Olympic Oval in Kearns. The Oval is my most-photographed location by far.

A screenshot shows a heat map from the Google Maps app showing the approximate location of 2,700+ photos taken in and around the Utah Olympic Oval in Kearns, Utah.
A screenshot shows a heat map from the Google Maps app showing the approximate location of 2,700+ photos taken in and around the Utah Olympic Oval in Kearns, Utah.

The heat map of the Utah Olympic Oval also exposes some inaccuracies with the location data. GPS can give a good approximation of where a photo was taken (or where your phone is at a given point), but it’s not 100% accurate all the time. Some of the photos that the map says were taken outside the building were very much taken _inside_ the building (the ice is usually a giveaway, particularly in the summer).

It’s amazing that over 27,000 of my photos have been tagged — it’s probably about 95% of the photos that I’ve ever taken in my life. The vast majority of photos have tags because I took them with a smartphone with GPS. That era essentially started for me in September 2013 with an iPhone 5 issued for work (I bought a Samsung Galaxy SII in April 2013 but it didn’t have a data plan).

For the earlier era where photos were taken with a film camera, a digital point-and-shoot, a “dumb phone” or a feature phone, I’ve had to go back and manually map where I took the photos. That process is inevitably incomplete — there are some places that I don’t remember exactly where I was or there are photos where it doesn’t make sense to tag (including photos of loved ones taken by others but I was not actually present for).

Before tonight, I didn’t know how many photos were tagged by Google based on its ability to identify landmarks. When I first discovered that Google was attempting to locate photos way back in 2017, the process could be hit-or-miss. For example, I had taken a photo of a train car during a 2009 trip through California. For some reason, Google thought the photo was taken in Kunming Shi, China. It’s northwest of Vietnam — and nearly 7,500 miles from the nearest Amtrak stop.

Just a bit off the mark. in 2017, Google Photos thought this photo of an Amtrak train car was taken in China — about 7,500 miles from where it was actually taken.
Just a bit off the mark. In 2017, Google Photos thought this photo of an Amtrak train car was taken in China — about 7,500 miles from where it was actually taken.

Since then, Google has appeared to get better as guessing where I might have been. It’s been interesting to take photos or videos during plane trips (when the phone is supposed to be in airplane mode) and to later see that Google had apparently located where they had been taken (again assuming that I wasn’t a dummy and I remembered to put the phone in airplane mode during the flight).

For example, here’s a screenshot of a photo I took of the Salton Sea. Google Photos was able to correctly identify the location.

A screenshot shows a photo taken in September 2021 from an airplane showing the Salton Sea and an estimated location by Google Photos.
A screenshot shows a photo taken in September 2021 from an airplane showing the Salton Sea and an estimated location by Google Photos.

Apparently, Google Photos has estimated locations for about 6,200 of my images. Interestingly, it doesn’t include some of the aerial photos of the St. George area that I thought Google was able to place. That must mean that I was an insensitive jerk and didn’t use airplane mode during that trip. Whoops.

By the way — It may seem creepy for Google to have so much location data and it can be. Unfortunately, it appears that Google Photos only has an option to remove the estimated location data. As Google notes, “If the location of a photo or video was automatically added by your camera, you can’t edit or remove the location.” In that case, it seems that the best option is to disable location data on the phone/camera _before_ taking the photo.

I don’t necessarily mind saving location data because I find it to be a useful set of information. I do try to be selective in the images that I share, but if I needed to be extra careful, I would need to take additional steps to ensure that people couldn’t find geodata (or identifiable landmarks) in the images.

While I find this data useful, it seems like it would be difficult for a stranger to really root through the information. Google Photos presents the location data in a heat map that someone needs to interact with to learn any additional context (such as dates). The map also doesn’t appear to be shareable.

Google has taken the additional step of turning off location sharing by default in some common ways of sharing information like shared albums or conversations (it can be turned on by the user and was apparently on prior to 2021).

I also know that the heat map is woefully incomplete — as I mentioned earlier, the map can only show places that I’ve taken photos of. I’ve visited or traveled through 42 states, but the heat map can’t show that because there simply aren’t many photos from my time living in the Southeast or my lengthy trips across the Sun Belt, Midwest or the Northeast (or visits to Canada and Mexico, for that matter).

The map can also exaggerate single trips across great distances. This can be seen with my single trip aboard the Empire Builder train across the Upper Midwest in 2009. It was memorable, but my only trip through North Dakota and Montana is depicted as a giant blue ribbon on the map.

The map is ultimately a fun look at where I’ve been. It is truly amazing that we have so much freedom to travel about the country. It’s a right that has sometimes been denied to Americans (and regrettably still is to some extent today). I certainly don’t wish to take this right for granted.

The map also indirectly shows places where I haven’t been. Even in areas where it looks like I’m well traveled, there are destinations that I haven’t yet been able to visit such as national parks in Utah.

I don’t know if I’ll get to some of these locales — such expeditions depend on time, money and planning. If I do, I’ll make sure that photos from those trips are added to my Google Photos map.