Adventures in AI, Part MMXIX: Google Photos’ wardrobe woes

It may surprise you to know that I am not a fashionista. I do wear “fast fashion” — clothes that I can put on quickly, although perhaps not as quickly as Mark Zuckerberg and his famous supply of hooded sweatshirts.

Despite my laissez-faire approach to my wardrobe, I was intrigued by Google Photos’ recent feature that catalogs your clothing and then allows you to try on outfits in a virtual dressing room.

I gave it a shot, but … stop me if you’ve heard this before about what we’re calling artificial intelligence … the feature had some pitfalls in initial setup and then assembled a virtual outfit that gave me a mobility aid that I don’t need.

Alas, it wasn’t a Hoveround.

Digging through digital drawers

It was easy to turn on the wardrobe feature, but it took about five days for the app to provide any results. The requirements for using this feature seemed reasonable. According to a Google Photos help page, the feature currently requires a user to have Face Groups (facial recognition) turned on and have more than 1,000 photos of themself.

There are some other requirements, including being located in either Brazil, India or the United States. It also appears that initial rollout went to Google AI Pro and Ultra subscribers and then to “other select users.”

I guess I was among the “other select users” when I found the feature on my recently acquired Pixel 10 Pro. Although I try to use AI elements sparingly because I appreciate human creativity (in addition to other moral and ethical concerns), I was tempted to see what Google Photos could come up with.

I turned on the wardrobe feature.

And waited.

After the five or so days, Google Photos was able to display all the items it was able to identify me wearing. It was a bit underwhelming.

A Google Photos screenshot shows some of the 18 articles of clothing identified by the Google Photos AI, including glasses, two hats, pants, a hoodie and a jersey.
A Google Photos screenshot shows some of the 18 articles of clothing identified by the Google Photos AI, including glasses, two hats, pants, a hoodie and a jersey.

Out of the 2,001 tagged photos I initially thought Google Photos scanned (more on that later), the app identified me wearing just 18 items —

  • a pair of glasses
  • a pair of curling shoes
  • a pair of pants
  • a T-shirt
  • one button-up shirt
  • two sports uniforms
  • three baseball caps
  • three jackets/sweatshirts
  • five polo shirts

The fact that Google Photos’ AI was able to extract these items from my photos is fascinating. The generated replicas range from spot on to only slightly off. It gave me some glee to see my clothing rendered as standalone items like they would be in an online store or catalog. All that was missing was a “Buy now” button.

The app actually handled most logos well — usually a massive and glaring shortcoming for AI. Most logos for Oval Curling Club, Real Salt Lake and the Utah Royals appeared to be accurate, although zooming in on some text displayed some of that ol’ fashioned AI greeking. It also gave up trying to render the USA Curling logo on a ball cap, omitted the Michelob red ribbon on another hat and goofed on the rendering of the wording on the Southern California Curling Center logo.

Additionally, Google Photos’ AI screwed up a couple pieces of clothing. In real life, the Real Salt Lake jersey featured a checkerboard pattern across the front, but Google Photos only caught a portion of it.

Likewise, an Oval Curling jersey was designed to feature a mountain motif with colors lightening as one moves up the shirt. For some reason, Google Photos reversed that on the short-sleeve jersey but got it mostly right on the jacket.

What’s missing

I have more than 18 pieces of clothing, so why didn’t Google Photos find other items (likely shirts) among the 2,001 tagged photos of me?

It is worth noting that Google Photos _didn’t_ analyze all 2,001 photos of me. The support page notes that the app will only scan photos taken within the last four years. Suddenly, that 2,001 becomes a much smaller 417 photos.

My working theory is that the vast majority of those photos are candid photos, geared to capture me and others in the moment. I tend not to take a lot of selfies or overly posed photos, especially not those that would clearly show my whole body and what I’m wearing.

As an example, here are some photos I took of myself at some events in March and April. Most of the shots are close-up shots from the shoulders up. I do have some wide shots, but those are usually in groups or in front of a unique backdrop.

A Google Photos screenshot shows recent photos featuring the author. Nearly all of the shots are close-ups, although two are wide shots in front of a unique curling backdrop.
A Google Photos screenshot shows recent photos featuring the author. Nearly all of the shots are close-ups, although two are wide shots in front of a unique curling backdrop.

Perhaps the feature would be more useful for those people who post their looks to Instagram on a near-daily basis. I can envision Google Photos having an easier time picking out possible articles of clothing from those sorts of photos.

Fashion show! Fashion show! Fashion show online!

Despite the lackluster crop of clothing, I was interested to see what it would be like to virtually try on an outfit.

I was a little disappointed to find that Google Photos wouldn’t recommend any specific outfits. I was hoping that the app would make some suggestions to help me rock the runway, like a personal edition of the old TV show “What Not to Wear.”

Alas, the “Outfits” feature allows you to pick items from the wardrobe but there’s no guidance. Even if Google Photos wouldn’t automatically generate a specific “look,” it might be nice to have the app make suggestions while the user is picking out items, like “don’t wear Crocs with a tuxedo” or “orange camo is not necessarily a good fit for a beach party.”

I could picture a “outfit advisor” similar to Google’s “Camera Coach” feature for the Pixel 10 Pro that uses its Gemini AI to analyze a photo preview to offer recommendations on the type of shot to take.

For my first outfit attempt, I picked glasses, pants and the RSL jersey. Once you save a specific outfit, Google Photos gives you the option to “try it on,” but warns that “results may be unexpected.”

Boy howdy, they weren’t kidding about the unexpected results.

Google Photos' first attempt to use generative AI to depict an avatar of the author wearing a virtual outfit, including a forearm crutch that the author doesn't use.
Google Photos’ first attempt to use generative AI to depict an avatar of the author wearing a virtual outfit, including a forearm crutch that the author doesn’t use.

The virtual dressing room assembled the outfit competently on a virtual representation of me. Everything looked pretty accurate, including my freckles on my arms, awkward facial expressions and being just a bit overweight. There was one glaring exception …

The AI outfitted my left arm with a forearm crutch — something I don’t need and have never been photographed using.

I found the situation amusing and puzzling, even while I paused to consider the ramifications of the app showing people using mobility aids. It’s far too common for popular culture to try to make disabled people and their situations invisible. It’s interesting that the Google Photos programming chose to render an assistive device (one that is typically used for longer-term care in the United States).

When I initially shared the rendering with Facebook friends, I suggested that the Google Photos AI may have analyzed photos of me holding a curling broom but incorrectly identified it as a crutch.

I tried a couple of different outfits to see what the AI would generate. One didn’t feature me holding anything at all, but a final rendering depicted me holding a curling broom!

It took three tries, but Google Photos finally got it right.

(As an aside, if you don’t select enough apparel, Google Photos will render clothing so your virtual self isn’t “Donald Ducking It” without pants.)

Will a virtual wardrobe contribute to the end of the world?

A later attempt by Google Photos to use generative AI to depict an avatar of the author wearing a virtual outfit more accurately shows the author holding a curling broom instead of an assistive device.
A later attempt by Google Photos to use generative AI to depict an avatar of the author wearing a virtual outfit more accurately shows the author holding a curling broom instead of an assistive device.

Ultimately, this wardrobe feature may be Google trying to find a useful application of so-called artificial intelligence. Part of the debate on AI adaption is developing beneficial features that the general public would want to use.

The other side of the debate are the ethical and moral ramifications of using the technology. I typically tend to favor the belief that technology is inherently neutral, but its applications can be good, evil or somewhere in between.

I don’t personally mind using large-language models to analyze my writing (going all the way back to IBM’s Watson), but I wouldn’t use it to generate stories or images based on my works. As someone who worked to report and present factual information, I’m also concerned about AI uses that generate falsehoods or harass people.

Also, companies’ push to encourage AI adoption en route to possible profitability appears to be leading to some questionable uses at considerable economic, environmental and social costs.

I think it’s important to highlight these dubious developments, which is why I started sharing “Adventures in AI” with links to stories on my former employer’s Slack workspace.

At the end of the day, being able to virtually don clothes in Google Photos is a mildly interesting use of generative AI. I don’t know what the economic and environmental costs of the technology are, but I don’t see it having a steep social cost given that feature is limited to the specific user.

I don’t necessarily think it’s something I would use a lot although it might be interesting to try on outfits that I don’t already own.

When it comes to dressing to the nines, I would give the Google Photos wardrobe feature a two out of four.

(Featured image: A screenshot from the Google Photos app on a Chromebook shows the option for the user to see their virtual wardrobe.)