One of the neat features of Google is its ability to search for images. I took this photo of Barack Obama, taken circa 1991 and asked Google to find it. Reader, please note that all of what follows are the actual Google image search results. This is not a spoof article. Truth is stranger than fiction.
And here’s what Google said:
Like father, like son so they say. Among the full results for the image are the actual original photo of President Obama that matches, and many of the Fake Columbia ID card with this picture, mentioned in the previous article. Now for the shocking part (and remember that I was searching with an image, not a name). The similar image display returned:
These photos are of Obama’s mother Ann Dunham, his father Barack Obama Sr., then the weird ones: poet Frank Marshall Davis and Ralph Waldo Emerson Dunham Sr., the President’s great grandfather.
High school photos? Let’s try:
And the similar results are:
So we have Obama photos and similar ones of Ann Dunham, and another high school yearbook photo of a girl named Hillary Clinton. OK, my head is like totally spinning at this point.
So where does this go? Here’s a nice young photo of the President.
And the results are:
Frank Davis is back, along with the unidentified girl, the caption beneath which is “Our dream is to live like Lenin.”
Are we having fun? Let’s spin the wheel one more time, buy a vowel, and see if you can guess this one, from an Obama toddler photo:
Lots of photos of Obama, his mother, Mitt Romney and Brigham Young.
Update:
Obviously those images weren’t selected just because they were “visually similar.” There’s got to be some textual context being taken into account. To add to Frank Marshall Davis, Hillary Clinton, Mitt Romney and Brigham Young, we get for:
George Clooney, Mayor Harold Washington, James Baldwin and Fidel Castro. If you repeat this Google search, avoid the 4th image which links to a suspected malware site.
I have used Google image search a fair amount, and based on what I’ve seen I’m pretty sure the “visually similar” images are not just visually similar — it seems clear that the matching also involves a similarity between text found in the vicinity of the pictures. If so, your results are not so surprising.
Have to agree with Dave. Clearly the Google image algorithm is recognizing the photo as a picture of Obama, and matching results accordingly.
My understanding is that images are stored by metadata and associated with URLs. But that’s a generic ‘image search’ approach. There my be more to Google particular implementation, and hopefully there is. This is a basic approach to producing satisfying results … but can also lead to seemingly random associations.
I’ve always thought the claim of “visually similar” is leading and vague. If the engine were truly analyzing image content, I’d expect similar results by color palette and composition.
Literally visually similar, you’d expect just that. Put in a grade school photo of a young african american kid, get other photos of young african-american kids. This kind of matching isn’t impossible, but over the entire web? Besides, this generates a different class of results. Results by name/context versus by genre.
You’re right. Even Google doesn’t have enough computing power to compare pictures. It’s clear that the matching comes from text associated with the picture.
I do a lot of searches on IMDB (the movie database) and you see it there when you search for a name and it comes back with any pictures in which the person’s name is noted in the attached text.
I know that when using Picassa to tag photos, an unknown photo is often tagged correctly when one is an adult and the other a child and it will tag relatives too. I should point out that the photo I initially searched was cropped by me and so doesn’t exactly match any photo on the Internet.
Maybe this page is a hint: It has one of the Obama photos found, the Romney Photo and the Clinton Photo:
http://pinterest.com/classmates/classmates-blog/
Still, how did Brigham Young get in there? And why Obama’s great grandfather and not his father?
My initial research on how Google Image search works, indicates that it does not use facial recognition, and I’ve found nothing to suggest that it uses text (unless you add some).
Any indication of how Google deals with the text that is contained in the related story or picture cut line?
The “unidentified girl” is, as the subtitle hints, from a Soviet propaganda film. It’s a bunch of school children literally singing the praises of Lenin. Best line: “We Are His Little Potatoes.” Maybe it has more gravitas in Russian.
It appears to have been tied to Obama as part of the frothing response to the idea of him giving a televised speech to the nation’s schools back in September 2009.
What’s interesting to me is that even the pictures that are clearly not of Obama are still monochrome headshots in approximately the same pose as the target picture. So it’s not just “pictures contextually associated with Obama,” it’s “pictures contextually associated with Obama and at least somewhat resembling the original.”
Which raises the question: how did it build the context? There’s an iterative process taking place here.
My friends at the “Fellowship of the Minds” blog are really good at “building context.” I’d love to see their take on this. If I gave Dr. Eowyn a link to Dr. C’s article, I’m sure this “proof” that Frank Marshall Davis is Obama’s father would appear on her blog, tout de suite! No doubt, she would also make mention of the “evidence” here that Obama was a creation of the Soviets. The amazing power of confirmation bias…
Note that this article is tagged: “articles I probably shouldn’t have written,.”
They talk of their computer vision approach as using features to produce queries, but I am finding it slightly tricky to nail down whether or not their results include other factors, even in the “visually similar” results. That would seem to be the case, that they include other factors.
For instance, check out these results from an image of Orly Taitz. I just searched on her name first, then selected an image of her, the top left image in the results (or very like it). Then I told Google to find visually similar images.
Taitz
Note the male doctor wearing a mask, note the couple images of Barack Obama. Note the image of a model. Notice the links and meta-data associated with these.
The top results start out as one might expect with some decent returns of other Taitz photos with similar features. But as you scan down the page, the results seem to get worse (visually).
Even if they start with features data, it looks like their query adds in results selected from their index that are not strictly *visually* similar.
I don’t think it uses text either; I think they tie everything together by the metadata in the image file, use that as an ID (the image itself is incidental), and index URLs the file is found at. Then returns search results based on number of matching URLs. Storing entire pages could be overkill, lead to to many possibilities. I mean, how many different kinds of stories has a particular image of Obama appeared with … in some case, the associations would be all over the map.
Well, worse they they already are LOL
I’d appreciate any help fellow anti-birthers would like to give over on:
http://www.westernjournalism.com/barack-obamas-impeachable-offense/#comment-180333
(Particularly on Harrison J. Bounel)
LOL, no worries, Doc. When I said I’d love to see their reaction, I was being sarcastic. I’m sure it would really make me feel ill. So Doc E will just have to find this on her own!
If birthers get tears in their eyes over finding a bogus 2010 skiptrace record, they’re going to be like incontinent chihuahuas over this. Guaranteed.
One buffoon over there is citing Weekly World News as the source that the House is about to being impeachment proceedings against Obama.
Given the link I just posted using Orly Taitz’s image as an example, what does that mean for Orly Taitz’s relationship to Barack Obama? Are they brother and sister? Twins separated at birth? Created in the same Soviet lab? How else to explain that Obama shows up as “visually similar” to her?
Ellen,
I admire your tenacity and I’ve tried as you have on many birther forums to get through to these nutjobs. The biggest problem is that they don’t want to believe anything other than what they have read on the birther and RWNJ sites. They NEED to believe that President Obama is evil. They have to blame the condition our country is in currently on President Obama only, otherwise, they might have to open their eyes and wonder just what led to the debacle Obama found in ’08 and why haven’t the RWers in Congress done anything to try to change things in a positive manner for our country.
I tried to speak to a few people lately about Indonesian Law & U.S. law and how it prohibited Barack Obama from becoming an Indonesian citizen, not that I believed it was ever attempted, but I thought if they saw it as a legal impossibility, they’d realize it can’t be true that he isn’t an American citizen. Orly told her followers that the only reason Barack Obama was removed from his mother’s passport was due to his becoming an Indonesian citizen so he could no longer be on her American passport. They didn’t want to believe the real reason and that was that he was likely traveling alone and needed his own passport. I don’t know what the cut-off is for ages on a family passport or children traveling under a parent’s passport but returning to the U.S. at the age of ten should have required his own passport. My children had their own as young as four years old.
These are the same people who believe that S.Ann Obama traveled to Kenya on her own, several months pregnant, despite being unable to receive necessary immunizations to enter Kenya due to being pregnant, let alone the simple fact that she didn’t apply for her first passport until she married Soetoro and planned to move to Indonesia with Barack. It makes more sense to them that a young woman would travel alone to a country she’d never been to before while several months pregnant, knowing she wasn’t welcomed by her husband’s family. Never mind the fact that her husband was still in the U.S.! They are crazy with hate towards President Obama and will believe anything no matter how outrageous it may be, as long as it is sufficiently defamatory and negative. There is just not getting through to them. However, they are so small in number that Orly has to lie about the number of signatures she has on her petition and refuses to acknowledge that many of them are duplicates or blatantly false names. They are far less than 0.001% of our population and although I argue with them and try to show them where they are wrong, it’s not worth getting upset over anymore. I’ll probably be back at it tomorrow, just like you, but tonight I’m simply laughing at their stupidity and that of their so-called leaders. When Orly Taitz is a leader, you know just how pitiful her followers must be.
Talking about pictures, I just found a picture of Orly on her way to court with her documents, enjoy!
http://static.onemansblog.com/wp-content/uploads/2007/09/funny_bike.jpg
Here’s an example of why I think text is involved. I start with a picture of Nixon and do an image search on it. On the resulting page, I click on “Visually similar images” and page down. Eventually I come to some pictures that are not Nixon. I click on one, and I come to a page that has no pictures of Nixon, but does have text that refers to him (turns out the picture I clicked on is a former director of the Nixon Library).
So the picture, which to my eye has no visual similarity to the picture I started from, does appear on a page that has the text “Richard Nixon.” That’s why I think text has something to do with it.
Google does use abstracted features to match images based on an index. It’s part of computer vision techniques. They write about it, explain it (somewhat). See:
http://insidesearch.blogspot.com/2011/07/teaching-computers-to-see-image.htmlaaaaa
In the Taitz results I linked to above, they definitely *start* using features to find, for example, where she tilts her head in exactly the same manner, but is wearing white instead of blue.
But as the results continue, the visual matching drops off, gets worse, more off-beat. This fall off may very well be a result of Google’s “secret sauce,” whereby their algorithm adds in other textually-based links to increase results (even in a search labeled as visually similar). This seems to fit Googles’s standard operating procedure.
In the Taitz link, we go from an uncanny match of specific Taitz images to a generic Taitz match. Then we end up with a model, President Obama, and two other dissimilar males. These all have textual references to Taitz.
With the Taitz image this progression happens in short order. With your Nixon example, it gets there “eventually.” I produced a similar result using the same process with a standard presidential image of Barack Obama.
Indeed, if you just do just a standard normal image search using the name “Barack Obama,” the proportions of “filler” start very low (with a high return of expected images) but then the filler proportion starts to go up and up, until we get something like this:
http://i.imgur.com/IoxFi.jpg
This article has been updated.