OK, so you’ve got this image, and you want to find something similar or related; and you want to find it with needle-like accuracy out there in the vast and windblown haystacks of the web.

Search engines, of course, come to mind. Those tireless workhorses of web direction (and misdirection) are always happy to give out their directions (and maybe a few ads) to help in your righteous quest, oh Grail Seeker.

But suppose you don’t have that magic word, the name of the painting or artist, What then?

Enter similarity based image search, the ability to search for other images based on the characteristics of a known image. There are several approaches to this, and several kinds of search engines to use.

In investigating them, I started out with an image that was common, but not too common, Titian’s portrait painting known as Man with a Blue Sleeve (The National Gallery, London, also called A Man with a Quilted Sleeve, which may or may not be a self-portrait, I think it is).

I got an interesting variety of results from several visual search engines with similarity based search features, though none of them returned the image I was hoping for (at least in the first round), Rembrandt’s similarly posed self-portrait (see my post on Web Gallery of Art).

The first kind of search engine can be particularly helpful in this kind of search, in that they allow you to actually upload an image you already have and search for similar images; completely image based, no search words necessary.

TinEye (images above, top) allows you to browse your desktop, upload an image, or enter the URL of an image file on the web, and search for similar images. It does a good job of finding other versions of the same image, which can be very helpful in identifying an image for which you don’t have a name or artist. It wasn’t that helpful for finding related images by the same or other artists, but you could start there and carry on with a more generalized search engine.

BYO Image Search Lab (above, 2nd down) from Idée Labs also lets you upload an image or enter a web URL, but the returns are quite different. I was actually very impressed with the engine’s returns of images with similar composition, color and value, and it returned fascinating images; but it was not particularly helpful in finding other images by Titian or similar images by other artists. It’s fun, though. Idée Labs also has searches based on a selection of colors, or combined tags.

The other engines require that you first find an image through traditional word-based search and then show you similar images to the ones found.

Incognia (above, third down) did a decent job of finding some other paintings by Titian, but was not extensive in its returns and doesn’t identify the source of the image on the thumbnail.

Pixolu (above, 4th down), which uses a Flash based interface, found a wider range of Titian images. It allows you to control the size of the thumbnail display, and drag multiple images to a sidebar and click “Refine” to focus your search. The thumbnails are not directly linked to their source, however. You have to click on them and then look to the right under the thumbnail size control, for the image URL.

Of course, the big boys want to play at this game too.

Venerable Google, which essentially owns web searching (something like 80-90% of all searches now) has a solid similar images labs feature (above, 5th down); probably the most useful and extensive of the lot (they’re #1 for a reason). Google found a range of Titian portraits, as well as similarly composed portraits by other old master artists, and displays the link to the original under the thumbnail. Subsequent pages of results started to stray more widely off topic, but retained a mix of relevant results for several pages. (Still didn’t turn up my Rembrandt, though. Perhaps that was too much to ask.)

Bing, Microsoft’s rebranded search engine, appears to have an automatic similar image feature. When searching for images, clicking on any one image brings up its source in a browser frame, and displays related images in another frame to the left. I don’t like the frame based interface, or the fact that it won’t let me Command-click (on the Mac) to open multiple images in additional browser tabs.

Bing did an OK job of finding some similar images (above, second from bottom); but as I scrolled down in the related images frame (above, bottom) some of the “similar images” just boggled my mind.

Correction: OK, I owe Microsoft an apology (sort of). Apparently the images to the left are not related by similarity, but are results from the original search term. It is evidently finding “Titan” in addition to “Titian”, thinking I’m too lame to have actually meant “Titian” (thanks for nothing, Clippy), which explains the colorful figure of the Boy Wonder and the football stuff. Still wonky, though; and now not related to the topic of this post. Sorry.

Addendum: I want to give you the benefit of one of the key things I’ve learned about searching in my 14 years on the web; and particularly over the last few years of researching topics for Lines and Colors: and that is that search engines are not where you begin and end your search.

Search engines are a great place to begin a search, particularly if you learn to use their more powerful advanced features, but search engines can only take you so far. Much of the web is the so called “deep web”, billions of pages beyond the reach of the major search engines because they are behind logins or database driven sites that don’t get indexed easily. (The last estimate I heard is that Google, by far the largest and most powerful of the major search engines, reaches less than 20% of the total pages on the web. Granted, most of this is corporate documentation, but it’s still a striking figure.)

Use search engines to start your search, but then focus in on the local search available within databased sites that may have the information (or image) that you’re seeking. For example, if go to the large online art sites like Art Renewal Center, Web Gallery of Art, CGFA, The Athenaeum, or Artcyclopedia and use their local search features, and you’ll often have greater access to their resources. The same applies to the online collection databases of museums.