Eye Candy for Today: Illustration by François Schuiten

Le panorama de Calvani, from Le Musee des Ombres by Francois Schuiten

Le panorama de Calvani, from Le Musee des Ombres by Francois Schuiten (details)

Le panorama de Calvani, François Schuiten, illustration for Le Musée des Ombres by Schuiten & Peters

A wonderful illustration by Belgian comics artist François Schuiten who is known for his striking drawings of imaginative architecture. From this page on the Alta Plana website.

 
FacebooktwitterredditpinterestlinkedinFacebooktwitterredditpinterestlinkedin

A few thoughts on “AI Art”

 Images generated by Stable Diffusion 1.5

I will begin as a curmudgeon and state that I think the term “artificial intelligence” is overly generous when applied to the current level of this technology, and would be better reserved for future systems that are more likely to fulfill the expectations created by the term “intelligence”.

In an effort to neatly sidestep the semantic/philosophical black hole that is the definition of “intelligence”, I will say that here I’m referring to the common perception most people have of the term — essentially, generalized human intelligence — the ability to think and reason as well as learn. In most cases, this carries an implied sense of self awareness, even when we assign the characteristic to other animals.

These “AI” systems — which I prefer to refer to simply as “machine learning systems” or “neural networks” — are indeed impressive in certain ways, but if they are intelligent, they are savants — extremely capable a few highly specialized areas.

However, I do wonder if they are adept in ways we haven’t likely considered, and I will pose the question: are neural networks capable of being creative?

How do machine learning text to image generators work?

There are three major text to image generators: Stable Diffusion, DALL-E, and Midjourney, as well a a number of less well known systems.

These systems are trained by being fed images — lots of images — from various sources. At present there is little restraint on sources. The images are tagged, or associated with text in some way, and accumulated into enormous datasets. The system can quickly access its dataset and identify various kinds of images by their associated words.

A user will log into a site from which one or more of the systems can be accessed, and give the system a text “prompt” from which to generate images. Typically, this is a set of words suggesting a kind of image, a subject and a style of rendering. The system will fly through its database of text-tagged images, compare them, and try to produce new images with a similar appearance based on the text cues.

Perhaps not as easy at it looks

In my attempt to learn more about text to image generators, and to understand how they work, I have been experimenting with the machine learning model known as Stable Diffusion 1.5.

I’m a reasonably accomplished artist, and in my role as a website designer and developer, quite comfortable with computers, to the point of writing certain kinds of code. However, my efforts in creating prompts for the system have produced rather mediocre results compared to some of the machine generated image examples I’ve seen. To me this indicates there is a level of effort and skill in crafting prompts that produce the best results from these systems.

The text prompt I used for the image above, left was: “beautiful young woman with straight red hair and bangs in front of elaborate art nouveau style decoration rendered in the style of alphonse mucha“.

For the image on the right it was: “beautiful young woman with straight, collar length bright red hair, bangs, green eyes, in the style of alphonse mucha“.

Though both images have a vague look of Art Nouveau, neither looks to me like the style of Mucha, nor do they quite fulfill the design feel of his posters that I was trying to achieve.

I’m hampered by the fact that I’m only writing prompts at the most basic level, and I have not learned the processes of iteration and other techniques more skilled users employ.

Eventually, I found that clicking on the original prompt text in the detail page for the image in my PlaygroundAI account profile would access a large number existing images posted by other users, presumably with prompts the system deemed related to mine.

Images generated by Stable Diffusion 1.5

Some of these were visually appealing, and obviously created by users with a greater degree of experience; others were aberrations that looked like an illustration for a science fiction story about a horrible teleportation accident. In each case, in these images the users had tagged as publicly viewable, the text prompt is there to read and learn from.

Images generated by Stable Diffusion 1.5

Are machine generated images that imitate the recognizable style of a contemporary artist theft? … or not?

When machine generated images seem to carry the style of a living artist, the “new images” part of that process is the at heart of a conundrum: if the generated images are not copies of existing copyrighted images, but are rendered in the recognizable style of an living artist (or a deceased artist whose works are still protected under copyright law), does this constitute theft?

Many of us will be quick to sail off to the conclusion that copying an artist’s style is theft, but on further reflection, will quickly run aground on the shoal of existing U.S. and international copyright law, which states that only existing works can be protected by copyright.

You cannot copyright a style.

However unethical it may seem, it is not against current law to copy a style, as long as the copyist is not misrepresenting the works as authentic works by the original artist.

What is more likely in question is the legality of the training methods of the machine learning models in “scraping” images from the web and other sources. So far they seem to be operating within generally accepted practices, as the “fair use” part of copyright law is, of necessity, vague.

Change the law?

The cry to “Change copyright law!” quickly runs into its own barriers. When given thought, (not a popular practice, I know), it becomes obvious this is not only a bog-like whirlpool of conflicting and amorphous concepts, it may well be an impossible task.

How would you go about defining copyright infringement of an artist’s style? In some of the most blatant cases, it seems obvious, but it’s in the dark, shifting fringes of this concept that the details, and the difficulties, lie.

As an artist, my own style is an accumulation of the influences I’ve encountered through my life — other artists whose work I have admired and, in many cases, studied.

If I admire the style of an artist whose work is under copyright — let’s say Alphonse Mucha — and I study his style and attempt to bring elements of it into my own work, at what point could I be accused of copyright infringement?

Can you see what a muddy slope this is already? How is this different from the history of art, in which artists have always learned from those who came before them?

Was Rembrandt guilty of theft in adopting the pose of a painting he admired by Titian?

Titian and Rembrandt
(Image above, left: Man with a quilted sleeve, Titian 1510, right: Self portrait at the age of 34, Rembrandt 1640; note: these are images of the real paintings, not machine learning imitations)

Learning from those who came before us is how human endeavor, whether artistic, scientific, literary or otherwise, has always progressed. As has often been said: “We stand on the shoulders of giants.”

So in what fundamental and legally definable way is a machine learning system creating new images based on the accumulated observation of existing images different from humans observing, and learning from art they have been inspired by?

In what way is this aspect of machine learning different from what we identify in humans as creativity, which has always consisted of combining existing material in new ways?

These seemingly simple, but challenging, questions are worthy of consideration.

Capitalism rears its greedy, leering grin

I have not yet mentioned the inexorable forces of commerce and the fact that a number of powerful and influential companies have a stake in making the commercial versions of these systems as powerful as possible.


(Image above: Stable Diffusion 1.5, text prompt: “fierce, threatening monster robot”)

Even more to the point is the “cost cutting” pressure on companies to use these systems in lieu of hiring artists and graphic designers who must be paid for their work.

On the hopeful side, I’m reminded of the “desktop publishing revolution” of the 1980s and 1990s, during which companies decided that a computer with lots of fonts and Microsoft Word meant that Kevin in Accounting could take over the design and publishing chores for the company, and that hiring a graphic designer was no longer necessary.

Endless centered-text multi-font Word documents later, the companies realized this was indeed an error of judgement.

How different the current situation may be is as yet unclear, but at this time, companies would have to pay someone who is skilled at manipulating one of these systems to produce acceptable results, so as yet, this doesn’t seem to be a Kevin in Accounting push-button threat to graphic designers.

However, machine learning systems are disrupting more areas of human endeavor than the arts; word-based systems like ChatGPT and Open AI Playground (not to be confused with Playground AI.com) are being used to write advertising copy, blog posts, term papers and computer code, and will be sought after to take over a variety of other jobs.

You may have noticed the prevalence of fake humans keeping you from talking to a real person when you try to get “customer service” on the phone, the “convenient self service checkouts” that encourage you to do a checkout clerk’s job for free, as well as taking orders from a machine, and robot voices in other aspects of modern life. All of these will become more sophisticated as machine learning makes its presence felt.

Companies love the fantasy of having to pay no employee salaries or benefits, instead having machines fulfill their roles in selling goods and services to consumers (who, one assumes, will be paid salaries by other firms that are less savvy).

What’s an artist to do?

For those artists concerned with protecting their own style of art from being adopted by these systems, what options are available?

If we focus on the imitations of existing copyright law, we find that in the U.S. copyright generally covers works for 95 years after the publication date.

If you tried to define an artist’s style for the purpose of copyright law, not only would defining a style be a daunting challenge, but how would you enforce such a regulation?

Many artists are urging that you contact your legislative representatives and demand that they do “something”.

The idea of involving legislators in this process just makes my blood run cold. Never have I seen a group more monumentally and almost universally ignorant and misguided in issues of technology than legislators — not that this has ever stopped them from sticking their fingers in the pie.

Could legal restrictions be placed on the types of content allowable for use by these systems in the training stage? Perhaps, but this is in itself a thorny, muddy issue, which may contain unintended consequences in the form of limitations on what we can access as humans. Can we really regulate machine access to images differently than what is available to humans?

The Concept Art Association it trying to rally the troops with a crowdfunding campaign and a list of suggested actions.

However, I think those suggesting that image collection for text to image generation be limited to opt-in, and otherwise restricted to public domain content, may find this is a more complex legal issue than it may seem at first, and are again casting wide nets that may well catch humans in unpredicted ways.


(Image above: Stable diffusion 1.5, text prompt: “fierce, threatening monster robot holding an artist’s palette and paintbrush”; image-to-image prompt: Self portrait by Élisabeth Louise Vigée Le Brun)

Leaving something like this in the hands of politicians would be at best ineffectual, and at worst disastrous. If there are solutions to be found, they must come from individuals who are intimately familiar with the complexities of the issues, the structure and use of these systems and the likely trajectory of their technological progression as well as the legal framework of copyright law.

At present there is some evidence that public opinion can have an effect on the creators of these systems. Already, Stability AI, the company behind the Stable Diffusion text to image generator, is offering an opt out to have your artwork excluded from the flood of images being fed into their training system for the next version of the software. This does, however, require that artists be proactive in opting out and requires an awareness of this option in the first place. Also, Stable Diffusion is only one of several systems in operation.

It’s worth noting that the creators of some of these systems are attempting to restrict the use of specific artists’ names in prompting the style of rendering.

There are also efforts being made to allow for digitally tagging images in a way that can be used to identify and exclude images from assimilation by the Borg, er,… I mean neural network training routines.

Meanwhile, putting the “NO AI” symbol up on social media accounts seems pretty weak sauce, though it may help raise a bit of awareness of the issue. (I can certainly understand the attempt to bring it to the attention of the owners of ArtStation.)

I will suggest, however, that artists would do well to raise their own level of awareness and become more informed abut the underlying technology and related copyright issues.

Being informed

I think that artists to whom this issue is important will benefit from taking a little time to log into one of these systems and spend a few minutes learning to write prompts, in order to understand what they do and how they are being used. It’s also worth noting how they can be individually further “trained” by uploading images from which the system can be prompted to create new variations.

If you can avoid a knee-jerk “I’m not having anything to do with this!” reaction, you can easily investigate image to text generation for yourself by going to PlaygroundAI.com, and creating an account, which requires only an email address. There, you will be able to use Stable Diffusion or DALL-E for free.

There is a 15 minute YouTube video here that will walk you through the process of creating prompts for these systems, as well as giving you a quick overview of their capabilities.

I’m not suggesting that you start using text to image generators going forward — or that a few minutes spent using one of these systems is likely to change your opinion — but I believe the experience will give you a better informed opinion.

It may also prompt you (if you’ll excuse the expression) to think about how you tag and classify your images when making them publicly available.

I will also suggest that artists will do well to become more informed about copyright, how it works, what its limitations are and what is meant by the public domain and fair use.

Can these systems be used ethically?

In my attempt to understand how these systems are trained to adopt a contemporary artist’s style, I tried to teach Stable Diffusion to imitate my own comics style by feeding it an image from my webcomic, Argon Zark! (image above, left) and playing with various text prompts. The results, though occasionally amusing, were far from successful.

That and my weak attempts to prompt the system to imitate the look of Alphonse Mucha convinced me that the image generator users who are successfully imitating a contemporary artist’s style are doing so not only deliberately, but with considerable effort and practice. If they are doing so to make money, this seems to me the focal point of unethical practice in this arena.

The loud voices in opposition to text to image generation in any form appear to assume that the only use of these systems is to appropriate without credit the hard work of living artists, ignoring the fact that there is a great deal of art, other images and writing that belongs in the public domain and is therefore fair game any way you look at it. If I ask a neural network to create an image in the style of Rembrandt, no one has reason to complain.

Image in the style of Rembrandt generated by Stable Diffusion

(Image above: Stable Diffusion 1.5, text prompt: “landscape etching in the style of Rembrandt”)

Where am I coming from, and where do we go from here?

For those of you who might assume from my reluctance to jump on the “text to image generation is the spawn of hell” bandwagon that I am a disinterested observer, I’ll point out that I’m a painter, illustrator, comics artist, and part-time art teacher, and the creator of intellectual property that I consider valuable.

Also, in my role as a graphic designer, I stand to lose business if these systems make website creation a job for neural networks rather than human designers.

I’m not without a stake in this discussion.

That being said, we have to acknowledge that this technology is here. It’s not going away, and it’s likely to rapidly become more sophisticated and effective in the near future.

We can rage against the machine, shake our fists at the sky and cry foul — and hide in our bunkers as Skynet becomes active — or we can turn around, examine the technology and its uses, and attempt to understand and adapt — and perhaps influence the outcome of some of these conflicts, or even find uses for some aspects of this technology in our own creative endeavors.

There may be no easy answers, but we can at least try to understand the questions.

 
FacebooktwitterredditpinterestlinkedinFacebooktwitterredditpinterestlinkedin

Alberto Varanda

Alberto Varanda, comics and illustration
Alberto Varanda, comics and illustration

Alberto Varanda is a French comics artist who has worked on a number of projects for different French and Belgian publishers. You can find English language versions of his Little Pierrot comics album on Bookshop.org and Amazon, and French editions of other books on Amazon.

His style can range from cartoon like children’s book illustration to various levels of comics illustration to intricate pen and ink renderings. I particularly enjoy the latter, as well as his looser figure sketches.

His website is available in both French and English versions, though some of the pages are only available in French.

 
FacebooktwitterredditpinterestlinkedinFacebooktwitterredditpinterestlinkedin

Bing image search vs. Google, Yahoo & Tin Eye

Bing image search interface

As you might imagine, in the course of writing Lines and Colors I do a fair bit of searching out art images on the web — whenever possible searching for the largest examples of images of artwork that I can find.

One of the ways I do this is to use the “image search” features of the major search engines. Unfortunately, Google Image Search, which used to be the standard, has been diminished in its usefulness, as Google, perhaps nervous about copyright issues, has gotten namby pamby about searching for large images and taken away the ability to search for images in extra large or custom, viewer chosen sizes.

I have of late switched the majority of my art image searching to another search engine.

Bing Image Search

OK, I hear you snickering (Bing, really? BING?). Yes, Bing, Microsoft’s seldom used (but actually decent) competitor to Google’s overwhelming dominance of the search arena.

While Google hamstrings its image search, Bing offers a full featured image search, that not only allows you to search for extra large images, but offers a few features Google’s version never did.

Unlike Google’s typically spare opening page, Bing Image Search is crowded with suggested images of pop stars, cute animals and a bunch of other pop culture garbage you’re sure to be fascinated by. (I think if you’re logged into a Microsoft account, it may remember your own recent searches.) The simple search field is at the top.

In the initial search term result (images above, with detail crop; I’ve searched for “Dutch landscape painting”), click on “Filter” to the right, and in the sub-navigation that drops down, click on “Image Size” at the left. You’ll have a choice for Small, Medium and Large, as with Google, but in addition you can choose “Extra Large”, or enter custom size parameters in the provided fields. I often search for 2000 x 2000 pixels. (The little icon in the right side of the search bar is for “search by image”.)

The filtered page will show large images with the size displayed over them. If you click on the hamburger menu at the upper right, you’ll have the option to display information from the page under the images.

Clicking on an image gives a close up. In the column to the right, the first two entries are ads, the third is the link to the originating age for the image, Under that are buttons for “Visit site”, “Pages” and Image sizes”, and below that similar images (related, but not the images in question) and related searches.

Clicking “Pages” produces a list of pages that display version of that image.

Clicking “Image sizes” organizes the image sources by the size of the image (largest is not always best as some may be watermarked or less accessible than others, you can also have multiple choices for the same size image).

The little icon in the search field at the top of the page (that I assume is supposed to be a camera) opens a Visual Search box. It offers you the option to upload an image, or enter a link to one, and search for other, hopefully larger, versions of that same image. It also allows you to search for a page with additional information about an image you’re trying to identify.

Google Image Search

Google image search interface

The Google Image Search initial returns on a search is similar to Bing’s. A link for “Tools” on the right drops down a sub-navigation from which you can choose “Image Size” on the left, with choices only for Small, Medium and Large as well as “All”.

The filtered returns show page location under them, image size is not available.

Google image search interface

Clicking on an image shows a preview in a right hand column, with the page name and link below it. In this case, the size is available by rollover. Under that are “Related images” (similar but not copies of the same image), and “Related Searches”.

Google image search interface

The Camera icon in the search bar is for visual image search. Upload or paste the URL of an image (the image itself, not a page containing an image), and returns an array of copies of the image with the source page underneath.

If you right click (or Control-click on Mac) on an image in Google Chrome, you will see a choice to “Search Googe for image”. There are plugins that provide the same functionality for other browsers.

Yahoo Image Search

Yahoo Image Search interface

Yahoo Image Search exists. Why, I’m not sure.

The initial search term results look much like Bing or Google, but there is no page or size information. Clicking on “Advanced” at right provides filters for color, size and image type. Sizes are S,M,L. (The others let you search by color as well, from the sub-menus.)

Clicking on an image returns a detail panel with the page and size info and the option to “Visit page” or View image”. There is no visual image search that I can find.

Tin Eye reverse Image Search

Tin Eye reverse Image Search interface

Tin Eye is a venerable visual image search engine that provided that service before the big guys, um… borrowed the idea. I mention it primarily out of respect for that. It still does a good job in its initial mission, but there is no provision for image size choices. Tin Eye offers plugins to put their reverse image search in browser menus. Tin Eye offers a service to track your own images and notify you if it finds they’re being used elsewhere, but the service is expensive, probably mostly of use to corporate intellectual property holders.

Tip for searching by site

All three of the above major search engines allow you to search for images (or other content) from a particular site. In the regular search bar, enter the search terms, followed by a space and then the word site, a colon (no space) and the URL of the site. For example: “dutch landscape paintings site:sothebys.com”. This will return a page with results for that topic only from the Sothebys auction site.

Other sources for high res art images

General search engines are just one avenue for searching out art images on the web. Another, often more fruitful way to find large art images is to do local searches on the sites of major museums, or on art image agglomeration sites, such as the Google Art Project, Wikimedia Commons of the Art Renewal Center. These should be the topic of another post.

Happy image searching!

(Oh yes, and Time Sink Warning!)

 
FacebooktwitterredditpinterestlinkedinFacebooktwitterredditpinterestlinkedin

Adrian Tomine

Adrian Tomine, New Yorker cover, Dec 7, 2020
Adrian Tomine, New Yorker cover, Dec 7, 2020

Originally from California, Adrian Tomine is an illustrator and cartoonist living and working in Brooklyn, NY. Tomine has taken to his adopted city so well that he has become a reader favorite contributor to the New Yorker.

His New Yorker covers, as well as many of his other illustrations and drawings, have that wonderful combination of evocative artwork and wry observation that exemplify the best of the magazine’s cover art. His artwork uses a streamlined line and color fill approach, reminiscent of the European ligne claire style of comics art.

As a case in point, his cover for the new December 7, 2020 issue of the New Yorker (images above, top) pretty well catches the whimsical side of the 2020 zeitgeist.

The New Yorker has a wonderful new online feature called Cover Story in which they give you background on the creation of the current issue’s cover; here is the one for Tomine’s December 7, 2020 cover.

Tomine is the author/illustrator of a number of books of drawings and comics, many of which are published by Drawn & Quarterly, and the latest of which is The Loneliness of the Long-Distance Cartoonist (Bookshop.org link).

There is a video overview of some of his titles by “panellogy 080” on YouTube.

Tomine’s website contains examples of his illustrations and information about his books and comics, as well as offering prints and original art for sale.

Unfortunately, his online gallery is of the wearisome “pop up and close, pop up and close” variety, which discourages casual browsing, and the images offered are small. You might find it helpful to augment your visit to his website with this Google image search I’ve set up for Tomine’s work on newyorker.com.

 
FacebooktwitterredditpinterestlinkedinFacebooktwitterredditpinterestlinkedin

R. Kikuo Johnson

R. Kikuo Johnson

R. Kikuo Johnson

Originally from Hawaii, R. Kikuo Johnson is an illustrator and comics artist based in Brooklyn, New York.

His illustration clients include Apple, Random House, Penguin Books, Marvel Comics, The Atlantic, The Guardian, The Wall Street Journal and the New York times, among others.

Johnson maybe best known for his wonderful covers for The New Yorker. The image above, top is an example. I don’t know about you, but it took me a second glance to catch the point of the illustration; I love that.

He studied at the Rhode Island School of Design, and has returned to teach there.

Johnson’s work ranges from dramatic to subtle, and often has something of a feeling of the ligne claire style of European comics — little variation in line width, flat color, but within those constraints producing a naturalistic feeling of time and place.

 
FacebooktwitterredditpinterestlinkedinFacebooktwitterredditpinterestlinkedin