Can Artistry Be Built Into a Machine?

One day lately, on a desk in Jean Oh’s lab within the Squirrel Hill neighborhood of Pittsburgh, a robotic arm was busy at a canvas. Slowly, as if the air had been viscous, it dipped a brush right into a pool of sunshine grey paint on a palette, swung round and stroked the canvas, leaving an inch-long mark amid a cluster of different brushstrokes. Then it pulled again and paused, as if to evaluate its work.

The strokes, principally completely different shades of grey, instructed one thing summary — an anthill, perhaps. Dr. Oh, the top of the roBot Intelligence Group at Carnegie Mellon University, wearing a sweatshirt bearing the phrases “There Are Artists Among Us,” seemed on with approval. Her doctoral pupil, Peter Schaldenbrand, stood alongside.

Dr. Oh’s work, which incorporates robotic imaginative and prescient and matters in autonomous aviation, typically touches on what is named the sim-to-real hole: how machines skilled in a simulated atmosphere can act in the actual world. In latest years, Mr. Schaldenbrand has led an effort to bridge the sim-to-real hole between subtle image-generation packages like Stable Diffusion and bodily artworks like drawings and work. This has primarily been manifest within the venture generally known as FRIDA, the most recent iteration of which was rhythmically whirring away in a nook of the lab. (FRIDA is an acronym for Framework and Robotics Initiative for Developing Arts, though the researchers selected the acronym, impressed by Frida Kahlo, earlier than deciding what it stood for.)

The means of transferring from language prompts to pixelated photographs to brushstrokes might be sophisticated, because the robotic should account for “the noise of the real world,” Dr. Oh stated. But she, Mr. Schaldenbrand and Jim McCann, a roboticist at Carnegie Mellon who additionally helped develop FRIDA, imagine that the analysis is value pursuing for 2 causes: It may enhance the interface between people and machines, and it may, by means of artwork, assist join individuals to at least one one other.