Wall Street Journal Today: On Virtual Orchestras

[funkychickendance]

This seemed too important to simply let go by, unremarked in KVR.

Fugue for Man & Machine
Classical musicians have bitterly opposed replacing human players with computers in the orchestra pit. Now, a small group is breaking ranks -- and arguing that it's the best hope for revitalizing the art. Cue the laptop.

By JACOB HALE RUSSELL and JOHN JURGENSEN
May 5, 2007; Page P1

Paul Henry Smith, a conductor who studied as a teen under Leonard Bernstein, hopes to pull off an ambitious performance next year: conducting three Beethoven symphonies back-to-back in a live concert. "Doing Beethoven's symphonies is how you prove your mettle," he says.


But Mr. Smith's proof comes with the help of a computerized baton. He will use it to lead an "orchestra" with no musicians -- the product of a computer program designed by a former Vienna Philharmonic cellist and comprised of over a million recorded notes played by top musicians.

Amid all the troubles facing the classical music world in recent years -- from declining attendance to budget cuts -- none has mobilized musicians more than the emergence of computers that can stand in for performers. Musicians have battled with mixed success to keep them out of orchestra pits in theaters, ballets and opera houses. Now, a new alliance of conductors, musicians and engineers is taking a counterintuitive stance: that embracing the science is actually the best hope for keeping the art form vital and relevant. They say recent technological advances mean the music now sounds good enough to be played outside the touring musicals and Cirque du Soleil shows it is typically associated with.

Among their arguments: Aspiring composers who couldn't otherwise afford to have their creations performed by an orchestra can now commission a high-quality computer-generated recording for a fraction of the price. For communities facing the loss of their orchestra, it could be a way to keep performances in town -- even if it means a computer stands in for half the players.


Experts Judge Virtual Music
We played four samples of Beethoven's Symphony No. 7 for two professors of music. Three were recordings of orchestra performances conducted by Roger Norrington, Fritz Reiner and David Zinman; the fourth was created on computer by Paul Henry Smith. We asked the professors to guess which was the computer recording. The results are below.
David Liptak, chairman of composition department, Eastman School of Music, Rochester, N.Y.
First guess: Incorrect. Picked the live performance conducted by Mr. Norrington as the computer-generated recording; Mr. Smith's was his second guess.
Verdict: Mr. Liptak zeroed in on the "false" tone of the clarinet in Mr. Smith's recording. When he heard a longer sample, he said that it was easy to catch evidence that it was synthetic.
Stephen Croes, dean of music technology, Berklee College of Music, Boston
First guess: Incorrect. Mr. Croes also picked Mr. Smith's as his second guess.
Verdict: "I'm not surprised at all that I could be faked out," he said. "The modern musician has tools that are so powerful, and they're looking for how to defeat all those giveaways."Critical to the push are new strides in computerized music. The latest software lets users pick from a massive library of digitally stored sounds, assemble them into a complete symphony and layer on texture and nuance. Picture a chef with an infinite variety of ingredients to choose from when creating a four-course meal.

Even some experts now find it hard to tell the difference. At the request of a Wall Street Journal reporter, David Liptak, chair of the composition department at the Eastman School of Music, listened to a 30-second passage of a Beethoven symphony created on a computer, as well as three versions recorded by live orchestras. On his first try at identifying the computerized version, Mr. Liptak guessed wrong. He says the difference became clear when he heard a longer clip (listen to the four sample passages2).

In 2003, computerized music sparked a big battle in New York's Broadway theaters. Musicians went on strike for four days, partly because producers had raised the idea of replacing some players with "virtual orchestra" computer programs. Musicians' unions have largely kept virtual orchestras out of Broadway orchestra pits, but on London's West End, they have been used in productions such as "The Sound of Music" and Cameron Mackintosh's revival of "Les Misérables." They have also been used in some U.S. touring musicals.

But there's a big difference between theatrical productions where the performers are mostly hidden from view in an orchestra pit, and symphony concerts, where concertgoers expect to see the musicians front and center. So far, the technology hasn't been used in traditional orchestra settings, although some advocates say it could be used to bring classical music to small towns without resident symphonies.

Computers are also being used in some more experimental classical performances. In Toronto this fall, an audience filled a concert hall to hear Bach's "Goldberg Variations," performed by Glenn Gould -- who died more than two decades ago. A company called Zenph Studios mounted the performance with a computer-controlled Yamaha grand piano that replicated the finger styling of the piano great.

Marin Alsop, incoming music director of the Baltimore Symphony Orchestra, says she supports the technology as a way for composers to get their music heard, but doesn't think it should replace working musicians. "It can be a great tool for moving up the ladder," she says. "But it's a slippery slope."

Headquarters for the high-tech music movement is the house in Hamden, Conn., where Mr. Smith, 43, lives and works. As a teenager, Mr. Smith spent summers studying under conductors like Bernstein at the Tanglewood Institute and Lukas Foss at the Eastern Music Festival in Greensboro, N.C.; he received his bachelor's and master's degrees in music from Oberlin and Brandeis. In 1995, he took a break from music to launch an Internet company, and later worked in Web development at Yale. In 2003, he bought his first edition of digital-orchestration software for about $5,000 and set out to pursue a new career.

Mr. Smith's studio is set up in a room that used to belong to his 3-year-old daughter (who's bunking for now with her 6-year-old brother). A silver keyboard is positioned in front of a computer monitor on his desk. To limit street noise and add a listening room for clients, Mr. Smith is planning to move the operation to a downstairs studio, which holds stacks of old music scores.

Mr. Smith has used computer programs to record themes for Nike and Adidas commercials, soundtracks for independent films and works by composers. He calls his business the Fauxharmonic Orchestra.

His main focus now is recreating Beethoven's symphonies on computer for a planned live performance, which he says he may conduct with a controller for the Nintendo Wii. He is making slightly unfinished recordings, so that on stage he will be able to interact with the music, modulating tempo and emotion. He says he has not yet secured a venue, but his goal is a fall 2008 performance, preferably in Asia where he says attitudes toward combining technology with high culture are more open.

At work on Beethoven's Symphony No. 7, Mr. Smith starts up a computerized rendition of the piece, which he's downloaded free online; it sounds roughly like a cellphone ringtone. Using a mouse, he selects a five-second snippet of a viola's part, which begins playing over and over in a loop. "This is what annoys everyone in the house," he says. With another click, he launches a digital library of recorded notes; for a single note there can be dozens of inflections to choose from. Mr. Smith selects "appassionata legato," or impassioned and smooth. The tinny tones are replaced with the sounds of a viola, which he tweaks further.

Five minutes have elapsed, and he has created five seconds of realistic-sounding viola music. Next, he'll need to repeat the process for other instruments, including the flute, oboe and bassoon. Beethoven's Seventh includes 18 musical parts, performed by as many as 70 players.

For Mr. Smith, this will be a work of art. But that's clearly not the view of the person who put an anonymous posting on Mr. Smith's Web site: "This man is evil. This project is evil. Die in hell." Mr. Smith responds that the advance of computerized music is inevitable, and that musicians are better off taking control of it than leaving it in the hands of producers and executives. "The genie's out of the bottle and you can't just outlaw it," he says.

He has founded a group called the Digital Orchestra League, aimed at bringing together leaders in virtual music. Two of the league's board members are controversial figures in the music world, Frederick Bianchi and David B. Smith, co-founders of Realtime Music Solutions. Their company is the maker of Sinfonia, a computer system that replaces some live musicians in the pits of Cirque du Soleil shows.

Teresa Marrin Nakra, another league member, is considered a pioneer in digital orchestra technology. She is best known for an invention dubbed the "conductor's jacket," which she began designing as a Ph.D. student at the Massachusetts Institute of Technology. The "jacket" -- a cyclist's jersey embedded with a dozen sensors -- maps conductors' movements and physiology. It measures everything from the flexing of a bicep to the electrical conductivity of the conductor's skin -- a gauge of excitement used in lie detectors. With the data, Ms. Nakra has created software that recognizes these movements and translates them to control a piece of music.

Keith Lockhart, conductor of the Boston Pops and the Utah Symphony, has donned the jacket in performances to help gather data. He says he's interested to see where the technology will lead, but has some reservations about its uses. "What makes an orchestra extraordinary is not just one person's creative take on a composition, but watching the interaction between musicians and the conductor and each other," he says. "That's subtle, but once you start removing it from the scene, that's when you start forgetting it was there."

Last year, Mr. Lockhart strapped on the jacket for a performance of the overture of Mozart's "Marriage of Figaro," but this time Ms. Nakra also placed sensors on five of the 75 musicians he was conducting, as well as 15 audience members. The goal was to see how precisely Mr. Lockhart's movements matched his players', and how the emotional peaks of the music affected them physically. Ms. Nakra is still crunching the data -- the first two minutes of the piece alone fill 16 feet of paper with spiky lines -- but already she can see points where heart rates raced during passages Mozart intended to be loud.

This is a difficult time for American orchestras. Their numbers have dropped slightly in recent years, to about 350; since 2002, nine orchestras have shut down, while about five have opened or resumed, according to the American Symphony Orchestra League.Meanwhile more graduates are filing out of music schools. Some, finding few openings at traditional orchestras, are turning to less traditional pursuits, from film scoring to marching-band music.

For a budding composer, the economics of a virtual orchestra are compelling. Matthew Fields, who has a doctorate in composition from the University of Michigan but now works as a computer programmer and writes music on the side, has spent $50,000 on a professionally produced recording by 18 musicians. Last year, he commissioned a recording from Mr. Smith's Fauxharmonic Orchestra for his complex six-minute work, "Fireheart," for about $800.

As a composer, getting players interested in his work is essential for building a reputation, Mr. Fields says. Without the recording, the piece "would simply be dismissed as unplayable and unworthy of playing," he says.

Music experts -- and even the biggest advocates of virtual-orchestra technology -- say a trained ear can typically tell computer-generated classical music from the real thing. Daniel Kellogg, a professor of music composition at the University of Colorado at Boulder, said he had no trouble recognizing a recording made by Mr. Smith as the work of a computer, in part because the woodwinds sounded the same every time. "There wasn't enough imperfection, " he said.

Mr. Smith says the technology still has limitations, but points to its dramatic improvement in recent years, and says he expects that it will rapidly become more realistic.

One of the people most responsible for recent strides in computerized music is Herb Tucmandl, formerly a cellist with the Vienna Philharmonic. He commands a small army of software engineers, recording technicians and freelance classical musicians, all working on the Vienna Symphonic Library, his digital archive of notes and musical patterns. A single note -- a violin playing a C-sharp, for instance -- is recorded hundreds of times; engineers edit that selection down to a few dozen.

Mr. Tucmandl pays musicians he knows from the Philharmonic, as well as other Viennese soloists and chamber musicians for the recordings. He says it took two years to record the samples for a solo violin, spanning 200 three-hour sessions.

For a musician, this would seem to be the ultimate conflict of interest -- helping to build a computer program that could one day eliminate one's own job. But Johann Schodl, a 39-year-old Austrian musician who has recorded trombone, euphonium and other horn parts for the library, says he doesn't see it as a threat to his livelihood. Live performing gigs are still relatively abundant in Europe.

Mr. Schodl says he was paid about $140 for each three-hour recording session for the digital library. He adds that he has often made more in a week of recording for the program than he would in a typical week of rehearsals for a European orchestra.

There have been other benefits, too, for Mr. Schodl, such as mastering the control to emit a perfect note while also quelling the sound of wind from his horn or his tongue. "It made me a better player," he says.


---

(c) Dow Jones. Buy the newspaper to get the photos, podcast links, hyperlinks, etc.

[Muzik 4 Machines]

really interesting article, thanks