Working with Quartz Composer in Leopard

Quartz Composer is a real-time visual programming environment. Real-time means that you can have a Viewer showing the output of your current program, running at full-speed and instantly updated while you work. There's no rendering or compiling required, which greatly reduces the time you'll spend waiting on your computer. With Quartz Composer, you can easily combine the capabilities of Cocoa, Quartz 2D, Core Image, OpenGL, and QuickTime, all using an approachable visual programming paradigm. Use Quartz Composer to prototype Core Image filters, build engaging screen savers, create custom user-interface widgets, make data-driven visual effects, and even perform live performance animations.

Quartz Composer is integrated into the very core of Mac OS X and can bring its wide array of features into a wide variety of programs—integrating animation, MIDI, RSS, Networking, Audio Input/Analysis and Computer Statistics into your existing and upcoming applications. Embed it into web pages and any WebKit-enabled programs for interactive, information-driven eye candy. Create filters, transitions and animations to use in Keynote, iChat, iMovie and Final Cut Studio. Whip up iTunes visualizers for your next house party. Design animated Screensavers that pull in RSS feeds to define and promote your brand. Develop an interactive performance system and go on tour as VJ with your favorite band.

This article provides an overview of the visual programming environment and introduces you to a few of the ways Quartz Composer has been used in real-world applications. You’ll meet some designers making good use of Quartz Composer in innovative ways. You will gain an understanding of how Quartz Composer is deeply integrated into the Mac OS, and how you can leverage its power and real-time feedback to improve your own projects. Finally, you will discover how you can write your own custom patches and extend Quartz Composer's possibilities even further.

The Visual Programming Environment

Quartz Composer offers a friendly and intuitive visual environment to work in. Most of your programming is done by physically drawing connections between nodes (called patches), twirling dials and entering values into input boxes. As you work with a composition, adding patches and connecting them, you can visually see the results in a viewer window. Each and every change you make is immediately reflected in the viewer—no compilation required. This results in a development experience like no other, as illustrated in Figure 1.

Figure 1 Quartz Composer windows.

Figure 1:  Quartz Composer windows.

A patch is similar to a subroutine in a traditional programming environment. You can provide inputs to the patch, which will then execute and produce some results. Circles on the left side of a patch represent the various inputs the patch will accept. Circles on the right side are the outputs. They are officially referred to as 'ports'. For example, a Random patch will accept Min and Max parameters and use them to create a Value output, as shown in Figure 2.

Figure 2 A simple patch.

Figure 2:  A simple patch.

The input values of a patch can be set in one of two ways. The first is to directly set the value by double-clicking on the input or by using the Patch Inspector. The second is to connect the output of another patch to the input. In Figure 3, the Inspector shows the input values for the Random patch while the Value output is connected to the Math patch. When run, the Random patch will generate a random number between 0 and 1 and pass that as its Value output.

Figure 3 Monitoring a connection.

Figure 3:  Monitoring a connection.

The overall programming task in Quartz Composer consists of selecting the patches that you want to use, organized in the Patch Browser, and connecting them together to achieve the desired effect.

Artist Success Stories

At this point, let's shift gears from talking about patches in the abstract and see how real-world artists use Quartz Composer.

MIDI Control

MIDI is a protocol that enables communication between hardware and software devices. It is mainly found in electronic musical instruments and music software. Quartz Composer supports a wide range of MIDI commands, making it simple for even amateur developers to integrate piano-style keyboards, drum sequencers and control surfaces covered in knobs and faders with their visual projects.

When artist Momo the Monster (Surya Buchwald) needed to create an arcade-cabinet-style video installation for The Orb's performance at the Walt Disney Concert Hall in Los Angeles, he found Quartz Composer to be a perfect fit. He designed a Quartz Composition that constantly drew a multi-layered 3-dimensional abstract pattern, with a MIDI box of knobs and faders built into the cabinet to control every movement in the system. Concert-goers freely manipulated the composition, watching their results projected on a facing wall of the architecturally striking building. Since Quartz Composer documents are so small (24KB zipped, in this case) it was a simple matter for the artist to make the project's source code available for any intrepid designers who were inspired by the system and wanted to build their own version.

This is an example composition of the installation from the Orb show. Although originally controlled by many MIDI controllers, this example gives an idea of what it looks like. If you click and drag in the viewer, you do affect a parameter of the composition.

Custom Patches

Quartz Composer comes with a vast array of patches that handle drawing, playing movies and sounds, generating streams of numbers, capturing and processing user input, analyzing sound and more. If you find that the functionality you need in your composition does not yet exist, then you can create your own patch. Quartz Composer has a plugin API, complete with Xcode templates for creating your own original patches, which can be used in your compositions as well as distributed to other users and developers. Using this API, you can greatly expand the universe of interactive graphics you can construct.

Programmer/Artist Vade (Anton Marini) has released a number of original Quartz Composer Custom Patches—v002 Screen Capture being a particularly innovative plugin. This patch captures the screen (a portion in any aspect, or the entire screen if desired) and passes the video output to its host program. For example, you could run the VJ program VDMX, and use the plugin to capture a Processing sketch and add it to your live mix, even though VDMX doesn't support direct loading of Processing sketches. As a bonus, Vade offers not only his plugin for free, but the XCode project and full source code so you can learn from his work.

Integration

Quartz Compositions excel at running as standalone presentations—providing high-resolution graphics with blazing frame rates that can run natively on any 10.5-capable system. In addition, the deep integration Quartz Composer enjoys with the Mac OS operating system creates a wide array of possibilities for behind-the-scenes power. You can run .qtz files directly in the Final Cut Studio suite of apps, automatically generating text and video that could take hours to animate by hand. Quartz Composer will also run in applications that support WebKit, as well as directly in Safari.

Developer Success Story

Dan Wood at Karelia needed a way for users to make stylish page headers in their WYSIWYG Web Page software Sandvox. The headers needed to be variable in size, use a variety of fonts, and wrap intelligently to multiple lines. Quartz Composer turned out to be a stellar engine for generating these stills—running text input from the user through a Quartz Composition and sampling one frame back into an image container to be embedded in the web page. In this way, they could also add effects like bubbles randomly generated through a particle system in Quartz Composer, so every header is a little different.

For an in-depth look at embedding Quartz Compositions in web pages and other WebKit documents, see the Introduction to Quartz Composer Web Kit Plug-in JavaScript Reference.

Hands On—Making 'Hello World'

In order to work with Quartz Composer, you'll need to have Xcode Tools installed. When installed, you can find the application in the /Developer/Applications/Graphics Tools folder, as shown in Figure 4.

Figure 4 Finding Quartz Composer.

Figure 4:  Finding Quartz Composer.

When you first launch Quartz Composer, you are presented with template choices. For this first project, start with a blank slate.

  1. Choose 'Blank Composition' from the Template List.

    You are presented with two windows at the start—the Editor, where you will be constructing your patch, and the Viewer, where you can see a realtime output from your composition. You can arrange all the windows however you see fit once you understand how they work—arrange them in 3 columns for now:

  2. Click the 'Patch Creator' button at the top left corner of the Editor window, and arrange your workspace with the Patch Creator and Editor on the left and the Viewer on the right.

  3. In the Patch Creator, type 'string' to narrow the list, and double-click on 'Image With String' to add this patch to your Editor.

    Alternatively, you can drag and drop patches into your Editor. Now add a Billboard on which to render the Image With String.

  4. In the Patch Creator, type 'bill' and drag-and-drop the Billboard patch just to the right of your Image With String patch.

  5. Drag-and-drop from the Image With String's 'Image' output port to the 'Image' input port on the Billboard to connect these two patches.

    It's working—you now see a black box with the words 'Hello World!' inside, on a checkerboard background.

  6. Using the Patch Creator, find and add a 'Clear' Renderer to the Editor. Arrange it so that it is just above the Billboard patch.

    Your screen goes black, because the Clear patch is Rendering on Layer 2, and the Billboard is now underneath it on Layer 1.

  7. Click directly on the the '2' in the upper-right corner of the Clear patch, and select 'Layer 1' from the menu that drops down.

  8. Select the 'Image with String' patch in the Editor and press command-i to open the Inspector.

    The Inspector is the final piece of the interface. It updates to show the changeable parameters for the currently-selected patch. In this case, you see String, Font Name, Font Size, etc. If you click on the left and right arrows at the top of the Inspector, you will see that the Image With String patch has 2 pages of useable items.

  9. Type a new string into the Inspector's 'String' field. Your viewer will update as soon as you press return.

  10. Use the Patch Creator to add a Particle System to your Editor, and connect the Image output of the Image With String patch to the Image input on the Particle System.

  11. Select the Particle System patch so its parameters will be visible in the Inspector. Find the 'Blending' parameter and choose 'Add' to blend the particles together.

  12. Find the Gravity parameter and twirl the dial clockwise until the Gravity field reads somewhere around '2'.

  13. Scroll up to find the 'Color' parameter and click on the white color field to open up the Mac OS Color picker. Click and drag in the color wheel and watch as your viewer updates your particles in real-time. Close the color window once you are happy with your choice.

  14. Add a 'Mouse' patch to your editor, and connect its 'X Position' and 'Y Position' outputs to the Particle System's 'X Position' and 'Y Position' inputs, respectively.

    Now you can hover your mouse over the viewer to move your particle system. This kind of hands-on interactivity is where Quartz Composer really shines.

  15. Connect the Mouse's 'Left Button' output to 'Z Position' in your Particle System.

Now you can click your mouse button for a quick particle explosion. Take some time to try changing parameters by hand in the Inspector—when you find one you like, hook it up to the mouse for real-time control.

Getting Started With Your Own Compositions

Once you’ve had a chance to use Quartz Composer, you will appreciate the elegance and power of this visual programming language. Explore some of the resources listed here to learn how to best utilize Quartz Composer and add it to your tool set.

Sample compositions:


Posted: 2008-10-28

Related Articles

Resources