S I L E N C E
Version 2.0

ScrnMuse.exe

ScrnMuse.exe is a part of Silence, a system for making music on computers by means of software alone. ScrnMuse requires the installation of Csound.exe and WCSound.dll on your computer.

The purpose of ScrnMuse is to serve as a utility for translating screen shots of images into music. A screen shot may be created at any time in Windows by pressing [Alt-Print Screen], i.e., by pressing the [Alt] key and the [Print Screen] keys at the same time. The screen is captured as a bitmap and stored on the Windows clipboard. ScrnMuse opens the clipboard, gains access to the captured bitmap and associated palette, and translates the data into a musical score or an additive synthesis file.

I created ScrnMuse mainly for translating Fractint images into music. Fractint is a popular public-domain program for generating high-resolution color images of a wide variety of fractals, chaotic dynamical systems, and other mathematical curiosities. As it happens, you can cannot capture a screen shot directly from Fractint. However, you can save any fractal image created with Fractint as a Graphics Interchange Format ("gif") file. A number of shareware images viewers and translators can be used to view the Gif file. Be sure you enlarge the image to occupy the entire screen so that menu bars, etc., will not be inadvertently translated into music.

Other programs that create interesting images can also be used to create music.

And, of course, you can draw images directly using an illustration program such as Adobe Photoshop, or even Paintbrush.

ScrnMuse can translate screen shots into either musical scores or sequences, or raw sound data.

Score Translation

In score translation, the horizontal dimension of the image represents time, and the vertical dimension represents pitch. The color of each pixel represents both the choice of instrument and its loudness. Hue is choice of instrument, and value (brightness) is loudness.

The algorithm for translation examines each column of pixels in the image and selects as many of the loudest (brightest) ones as can be satisfied by the pool of available voices. Once a voice begins playing a note, it continues to sound until a black pixel (color 0) is reached.

As a consequence of this logic, different numbers of voices must be used to translate different types of images. Not all images translate satisfactorily. It is necessary to experiment.

To translate an image into a score:

1. View the image to be translated in full-screen mode.
2. Press [Alt-Print Screen] to capture the image on the Windows clipboad.
3. Run the ScrnMuse.exe program.
4. Select a score template file. It should be a Silence .osc file containing Csound instrument definitions and wavetable definitions.
5. Enter the number of voices that are allowed to sound simultaneously. In general, from 4 to 40 voices are a good choice; the more complex the image, the higher the number of voices required to achieve a satisfactory rendering.
6. Enter the number of instruments to which the voices are to be assigned in the relevant edit boxes. This depends entirely upon the desired musical texture.
7. Enter the desired duration of the score in seconds.
8. Enter the number of divisions per octave to be used for equally tempered tuning. If you enter 0, then ScrnMuse will translate the pixel rows into exact pitches.
9. Click on the "Csound score / Midi file" button.

A sample file for score translation is provided in Sample.gif.

Sound Translation

In this mode, ScrnMuse translates the image directly into sound. This requires a powerful computer with a lot of memory. I have a Pentium 166 MHz with 40 megabytes of RAM, and it still takes more than a few minutes to translate an image and render it as sound. This mode also requires the Silence version of Csound which has been specially modified to permit up to 1,024 partials in the adsyn opcode.

Each horizontal row of pixels represents a series of amplitude breakpoints for a single sine wave oscillator. The oscillators are spaced logarithmically in frequency, that is, the lower rows are closer together in frequency than the higher rows. In effect, the image represents a sort of sonogram of the desired sound. ScrnMuse translates the image into a Csound additive synthesis analysis .ads file, and runs Csound to synthesize the sound using the adsyn opcode.

To translate an image into a soundfile:

1. View the image to be translated in full-screen mode.
2. Press [Alt-Print Screen] to capture the image on the Windows clipboad.
3. Run the ScrnMuse.exe program.
4. Enter the desired duration of the score in seconds.
5. Click on the "Csound additive synthesis" button.

ScrnMuseproduces a filename.ads additive synthesis breakpoint file, a filename.orc Csound orchestra file, and a filename.sco Csound score file, runs Csound to realize the score, and plays the soundfile using the Windows shell "play" action.

If you are not happy with the soundfile, you can edit the .sco file to change the duration or pitch of the sound. The .orc file assumes that the .ads file represents a sound whose pitch is middle C (octave 8.0) at 80 decibels, so you can raise the pitch an octave by changing the pitch of the note from 8 to 9, or lower it an octave by changing it to 7.

You can also edit the .orc file if you know the actual pitch or loudness of the sound. In this way, you can use the generated instrument definition in the .orc file as a sampler for Csound. You can then make notes of any pitch, loudness, or speed from the sound.

A sample to translate into sound is provided in Sample.gif. Try making a relatively long (say 240 seconds, i.e. 4 minutes) sound out of this.