<div id="popup_box_thanks" style="display:none" onClick="close_popup_thanks('popup_box_thanks', 'ts')"><br>Thanks for submitting your tip! All submissions are moderated by an editor before appearing online. We've reset the form so you can enter another tip. Or you can close the tip submission box. <div class="x_close" id="thanks_upper_right"><a href="javascript:void(0)" onmousedown="close_popup_thanks('popup_box_thanks', 'ts'); return true;">Close</a></div></div>
<div class="tbf_row"><div class="tbf_wide_extra_top not_bold">Please submit only technical tips that will help other TidBITS readers better use their Macs, iPhones, and related software and hardware. All product announcements should be sent to <a href="mailto:releases@tidbits.com">releases@tidbits.com</a>.</div></div>
<div class="tbf_left">URL</div><div class="tbf_right"><input type="text" value="" name="tip_link_url" tabindex="3"><span class="tip_description"><br>Enter the URL to a Web page that supports your tip.</span></div>
</div>
<div class="spacer"></div>
<div class="tbf_row">
<div class="tbf_left">Linked text</div><div class="tbf_right"><input type="text" value="" name="tip_link_label" tabindex="4"><span class="tip_description"><br>Enter the name of the page linked above.</span></div>
<div class="tbf_wide"><input type="submit" value="Preview Your Tip" name="preview_tip" onClick="fill_preview('tipbits_enclosure_preview', 'ts', this.form); return false;" tabindex="7"> <input type="submit" value="Send Us Your Tip!" name="submit_this_tip" onClick="handle_tip_submission('ts', '', this.form, 'tip'); return false;" tabindex="8"></div>
</div>
<div class="spacer"></div>
<div class="tbf_row">
<div class="tbf_wide"><span class="fine_print">When you submit a tip, you give us permission to use it. Read <a href="javascript:void(0)" onClick="generic_show_hide('tip_terms')">our terms</a> for more details. All submissions are reviewed before publication.</span></div>
<div class="tbf_wide"><span class="fine_print">Our terms: By submitting a tip, you agree to assign TidBITS Publishing Inc., a non-exclusive, worldwide, perpetual license to reproduce, publish, and distribute your tip in connection with the TidBITS Web site and associated products in any media. You agree that you created the content you submitted, and that you have the right to assign us this license. You give us permission to use your name, but your email address won't be publicly displayed or shared. We review all submissions before publication, and reserve the right to select which submissions we feel are appropriate for our readers and to edit those we publish.</span></div>
<div id="comment_thanks" style="display:none" onClick="close_popup_thanks('comment_thanks', 'comm')"><br>Thanks for submitting a comment! Please check your email for a link that, when clicked, will verify that you're a real person and cause your comment to appear immediately. <div class="x_close" id="comment_upper_right"><a href="javascript:void(0)" onmousedown="close_popup_thanks('comment_thanks', 'comm'); return true;">Close</a></div></div>
<div class="tbf_wide"><span class="fine_print">Our terms: We reserve the right to edit or delete any comment, so please post thoughtfully. We use your email address <i>only</i> to send you a one-time verification message confirming that you posted this comment. We also store your address to allow you to verify using other Web browsers in the future. For more info, see our <a href="http://db.tidbits.com/privacy.html">privacy policy</a>.</span></div>
<li><a href="/feeds/tidbits.rss" title="Subscribe via RSS" class="gettb">RSS <img src="/images/feed-icon-12x12.gif" width="12" height="12" border="0" class="nav_img" alt="Subscribe via RSS"></a></li>
<li><a href="http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=276986548" title="Subscribe to the podcast" class="gettb">Podcast <img src="/images/feed-icon-12x12_podcast.gif" width="12" height="12" border="0" class="nav_img" alt="Subscribe to the postcast"></a></li>
<li><a href="http://www.twitter.com/TidBITS" title="Get Article Updates via Twitter" class="gettb">Twitter <img src="/images/feed_icon_12x12_twitter.png" width="12" height="12" border="0" class="nav_img" alt="Get Article Updates via Twitter"></a></li>
<li><a href="http://www.facebook.com/pages/TidBITS/195314925519" title="Go to the TidBITS Page at Facebook" class="gettb">Facebook <img src="/images/feed_icon_12x12_facebook.gif" width="12" height="12" border="0" class="nav_img" alt="Go to the TidBITS Page at Facebook"></a></li>
<li><a href="javascript:void(0)" title="Sections" class="tabhead" onClick="return showhide('articleslist')">Sections <span id="articleslist_triangle"><img src="/images/nav_triangle_open.gif" width="9" height="9" border="0" class="navtriangle" id="articleslist_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="javascript:void(0)" onClick="return showhide('stafflist')" title="Staff" class="tabhead">Staff <span id="stafflist_triangle"><img src="/images/nav_triangle_closed.gif" width="9" height="9" border="0" class="navtriangle" id="stafflist_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="javascript:void(0)" title="Issues" class="tabhead" onClick="return showhide('issuelist')">Weekly Issues <span id="issuelist_triangle"><img src="/images/nav_triangle_closed.gif" width="9" height="9" border="0" class="navtriangle" id="issuelist_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="javascript:void(0)" onClick="return showhide('abouttidbits')" title="About TidBITS" class="tabhead">About TidBITS <span id="abouttidbits_triangle"><img src="/images/nav_triangle_closed.gif" width="9" height="9" border="0" class="navtriangle" id="abouttidbits_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<div class="center_top">Thoughtful, detailed coverage of the Mac, iPhone, and iPad, plus the best-selling <a href="http://www.takecontrolbooks.com/?pt=TB-TAGLINE" style="color:yellow">Take Control</a> ebooks.</div>
<!-- begin centercolumn -->
<div id="centercolumn">
<!-- begin rightcolumn_container -->
<div id="rightcolumn_container">
<!-- begin rightcolumn -->
<!-- rightcolumn is embedded within centercolumn so featured text wraps around it -->
</div><!-- end tearoffbox_wide_container for watchlist items -->
<!-- begin tearoff box wide -->
<div class="tearoffbox_wide_container">
<div class="tearoffbox_wide_tips">
<div class="tip_display">
<div class="tips_sponsor_logo">
</div>
<h6>Sleep (and Lock) Your Screen</h6>
<p><p>When you are walking away from your computer, it's fairly common practice to start your screen saver and lock your screen. But did you know that there is a built-in keyboard shortcut in Mac OS X to sleep the screen?</p><p>Press Control-Shift-Eject and your monitor sleeps without engaging the screen saver.</p></p>
<p>Visit <a href="http://discussions.apple.com/thread.jspa?messageID=10350660&tstart=0">Apple Support Discussions</a></p>
</div>
<div class="tearoffbox_wide_bottom_tips">
<div style="padding-bottom:35px"><div class="tip_display" style="float:left"><p><br><a href="/tipbits/169">Link to this tip</a></p></div><div class="tip_display" style="float:right; width:150px">
<div class="tbf_wide_80" id="hc_rc_5716">To help us avoid automated posts and misuse of our site, please enter the words below.</div><div class="x_close_row" id="hc_upper_right2_5716"><a href="javascript:void(0)" onmousedown="HidePopupContent('hc_5716', 'hc', '5716'); return true;">Close</a></div>
<div class="featured_meta"><div class="meta_article">10 Mar 2003 | <a href="/article/7102?print_version=1">Print <span class="shift_up"><img src="/images/printer_icon.gif" alt="Printer-Friendly Version of This Article" border="0" width="9" height="10"></span></a></div></div>
<div id="article_box_5716"><P>This is the tale of a U.S. government statute, a certain Cupertino-based computer company, and a small revolution taking place quietly on your computer, if you use Mac OS X 10.2 Jaguar.</P><P>To understand this revolution, you need to know what a macro program is. I've been writing in TidBITS about such programs for years. Simply put, there are certain frequent or repetitive computer tasks that one would like to be able to automate. This is often straightforward if the program that performs these tasks is scriptable, meaning that it responds to a repertory of AppleScript commands: you just write a script expressing what you want done. But what if the program is not scriptable?</P><P><STRONG>Of Macs and Macros, Take Two</STRONG> -- One solution that worked pretty well for me in previous incarnations of the Mac OS was to use a macro program. Such a program must hack into the Mac OS itself so as to be able to simulate the presence of a "ghost" user who can press keys and move and click the mouse just as a real user would. By stringing together such actions, the computer could sometimes be forced to make up for the lack of scriptability in certain applications. Macro programs that I formerly used include PreFab Player, QuicKeys, OneClick, and KeyQuencer.</P><P><<A HREF="http://db.tidbits.com/article/00801">http://db.tidbits.com/article/00801</A>><BR><<A HREF="http://db.tidbits.com/series/1044">http://db.tidbits.com/series/1044</A>></P><P>Now wind the clock forward to Mac OS X. The trouble here is the word "hack" in the previous paragraph. One of the main points of Mac OS X is that you can't hack the operating system; to preserve stability, Apple has abolished the kind of extensions that used to make such hacks possible. Unfortunately, this means that macro programs don't work all that well on Mac OS X. When QuicKeys was migrated to Mac OS X, I was appalled how poorly it worked. It had trouble performing certain kinds of actions, such as choosing from menus and pushing buttons; and, more important, it couldn't "see" what menu items and buttons were present, so it couldn't make intelligent decisions or wait for the right window to appear.</P><P><<A HREF="http://db.tidbits.com/article/06603">http://db.tidbits.com/article/06603</A>></P><P>These limitations, of course, were not the fault of QuicKeys. Since QuicKeys couldn't hack into Mac OS X, it could see and act only in ways permitted to it by the various legitimate application programming interfaces (APIs) provided by Apple. And Apple, concerned with stability, wasn't about to let just any old program reach in and start meddling with some other program's windows and menus. Knowing this, however, did not solve the problem; and users were left staring at QuicKeys, waiting for it to improve. It did; but not much.</P><P><<A HREF="http://db.tidbits.com/article/07022">http://db.tidbits.com/article/07022</A>><BR><<A HREF="http://db.tidbits.com/article/06786">http://db.tidbits.com/article/06786</A>></P><P>Now for the revolution. While we were all staring the wrong way - at QuicKeys - the problem was being solved by Apple itself. And the solution (in general, and for future versions of QuicKeys as well) is in place, right now. You can script unscriptable applications, manipulating their menus and windows and buttons, and typing text, right now. It's just that Apple hasn't bothered to say much about it, so few people know.</P><P><STRONG>Enter Uncle Sam</STRONG> -- How could this happen? And why did it happen? It all goes back to a U.S. statute called the Workforce Investment Act of 1998, commonly known as Section 508. This statute says that when federal agencies procure electronic information technology, that technology must give disabled people access to information that's comparable to the access it gives non-disabled people. In plain English, if you manufacture computers, and you want any government agency to be able to buy any of them, they must be fully operable by people with disabilities.</P><P><<A HREF="http://www.usdoj.gov/crt/508/508law.html">http://www.usdoj.gov/crt/508/508law.html</A>></P><P>To see what this statute might mean for Apple Computer, Inc., let's put ourselves inside the mind of someone at Apple Computer, Inc. This person is imagining a Mac being operated by someone with a disability. And he or she is thinking: "Suppose this disability is such that the user can't work with the keyboard or the mouse. Holy cow! Unless we want our government sales to be zero, we'd better provide a way, in Mac OS X, for some other device, such as one of those sip-and-puff joysticks, to be able to operate all parts of the user interface." Clearly such operation implies something far beyond merely a few keyboard shortcuts that substitute for the mouse some of the time, as in the Universal Access and Keyboard preferences panes. It means that the system itself must be able to "see" and access all the individual interface elements of any application, so as to provide a pathway whereby a joystick, or any other assistive device, can press every button and click in every text box and somehow reach every interface element that appears on the screen.</P><P><<A HREF="http://www.apple.com/macosx/jaguar/universalaccess.html">http://www.apple.com/macosx/jaguar/ universalaccess.html</A>><BR><<A HREF="http://www.abilityhub.com/mouse/joystick.htm">http://www.abilityhub.com/mouse/joystick.htm</A>></P><P>The idea, then, is this. Given any application on Mac OS X, it needs to be possible for some other application to discover what interface elements it is displaying. The other application needs to be able to "read" these elements ("There's a button that says OK") and it needs to be able to access them (click the OK button). Apple went to work on the problem, and the result, which first emerged to public view in Jaguar, is the Accessibility API. Actually, there are two things going on. Under the surface, Apple has provided some deep magic, in the operating system itself, that does the reading and the accessing of all the interface elements present on the screen. Closer to the surface, Apple has provided the API, which is a set of commands a programmer can use to take advantage of this magic.</P><P><<A HREF="http://developer.apple.com/techpubs/macosx/Cocoa/TasksAndConcepts/ProgrammingTopics/Accessibility/">http://developer.apple.com/techpubs/macosx/ Cocoa/TasksAndConcepts/ProgrammingTopics /Accessibility/</A>><BR><<A HREF="http://developer.apple.com/techpubs/macosx/Carbon/HumanInterfaceToolbox/Accessibility/MakingAppsAccessible/">http://developer.apple.com/techpubs/macosx/ Carbon/HumanInterfaceToolbox/ Accessibility/MakingAppsAccessible/</A>></P><P>Now stop and think about what I just said. This is all present in Jaguar, right now. So anyone can write a program that can see and access the interface elements of any other program, right now. But such a program would, in effect, be a macro program! In short, the various things QuicKeys couldn't do when it first appeared on Mac OS X, because it couldn't hack into the Mac OS, any program can now do, thanks to the Accessibility API.</P><P>But it gets better. Read on.</P><P><STRONG>Blessed (System) Events</STRONG> -- At some point, someone at Apple put two and two together, as follows. "On the one hand, we've got the Accessibility API, which lets any program access the interface elements of any other program. On the other hand, we've got AppleScript, which lets any user write a script to give commands to any scriptable program. So what would happen if we put them together? Any user would be able to write a script to give commands to a scriptable program which would use the Accessibility API to access the interface elements of any program." To see the significance of this, just take out the middle terms, and what have you got? Any user can write a script that can access the interface elements of any program. Any user can script an unscriptable program, using AppleScript! You wouldn't need any special macro program, because AppleScript itself, which is already on your computer, would become a macro language.</P><P>Of course, for this to work, there has to be that "middle term" - the go-between, the application that receives AppleScript commands and talks to the Accessibility API. That application is called System Events. It's on your computer right now, in /System/Library/Core Services. But the version of System Events that knows about the Accessibility API is probably not on your computer, because it's in beta and is not part of the standard Mac OS X release. You must download and install it separately, from Apple's GUI Scripting Web page.</P><P><<A HREF="http://www.apple.com/applescript/GUI/">http://www.apple.com/applescript/GUI/</A>></P><P>Let's try it out! There are two preliminary steps.</P><OL><LI><P>Download and install the beta version of System Events.</P></LI><LI><P>In the Universal Access preferences pane, check the box at the bottom that says "Enable access for assistive devices." This is crucial because it throws the virtual switch that brings the Accessibility API into play.</P></LI></OL><P>We'll also need an unscriptable application to operate on. For this example, we'll use my own freeware MemoryStick, so download it (and shame on you) if you haven't got it.</P><P><<A HREF="http://pages.sbcglobal.net/mattneub/downloads/MemoryStick%20Installer.dmg.gz">http://pages.sbcglobal.net/mattneub/downloads/ MemoryStick%20Installer.dmg.gz</A>></P><P>MemoryStick isn't scriptable, and was never intended to be. Nevertheless, as if by magic, we're going to open its Preferences window, find the Poll tab item, read how often MemoryStick is set to poll the system, and click the spinner up or down the right number of times so that the setting ends up at "5 seconds" - and then close the Preferences window. Ready?</P><P>Start up MemoryStick if it isn't running. Start up Script Editor (it's in /Applications/AppleScript). Paste in the following script:</P><PRE><SMALL>tell application "MemoryStick" to activatetell application "System Events" tell application process "MemoryStick" click menu item "Preferences..." of menu "MemoryStick" of menu bar 1 tell tab group 1 of window "MemoryStick Preferences" click radio button "Poll" get value of static text 3 copy the result as number to theVal set theDec to theVal - 5 if theDec > 0 then repeat theDec times decrement incrementor 1 end repeat else if theDec < 0 then set theDec to 0 - theDec repeat theDec times increment incrementor 1 end repeat end if end tell click button "Done" of window "MemoryStick Preferences" end tellend tell</SMALL></PRE><P>Now run the script, and watch the fun. Don't blink, or you'll miss it!</P><P><STRONG>Future Directions</STRONG> -- By now, you're probably saying, "Wow! That was great! How can I learn to give these sorts of commands to any application, so that I can script unscriptable applications and turn AppleScript into a macro programming language?"</P><P>Luckily, there's a splendid way to do this. It's called PreFab UI Browser, the work of Scott Lawton of PreFab Software, who brought us PreFab Player and TextMachine, and Bill Cheeseman, the well-known AppleScript guru and author of Cocoa Recipes for Mac OS X. This brilliant little program uses the Accessibility API to "look" at all the interface elements of any running application, and generates the AppleScript commands you'd use to click them, read them, type into them, or whatever. It's $25 (until mid-April, when the price goes up), and a 30-day demo is available.</P><P><<A HREF="http://www.prefab.com/uibrowser/">http://www.prefab.com/uibrowser/</A>><BR><<A HREF="http://db.tidbits.com/article/04198">http://db.tidbits.com/article/04198</A>><BR><<A HREF="http://www.applescriptsourcebook.com/">http://www.applescriptsourcebook.com/</A>><BR><<A HREF="http://www.amazon.com/dp/0201878011/?tag=tidbitselectro00">http://www.amazon.com/exec/obidos/ASIN/ 0201878011/tidbitselectro00</A>></P><P>It will be interesting to see what further applications will emerge that take advantage of the Accessibility API. For example, one of my favorite utilities in earlier days was an extension that listed all the windows in all running applications, so that you could switch directly to the right window, no matter what application you were in now. Before the Accessibility API, such a utility was impossible in Mac OS X, because no application could "see" another application's windows, let alone switch amongst them - the Dock could do it, but only because it belonged to Apple and was privy to System-level secrets no one else had. But now, writing such a utility should be easy.</P><P>The story is not over, not least because both the Accessibility API and the Scripting Events beta are still young, still in development, and still buggy. Nevertheless, they do work, as the example proves; and eventually, perhaps as part of the next major update to Mac OS X, I would expect the special version of System Events to come out of beta and be made part of the standard release. Meanwhile, now that you know the secret, you can start playing with AppleScript in its wonderful new role as a macro language on Mac OS X.</P><P>As you do, keep in mind that even the Accessibility API can't perform miracles; it can't detect interface items that are not constructed by way of Mac OS X's built-in toolbox. A program that uses non-standard interface items can be written deliberately to provide Accessibility API access to them; but if it isn't, the Accessibility API is blind. For example, the Accessibility API can't see any of Microsoft Word's menus, toolbars, or even the text of a document.</P><P>Not that this matters, of course, because Microsoft Word is completely scriptable already. Which raises another point: There's no substitute for true scriptability. To write an application of any serious power without giving it the ability to be driven with AppleScript is simply poor programming practice. So, if you're able to automate some previously unscriptable area of your workflow through Accessibility API scripting, that's splendid; but if the target program is still being actively developed, please do also write to the developers and ask that they make that same functionality genuinely scriptable.</P><P class="paybits">PayBITS: If Matt's article makes a difference for what you<BR> can script, consider sending him a few bucks via PayPal!<BR> <<A HREF="https://www.paypal.com/xclick/business=matt%40tidbits.com">https://www.paypal.com/xclick/ business=matt%40tidbits.com</A>><BR> Read more about PayBITS: <<A HREF="http://www.tidbits.com/paybits/">http://www.tidbits.com/paybits/</A>></P><!-- Scripting the Unscriptable in Mac OS X Matt Neuburg --></div>
<!-- end article text -->
<!-- PayBITS -->
<p> </p><div class="sponsorbox">
<div class="sponsortext"><A HREF="http://www.usefulfruit.com/tb"><IMG SRC="http://db.tidbits.com/images/badges/pear-note-icon50x50.png" ALT="" HEIGHT="50" WIDTH="50" BORDER="0" ALIGN="left"></A>Pear Note 2: More complete, understandable notes on your Mac.<br />Typed notes are blended with recorded audio, video, and slides<br />to create notes that make more sense when you need them most.<br />Learn more at <<a href="http://www.usefulfruit.com/tb">http://www.usefulfruit.com/tb</a>>!</div>
