Some people cannot use the keyboard and mouse at all due to a physical impairment, and must use some specially designed input device (e.g. an augmentative or alternative communication [AAC] device) while accessing a computer. Many AAC devices are capable of connecting to a computer's serial port (the connector labeled "modem", typically on the rear of your Macintosh). However, the computer is designed to receive input from the user via the keyboard and mouse, not through the serial port. Therefore, simply connecting an AAC device to the computer's serial port will not allow the user to operate the computer.
The Macintosh SerialKeys software is designed to allow the AAC device to communicate to the computer through a serial port, and changes that information into keyboard or mouse events, just as if the user were typing on the keyboard or using the mouse. Macintosh SerialKeys follows the connection and communication protocol outlined in the General Input Device Emulating Interface (GIDEI) Proposal. Please contact the Trace R&D Center if you would like more information regarding the GIDEI. Note, the last page of this introductory memo has a list of "Key Names for the Enhanced Apple Keyboard" which SerialKeys for the Macintosh supports.
II. Setting up SerialKeys on your Macintosh
The following instructions listed in steps 1 through 8, tell you how to set up SerialKeys for the Macintosh with an AAC device. These instructions do not tell you how to program your AAC device.
Step 1: Determine if your AAC device will work with SerialKeys
The following questions should help you determine if your AAC device is capable of accessing the computer via SerialKeys.
1) Can you program selections on your AAC device to have unique words and sentences stored under them?
2) Can your AAC device store at least as many programmable selections as there are keys on your computer keyboard, while also saving a few programmable selections for mouse actions? Remember, the AAC device user will want to operate the computer just as if they were using the keyboard and mouse, and therefore will need all the capability of the keyboard and mouse.
3) Does your AAC device have a serial port? This is the connection sometimes used to hook up a printer and maybe called an RS-232 serial port in your device user manual.
4) Can you program selections on the AAC device such that when they are selected, they will automatically be sent out the serial port?
5) Does your Macintosh computer have an available serial port (e.g., modem or printer port)? If you are using some other device on your Macintosh modem or printer port, SerialKeys will not be able to operate. Please note, internal modems use the modem or serial port drivers on the Macintosh, making the port un-usable for SerialKeys while AppleTalk tends to make use of and tie up the printer port. Also note, while SerialKeys allows you to use the printer port, the Macintosh computer itself may not fully support the higher baud rates. Therefore, if you are loosing characters while using the printer port, you may be forced to either lower your baud rate or send shorter character strings.
6) Can your AAC device send information through its serial port at the rate of at least 300 baud? (The term "baud" refers to the speed at which characters are transmitted.) SerialKeys for the Macintosh can "talk to" an AAC device that is sending information at 300, 600, 1200, 2400, 4800, 9600, or 19200 baud, provided it is supported by the Macintosh computer you are connecting to.
Step 2: Connect the AAC device to your computer
Plug one end of a serial cable into the serial port on your AAC device and the other end into the serial (e.g., modem or printer) port on your Macintosh computer. Most AAC devices will require a special serial cable known as a NULL serial cable for attaching to the Macintosh. (Example cable can be purchased from PC/MAC Connection (800) 800-0021, #1096, AB MAC+PC, 6 feet)
Step 3: Configure the AAC device for serial transmission
Your AAC device and SerialKeys must be talking to each other at the same speed (baud) for them to initiate communication. We recommend choosing a speed of 300 baud to get started. You must also make sure your AAC device is configured correctly. It must be communicating with the following format:
- 1 start bit
- 8 data bits
- 1 stop bit
- no parity.
Step 4: Install SerialKeys onto your Macintosh hard drive.
The Beta version of Macintosh SerialKeys you've just received on disk, can be installed onto your Macintosh hard drive by using standard Macintosh click and drag or "copy" procedures (e.g., there is no fancy installation routine). The diskette contains four files, besides this readme file, one titled "serialkeys", another titled System7.0-NOTICE, another titled "modem.skeys" and the last titled "printer.skeys".
The Macintosh SerialKeys program is actually the file "serialkeys." However, double-clicking the mouse on either the "modem.skeys" or "printer.skeys" file will launch the "serialkeys" program with some default settings, which you may want to use after you have experimented with the program, or you can create your own default or user files.
If you want Serialkeys to be active when you first power on your Macintosh, drag the file "serialkeys" to the StartUp Items folder inside of your System 7.x folder and restart your Macintosh.
If you do not want Serialkeys to run when your first turn your Macintosh on, you can copy the files to any folder or area on your hard disk that you wish to store them. The program can be launched by either double clicking on the Serialkeys icon, or on one of the two supplied data files (e.g., modem.skeys and printer.skeys), or on any other data file you create while running Serialkeys. If Serialkeys is already running, double clicking on a Serialkeys data file will load the data in that data file. If Serialkeys is not running, double clicking on a data file will launch Serialkeys and use that data file's data. If Serialkeys cannot run on your computer, it will supply you with an error message(s). Please inform us of this condition.
When Serialkeys does launch, assuming it encounters no difficulty, it will display a small window in the upper left corner of your Macintosh screen with a window name of "Serial Port Window". When ever Serialkeys is the active on-top application, this small window will appear, allowing the user to test their serial connection. If the user has a serial connection, and they are sending characters at the proper baud, the characters being sent should be appearing in the small serialkeys window. If the characters are not appearing in the serialkeys window, then something is wrong with the AAC device, cable, connection, etc. If you cannot get characters to appear in the serialkeys window, do not attempt to operate the Macintosh, since your connection is not correct, and you need to stop and examine your AAC device, connection, cable, etc., prior to proceeding. If you are sending characters from your AAC device and they are appearing correctly (what you send is visible), Serialkeys is operating properly and you may then navigate around the Macintosh desktop in what ever fashion you choose, keyboard, mouse, etc., and launch and use other applications just as if you were typing on the Macintosh keyboard or using the Macintosh mouse.
The Serialkeys program has only 3 menus, "File, Edit, and Serial Setup". Under the "File" menu, you have the ability to "Open, Save, and Save As..." files and "Quit" the program. Typically, you would "Open" a file you've created using the SerialKeys program, which contains pre-stored parameters like the baud you wish to communicate at, and any "anchor" locations you wish to use with your mouse (see later section on mouse "anchor"). There are no user options for Serialkeys under the "Edit" menu. Under the "Serial Setup" menu, the user can manually select a port and baud change. If the user does select to manually change the baud, they must remember to also change the sending baud on their AAC device to match the Serialkeys program.
Step 5: Programming keys and mouse actions on your AAC device
There are four types of SerialKeys actions you can program into any selection (position) on your AAC device:
1) basic keys,
2) special keys,
3) modifier keys,
4) mouse movements.
Typing basic keys
Each key on your computer keyboard is given a key name (see later section for key names which are longer than a single character), and it is this key name that you must program into your AAC device if you want to be able to type that key. Key names which consist of a single character are called "basic keys."
` 1 2 3 4 5 6 7 8 9 0 - =
q w e r t y u i o p [ ] \
a s d f g h j k l ; '
z x c v b n m , . /
To type one of these keys, you just send the character out the serial port of the AAC device. For example, to type the word "hello", you simply select the "h" key name, the "e" key name, the "l" key name, another "l" key name, and finally an "o" key name. Most AAC devices have all of these single character key names pre-programmed into them, so there is no additional programming for you to do for these keys.
Step 6: Typing special keys
If you look at the computer keyboard, you will find that there are many keys that do not have single character key names; for example, the arrow keys and the "num lock" and "return" keys. We call these keys "special keys", since you have to do something special or different if you want to type them using your AAC device.
To type special keys, you must program a string of characters, called an escape sequence, into a single selection on your AAC device. The reason this string of characters is called an "escape sequence", is because the string begins with the escape character. The escape character is a special character, created differently on the various AAC devices. For some AAC devices, the escape character can be created by selecting the control function along with the left bracket (^[) while on other AAC devices, the escape character is part of a special programming overlay. For the remainder of this introductory memo, we will refer to the escape character like this, <esc>. Remember, this is just a single character.
An escape sequence follows two rules:
1) The escape sequence must begin with the <esc> character.
2) The escape sequence must end with a period <.> character.
For example, to type the "return" key, the escape sequence would be:
<esc> return .
The escape character comes first <esc>, followed by the key name for the key that you want to press, followed by a period <.>. All the key names listed at the end of this document (with the exception of the four keys talked about in the next section) would be typed using this same method.
<esc> key name .
Step 7: Typing Modifier Keys (Shift, Control, Option, and Command)
There are four special keys on the Macintosh computer keyboard, the Shift, Control, Option, and Command keys, which have to be typed differently from the other special keys. These four keys are called "modifier" keys, since they don't do anything themselves, but modify the action of another key. On the keyboard, a user presses and holds pressed a modifier key while another key (e.g. the key to be modified) is pressed. For example, if the Shift is pressed and held down while the user types an "a" key, the result will be turned into a capital "A." SerialKeys imitates or emulates this action by adding a command called "hold" to the escape sequence you've just read about above. For example, the escape sequence you have to store or type on your AAC device in order to use a modifier key is as follows:
1) escape character <esc>
2) a comma <,>
3) the word hold
4) a comma <,>
5) the name for the modifier key
6) a period <.>
Now, the next key typed would be "modified", just as if the user was typing on the keyboard and holding down the modifier key or Shift key as in our example above.
Example: (shift or "modifier" key escape sequence)
<esc> , hold , shift .
would be programmed into a selection. Now selecting an "a" on the AAC device after selecting the above shift escape sequence would result in a capital "A" being typed at the computer via SerialKeys. All four Macintosh modifier keys (e.g., Shift, Control, Option, and Command keys,) should be stored on your AAC device in this fashion.
Step 8: Moving and clicking the mouse (mouse escape sequences)
All mouse activity is done through the use of escape sequences which contain specific mouse commands. One of the first commands you should use, before you do any mouse functions through SerialKeys, is the mouse reset (e.g. moureset) command. You should send a "moureset" command whenever you start up an application program. After this command is sent from your AAC device, the mouse cursor should move to the upper left hand corner of the screen. This command should also release any "locked" mouse buttons.
***** Note, you need to open the "Mouse Control Panel" and choose "tablet" mode (e.g., Very Slow), to prevent any mouse acceleration when you are doing mouse movements using SerialKeys. *****
Example:
<esc> , moureset .
Saving mouse cursor locations on the screen
The mouse "anchor" command can be used after the user has identified locations on the screen that they wish to return to on a frequent basis. An example might be the Macintosh "File" menu, the MultiFinder icon, or perhaps the "trash" icon. The user can save 26 separate anchor locations in each user file they create, each anchor locations being identified by a single lowercase alphabetic letter. To use the anchor command, position the mouse cursor on the computer screen where you would like to label an anchor, and then issue the anchor command followed by a single letter name. Use the mouse "goto" command when you wish to return to your previously saved anchor locations.
Example:
- first, move the mouse cursor to the desired location
- next, select a lower case letter to represent this location
- example, an "a" now becomes the label of the current location
of the mouse cursor
- to return to the anchor location labeled "a", issue a goto
escape sequence command, <esc> , goto .
- next, select the location label, in this example it was "a" and the mouse cursor should return to the location which was
previously identified and stored as label "a"
You can only have 26 anchor points stored in each user file. However, Serialkeys does allow the user to create and save separate user files, each containing the current baud, port, and defined anchor points. Therefore, a user could create and save 26 anchors per each application they wish to use, and load the respective file as they run each application.
Moving the mouse
You can move the mouse cursor by sending a "move" command from your AAC device to the computer. The move command takes two numbers after it, the motion in the horizontal direction followed by the motion in the vertical direction. There must be either a plus (+) or a minus (-) sign before each number, unless the number is a zero. Positive numbers move the mouse cursor to the right or down. Negative numbers move the mouse cursor left or up. For example, the escape sequence you have to store or type on your AAC device in order to move the mouse using the "move" command follows:
1) the escape character <esc>
2) a comma <,>
3) the word move
4) a comma <,>
5) a plus or minus sign followed by a number (for horizontal movement)
6) a comma <,>
7) a plus or minus sign followed by a number (for vertical movement)
8 a period <.>
Example:
<esc> , move , +10 , -20 .
Another command you can use to move the mouse cursor is the mouse "goto" command. The mouse "goto" command has two uses. First, the mouse "goto" command is used to move the mouse cursor to a specific location on the screen.
Example A:
<esc> , goto , 20 , 25 . causes the mouse cursor to goto screen location 20 units in the horizontal and 25 units in the vertical, usually in relationship to the position zero on the screen, which is the upper left hand corner.
A second use of the mouse "goto" command is when you wish to return the mouse cursor to a screen location the user saved previously with the mouse "anchor" command. (see "anchor" command example above)
Example B:
<esc> , goto .
the next single lowercase character is the name of the previously saved screen location (see mouse "anchor" command above).
Clicking the mouse button
To click or double click a button on the mouse, you use the "click" and "dblclick" commands.
Example:
<esc> , click .
<esc> , dblclick .
Clicking and dragging with the mouse
SerialKeys lets you do the equivalent of holding the mouse button down while moving the mouse. This is done using the "moulock" (mouse lock) and "mourel" (mouse release) commands, together with a mouse movement (e.g. move, goto) command. If, for example, you want to select text in a word processing program that uses the mouse, you would:
1) Move the mouse to the beginning of the text you want to select.
2) To press and keep a mouse button down, use the "moulock" command.
Example:
<esc> , moulock .
3) Move the mouse to the end of the text you want to select, which in most word processors "high-lites" the text for cut-and-paste actions.
4) To release the mouse button, use the "mourel" command.
Example:
<esc> , mourel .
Additional SerialKeys commands
Lock and Release of Keyboard Keys (Keyboard Commands)
The keyboard "lock" and "rel" commands can be used to hold a key down and lift it back up again as separate actions. Its primary usefulness is to allow SerialKeys, in combination with MouseKeys, to provide the AAC device user a method for continuous mouse cursor motion.
1) Turn on Easy Access MouseKeys
(******Note: Easy Access MouseKeys has to be turned on before Macintosh SerialKeys is activated for this feature to work.)******
2) Decide which direction you want to move the mouse cursor and find out what keypad key moves the mouse in that direction in Easy Access MouseKeys.
3) Send a lock command with that key name. For example, to move the mouse cursor to the right, use:
<esc> , lock , kp6 .
4) When the mouse cursor has moved as far as you want in that direction, send the release command.
<esc> , rel .
Baudrate Command (Miscellaneous Commands)
The baudrate command allows you to change the baud of SerialKeys from your AAC device. The possible bauds are 300, 600, 1200, 2400, 4800, 9600, and 19200.
Example:
<esc> , baudrate , 9600 .
Additional Macintosh SerialKeys ONLY feature(s) (experimental)
The Macintosh Operating System 7 or newer, supports the capability to switch between active processes that are running on the computer desktop. For this discussion, each process can be thought of as an application (e.g. SerialKeys would be one process or application, MultiFinder another, etc.) and the active process is defined as the process or application on top, receiving keystrokes. For evaluation purposes, the ability of an AAC user to quickly switch between active processes may assist them greatly in their ability to efficiently operate the Macintosh. (For example, from the AAC device having the option of using the keyboard to switch from a word processor to a mail program and back, instead of having to move the mouse to each application and click, or having to open MultiFinder and choose one or the other application using the mouse.) Therefore, as an experiment, this beta version of Macintosh SerialKeys (e.g., v0.93, Nov. 1995) has the ability to switch between processes if the "option + escape" keys are detected. The "option + escape" means to hold the option key pressed while also pressing the escape key. This only functions for keys received via SerialKeys (e.g., keys sent from the AAC device), not the standard keyboard.
To achieve this using an AAC device, the best method would be to have the "option" key pre-programmed as a special or modifier key as explained above in Step 7 (e.g., <esc> , hold , option .). If the user then selects the "option" key, followed by the escape key (e.g. <esc> escape.), the process switch would be activated. For those AAC users switching between a PC running Windows, and the Macintosh, the task switching sequence used in Windows (e.g., alternate + escape) should also work for process switching on the Macintosh, since the alternate key on a PC keyboard translates to the "option" key on the Macintosh Extended Keyboard. Therefore, users in this situation should not have to re-program their AAC devices.
Please note, that switching through the desktop (e.g., the Operating System and MultiFinder) requires that the "option + escape" sequence be pressed twice. Also note, that if only the Operating System, MultiFinder and SerialKeys are running, the process switching may not appear to function, since there is little to distinguish the Operating System from MultiFinder and SerialKeys prefers to remain a background application. Also note, that applications such as control panels, desk accessories, etc., may not be considered separate processes. In limited testing, the best results occur when other applications such as a mail program, a word processor, etc. are running.
Users and testers should note that this feature is not part of the GIDEI Standard, which is mentioned in Part I, Introduction to SerialKeys, above. Also, since this capability tends to go against the normal Macintosh User Interface, any feedback regarding this experimental feature is welcome and encouraged.
Contact Information
Mark Novak (608) 262-6966
Trace R&D Center (608) 262-8848 (fax)
1500 Highland Avenue (608) 263-5408 (tdd)
Madison, WI 53705-2280 menovak@facstaff.wisc.edu
A
alt
apostrophe
B
backslash
backspace
bslash
bspace
C
capslk
capslock
clear
cmd
comma
command
control
copy
ctrl
cut
D
del
delete
divide
dn
down
E
eight
end
enter
equal
esc
escape
F
f1
f10
f11
f12
f13
f14
f15
f2
f3
f4
f5
f6
f7
f8
f9
five
four
G
grave
H
help
home
hyphen
I
ins
insert
K
kp*
kp+
kp-
kp/
kp0
kp1
kp2
kp3
kp4
kp5
kp6
kp7
kp8
kp9
kp=
kpdel
kpdelete
kpdivide
kpdn
kpdown
kpdp
kpend
kpenter
kpequal
kphome
kpins
kpinsert
kpleft
kpmidl
kpminus
kpperiod
kppgdn
kppgup
kpplus
kpright
kpslash
kpstar
kptimes
kpup
L
lalt
lbracket
lcmd
lcommand
lcontrol
lctrl
left
lopenapple
loption
lshift
M
minus
multiply
N
nine
numlk
numlock
O
one
openapple
option
P
pagedown
pageup
paste
pause
period
pgdn
pgup
print
printscreen
prtscr
R
ralt
rbracket
rcmd
rcommand
rcontrol
rctrl
reset
ret
return
right
ropenapple
roption
rshift
S
scroll
scrolllock
semicolon
seven
shift
six
slash
space
T
tab
three
two
U
undo
up
Z
zero
Macintosh key names (partial list, from GIDEI2 Proposal)
