home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Black Box 4
/
BlackBox.cdr
/
progc
/
alib12.arj
/
CONSULT.DOC
< prev
next >
Wrap
Text File
|
1992-03-30
|
17KB
|
442 lines
CONSULT.DOC
Contents
Consulting/Contracting Advertisement . . . . . . . . . . . . . . 1
Consulting/Contracting Advertisement
I am interested in creating an automated test suite for your product
using this tool. I have developed the following flow chart to assist
you on your decision to hire me. Your answers must be 'yes' for a
good fit. However, before answering, please read the detailed
information on each question, which follows this flow chart.
Also, if you need more help in determining the answers to these
questions, contact me.
--------------------------
- QUESTION 1: -
- Are you interested -
- in hiring a contractor -
- to write an -
- automated test suite? -
--------------------------
|
| Yes
|
V
-------------------------
- QUESTION 2: -
- Is it possible -
- to test your product -
- using the AutoLibrary -
- Program-Playback tool?-
-------------------------
|
| Yes
|
V
------------------------
- QUESTION 3: -
- Do you want your -
- test suite written -
- in the C programming -
- language? -
------------------------
| Yes (Go to next page)
1
|
|
V
------------------------
- QUESTION 4: -
- Will I be able to -
- test your product -
- from my home office? -
------------------------
|
| Yes
|
V
--------------------
- QUESTION 5: -
- Will you benefit -
- by having your -
- tests automated? -
--------------------
|
| Yes
|
V
-----------------------
- QUESTION 6: -
- Does your company -
- and management -
- support the idea of -
- test automation? -
-----------------------
|
| Yes
|
V
-------------------------
- You may want to hire -
- me to write an -
- automated test suite -
- for your product. -
- (Please read the -
- following text.) -
-------------------------
If you are interested, please give me a call or send email. There is
no charge for initial consultations. My qualifications include a
degree in Computer Science from Washington State University and
industry experience since 1983. I am an experienced user of MS-DOS
and the Unix operating system.
2
My industry experience includes developing an automated network access
program, developing an automated test suite for a target debug
monitor, developing 2 real-time operating systems, and sustaining a
parallel operating system.
I have been working on automation in one form or another since 1985
and on the AutoLibrary(tm) Program-Playback tool since 1989.
I hope you decide to give me a call.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
QUESTION 1:
Are you interested in hiring a contractor to write an automated test
suite?
Before answering this question, please consider some of the advantages
of having an independent contractor test your product.
If your answer is yes, great, please continue the flow chart.
If your answer is no, the flow chart may still help if you are making
a decision on using this tool in your test automation plans.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
QUESTION 2:
Is it possible to test your product using the AutoLibrary(tm) Program-
Playback tool?
The answer to this question could very well be no, since the
AutoLibrary(tm) Program-Playback tool was written to address a
specific type of testing. The following survey will allow you to
3
determine if your testing requirements fit within the parameters of
this tool.
A) Does your product use serial communication? Yes ___ No ___
Your product must be testable via some type of serial connection.
RS-232 is supported directly and is quite common. Other
protocols, like RS-422 and IEEE-488, can be supported by
modifying the interface routines. The interface routines can
also be modified to support networks. See the documentation for
CommInterfaceOpen, CommInterfaceClose, CommInterfaceRead, and
CommInterfaceSend in this manual for details of supporting other
communication protocols.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
B) Does your product have a human interface? Yes ___ No ___
Does your product have a command line prompt? Yes ___ No ___
Human interfaces wait for commands to be typed in at the command
prompt. The response to commands are then displayed. I have
split this question into two parts because it is possible to
implement a command line interface that is meant for other
computers instead of for humans. This tool will work fine for
both types.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
C) Can your product be tested *by* an IBM PC or compatible?
Yes ___ No ___
The key word here is *by*, since, the system under test cannot be
the PC that is executing the AutoLibrary(tm) automation program.
4
The system under test is connected to a PC via a serial port.
The following list of questions will illustrate this point.
a) Can a modem be tested using this tool?
Yes. The PC can be connected to the modem via a serial
port. A test program can be written using the
AutoLibrary(tm) Program-Playback tool. Such a program would
send commands to the modem and capture expected responses.
The captured responses would then be compared to expected
responses.
b) Can an ASCII terminal be tested using this tool?
Yes. The PC can be connected to the terminal via a serial
port. Most terminals have host commands (ESC sequences)
that allow the host to change the operating features of the
terminal. An AutoLibrary(tm) automation program could be
written to test these features. (The Wyse demo, in the
self-extracting file ex02.com, does this.)
For terminals, it is not possible to automatically type in
characters. However, it is possible to use this tool to
semi-automate typed commands. For example, the automation
program could prompt a user on what to type in. Then the
automation program could verify that the terminal responded
correctly. The automation program would also track what
features have been tested and remind the operator on the
remaining tests to be run.
c) Can special equipment, such as medical instruments, be
tested using this tool?
Yes, provided that there is some way to communicate to the
instrument via a serial connection. Many times a special
RS-232 port is designed in specifically for test automation.
Note, however, that the serial connection does not have to
be RS-232. Interface routines can be written to communicate
to any protocol.
d) Is it possible to test programs that run on operating
systems like DOS and Unix? What if the programs use a
mouse?
Mostly Yes. This tool works great for Unix and other
operating systems, like VMS, that support serial connections
via terminals. Your automation program, which runs on a PC,
would connect to one of the Unix operating systems serial
5
ports. Unix would just see the connection as another user
and would not be aware of the automation.
This tool cannot test DOS programs, since it is a DOS
program itself. However, with two PC's it may be possible.
The first PC would execute the program to be tested and the
other PC would have the automation program. Remote software
(like pcAnywhere) would be used to redirect all input and
output through a serial connection. I have not tried this
technique at the time of this printing. However, given my
understanding of how remote software tools work, I believe
it would work fine. The key is to somehow get all input and
output flowing through a serial connection from a PC.
I haven't tried this tool with a mouse. However, given my
understanding of a serial mouse, automation with this tool
is possible. Your automation program would open two
AutoLibrary(tm) automation ports, one for typed commands and
the other for mouse commands. The mouse port would send
mouse commands and scan for responses just like the command
port.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
QUESTION 3:
Do you want your test suite written in the C programming language?
C programs, using this tool, are written to automatically send
commands to the system under test. The automation program then
captures and compares the results to expected results. Automation
sequences that are common can be written as a subroutine. Formal
parameters can be added to subroutines to accommodate variations in
the testing environment. Automation programs read like real C
programs: with subroutines, variables, data structures, flow control,
and file I/O. Also, automation programs written with this tool have
the additional advantage of using the C libraries and commercial add-
on libraries. All of this allows automation programs to be flexible
and effective.
The AutoLibrary(tm) Program-Playback tool is not a capture-playback
tool.
6
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
QUESTION 4:
Will I be able to test your product from my home office?
This answer must be yes.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
QUESTION 5:
Will you benefit by having your tests automated?
Since test automation is expensive, you need to justify this cost with
future benefits. The following list of questions will help you
decide.
a) Will an automated test suite help you with future releases
of the software?
You must compare the cost of developing the automated test
suite against the cost of running the regression tests
manually over the life of the software.
If you plan to sustain the software and produce updates with
enhancements and bug fixes, and you are concerned about new
enhancements and fixes potentially breaking something that
once worked, an automated regression test suite would be
helpful.
7
b) Do you have a requirement to supply each customer with an
acceptance test?
If you have a requirement to supply your product with an
acceptance test, you may want an automated acceptance test
to document the features being tested. An automated
acceptance test will give your product a professional look.
You can even provide your customers with the source code to
the acceptance test for their scrutiny.
c) Does you product need stress testing?
You may have a product that needs stress testing. An
automation program will be able to send command sequences
faster than humans and be able to simulate many users.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
QUESTION 6:
Does your company and management support the idea of test automation?
This is an important question to analyze honesty. I have found that
*everyone* likes the idea of test automation. However, I have also
found that most people don't understand the cost of automating test
suites. Automation requires significant time and effort to create.
Then, once created, automated test suites need to be sustained along
with the software. So, there is a real cost to develop automated test
suites and real costs to maintaining them. Everyone must agree that
this cost is outweighed by the benefits gained.
Notes _____________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
8