NetNews Usenet Archive 1992 #18/NN_1992_18.iso/spool/comp/lang/rexx/738

Text File | 1992-08-19 | 17.7 KB | 403 lines

Comments: Gated by [e-mail]
Path: sparky!uunet!charon.amdahl.com!pacbell.com!mips!darwin.sura.net!paladin.american.edu!auvm!DKNKURZ1.BITNET!RZOTTO
X-Acknowledge-To: <RZOTTO@DKNKURZ1>
Message-ID: <REXXLIST%92081920041037@DEARN>
Newsgroups: comp.lang.rexx
Date:         Wed, 19 Aug 1992 18:17:10 MEZ
Sender:       REXX Programming discussion list <[e-mail]>
From:         Otto Stolz <[e-mail]>
Subject:      Re: Capturing System Command Output (was: SH Backquote)
In-Reply-To:  Message of Tue,
              18 Aug 92 22:32:35 LCL from <[e-mail]>
Lines: 389

In article <[e-mail]> [e-mail]
(Steve Bacher) writes:
> [...] they all have different conventions for
> specifying how to route command output to the stack.

On Tue, 18 Aug 92 22:32:35 LCL Anders Christensen said:
> [...] The three different _conceptual_ methods that I am aware of are

Below, I'll suggest a fourth method. But first, I'd like to make a few
remarks on methods 1 through 3 of Anders' excellent article, including
proposals for variants of these methods. I'll try to quote only small
parts of the article; particularly, I'll not quote those parts I agree
with and have nothing to add to.

If you do not want to scrutinise all of my considerations and examples,
skip to a line containing the word Summary.

> (supposing that 'ls' is an opsys command that takes a file-spec as
> parameter, which may contain wildcards, and returns all matching
> filenames, one per line):

Apparently, Anders' examples assume that ls rather returns the filenames
one per word; I'll leave the ls examples that way (including my own
variants). In other examples, I'll use the actual conventions of the
respective environments; note that the one-per-line convention (while
permitting almost arbitrary items in the result) will render the examples
a bit more verbose.

> 1) CALLED AS A FUNCTION
>       files = 'ls'(directory)
>       say 'There are' words(files) 'files'
>       do i=1 while files \= ""
>          parse var files file files
>          say 'file number' i 'is' file
>          end
This would require that any command could be invoked as an external REXX
function, inducing possible name-conflicts between REXX functions and
system commands. Even if these conflicts are resolved by a visibility
rule (aka search order), the REXX programmer will have to consider the
system commands when choosing names for REXX functions (both internal and
external ones).

A less cumbersome variant would be to have *one* built-in REXX function
invoke *any* system command. This general routine would take a complete
system command as its argument, and yield its standard output, as in
        files = command("ls" directory)
        say "There are" words(files) "files:"
        do i = 1 while files ^= ""
           parse var files   file files
           say "File number" i "is" file
        end i

An actually implemented example of this scheme: In REXX under CMS,
CP commands may be invoked using the DIAG, or DIAGRC, function with
8 as its 1st, and the desired CP command as its 2nd (and optionally the
buffer length for the expected result as its 3rd) argument, as in:
        parse value diag(8, "QUERY FILES") ,
              with . r .
        if   datatype(r, "W")
        then r = format(r)
        else r = 0
        say "There are" r "files in your Virtual Reader:"

        nl = '15'x                     /* new-line character */
        parse value diag(8, "QUERY RDR ALL", (r+4)*81) ,
              with (nl) files          /* remove header line */
        say left("Id",4) left("Origin",8) left("Fname",8) left("Ftype",8)
        do while files ^= ""
           parse var files orig id . . . . . . . fn ft . (nl) files
           say left(id,4) left(orig,8) left(fn,8) left(ft,8)
        end /* files ^= "" */

An even more general variant would be a built-in function to invoke any
command in any environment. This function would take the environment, and
a suitable command, as its arguments, and yield the command's standard
output, as in
        files = invoke("command", "ls" directory)
        say "There are" words(files) "files."
        do i = 1 while files ^= ""
           parse var files   file files
           say "file number" i "is" file
        end i

        nl = '15'x
        parse value invoke("XEDIT", "QUERY RING") ,
              with  . r . (nl) files
        say "You are editing" r "files."
        do i = 1 while files ^= ""
           parse var files    file (nl) files
           say "file number" i "is" subword(file,1,5) subword(file,7,1)
        end i

You see, and appreciate, the uniformity (aka Principle of Least
Astonishment)? (Cf. my general remark on the remaining discrepancies,
at the end of this article).

> 2) EXTRA TERM IN COMMANDS [to] redirect the output [to] the stack:
>       'ls' directory ' >LIFO'     /* or "|rxstack" */
>       say 'There are' queued() 'files'
>       do i=1 while queued()>0
>          parse pull file
>          say 'file number' i 'is' file
>          end

This is actually implemented
- in OS/2 as "| rxstack",
- in Personal REXX as ">STK:",
- in CMS as "( LIFO", "( FIFO", or "( STACK" -- though only for
  particular commands,
- and in CMS Rel 7 (and older CMS Releases with CMS-Pipes added) as
  "PIPE cms" ... "| stack", as in
       "PIPE cms ls" directory "| stack".

Excursus:
As the stack is a global resource, it has to be used with care: every
program should take pains to leave the stack in the state expected by
other programs. Particularly, a program that uses the stack for its own,
internal, purposes (as the program fragment above) should leave the
stack unaltered. Hence, I always recommend the following "stack dis-
cipline" for this sort of programs:
        stack_level = queued()
        "ls" directory " >LIFO"     /* or "|rxstack" */
        say "There are" queued() - stack_level "files"
        do i = 1 to queued() - stack_level
           parse pull file
           say "file number" i "is" file
        end i

>    This is, of course, closely related to how it is done under CMS,

Minor correction:
It used to be that way until (around Release 4) CMS commands learned to
use the EXECCOMM interface, cf. method 4, infra.

>    where commands have options to flush the output to the stack, and
>    where EXECIO can be used as a general purpose interface to the
>    stack for operations that does not know about the stack.

Correction:
EXECIO does this only for CP commands. There used to be no solution
(short of re-reading the console output bound for the printer) for those
CMS commands that neither deliver their results on the stack nor via
EXECCOMM. Now, CMS-Pipes does the trick, uniformly, for all sorts of
commands.

> 3) READING COMMANDS WITH LINEIN.  A command is started with a builtin
>    function, and the output is read using the LINEIN() function.
>       rcode = popen( "ls" directory,, "ls-output" )
>       say 'There are' lines('ls-output') 'files'
>       do i=1 while lines() \= 0
>          say 'file number' i 'is' linein('ls-output')
>          end

How will the interaction between the REXX program and its environment
(or environments) be handled, in this scheme? Note that the particular
strength of REXX lies in this very interaction: REXX programs issue
commands depending on the results of preceding ones. Take the recent FTP
thread as a vivid example: you would like to proceed only when the con-
nection could be established properly, or you would like your REXX pro-
gram to query the contents of some directory then decide what files to
transfer. Hence, we need a particularly safe, convenient, and efficient
scheme for this sort of interaction.

To enhance the preceding example, let fs. be a family of REXX variables
containing a sequence of file-specs (possibly containing wildcards), and
let us try to list all files matching those specs.

Will we have to open and close the stream for every command to be issued?
This would naturally provide for multiple-line responses, but it would
probably be horribly inefficient. Example of the possible syntax:
        files = ""
        do i = 1 to fs.0
           call popen "ls" fs.i, "ls-output"
           files = files linein("ls-output")
           call pclose "ls-output"
        end i
        say "There are" words(files) "files."
        do i = 1 to words(files)
           say "file number" i "is" word(files,i)
        end

Another approach would be to open, and close, the stream once and for
all. Then we would need another built-in function, say pcommand, to issue
subsequent commands, as in:
        call popen "ls" fs.1, "ls-output"
        files = linein("ls-output")
        do i = 2 to fs.0
           call pcommand "ls" fs.i, "ls-output"
           files = files linein("ls-output")
        end i
        call pclose "ls-output"
        say "There are" words(files) "files."
        do i = 1 to words(files)
           say "file number" i "is" word(files,i)
        end

But then, why invent so many new functions, at all? Let us issue the
commands in the conventional, REXXish, way via the address statement!
All we need for method 3 is a connection between the standard-output
(generated by some environment) and some input stream. Most naturally,
the input stream and the environment could be associated via a common
name. However, we cannot unconditionally bind the command output to
a REXX input stream, as this would invalidate existing REXX programs,
viz. those invoking a command in order to have its output presented
directly to the user, as in
           address command "ls" directory

Hence, I propose to bind the command output explicitely to the cor-
responding REXX input stream. We could achieve this via an enhanced
address statement, e.g. a new keyword, separated by a comma. With this
proposal implemented, two of the above examples would burn down to:
        files = ""
        do i = 1 to fs.0
           address command "ls" fs.i, Linein /* explicit address stmt. */
           files = files linein("command")
        end i
        say "There are" words(files) "files."
        do i = 1 to words(files)
           say "file number" i "is" word(files,i)
        end

        address "XEDIT", Linenin
        "QUERY RING"  /* cf. supra, method 1 */    /* implicit address */
        parse value linein("XEDIT") ,
              with  . r .
        say "You are editing" r "files."
        do i = 1 while files ^= ""
           file = linein("XEDIT")
           say "file number" i "is" subword(file,1,5) subword(file,7,1)
        end i

The basic idea is an explicit binding of the command output (from any
environment) to a linein-stream; the particular syntax is negotiable :-)

> No doubt, there are several conceptual additional methods ... anyone?

4) Exploit the Variable Pool Interface:
   A command invoked via the usual (explicit or implicit) address in-
   struction will deliver its results to a REXX stem or variable.

   Section 5 of Cowlishaw's "The REXX Language" (TRL) states that a
   command to an external environment "may have side-effects". One con-
   ceivable side effect (though not explicitely deemed so in TRL) would
   be the setting of a REXX variable, such as RC which is set in any
   case.

   The official IBM definitions of REXX,
   - SC26-4358 "SAA CPI Procedures Language Reference", and
   - SC26-5549 "SAA CPI Procedures Language Level 2 Reference",
   carry this concept one step further: under the heading of
   "Variable Pool Interface", they state that every SAA conforming REXX
   implementation "provides an interface whereby called commands can
   easily access and manipulate the current generation of Procedure
   Language [i.e. REXX] variables".

   In CMS, this interface is called EXECCOMM (cf. SC24-5239-2, p 159).
   In TSO/E, it is called IRXEXCOM (cf. SC26-1883-0, p. 240). In OS/2,
   it is called Variable Pool Interface (cf. OS/2 Procedures Language
   Reference, Chapter on "Applications Programming Interface" -- I have
   neither the IBM form number nor the exact page available; it should
   be somewhere around page 180). In Personal REXX (non-IBM product), it
   is called Shared Variable Interface (cf. "Personal REXX User's Guide"
   Ver 1.60, p. 145 and p. 162). I don't know about possible Variable
   Pool Interfaces in other REXX implementations. Any volunteers to
   tell us?

   In CMS, this interface has been used in several ways that may serve
   as general examples.

4.1) Implicit REXX variable:
   Back in CMS Rel 3, the XEDIT learned to set REXX variables.
   One of the above examples would rather be programmed as:
        address XEDIT "COMMAND EXTRACT /RING/"
        say "You are editing" ring.1 "files."
        do i = 2 to ring.0
           say "file number" i-1                                    ,
               "is"          subword(ring.i,1,5) subword(ring.i,7,1)
        end i

   The result of the EXTRACT command is placed in the (implicitly
   chosen) variables ring.0, ring.1, ring.2, etc.

   In CMS Rel 5, particular commands (PARSECMD, XMITMSG), designed for
   close interaction with REXX code, were added. They deliver their
   results in the MESSAGE., CODE., and TOKEN., stems. I gave several
   examples in the proceedings of the SHARE Europe Spring Meeting
   1990, Volume 1, pages 197 through 224, which I won't repeat here.
   Whoever is interested in those particular commands may ask privately
   for a copy of my presentation.

4.2) Arbitrary, explicitely specified, variable:
   In CMS Rel 4, the EXECIO command was enhanced to deliver its results
   in an arbitrary REXX stem, or variable. CMS-Pipes (CMS 7, or add-on
   to lower releases) can deliver results from any command in an arbi-
   tray REXX stem or variable. Example:
        address command "PIPE cms ls" directory | var files"
        say "There are" words(files) "files:"
        do i = 1 while files ^= ""
           parse var files   file files
           say "File number" i "is" file
        end i

Summary of conceivable methods to access command output:

1. Command output is yielded as a function result.
1.1 System command verb is used as REXX function name;
    command argument string is given as REXX function argument.
1.2 Built-in REXX function invokes system command;
    entire command is given as REXX function argument.
1.3 Built-in function invokes command in any environment;
    environment and entire command, are given as REXX function arguments.

2. Command output is delivered on program stack.
2.1 Command syntax of environment allows
    to specify stack as a target for command output.

3. Command output is delivered to a REXX input stream
   (to be read via LINEIN).
3.1 Built-in function issues system command,
    and binds its output to a particular REXX input stream;
    entire command, and stream-id, are given as REXX function arguments.
3.2 Built-in function issues 1st system command
    and binds system command's output to particular REXX input stream;
    entire command, and stream-id, are given as REXX function arguments.
    Another built-in function to issue subsequent commands,
    while preserving the output binding;
    entire command, is given as REXX function arguments.
3.3 Explicit binding of REXX command environment
    to REXX input stream.

4. Command output is delivered to REXX variables.
4.1 Implicit REXX stem or variable, depending on command issued.
4.2 Command syntax allows to specify target stem or variable.

I'd prefer methods 4.2, 1.3, 2.1 and 3.3, in this order.

In article <[e-mail]> [e-mail]
(Steve Bacher) writes:
> The best approach available to date, hinted at by one of the posts, is
> to write a general-purpose exec that will interrogate the type of REXX
> implementation environment and do the appropriate thing.

On Tue, 18 Aug 92 22:32:35 LCL Anders Christensen said:
> This is not an acceptable permanent solution, since any program is
> outdated the very moment a new rexx interpreter is introduced.

Not quite so: Only Steve's general-purpose exec (i.e. external REXX
function) would be outdated; when it is amended to cover the new imple-
mentation, all programs would be valid again. You may view that general-
purpose exec as a private implementation of either method 1.2 or method
1.3, above.

> A better solution might be to label the _methods_ rather than the
> _interpreters_, then it would be possible to check whether an
> interpreter supports e.g. the popen-based method.

This would require an amendment to the REXX standard and to current REXX
implementations, while Steve's proposal can be immediately implemented
by any REXX programmer, exploiting the existing Parse Source and Parse
Version statements.

Remaining discrepancies:
Even if one or several of the methods outlined above would become
standard, REXX programs could not be more portable than their environ-
ments would allow them to be! (viz. virtually not portable :-( )

As a simple example, consider the syntax of file identifiers in various
systems: In CMS, it is 3 words (but only for a disk file), in MS-DOS
it is one word, possibly containing a colon and several backslashes, in
Unix you have slashes instead, etc etc. Some of these discrepancies are
more than mere syntactic idiosyncracies: they reflect different file
system concepts, requiring different approaches and algorithms to deal
with. How can a REXX program be portable between such diverging environ-
ments?

Moreover, the desired results from the command may be separated in
various ways (by blank space, by line-breaks, by syntactic marks), or
imbedded in irrelevant information.

The only way for the REXX programmer to strive for portability is to
invent his or her own, system independend, set of primitives, implement
them seperately for every system the programs are intended for, and use
them in the rest of the programs in a system independend way.

Thank you for reading this far (385 lines).

Best wishes,
                    Otto Stolz <[e-mail]>
                               <[e-mail]>