Tools: MPW
Advanced Search
Apple Developer Connection
Member Login Log In | Not a Member? Support

MPW Command Reference


StreamEdit

Tool

SYNTAX

StreamEdit file1 [file2]… [-d] [-e scriptString] [-o outputFile]
[-p] [-s scriptFile] [-set variable[=value]]

DESCRIPTION

The StreamEdit tool allows you to modify the text that appears in the specified files by executing one or more scripts (-e or -s) containing text-editing statements. Each statement is applied to each line of text and, after all the statements have been processed, the modified input files are written to either the specified output file or to standard output.

StreamEdit provides a set of commands that can be used in conjunction with address expressions in statements to perform text matching and editing. This makes StreamEdit useful for handling repetitive changes to source files, such as extracting comments or replacing one identifier with another.

Note
StreamEdit is a noninteractive text editor similar in function to the UNIX® tool sed. However, StreamEdit is not compatible with sed or awk. •

The StreamEdit tool operates by maintaining three text buffers:

The edit buffer holds the current line of input. Most StreamEdit expressions and commands operate on this buffer.

The insert buffer holds text to be printed before the current input line. This buffer is set by the Insert command.

The append buffer holds text to be printed after the current input line. This buffer is set by the Append command.

For each line of text in the input, StreamEdit does the following:

1. Reads the next input line into the edit buffer.

2. Evaluates each address expression in the script with respect to the contents of the edit buffer and, if the expression matches, executes the commands that follow the expression.

3. Concatenates the insert, edit, and append buffers and writes them to the output file after all address expressions have been evaluated.

StreamEdit has its own language for text manipulation. Each part of the language is explained in the special Statements section, which follows the Options section.

INPUT

Standard input, unless you specify one or more text files (file) on the command line.

OUTPUT

Standard output, unless you specify an output file using -o or -to file or -appendto file after a command.

STATUS

StreamEdit can return the following status codes:

0

no errors

1

syntax error

2

execution error

3

system error

Note
You can also specify a status code with the Exit command (see Commands in the Statements section). •

PARAMETERS

file1 [file2]…

Specifies one or more input text files to be modified. If you do not specify any input files, StreamEdit reads standard input.

In the following example the text from the Words text file is modified by the StreamEdit statements in the Alter script:

StreamEdit Words -s Alter

OPTIONS

Note that you can specify more than one script; scripts specified by -e or -s are concatenated in the order in which they appear on the command line. If you do not specify any scripts and do not specify -d, StreamEdit merely copies all of the input lines to output.

-d

Deletes all input lines, leaving only output from Print or Next commands. This filters the initial input lines so that they are not written to the output unless a Print or Next command explicitly prints.

-e scriptString

Reads text-editing statements from the script contained in the string scriptString. In addition to placing more than one statement in a single string, you can use -e more than once on a command line and can specify it in conjunction with -s.

Remember to use semicolons to separate multiple statements within the string and, as always, place quotation marks around the entire string argument to this option. Thus a command such as

StreamEdit -s script1 -d

where the script1 file contains

1 print 'ONE'
2 print 'TWO'

can be written using the -e option as

StreamEdit -e "1 Print 'ONE'; 2 Print 'TWO'" -d

-o outputFile

Directs output to the specified output file instead of standard output. StreamEdit does not produce any output until all the input is read. This allows you to specify one of the input files as the output file.

In the following example, StreamEdit reads the entire input file, Words, before writing text modifications to the same file:

StreamEdit Words -s Alter -o Words

-p

Writes progress information to diagnostic output.

-s scriptFile

Reads text-editing statements from a script contained in the file scriptFile. Statements within the script can be terminated by either a newline or a semicolon. This option can be used more than once and in conjunction with -e.

In the following example, StreamEdit executes the statements found in the MyScript script file.

StreamEdit -s MyScript

-set variable[=value]

Sets a variable's initial value before StreamEdit processes any statements. If you omit value, variable is set to the empty string.

STATEMENTS

Statements contain an address expression, which tells StreamEdit exactly what line or range of lines to modify, and one or more commands, which tell it what changes to perform on those lines. An optional "count" can also be specified which tells StreamEdit to only change the first "count" matches.

These statements have the syntax

addressExpression [-c count] command [;command]…

where addressExpression is an expression that uses line numbers and regular expressions to match one or more lines, and command is one of several StreamEdit commands, most of which accept a text argument. In a script file (-s), you can use newlines or semicolons to separate statements and commands. In a script string (-e) only the semicolon can be used. These syntax elements are covered in detail in Address Expressions, Text Arguments, and Commands below.

Note
Although semicolons usually indicate breaks (as in the MPW Shell) and are used to terminate commands, a semicolon located in the first column of a script line receives special treatment. When a semicolon is found in this position, StreamEdit considers the entire line to be a comment. This allows you to write StreamEdit scripts that also contain MPW Shell commands (see the second example in Examples). In all other cases, comments must begin with a # sign. •

 

Address Expressions

Every command must have a line address; however, a single address may be followed by a number of commands, terminating only when a new address is specified.

You can specify addresses numerically (by line number), by context matching using regular expressions, or by simple Boolean functions of the two. Boolean functions can be created by using numerical and regular expression addresses in conjunction with address operators.

The following list describes address operators in decreasing order of precedence:

(address)

Parentheses control precedence.

!address

The address expression must not match; this uses the NOT operator.

address&&address

Both address expressions must match; if the first address expression fails, the second address expression is not evaluated.

address||address

Either address expression may match; if the first address expression succeeds, the second address expression is not examined.

address,address

The comma operator matches the inclusive range of lines between the first address expression and the second address expression.

/regular expression/

The slash operator matches the text pattern defined by the regular expression.

For more information about regular expressions, refer to the chapter "Editing With Commands" in Introduction to MPW.

Note
Regular expressions supported by the MPW Shell have been extended for StreamEdit to include the following three:

ç (Option-C)

Causes a case-sensitive match of the text pattern indicated by the regular expression when ç is the first character in a regular expression. Normally, matches are not case sensitive.

When using a bullet character to indicate a match at the beginning of the line, the bullet must follow ç. In other words, you use

/ç•moo/

instead of

/•çmoo/ # Not This!

//

Indicates the last regular expression matched. Note that the last regular expression matched may be different from the most recent regular expression in a script due to the conditional evaluation that occurs with the && and || operators.

For instance, this expression replaces all occurrences of 1 or 2 with 3:

/1/ || /2/ Replace // "3" -c ∞

variable

Matches the value of the specified variable. This allows you to reference the value of a variable by enclosing the variable between ≤ and ≥ symbols. Notice the similarity to the way braces work in the MPW Shell.

The variable's value is not treated as a regular expression but as a string argument to be matched. An empty variable (that is, a null string) matches nothing. This is important to remember since variables are empty until they are set!

The following script sets the variable VAR to "moo" at the beginning of the text file and then prints all occurrences of "moo" to standard output:

• Set VAR "moo"
/≤VAR≥/ print

For a case-sensitive variable match you replace /≤VAR≥/ with /ç≤VAR≥/.

n

Matches input line number n.

• (Option-8)

Matches the point prior to the first input line, when specified alone.

∞ (Option-5)

Matches the point after the last input line, when specified alone.

$

Matches the last input line.

You can use these operators to create complex addresses. For example, this address will match any line containing either the text "trillian", or both "zaphod" and "beeblebrox" (in any order), as long as it is not found on line 42 of input:

(/trillian/ || /zaphod/ && /beeblebrox/) && !42

Although extremely useful, the range operator (the comma) can be tricky because it represents a two-state system rather than a continuous test for a condition. When the condition for the start of the range is met, source lines are considered to match up to and including the line at which the condition for the end of the range is met. If the first condition is not met, StreamEdit does not consider any lines in the range; if the last condition is not met, StreamEdit considers the succeeding lines to be a match forever.

For example, this address will match any line containing the word "Copyright" if it is found in one of the first ten lines of the input file:

(1,10) && /Copyright/

However, if the order is reversed, as in the following line, StreamEdit matches the range /Copyright/ && 1 to /Copyright/ && 10:

/Copyright/ && (1,10)

If "Copyright" does not appear on the tenth input line, the range stays active until the end of input. If it does not appear on the first input line, the range is not activated at all.

 

Text Arguments

Most commands accept text arguments that allow you to modify the input lines with specific text. These text arguments can take any of the following forms:

"string" or 'string'

Indicates a string. Special characters within the string are interpreted.

. (period)

Represents the current input line, minus its newline.

variable

Represents the contents of a variable. The StreamEdit tool considers variables to be names for strings and allows variable names that follow the same rules as case-ignored C identifiers (that is, [a-z_] [a-z0-9_]*). Remember that variables are empty until set.

®n

Specifies expression n, from the most recent regular expression match.

-from filename

Reads the next line of input from the specified file, minus any trailing newline. As with the options -to and -appendto, the filename argument can take any text argument form, including strings, variables, and ®-variables.

The StreamEdit tool can read any number of files, because it is not limited by the system's file control block (FCB) count, and can also read a file that is being written to without changing the read position.

When quoting special characters within strings and regular expressions, you use the backslash (\) or MPW Shell quote (∂) characters. You will often need to quote special characters for text-matching purposes, as in the Delete example in which StreamEdit searches for any number of spaces or tabs ( ∂t)* before a comment mark.

The quoted forms of other special characters and their meanings follow:

∂n or \n

Newline

∂t or \t

Tab

∂\ or \\

Backslash

∂∂ or \∂

MPW Shell quote

 

Commands

Commands can modify the current input line, set variables, print text, and affect the flow of control within scripts (that is, using the Next and Exit commands allows you to control the way in which StreamEdit processes scripts). Note that if the current input line is modified, subsequent address matches search for the modified line instead of the original input line.

As noted earlier, statements and individual commands can be separated by either semicolons or newlines. However, for scripts contained in a string (-e scriptString), statements and commands can be separated only by semicolons. In addition, although empty commands are allowed and do not produce any output, the first command following an address expression cannot be empty.

App[end] text [-n]

Appends the specified text to the append buffer, to be written at the end of the current cycle.

text

Specifies the text that is appended to the addressed line.

-n

Inhibits adding a newline. Normally StreamEdit always adds a newline after a change is made.

In the following example, the first statement causes the text argument to be appended to the last input line, while the second causes the text argument to be appended to line 42 without adding a newline:

$ Append "This follows the last line of input"

42 App "This is line 43==>" -n

Ch[ange] text [-n]

Replaces the contents of the edit buffer with the specified text.

text

Specifies the text that replaces the text at the addressed line. Omitting this parameter performs the same function as the Del[ete] command.

-n

Inhibits adding a newline. Normally StreamEdit always adds a newline after a change is made.

The first example changes line 42 of input to read "This is line 42", while the second changes the line containing the string "droid" to "These are not your droids!":

42 Change "This is line 42"

/droid/ Ch "These are not your droids!"

Del[ete]

Deletes the contents of the edit buffer. Executing Change without a text argument performs the same function.

The first example deletes every line of the input file, while the second deletes every MPW Shell comment it finds by searching for any number of spaces or tabs followed by a # character at the beginning of a line:

1,$ Delete

/•[ ∂t]*#/ Delete

Exit [status]

Stops processing and exits with the specified status code, where status is any number. If you do not specify status, StreamEdit exits with a default status of 0.

In the first example StreamEdit exits with a status of 42, while in the second example, it exits with a default status of 0 at line 3:

Exit 42

3 Exit

Ins[ert] text [-n]

Appends the specified text to the insert buffer, to be written at the end of the current cycle.

text

Specifies the text to be inserted at the line address.

-n

Inhibits adding a newline. Normally StreamEdit always adds a newline after a change is made.

In the following two examples, text is inserted at the beginning of the line addressed, but the trailing newline is discarded in the second:

1 Insert "This precedes the first line"

42 Ins "This is line 42==>" -n

Next

Stops the StreamEdit process and fetches the next input line immediately. (Normally, StreamEdit matches the current input line against each address expression in the script.) Note that the text prior to the line address is printed only once, regardless of the presence or absence of -d on the command line.

The following example would be useful if you want to ignore input until "moo" is found. When the script reaches this statement, the next line is retrieved and the lines ranging from line 1 to the line that contains the string "moo" can no longer be altered.

1,/moo/ Next

Opt[ion] Auto[Delete]

Deletes all input lines, leaving only the output from Print commands. This is useful for writing scripts to filter the input.

Appending the following command to the end of the script or specifying -d on the command line are equivalent in effect to using Opt[ion] AutoDelete.

/≈/ Delete

Pr[int] text [-appendto file] [-to file]

Prints the specified text to standard output or to a file specified by -to or -appendto. The text prints immediately, without modification from other commands at that address.

text

Specifies the text to be printed. If you do not specify any text, the contents of the current line prints without modification.

-appendto file

Appends the output of the Print command to the specified file. You can specify the output file file with a string, a variable, or an ® variable.

If the file does not already exist, StreamEdit creates it; however, if the filename is an empty string (that is, an uninitialized variable), nothing is printed.

-to file

Writes the output of the Print command to the specified file, truncating the file the first time it is written to. You can specify the output file file with a string, a variable, or an ® variable.

If the file does not already exist, StreamEdit creates it; however, if the filename is an empty string (that is, an uninitialized variable), nothing is printed.

The first example below prints lines 1-10 of the input file to standard output. The second example prints each string to a file of the same name as the string.

1,10 Print

/([a-z0-9]+)®1/ Print -to ®1

Rep[lace] [-c count] /pattern/ replacementText

Replaces the text described by the regular expression pattern in the edit buffer with replacementText.

/pattern/

Specifies the text to be replaced, using the pattern-matching capabilities of regular expressions.

replacementText

Specifies the string that replaces the text described by /pattern/.

Note
Unlike its behavior in other commands, an ® variable in the replacement text refers to the value set when StreamEdit processes the regular expression, /pattern/. It does not refer to an ® variable in the line address. •

-c count

Indicates the number of times /pattern/ is replaced in each matched input line by the new text, replacementText. You can specify the count with either a number or ∞ (infinite count).

The first example replaces the number 42 with the text "Meaning of Life" when it is found on line 42. The second example appears almost identical to the first, yet the number 42 is replaced by the contents of the variable MeaningOfLife. In the third example, the number 42 is replaced by "line 42".

42 Replace /42/ "Meaning of Life"

42 Replace /42/ MeaningOfLife

42 Replace /(42)®1/ "line "®1

Note
Be careful not to confuse variables (unquoted) with strings (quoted). Since variables are empty until they are set, accidentally specifying a variable instead of a string (for example, MeaningOfLife instead of "Meaning of Life") deletes the original text. •

Set variable text [-a] [-i]

Sets a variable's value to the specified string, without adding an automatic newline.

variable

Specifies the variable whose value is set to text.

text

Specifies the text string to which variable is set.

-a

Appends the text string, text, to the variable's current contents.

-i

Inserts the text string, text, before the variable's current contents.

The first example sets the Line variable to the contents of the current line and explicitly adds a newline. Note that the period (.) represents the current input line (see Text Arguments, earlier).

Set Line . "\n"

In the following example the text "Dirk Gently" is inserted before the other contents of the DouglasAdams variable.

Set -i DouglasAdams "Dirk Gently"

EXAMPLES

The following two examples demonstrate how you can use StreamEdit to solve text-editing problems.

 

Extracting the Leaf

It is sometimes necessary to extract the leaf part of a complete pathname in an MPW Shell script. You can do this by putting the following statement into a file named FilterLeaf :

StreamEdit -d -e '/(≈:)*([¬:]*)®1/ print ®1'

The regular expression /(≈:)*([¬:]*)®1/ sets ®1 to the part of the filename following the last colon, or to the whole filename if yours does not contain a colon.

To test the script you can type

Echo "The:I:Is:Silent:MyFile" | Filterleaf

This produces the output

MyFile

 

Generating Inline Function Declarations

This script generates MPW C or C++ inline function declarations from assembly-language source code. It automates the process of extracting object codes by taking advantage of the format of a listing file and makes the job quicker and less error prone than cutting and pasting. See Listing 1 below.

Note
The character ¥ in the assembly-language code comment marks the declaration. In principle, any unique character or string can be used. •

The StreamEdit script needs to generate inline declarations that look like this:

void pascal_string_copy(char* src, char* dest) =
{0x4CD7, 0x0300, 0x7000, 0x1011, 0x6002, 0x10D9, 0x51C8, 0xFFFC};

You can begin automating the process that generates this inline declaration by typing the script shown in Listing 2 into a new file named MakeCInline.

Listing 1: A typical assembly-language listing

MC680xx Assembler - Ver 3.2 18-Feb-91           Page 1
Copyright Apple Computer, Inc. 1984-1990

Loc   F Object Code  Addr  M  Source Statement

   ;+
   ; Inline Pascal string copy
   ;
   ;¥ void pascal_string_copy(char* src, char* dest);
   ;
   ;-

00000                         strcopy proc
00000   4CD7 0300                     movem.l  (SP), A0-A1
00004   7000                          moveq    #0, D0
00006   1011                          move.b   (A1), D0
00008   6002         0000C            bra.s    @2
0000A   10D9                  @loop:  move.b   (A1)+, (A0)+
0000C G 51C8 FFFC    0000A    @2:     dbra     D0, @loop
00010                                 endproc
                                      end
Elapsed time: 0.11 seconds.
Assembly complete - no errors found.  16 lines.

Listing 2: MakeCInline inline declaration

# MakeCInline--make C assembly language inline declarations
# Shell commands:--Assemble listing file,
# pass listing file to StreamEdit and delete listing file
; Asm "{1}" -l
; StreamEdit -d -s "`which "{0}"`" "{1}".lst
; Delete "{1}".lst "{1}".o
; Exit

# StreamEdit commands
# Print declaration line and inline format
  /;¥[ ∂t]*([¬;]*)®1/
  Print -n ®1 " =∂n{"
  Set PrecedingComma ""
  Delete
  /[ ∂t]ENDP/
  Print "};∂n"
  Delete

# Delete unnecessary lines, text before hex constant
!/•[0-9a-f]+ [g] [0-9a-f]«4»/ Delete
1,$ Replace /[0-9a-f]+ [g] / ""

# Print hex constant, preceded by standard 0x
/•([0-9a-f]«4»)®1 /
Print -n PrecedingComma "0x"®1
Set PrecedingComma ", "
Replace // ""

# Convert remaining words on line
/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

When you want to execute the script for any assembly-language source file use the following command line:

MakeCInline assembly.a > outputFile

MakeCInline writes the inline declarations from the assembly-language listing file to the specified output file.

How does this work? The MakeCInline script has two parts.

The first part of the script, "# Shell commands…", is purposely commented out from StreamEdit to force the MPW Shell to perform certain tasks. These commands are executed by the MPW Shell but are ignored by StreamEdit because the semicolons preceding each line signal comments to StreamEdit.

The MPW Shell first runs the assembler on the input source file and produces a listing file (see the Asm command):

;Asm "{1}" -l

StreamEdit then uses the listing file ("{1}".lst) as input for the currently executing script, {0}:

;StreamEdit -d -s "{0}" "{1}".lst

The following lines delete the listing and object files before exiting the MPW Shell portion of the script.

;Delete"{1}".lst" {1}".o
;Exit

The second part of the MakeCInline script consists of several sections of StreamEdit commands that extract the object codes from the assembly-language listing and put them in the appropriate format for an inline declaration.

The first of these sections, "# Print declaration line…", sets the format for the object codes. Its first task is to extract the declaration and copy it to output for the inline declaration.

The regular expression in the first line finds the inline declaration by virtue of the marker "¥", which carries over to the assembly-language listing. The following StreamEdit command line extracts the line containing a ¥, followed by any number of spaces or tabs, then by text possibly ending in a semicolon. The text, excluding a possible trailing semicolon, is stored in variable ®1.

/;¥[ ∂t]*([¬;]*)®1/

The StreamEdit code

Print -n ®1 " =∂n{"

prints the contents of ®1, and follows it with an equal sign, a newline, and an opening brace for the second line of the inline declaration.

Since the output will contain hex constants--which must be separated by commas--the final lines of the script use the PrecedingComma variable. The PrecedingComma variable is set initially to empty, so there will be no comma before the first object code. After ENDP is found the script prints a closing brace and a newline.

Set PrecedingComma ""
Delete
/[ ∂t]ENDP/
Print "};∂n"
Delete

So far, the script produces the following part of the inline declaration:

void pascal_string_copy(char* src, char* dest) =
{ };

The next section, "Delete unnecessary lines…", deletes unnecessary information from the listing file. Anything not containing object codes is deleted first; acceptable lines contain a hexadecimal constant starting in the first column, several spaces optionally followed by "G", and then at least one 2-byte constant.

# Delete unnecessary lines, text before hex constant
!/•[0-9a-f]+ [g] [0-9a-f]«4»/ Delete
1,$ Replace /[0-9a-f]+ [g] / ""

The section, "Print hex constant…" prints the first hexadecimal constant found at the beginning of a line, preceded by 0x and followed by a comma. At this point the PrecedingComma variable gets set to comma-space and printed along with the hexadecimal constant. The hexadecimal constant is then replaced with a null string, which effectively deletes it.

# Print hex constant, preceded by standard 0x
/•([0-9a-f]«4»)®1 /
Print -n PrecedingComma "0x"®1
Set PrecedingComma ", "
Replace // ""

Since the last hexadecimal constant was deleted, another can become a match and be extracted from the assembly-language listing.

The last sections, "Convert remaining…", are exact duplicates of the preceding section. They continue to delete unnecessary lines and text before the opcodes and then print them. StreamEdit has no control structures for looping so the code is repeated for up to four additional words on a line--the maximum number produced by the current assembler.

# Convert remaining words on line
/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

/•([0-9a-f]«4»)®1/
Print -n ", 0x"®1
Replace // " "

LIMITATIONS

Lines of more than 1000 characters are split without warning.

There is no easy way to do a Replace operation on a variable, since variables are used for pattern matches rather than addresses. (See the Replace command.)

The StreamEdit tool does not allow true expressions, so arithmetic expressions are impossible.

Conditionals and other control structures are not supported.

 
 


Last Updated July 2000