AmigActive 15

home *** CD-ROM | disk | FTP | other *** search

/ AmigActive 15 / AACD15.ISO / AACD / Programming / SED / SED.doc < prev next >

Wrap

Text File | 2000-10-15 | 18.0 KB | 466 lines

Amiga SED An Amiga Stream Editor © THOR-Software (Thomas Richter) ______________________________________________________________________________ Purpose of this program: SED takes an input file, checks each line of this file against a pattern supplied on the command line, and generates a new line from this pattern match in the destination. This could either mean that the matching line is removed completely from the output, replaced by a different line, or changed according to the specifications of SED. SED is an approximation of the Unix "stream editor" sed. It is not quite as powerful as sed because its command set is currently very limited, and it does not support command files. Its pattern syntax is different, too. It still looks like "line noise" to me - I didn't want to break with this tradition - but it's at least the Amiga kind of line noise. Its pattern matching rules are a superset of the AmigaOs patterns, with some additional features like "captured expressions" and more powerful "character classes" and "escaping". SED is useful for automatic processing of text files, e.g. the modi- fication of the startup-sequence. SED can also be run as a "filter" in which case it reads its input from stdin and prints output to stdout. Combine this feature with pipes and you get a very powerful text processing tool. A warning: Pattern matching looks simple, but is full of hard to gasp traps. This tool is therefore thought to be for "expert usage". In case you think SED doesn't process your pattern correctly, think twice! ______________________________________________________________________________ SYNOPSIS: SED FROM,TO,MATCH/A,REPLACE,CHANGE,DELETE/S,USECASE/S,ALL/S,VERBOSE/S FROM An (AmigaOs) pattern specifying the input file(s) to process. If not given, SED reads from the standard input. TO The output file where to write the processed lines to. If not given, SED writes to the standard output. MATCH A pattern specification used for filtering the input lines. More on the pattern rules below. The next options specify what to do with the lines matching the pattern: REPLACE Replace matching lines in the input by this replace rule, do not write non-matching lines to the output. The replacement rules are given below. CHANGE Replace matching lines by the replace rule given by this ex- presson. In contrast to REPLACE, non-matching lines are placed in the destination without change. Useful for modifying a file according to a pattern rule. DELETE Print all lines except those matching the pattern. This effectively removes the matching lines from the input file. USECASE Be case-sensitive. By default, SED is case-insensitive. Note that SED differs in this detail from the Un*x sed. ALL In case the FROM pattern is a wildcard, enter sub-directories recursively. VERBOSE Print information about the file currently scanned, and upon entering a directory. By default, SED is quiet. Note that "Search" is by default not quiet. ______________________________________________________________________________ Pattern specification: In the following, the syntax of the patterns is specified. By good tradition, this is in-comprehensively. I present first a "quick and dirty" presentation of the available patterns as a quick reference which might give you an impression about the possibilities. It is all but sufficient to work with SED. Then, a detailed and more precise, but also more confusing presentation follows. ______________________________________________________________________________ Quick guide to patterns: SED patterns work much like Amiga patterns. Unlike in "Search", a pattern is applied to a FULL line, and not to sub-strings of this line. Which means that the pattern "hello" matches ONLY the line containing the single word "hello", nothing more. If you want to match lines containing "hello", use "#?hello#?" instead - see below for what "#?" means. Unlike Un*x sed, there are no special characters to match the start or the end of a line. They are not required in the SED approach. Standard patterns: ? Matches a single arbitrary character. # Matches zero or more repetitions of the following symbol in the AmigaOs sense. Note that # may match zero(!) characters as well. Therefore, #? matches an arbitrary sequence of at least zero characters, hence any string. + Matches one or more repetitions of the following symbol New to AmigaOs, standard Un*x regular expression. * Matches zero or more arbitrary characters in the sense of MS-DOS. Note that this is a functional difference to Un*x regular expressions where * has the meaning of #. Note that you need to write ** instead of * if you enclose the pattern in double quotes on the shell command line. This is because * is also the BCPL escape character. Messy. (...) Groups the characters in the bracket to a single symbol. As for example, #(ab) would match an arbitrary repetition of "ab", as the empty string, "ab", "abab" or "ababab", but not "aba". Brackets can be nested. (..|..) The vertical bar means "or". Matches either the left or the right string. The bar is only valid within brackets. {...} Groups expressions much like (...) but captures the contents of the sub string that matched the brackets. This captured expression is then available for the ouput replacement rules, see below for more information. For example, SED MATCH {#?}.c REPLACE {1}.o would match all lines ending on ".c", and would capture the string in front of the ".c". The "{1}" in the replace pattern would insert this string, and would append an ".o". Namely, the above replaces all lines ending on ".c" by a similar line ending by ".o". Works very much line the Un*x $..$ matching. {..|..} The vertical bar works right the same way here as described above. Matches either the left or the right expression, and captures the expression that fits. % Matches the empty string. Useful for patterns like "#?(.c|.o|%)" which could be used to match the source, the object code and the final executable of a C project, for example. ~ Means "not" and matches all symbols that do not match the following symbol. Be warned, ~ is full of traps, see below for the full description. [..] Character classes. Matches a single character on a range of valid characters specified in the interior of the bracket. For example, "[ac]" would match the single character "a" or "c". [..|..] Matches either the left or the right character range. Hence, [a|c] is equivalent to [ac]. [..,..] Another equivalent formulation of the above. [a,c] is the same as [ac] or [a|c]. [..-..] Matches a character range. [a-z] matches all letters - except language specific "Umlaute", though, which have different en- codings. Several ranges can be grouped much the same way as single characters. [a-z|0-9] means "any character or any digit" but nothing else. [-..] Matches all characters up to the specified character. Hence, [-z] means "all characters up to z". Note that unlike in Un*x implementations, there are no messy rules concering the "[" itself as character. The escape character "\" must be used to specify "[" or "]" itself, see below. This syntax can be combined freely with "|" or "," to specify more than one range. [..-] Matches all characters starting at the given ASCII value. Can be combined freely with "," and "|". There are no messy rules concerning "-" in the middle or the end of a character range, proper escaping must be used if "]" should be matched. [a-] matches therefore all characters "a" and up. [~..] Matches all characters not in the following range. ~ is applied up to the next "|" or ",", unlike in the standard AmigaOs (Arp) expression matching. Therefore, [~ab] matches all characters except "a" and "b" and is equivalent to [~a,~b] and [~a|~b]. \ Escape character. Specifies a character to be matched: \t Tabulator \v Vertical TAB \b Backspace \r CR \f Form Feed \a Bell \n is INVALID since the end of the line is matched by the end of the pattern itself. \x.. The character encoded by the hex value following the "x". In case this specification is ambigious, the number might be terminated by a dot ".". Hence, "\x9.0" matches a tabulator sign and the digit "0", whereas "\x90" matches the ASCII char- acter of the code hex 90. Note that this rule differs from the ANSI-C rule. \0.. The character of the ASCII code encoded as an octal number. The dot is used as above as separator, unlike in ANSI-C. \$.. Identical to \x.., matches the digit encoded by the ASCII code in hex. \d Matches the dollar sign since \$ has a different meaning already. \#.. Matches the character encoded by the ASCII code in decimal notation. \h Matches the hash-mark since \# has a different meaning already. Everything else: The character following the backslash itself. Especially, \\ is the backslash itself and \" is the double quote. Note that you must use the backslash to match characters which are otherwise part of the pattern syntax, as for example "\(" to match the bracket. Note that "#" and "$" are special in this sense since "\$" and "\#" are used to specify characters by ASCII code. !,",§,$,&,= -,^,',`,<,> are reserved for future use AND MUST NOT be used at all. Escape them if you need them. However, the dot (".") is free, unlike Un*x regexp, same goes for "@" and "/". .. Everything else: Matches the character itself. Hence "a" matches a single "a" much like "[a]". ______________________________________________________________________________ Replacement rules: The arguments of REPLACE and CHANGE specify what do with the lines which matched the specified pattern. Unlike the pattern specification, only the special operators \ and {..} are allowed. All other operators from the above list are forbidden and generate an error. \ Escape character, works identically to the \ in the pattern and places the single character encoded by the sequence following the backslash on the output directly. {..} Specifies a captured expression to be inserted into the output stream. The brackets take up to three arguments: The couting number of the regular expression, and optionally two arguments how to format the regular expression separated by a dot ".". These numbers work very much the same way like the arguments to the %s format specifier in ANSI-C. The first number in the bracket describes which captured expression to insert. If it is a positive number, the number is simply the index of the captured expression, counting from one upwards. Each opening bracket "{" in the input pattern starts a new captured expression, hence in nested expressions the other- most bracket has the lowest index. If this number is negative, it counts the captured ex- pressions downwards from the last expression. If the specified expression does not exist, the brackets expand into an empty string that is formatted according to the rules given by the next three arguments. {1} is the first captured expression, {3} is the third expression, {-1} is the last expression, {-2} is the second to last expression. The next number is the field with to print the captured ex- pression in. At least the specified number of characters are printed, or more if the expression is longer. If the ex- pression is too short, the field is padded with blank spaces. The expression is right-justified into this field, unless the field width is negative in which case the expression is left-justified. The sign of the field width is otherwise ignored. Defaults to 0, i.e. the field is always as small as possible. The last number is the size limit of the expression. The expression will be cut down if it is longer than the specified limit. SED will cut the end of the string if this argument is positive, or the start of the string if it is negative. The sign of the limit is otherwise ignored. If the limit is 0, which is the default, the expression will not be cut down at all. {1.10} is the first captured expression right justified in a field of ten characters or longer. {2.-5.7}is the second captured expression, left justified in a field of five characters. At most seven characters of the expression will be printed. .. Everything else: The character itself is printed on the ouput. ______________________________________________________________________________ Detailed pattern matching rules: And now for the detailed rules to confuse you completely: - A SYMBOL is either a single character, one of the following operators followed by its arguments, a character class [..] or a (..) or {..} sequence. - A PATTERN is a sequence of SYMBOLs. - The POSTFIX of a symbol in a pattern is the subsequence of the pattern following the symbol, not including the argument of the symbol itself. ? Matches a single character except the end of a string. # Matches as many repetitions of the following symbol, but at least zero such that the postfix of the symbol matches the postfix of the input. Hence, "#" is greedy. There is currently no non-greedy form. + Matches as many repetitions of the following symbol but at least one such that the postfix of the symbol matches the postfix of the input. "+" is greedy. * is fully equivalent to "#?" and therefore greedy. (...) groups the pattern up to the next | or ) into a symbol which matches if the contents of the brackets match. (..|..) An or-combined symbol matches if one of the expressions in the bracket match such that the postfix symbol matches the postfix of the input. {...},{..|..} Similar to the above except that the matched string is captured. % Is completely ignored as pattern and gobbles no character from the input sequence at all. ~ Matches the longest subsequence or at least zero characters that does not match the following symbol such that the postfix of the symbol still matches the postfix of the input. "~" is greedy and will try to match as many characters first. Note that a symbol could either be a single character or a sequence of characters grouped by () or #. Since a single character cannot match a string larger or smaller than one character, ~ followed by a one-character symbol will match all subsequences except those whose postfix either don't match the postfix of the character, or which match the character and the postfix. This is *very* tricky and you should think about the con- sequences of this rule twice. More examples below. [..] Character classes. Groups a range of characters into a symbol that matches exactly a single character, but never the empty string. ~ in character classes is special: If there is a not-sequence in a character class, it matches if all not sequence match at once and one of the normal sequences match. Hence [~p,~q,a-z] matches all letters except p and q. .. Everything else matches exactly the the one character that it represents. They will not match the empty string. ______________________________________________________________________________ Some examples of patterns to think of: % Matches only empty lines in the input. ~% Matches only non-empty lines in the input. #?.c Matches all lines ending on ".c" The#? Matches all lines starting with "The". #?Example#? Matches all lines containing the word "Example". Example#? Matches all lines starting with the word "Example". Example Matches all lines consisting of the single word "Example". #? Matches all lines. #?(.c|.o|%) Matches all lines (think about why!). foo(.c|.o|%) Matches all lines consisting entirely of the word "foo", "foo.c" or "foo.o". foo(.c|.o|) Just the same. ~(Example) Matches all lines except the line consisting of the single word "Example". ~(#?Example#?) Matches all lines that do not contain the word "Example". ~(ab)cd Matches all lines that do not start with "ab" and that end on "cd". Especially, this would match "bccd". It would also match the line "cd" since "ab" does not match the empty sequence in front of "cd". (think about this!) ~#a.c Matches all lines ending by ".c" except those where the ".c" is prefixed by an arbitrary number of a's, including zero a's. Hence, it would match "bc.c" and even "ab.c", but not "a.c" or ".c" as the last consists of zero a's and one ".c". It would not match "bc.o". This is identically to ~(#a).c since # binds the following a. ~(#ab)#? Matches all lines except those starting with a possibly empty sequence of a's followed by a single b. Hence, does not match aaabccc or bccc. ~(#[ ,\t];)#? Matches all lines except those starting with a possibly empty sequence of blanks or tabulators followed by a colon. Hence, for a shell script, this would match all non-comment lines. ~(#[ ,\t];)if#? This is a tricky one. Unlike what you might think, this does not match all non-comment lines starting with if. It also matches lines starting with a semicolon provided the string "if" is in the line and not directly behind the semicolon. Note that this is the intended behaivour. For example, it would match ;aifb The reason is simple: This is a string ";a" that does not match the symbol #[ ,\t]; followed by "ifb" which matches if#?. What you want here instead is #[ ,\t]if#? which matches all if-lines with additional, at least zero, blanks or tabs in front. The above example shows again the tricky nature of pattern matching. A real life example would be sed from S:Startup-Sequence match "{#[ ,\t]}RunBack{#?}" change "{1}Launch{2}" which would replace all invocations of "RunBack" in the Startup-Sequence by similar invocations of "Launch". Another example to think about as exercise: ({}|{#[~;]+[ \t]}#[~; \t][/:]|{}#[~; \t][/:]|{#[~;]+[ \t]})FooBar{|[ \t;]#?} Yes, this pattern is useful. Consider again S:Startup-Sequence as input file and think about what this could possibly do. Note that some expressions are captured. (Hey, I said this would look like line noise!) ______________________________________________________________________________ Thomas Richter, October 2000