Nebula/nebula.bin/Documents/FAQ/Unix/Unix.fq2

Text File | 1993-01-27 | 29KB | 831 lines

Archive-name: unix-faq/part2
Version: $Id: part2,v 1.4 92/03/19 14:07:31 tmatimar Exp $

These four articles contain the answers to some Frequently Asked
Questions often seen in comp.unix.questions and comp.unix.shell.
Please don't ask these questions again, they've been answered plenty
of times already - and please don't flame someone just because they may
not have read this particular posting.  Thank you.

These articles are divided approximately as follows:

      1.*) General questions.
      2.*) Relatively basic questions, likely to be asked by beginners.
      3.*) Intermediate questions.
      4.*) Advanced questions, likely to be asked by people who thought
       they already knew all of the answers.

This article includes answers to:

      2.1)  How do I remove a file whose name begins with a "-" ?
      2.2)  How do I remove a file with funny characters in the filename ?
      2.3)  How do I get a recursive directory listing?
      2.4)  How do I get the current directory into my prompt?
      2.5)  How do I read characters from the terminal in a shell script?
      2.6)  How do I rename "*.foo" to "*.bar", or change file names
              to lowercase?
      2.7)  Why do I get [some strange error message] when I
              "rsh host command" ?
      2.8)  How do I {set an environment variable, change directory} inside a
              program or shell script and have that change affect my
              current shell?
      2.9)  How do I redirect stdout and stderr separately in csh?
      2.10) How do I tell inside .cshrc if I'm a login shell?
      2.11) How do I construct a shell glob-pattern that matches all files
            except "." and ".." ?
      2.12) How do I find the last argument in a Bourne shell script?
      2.13) What's wrong with having '.' in your $PATH ?

    If you're looking for the answer to, say, question 2.5, and want to skip
    everything else, you can search ahead for the regular expression "^5)".

While these are all legitimate questions, they seem to crop up in
comp.unix.questions on an annual basis, usually followed by plenty
of replies (only some of which are correct) and then a period of
griping about how the same questions keep coming up.  You may also like
to read the monthly article "Answers to Frequently Asked Questions"
in the newsgroup "news.announce.newusers", which will tell you what
"UNIX" stands for.

With the variety of Unix systems in the world, it's hard to guarantee
that these answers will work everywhere.  Read your local manual pages
before trying anything suggested here.  If you have suggestions or
corrections for any of these answers, please send them to to
[e-mail].

1)  How do I remove a file whose name begins with a "-" ?

    Figure out some way to name the file so that it doesn't
    begin with a dash.  The simplest answer is to use

        rm ./-filename

    (assuming "-filename" is in the current directory, of course.)
    This method of avoiding the interpretation of the "-" works
    with other commands too.

    Many commands, particularly those that have been written to use
    the "getopt(3)" argument parsing routine, accept a "--" argument
    which means "this is the last option, anything after this is not
    an option", so your version of rm might handle "rm -- -filename".
    Some versions of rm that don't use getopt() treat a single "-"
    in the same way, so you can also try "rm - -filename".

2)  How do I remove a file with funny characters in the filename ?

    If the 'funny character' is a '/', skip to the last part of
    this answer.  If the funny character is something else,
    such as a ' ' or control character or character with
    the 8th bit set, keep reading.

    The classic answers are

    rm -i some*pattern*that*matches*only*the*file*you*want

    which asks you whether you want to remove each file matching
    the indicated pattern;  depending on your shell, this may
    not work if the filename has a character with the 8th bit set
    (the shell may strip that off);

    and

    rm -ri .

    which asks you whether to remove each file in the directory.
    Answer "y" to the problem file and "n" to everything else.
    Unfortunately this doesn't work with many versions of rm.
    Also unfortunately, this will walk through every subdirectory
    of ".", so you might want to "chmod a-x" those directories
    temporarily to make them unsearchable.

    Always take a deep breath and think about what you're doing
    and double check what you typed when you use rm's "-r" flag
    or a wildcard on the command line;

    and

    find . -type f ... -ok rm '{}' \;

    where "..." is a group of predicates that uniquely identify the
    file.  One possibility is to figure out the inode number
    of the problem file (use "ls -i .") and then use

    find . -inum 12345 -ok rm '{}' \;

    or
    find . -inum 12345 -ok mv '{}' new-file-name \;

    "-ok" is a safety check - it will prompt you for confirmation of the
    command it's about to execute.  You can use "-exec" instead to avoid
    the prompting, if you want to live dangerously, or if you suspect
    that the filename may contain a funny character sequence that will mess
    up your screen when printed.

    What if the filename has a '/' in it?

    These files really are special cases, and can only be created
    by buggy kernel code (typically by implementations of NFS
    that don't filter out illegal characters in file names from
    remote machines.)  The first thing to do is to try to
    understand exactly why this problem is so strange.

    Recall that Unix directories are simply pairs of
    filenames and inode numbers.  A directory essentially
    contains information like this:

    filename  inode

    file1      12345
    file2.c      12349
    file3     12347

    Theoretically, '/' and '\0' are the only two characters that
    cannot appear in a filename - '/' because it's used to separate
    directories and files, and '\0' because it terminates a filename.

    Unfortunately some implementations of NFS will blithely create
    filenames with embedded slashes in response to requests from remote
    machines.  For instance, this could happen when someone on a Mac or
    other non-Unix machine decides to create a remote NFS file on
    your Unix machine with the date in the filename.  Your Unix
    directory then has this in it:

    filename  inode

    91/02/07  12357

    No amount of messing around with 'find' or 'rm' as described above
    will delete this file, since those utilities and all other Unix
    programs, are forced to interpret the '/' in the normal way.

    Any ordinary program will eventually try to do unlink("91/02/07"),
    which as far as the kernel is concerned means "unlink the file 07
    in the subdirectory 02 of directory 91", but that's not what we
    have - we have a *FILE* named "91/02/07" in the current directory.
    This is a subtle but crucial distinction.

    What can you do in this case?
    The first thing to try is to return to the Mac that created this
    crummy entry, and see if you can convince it and your local NFS
    daemon to rename the file to something without slashes.

    If that doesn't work or isn't possible, you'll need help from your
    system manager, who will have to try the one of the following.
    Use "ls -i" to find the inode number of this bogus file, then
    unmount the file system and use "clri" to clear the inode, and
    "fsck" the file system with your fingers crossed.  This destroys
    the information in the file.  If you want to keep it, you can try:

    create a new directory in the same parent directory as the one
    containing the bad file name;

    move everything you can (i.e. everything but the file with
    the bad name) from the old directory to the new one;

    do "ls -id" on the directory containing the file with the
    bad name to get its inumber;

    umount the file system;

    "clri" the directory containing the file with the bad name;

    "fsck" the file system.

    Then, to find the file,

    remount the file system;

    rename the directory you created to have the name of
    the old directory (since the old directory should have
    been blown away by "fsck")

    move the file out of "lost+found" into the directory
    with a better name.

    Alternatively, you can patch the directory the hard way
    by crawling around in the raw file system.
    Use "fsdb", if you have it.

3)  How do I get a recursive directory listing?

    One of the following may do what you want:

    ls -R             (not all versions of "ls" have -R)
    find . -print        (should work everywhere)
    du -a .            (shows you both the name and size)

    If you're looking for a wildcard pattern that will match
    all ".c" files in this directory and below, you won't find one,
    but you can use

    % some-command `find . -name '*.c' -print`

    "find" is a powerful program.  Learn about it.

4)  How do I get the current directory into my prompt?

    It depends which shell you are using.  It's easy with some shells,
    hard or impossible with others.

    C Shell (csh):
    Put this in your .cshrc - customize the prompt variable
    the way you want.

        alias setprompt 'set prompt="${cwd}% "'
        setprompt        # to set the initial prompt
        alias cd 'chdir \!* && setprompt'

    If you use pushd and popd, you'll also need

        alias pushd 'pushd \!* && setprompt'
        alias popd  'popd  \!* && setprompt'

    Some C shells don't keep a $cwd variable - you can use
    `pwd` instead.

    If you just want the last component of the current directory
    in your prompt ("mail% " instead of "/usr/spool/mail% ")
    you can use

        alias setprompt 'set prompt="$cwd:t% "'

    Some older csh's get the meaning of && and || reversed.
    Try doing:

        false && echo bug

    If it prints "bug", you need to switch && and || (and get
    a better version of csh.)

    Bourne Shell (sh):

    If you have a newer version of the Bourne Shell (SVR2 or newer)
    you can use a shell function to make your own command, "xcd" say:

        xcd() { cd $* ; PS1="`pwd` $ "; }

    If you have an older Bourne shell, it's complicated but not impossible.
    Here's one way.  Add this to your .profile file:

        LOGIN_SHELL=$$ export LOGIN_SHELL
        CMDFILE=/tmp/cd.$$ export CMDFILE
        # 16 is SIGURG, pick some signal that isn't likely to be used
        PROMPTSIG=16 export PROMPTSIG
        trap '. $CMDFILE' $PROMPTSIG

    and then put this executable script (without the indentation!),
    let's call it "xcd", somewhere in your PATH

        : xcd directory - change directory and set prompt
        : by signalling the login shell to read a command file
        cat >${CMDFILE?"not set"} <<EOF
        cd $1
        PS1="\`pwd\`$ "
        EOF
        kill -${PROMPTSIG?"not set"} ${LOGIN_SHELL?"not set"}

    Now change directories with "xcd /some/dir".

    Korn Shell (ksh):

    Put this in your .profile file:
        PS1='$PWD $ '

    If you just want the last component of the directory, use
        PS1='${PWD##*/} $ '

    T C shell (tcsh)

    Tcsh is a popular enhanced version of csh with some extra
    builtin variables (and many other features):

        %~        the current directory, using ~ for $HOME
        %d or %/    the full pathname of the current directory
        %c or %.    the trailing component of the current directory

    so you can do

        set prompt='%~ '

    BASH (FSF's "Bourne Again SHell")

    \w in $PS1 gives the full pathname of the current directory,
    with ~ expansion for $HOME;  \W gives the basename of
    the current directory.  So, in addition to the above sh and
    ksh solutions, you could use

        PS1='\w $ '
    or
        PS1='\W $ '

5)  How do I read characters from the terminal in a shell script?

    In sh, use read.  It is most common to use a loop like

        while read line
        do
            ...
        done

    In csh, use $< like this:

        while ( 1 )
        set line = "$<"
        if ( "$line" == "" ) break
        ...
        end

    Unfortunately csh has no way of distinguishing between
    a blank line and an end-of-file.

    If you're using sh and want to read a *single* character from
    the terminal, you can try something like

        echo -n "Enter a character: "
        stty cbreak        # or  stty raw
        readchar=`dd if=/dev/tty bs=1 count=1 2>/dev/null`
        stty -cbreak

        echo "Thank you for typing a $readchar ."

6) How do I rename "*.foo" to "*.bar", or change file names to lowercase?

    Why doesn't "mv *.foo *.bar" work?  Think about how the shell
    expands wildcards.   "*.foo" and "*.bar" are expanded before the mv
    command ever sees the arguments.  Depending on your shell, this
    can fail in a couple of ways.  CSH prints "No match." because
    it can't match "*.bar".  SH executes "mv a.foo b.foo c.foo *.bar",
    which will only succeed if you happen to have a single
    directory named "*.bar", which is very unlikely and almost
    certainly not what you had in mind.

    Depending on your shell, you can do it with a loop to "mv" each
    file individually.  If your system has "basename", you can use:

    C Shell:
    foreach f ( *.foo )
        set base=`basename $f .foo`
        mv $f $base.bar
    end

    Bourne Shell:
    for f in *.foo; do
        base=`basename $f .foo`
        mv $f $base.bar
    done

    Some shells have their own variable substitution features, so instead
    of using "basename", you can use simpler loops like:

    C Shell:

    foreach f ( *.foo )
        mv $f $f:r.bar
    end

    Korn Shell:

    for f in *.foo; do
        mv $f ${f%foo}bar
    done

    If you don't have "basename" or want to do something like
    renaming foo.* to bar.*, you can use something like "sed" to
    strip apart the original file name in other ways, but
    the general looping idea is the same.  You can also convert
    file names into "mv" commands with 'sed', and hand the commands
    off to "sh" for execution.  Try

    ls -d *.foo | sed -e 's/.*/mv & &/' -e 's/foo$/bar/' | sh

    A program by Vladimir Lanin called "mmv" that does this job nicely
    was posted to comp.sources.unix (Volume 21, issues 87 and 88) in
    April 1990.  It lets you use

    mmv '*.foo' '=1.bar'

    Shell loops like the above can also be used to translate
    file names from upper to lower case or vice versa.  You could use
    something like this to rename uppercase files to lowercase:

    C Shell:
        foreach f ( * )
        mv $f `echo $f | tr '[A-Z]' '[a-z]'`
        end
    Bourne Shell:
        for f in *; do
        mv $f `echo $f | tr '[A-Z]' '[a-z]'`
        done
    Korn Shell:
        typeset -l l
        for f in *; do
        l="$f"
        mv $f $l
        done

    If you wanted to be really thorough and handle files with
    `funny' names (embedded blanks or whatever) you'd need to use

    Bourne Shell:

        for f in *; do
              g=`expr "xxx$f" : 'xxx\(.*\)' | tr '[A-Z]' '[a-z]'`
              mv "$f" "$g"
        done

    The `expr' command will always print the filename, even if it equals
    `-n' or if it contains a System V escape sequence like `\c'.

    Some versions of "tr" require the [ and ], some don't.  It happens
    to be harmless to include them in this particular example; versions of
    tr that don't want the [] will conveniently think they are supposed
    to translate '[' to '[' and ']' to ']'.

    If you have the "perl" language installed, you may find this rename
    script by Larry Wall very useful.  It can be used to accomplish a
    wide variety of filename changes.

    #!/usr/bin/perl
    #
    # rename script examples from lwall:
    #       rename 's/\.orig$//' *.orig
    #       rename 'y/A-Z/a-z/ unless /^Make/' *
    #       rename '$_ .= ".bad"' *.f
    #       rename 'print "$_: "; s/foo/bar/ if <stdin> =~ /^y/i' *

    $op = shift;
    for (@ARGV) {
        $was = $_;
        eval $op;
        die $@ if $@;
        rename($was,$_) unless $was eq $_;
    }

7)  Why do I get [some strange error message] when I "rsh host command" ?

    (We're talking about the remote shell program "rsh" or sometimes "remsh";
     on some machines, there is a restricted shell called "rsh", which
     is a different thing.)

    If your remote account uses the C shell, the remote host will
    fire up a C shell to execute 'command' for you, and that shell
    will read your remote .cshrc file.  Perhaps your .cshrc contains
    a "stty", "biff" or some other command that isn't appropriate
    for a non-interactive shell.  The unexpected output or error
    message from these commands can screw up your rsh in odd ways.

    Here's an example.  Suppose you have

    stty erase ^H
    biff y

    in your .cshrc file.  You'll get some odd messages like this.

    % rsh some-machine date
    stty: : Can't assign requested address
    Where are you?
    Tue Oct  1 09:24:45 EST 1991

    You might also get similar errors when running certain "at" or
    "cron" jobs that also read your .cshrc file.

    Fortunately, the fix is simple.  There are, quite possibly, a whole
    *bunch* of operations in your ".cshrc" (e.g., "set history=N") that are
    simply not worth doing except in interactive shells.  What you do is
    surround them in your ".cshrc" with:

        if ( $?prompt ) then
            operations....
        endif

    and, since in a non-interactive shell "prompt" won't be set, the
    operations in question will only be done in interactive shells.

    You may also wish to move some commands to your .login file; if
    those commands only need to be done when a login session starts up
    (checking for new mail, unread news and so on) it's better
    to have them in the .login file.

8)  How do I {set an environment variable, change directory} inside a
      program or shell script and have that change affect my
      current shell?

    In general, you can't, at least not without making special
    arrangements.  When a child process is created, it inherits a copy
    of its parent's variables (and current directory).  The child can
    change these values all it wants but the changes won't affect the
    parent shell, since the child is changing a copy of the
    original data.

    Some special arrangements are possible.  Your child process could
    write out the changed variables, if the parent was prepared to read
    the output and interpret it as commands to set its own variables.

    Also, shells can arrange to run other shell scripts in the context
    of the current shell, rather than in a child process, so that
    changes will affect the original shell.

    For instance, if you have a C shell script named "myscript":

    cd /very/long/path
    setenv PATH /something:/something-else

    or the equivalent Bourne or Korn shell script

    cd /very/long/path
    PATH=/something:/something-else export PATH

    and try to run "myscript" from your shell, your shell will fork and run
    the shell script in a subprocess.  The subprocess is also
    running the shell; when it sees the "cd" command it changes
    *its* current directory, and when it sees the "setenv" command
    it changes *its* environment, but neither has any effect on the current
    directory of the shell at which you're typing (your login shell,
    let's say).

    In order to get your login shell to execute the script (without forking)
    you have to use the "." command (for the Bourne or Korn shells)
    or the "source" command (for the C shell).  I.e. you type

    . myscript

    to the Bourne or Korn shells, or

    source myscript

    to the C shell.

    If all you are trying to do is change directory or set an
    environment variable, it will probably be simpler to use a
    C shell alias or Bourne/Korn shell function.  See the "how do
    I get the current directory into my prompt" section
    of this article for some examples.

9)  How do I redirect stdout and stderr separately in csh?

    In csh, you can redirect stdout with ">", or stdout and stderr
    together with ">&" but there is no direct way to redirect
    stderr only.  The best you can do is

        ( command >stdout_file ) >&stderr_file

    which runs "command" in a subshell;  stdout is redirected inside
    the subshell to stdout_file, and both stdout and stderr from the
    subshell are redirected to stderr_file, but by this point stdout
    has already been redirected so only stderr actually winds up in
    stderr_file.

    Sometimes it's easier to let sh do the work for you.

    sh -c 'command >stdout_file 2>stderr_file'

10) How do I tell inside .cshrc if I'm a login shell?

    When people ask this, they usually mean either

    How can I tell if it's an interactive shell?  or
    How can I tell if it's a top-level shell?

    You could perhaps determine if your shell truly is a login shell
    (i.e. is going to source ".login" after it is done with ".cshrc")
    by fooling around with "ps" and "$$".  Login shells generally have
    names that begin with a '-'.  If you're really interested in the
    other two questions, here's one way you can organize your .cshrc to
    find out.

    if (! $?CSHLEVEL) then
        #
        # This is a "top-level" shell,
        # perhaps a login shell, perhaps a shell started up by
        # 'rsh machine some-command'
        # This is where we should set PATH and anything else we
        # want to apply to every one of our shells.
        #
        setenv      CSHLEVEL        0
        set home = ~username        # just to be sure
        source ~/.env               # environment stuff we always want
    else
        #
        # This shell is a child of one of our other shells so
        # we don't need to set all the environment variables again.
        #
        set tmp = $CSHLEVEL
        @ tmp++
        setenv      CSHLEVEL        $tmp
    endif

    # Exit from .cshrc if not interactive, e.g. under rsh
    if (! $?prompt) exit

    # Here we could set the prompt or aliases that would be useful
    # for interactive shells only.

    source ~/.aliases

11) How do I construct a shell glob-pattern that matches all files
    except "." and ".." ?

    You'd think this would be easy.

    *        Matches all files that don't begin with a ".";

    .*         Matches all files that do begin with a ".", but
         this includes the special entries "." and "..",
         which often you don't want;

    .[!.]*   (Newer shells only; some shells use a "^" instead of
         the "!"; POSIX shells must accept the "!", but may
         accept a "^" as well; all portable applications shall
         not use an unquoted "^" immediately following the "[")

         Matches all files that begin with a "." and are
         followed by a non-"."; unfortunately this will miss
         "..foo";

    .??*     Matches files that begin with a "." and which are
         at least 3 characters long.  This neatly avoids
         "." and "..", but also misses ".a" .

    So to match all files except "." and ".." safely you have to use
    3 patterns (if you don't have filenames like ".a" you can leave out
    the first):

    .[!.] .??* *

    Alternatively you could employ an external program or two and use
    backquote substitution.  This is pretty good:

    `ls -a | sed -e '/^\.$/d' -e '/^\.\.$/d'`

    (or `ls -A` in some Unix versions)

    but even it will mess up on files with newlines, IFS characters
    or wildcards in their names.

12) How do I find the last argument in a Bourne shell script?

    Answer by:
    Martin Weitzel <@mikros.systemware.de:[e-mail]>
    Maarten Litmaath <[e-mail]>

    If you are sure the number of arguments is at most 9, you can use:

    eval last=\${$#}

    In POSIX-compatible shells it works for ANY number of arguments.
    The following works always too:

    for last
    do
        :
    done

    This can be generalized as follows:

    for i
    do
        third_last=$second_last
        second_last=$last
        last=$i
    done

    Now suppose you want to REMOVE the last argument from the list,
    or REVERSE the argument list, or ACCESS the N-th argument directly,
    whatever N may be.  Here is a basis of how to do it, using only
    built-in shell constructs, without creating subprocesses:

    t0= u0= rest='1 2 3 4 5 6 7 8 9' argv=

    for h in '' $rest
    do
        for t in "$t0" $rest
        do
            for u in $u0 $rest
            do
                case $# in
                0)
                    break 3
                esac
                eval argv$h$t$u=\$1
                argv="$argv \"\$argv$h$t$u\""    # (1)
                shift
            done
            u0=0
        done
        t0=0
    done

    # now restore the arguments
    eval set x "$argv"                    # (2)
    shift

    This example works for the first 999 arguments.  Enough?
    Take a good look at the lines marked (1) and (2) and convince yourself
    that the original arguments are restored indeed, no matter what funny
    characters they contain!

    To find the N-th argument now you can use this:

    eval argN=\$argv$N

    To reverse the arguments the line marked (1) must be changed to:

    argv="\"\$argv$h$t$u\" $argv"

    How to remove the last argument is left as an exercise.

    If you allow subprocesses as well, possibly executing nonbuilt-in
    commands, the `argvN' variables can be set up more easily:

    N=1

    for i
    do
        eval argv$N=\$i
        N=`expr $N + 1`
    done

    To reverse the arguments there is still a simpler method, that even does
    not create subprocesses.  This approach can also be taken if you want
    to delete e.g. the last argument, but in that case you cannot refer
    directly to the N-th argument any more, because the `argvN' variables are
    set up in reverse order:

    argv=

    for i
    do
        eval argv$#=\$i
        argv="\"\$argv$#\" $argv"
        shift
    done

    eval set x "$argv"
    shift

13) What's wrong with having '.' in your $PATH ?

    A bit of background: the PATH environment variable is a list of
    directories separated by colons.  When you type a command name
    without giving an explicit path (e.g. you type "ls", rather than
    "/bin/ls") your shell searches each directory in the PATH list in
    order, looking for an executable file by that name, and the
    shell will run the first matching program it finds.

    One of the directories in the PATH list can be the
    current directory "." .  It is also permissible to use
    an empty directory name in the PATH list to indicate
    the current directory.  Both of these are equivalent

    for csh users:

    setenv PATH :/usr/ucb:/bin:/usr/bin
    setenv PATH .:/usr/ucb:/bin:/usr/bin

    for sh or ksh users

    PATH=:/usr/ucb:/bin:/usr/bin export PATH
    PATH=.:/usr/ucb:/bin:/usr/bin export PATH

    Having "." somewhere in the PATH is convenient - you can type
    "a.out" instead of "./a.out" to run programs in the current
    directory.  But there's a catch.

    Consider what happens in the case  where "." is the
    first entry in the PATH.  Suppose your current directory is a
    publically-writable one, such as "/tmp".  If there just happens to
    be a program named "/tmp/ls" left there by some other user, and you
    type "ls" (intending, of course, to run the normal "/bin/ls"
    program), your shell will instead run "./ls", the other
    user's program.  Needless to say, the results of running
    an unknown program like this might surprise you.

    It's slightly better to have "." at the end of the PATH:

    setenv PATH /usr/ucb:/bin:/usr/bin:.

    Now if you're in /tmp and you type "ls", the shell will
    search /usr/ucb, /bin and /usr/bin for a program named
    "ls" before it gets around to looking in ".", and there
    is less risk of inadvertently running some other user's
    "ls" program.  This isn't 100% secure though - if you're
    a clumsy typist and some day type "sl -l" instead of "ls -l",
    you run the risk of running "./sl", if there is one.
    Some "clever" programmer could anticipate common typing
    mistakes and leave programs by those names scattered
    throughout public directories.  Beware.

    Many seasoned Unix users get by just fine without having
    "." in the PATH at all:

    setenv PATH /usr/ucb:/bin:/usr/bin

    If you do this, you'll need to type "./program" instead
    of "program" to run programs in the current directory, but
    the increase in security is probably worth it.
--
Ted Timar - [e-mail]
Omron Corporation, Shimokaiinji, Nagaokakyo-city, Kyoto 617,  Japan