NetNews Usenet Archive 1993 #1/NN_1993_1.iso/spool/comp/sys/sgi/18671

Internet Message Format | 1993-01-11 | 3.6 KB

Xref: sparky comp.sys.sgi:18671 comp.unix.admin:7028 comp.unix.internals:2111 comp.unix.misc:4916 comp.unix.bsd:11122
Path: sparky!uunet!zaphod.mps.ohio-state.edu!usc!howland.reston.ans.net!spool.mu.edu!olivea!sgigate!sgi!rhyolite!vjs
From: [e-mail] (Vernon Schryver)
Newsgroups: comp.sys.sgi,comp.unix.admin,comp.unix.internals,comp.unix.misc,comp.unix.bsd
Subject: Re: Problem to transfer large data via socket communication - Need Help
Message-ID: <[e-mail]>
Date: 11 Jan 93 17:09:24 GMT
References: <[e-mail]> <[e-mail]>
Organization: Silicon Graphics, Inc.  Mountain View, CA
Lines: 63

In article <[e-mail]>, [e-mail] (Thomas Sippel - Dau) writes:
> In article <[e-mail]>, [e-mail] (Oliver Kretzschmar) writes:
> .....
> -   to send data from socket to socket (stream):
> -
> -         write(socket,(char*)data,buffer)
> -         read(socket,(char*)data,buffer)
> -
> -   But now we've problems sending/receiving large data for
> -   example 50*50*50 matrix of floats or larger. Is there any
> -   transfer-bufferlimitation by using streamsockets, that we can
> -   manipulate ?
>
> Yes, there is a number of streams buffers configured in the kernel, see
> (on IRIX) /usr/sysgen/master.d/kernel.

No, the number of streams buffers configured in the Silicon Graphics
version of UNIX, "IRIX", have absolutely nothing to do with such
uses of TCP/IP.  IRIX uses STREAMS buffers only for the MIDI and some
audio code, the keyboard, and some other serial I/O devices.

Under certain circumstances, the number of MBUFs can affect pure TCP/IP
uses, but almost certainly none concerning the original questioner.

The use of both mbufs and STREAMS buffers, events and queues can
be monitored with `netstat`.  However, that is almost their only
connection in IRIX.  Other versions of UNIX, especially those using
STREAMS based TCP/IP are different from IRIX, which includes fairly
recent versions of 4.3BSD style network code--i.e. "sockets".


> However, you main error is to use read/write, which is for "unbuffered"
> I/O and has generally been designed to be fast and for people who know
> what they are doing and are willing to cope with hardware limitations
> themselves.
>
> For applications programers the latter is not generally true, so use
> fread/fwrite and swallow the extra layer of software for the benefit
> of generality.


In general, UNIX hides all considerations of "hardware limitations"
from application programmers.  The need to use various hacks like
"physio" to get to the hardware is both an advantage and disadvange of
most varients of UNIX.

Stdio would add a very costly layer in such an application as I
understood to be the original questioner's.  Stdio would require at
least one extra copy of the data.  Since the current IRIX version of
stdio defines BUFSIZ to be 8K, the reads and writes of the application
might be much smaller.  As wrote recently, 8K buffers are "below the
knee" of the pure TCP/IP performance curve of IRIX, at least on fast
media such as FDDI.

The fread() and fwrite() functions of the stdio package in IRIX does no
more than some simple buffering on top of the read() and write() system
calls.  One generally uses fread() and fwrite() only when performance
is not an issue, and either when you don't want to deal with the short
read()'s caused by the nature of TCP/IP or when you are also using
formatting or scanning functions such fprintf() or fscanf().


Vernon Schryver,  [e-mail]