NetNews Usenet Archive 1992 #16/NN_1992_16.iso/spool/comp/lang/c/11527

Text File | 1992-07-23 | 4.0 KB | 83 lines

Newsgroups: comp.lang.c
Path: sparky!uunet!uunet.ca!hyper.hyper.com!hurst
From: [e-mail] (Graham Hurst)
Subject: Re: good memory allocation strategies?
Message-ID: <[e-mail]>
Keywords: SGI kernel
Reply-To: [e-mail] (Graham Hurst,,)
Organization: HyperCube Inc.
References: <[e-mail]> <[e-mail]> <boehm.711834865@siria>
Date: Thu, 23 Jul 1992 17:10:14 GMT
Lines: 70

In article <boehm.711834865@siria> [e-mail] (Hans Boehm) writes:
>[e-mail] (Graham Hurst) writes:
>>We recently discovered a "feature" of the default SGI kernel that makes
>>dynamic memory allocation unreliable - it never turns down a request for
>>memory!
>
>>Since IRIX 3.3.2, the default SGI kernel will cause malloc/realloc
>>(via sbrk) to return a non-zero pointer even if the requested memory size is
>>greater than the sum of physical memory and swap space! If your app tries
>>to use enough of this memory, the kernel sends a SIGKILL to kill your app
>>and puts a message on the console about running out of swap space.
>
>This does increase the probability of crashes due to running out of
>swap space.  But the traditional way of checking malloc results is
>already somewhat unreliable under most circumstances.  Consider
>the following scenario:
>
>malloc(enough space to barely fit in swap space);
>call deeply recursive procedure;

This example shows that a program using a "reliable" malloc can still
crash (via blowing the stack). It doesn't show that "the traditional way
of checking malloc results is already somewhat unreliable under most
circumstances".

I agree that there are circumstances where there is no other choice but
to have the operating system kill your application, particularly if the
swap space is full.  However I still don't think the OS should give you
requested memory (i.e. a successful malloc) that you cannot then use.
I also think this is at odds with the ANSI standard.

>On most systems that I know of, your process now gets sent a bus error due
>to the inability of the kernel to grow your process stack sufficiently.
>You can try to avoid this by growing the stack sufficiently in the beginning,
>and catching any bus errors that occur; but if it weren't for the
>particularly poor choice of SIGKILL, you could do the analogous thing
>for malloc.  (AIX gives you a warning signal first, before it sends
>SIGKILL, I believe.  This initially struck me as silly, but I'm begginning
>to believe that it's more useful than any of the other alternatives,
>including the one in which malloc preallocates swap space.)

The concept of "preallocation" implies that you may be asking for memory
before you need it or without knowing how much of it you'll need.  This is
not the case for my applications (large scale number crunching for molecular
modelling), where I know exactly how much space I need and will use all
of it - i.e. if I can't be certain that I can use all of it, I want to know
when I call malloc. Again this is how I interpret (via K&R II) the
ANSI C standard library definition of malloc.  (I don't care if the OS
procrastinates about physical allocation, just that I am guaranteed as
much as I have asked for.)

I realize not all apps can or should be written this way, and that the
other consumers of swap generally can't, so I think the solution has
to be a richer API for memory allocation, where the programmer decides
whether he wants reliable memory or not.  Microsoft adopted this
approach with NT, and I've advocated this to SGI as well.

Of course any function called "malloc" should work the way I expect! ;-)
>
>Note that similar problems occur if there are other crucial processes
>allocating swap space on the same machine.  Copy-on-write pages for
>shared libararies are yet another problem.  Malloc is only one consumer
>of swap space.
>
>Hans-J. Boehm
>([e-mail])

Graham Hurst
Hypercube Inc, 7-419 Phillip St, Waterloo, Ont, Canada N2L 3X2 [phone]
internet: [e-mail]