Re: [MiNT] memory fragmentation

[Guido Flohr <gufl0000@stud.uni-sb.de>]

Hi,

On Sat, Aug 14, 1999 at 06:11:48PM +0000, kellis wrote:
> 
> Hi, been almost 4 days uptime. (so it IS stable).
> 
> My question is:
> 19641712 bytes (19181K) in 112 blocks free, largest free block: 2383920
> (2328K)
> 
> As you can see, i have 19meg free, and only 2 meg for largest block.
> huge fragmentation. 
> 
> What can I do to prevent that as little as possible?
> How do other OS do?

As Jo already said, the proper solution would be to make the kernel
use the MMU what it had been designed for, for virtual memory 
management.

I don't see how shared libs could reduce fragmentation.  I would 
rather expect the opposite effect.

A strategy for now to reduce fragmentation:  Finish your programs
from time to time, especially memory hungry stuff.  This is especially
advisable for QED after you have loaded huge files.  QED has a
"hand-optimized" memory management (not invented by Christian, he
is unhappy with it too) that will prevent memory to be released
once it has been allocated.  In other words: QED will only free
memory at program termination.

Apart from the need for VM I think that there are really some things
that could at least reduce fragmentation.  My first idea shouldn't be
that hard to implement: Under Magic the kernel evaluates a program
flag that tells the kernel that the program has reserved space for
the stack in the BSS.  This makes a big difference: If this flag is
not set the kernel will have to look for the largest block of free
memory and load the program image there before it gets relocated
and run.  This is usually pure waste because the very first thing
that the program will do is to shrink to the size really needed.
Depending on the order that processes get started this will lead
to fragmentation.

This technique could easily be implemented in the kernel but it would
require a rewrite of the startup code and stack handling of the MiNTLib
(I think the Magic approach is focussed against PoorC).  But the
kernel should really support this feature, it will help already
with PoorC linked applications.

For gcc compiled stuff I would suggest a different approach:  The new
program format would allow the kernel to read the stack size of
the program image (even without reading the symbol table).  In the
end that has the same effect as the Magic approach: The kernel could
deduce the optimum size for the TPA at runtime, thus making more
efficient use of smaller memory blocks.

I will see if I understand enough of the kernel sources to implement
that.  Once implemented, shared libraries would also reduce 
fragmentation because the use of shared libs would reduce the
initial memory consumption.

My next idea is probably harder to implement, maybe nonsense, maybe
even harder to realise than VM:  What if the kernel distinguished between
four different kinds of malloc requests: kernel malloc, large malloc,
medium malloc and small malloc.  Depending of the kind of malloc request
the kernel would choose the preferred region from where the memory
should be taken.  If we could find some appropriate heuristic this might
also help.

Ciao

Guido
-- 
http://stud.uni-sb.de/~gufl0000/
mailto:gufl0000@stud.uni-sb.de