Re: Shared Libraries.

[Stephen Usher <steve@earth.ox.ac.uk>]

>>Hmm... why are you thinking about using such a mixed bag of formats? Why did
>
>? Both shared libraries and programs linked with them would be DRI object
>files. The "GST long name symbol table" is a (minor and quite widely
>supported, including by gcc) extension to the DRI symbol table format.

You mean standard TOS executable format?

This doesn't HAVE to be so as programs which use shared libraries will have
to have a different magic number so as to stop them being run under the
wrong environment.

>
>Unix "ar" format wouldn't be suitable because a shared library isn't a
>"library" in the sense of an ar file, but rather it's a single object
>(more like an object file).

I understand this, I was just taking a format off the top of my head. The
important thing, however, is that there is an index of modules (routines)
which can be loaded individually, this index being loaded into the kernel's
module lookup cache in the shared library mode lookup table.

If you have the whole library as one big object which has to be loaded in
totallity then we might as well forget the whole idea of shared libraries!

>
>>What you suggest seems fine for the 68000 based machines, but for the 68030
>>based machines it seems a bit of a munge, why not give all processes on
>>these machines the same virtual address space and map the shared library
>>objects into that address space?
>
>I don't want to have to implement shared libraries twice. Obviously it
>would be much easier to throw out 68000 support altogether, but I really
>don't want to do that.

I understand this, that's why I suggested below a scheme which could be
implemented on both architectures, as long as memory protections were sorted
out (eg. making the shared library modules in memory global read-only).
Later on, once MiNT has processes with their own virtual address space, the
modules could be mapped in using the MMU.

>
>>>------------------------------------------------------------------------------
>>>| A's data |   FOO's data |   C's data  |  D's data  |  more of FOO's data   |
>>>------------------------------------------------------------------------------
>
>>What happens if FOO needs A, B, C & D? This scheme also means that all the
>>library modules will have to be loaded at load time instead at first call
>>time (doesn't it?).
>
>If FOO needed library B, then it's data would have gone in between A's
>data and C's (i.e. all of FOO's own data would have been contiguous).
>
>It is true that all the library modules for a program would have to be loaded
>at program load time, rather than when the library is first called. I don't
>think that this would generally be much of a problem; I doubt there are all
>that many cases where a shared library is linked but never called at run time.

I'm sure there are quite a few cases, just look what scanf could pull in, it
could pull in all sorts of floating point code when all you wanted was to
parse a mixture of text and integers. (I know you could use an integer
version, but this integer version will fill up your disk and should not be
necessary.)

>
>>How about an alternative....
>
>>(1) Have in the kernel a table of currently loaded modules, each with a
>>	reference count.
>>(2) Modules are loaded into an arbitary memory location with a read-only
>>	copy of their initialised data space at the time of first reference
>>	and a copy of the initialised data space is placed in the process'
>>	shared lib data segment, which is separate from it's own data
>>	segment. The total size of the shared library data segment is 64K on
>>	the 68000 based machines and "unlimited" on 68030 based machines,
>>	there is no limit imposed upon the normal data segment. Shared data
>>	is accessed via an offset table.
>>(3) Modules are deleted once the last program to use them exits, or
>>	alternatively... The kernel table has a timer which is added to
>>	every time a module is accessed. After n context switches, any
>>	module which has not been accessed in that time could be deleted
>>	from memory and a flag in the table set to say that the module is
>>	active but not in memory. When the module is next accessed, it is
>>	re-loaded, but the data space is not copied to the currently running
>>	program. This would improve memory usage no end, especially if
>>	modules were of a fixed size or a multiple of a fixed size, no
>>	memory fragmentation! This would give a sort of virtual memory even
>>	for 68000 based machines.
>
>(1) is obviously necessary for any shared library scheme :-).
>(2) is reasonable, albeit slower (because of the extra indirection). The
>shared library data segment would actually be limited in size to 64K items,
>rather than 64K bytes (hmmm, actually 16K items, I guess, given 32 bit
>offsets).
>(3) is attractive, but I don't think it's workable. Or rather, it may not
>be as useful as it seems. The module memory can't be re-used for anything
>else (including program malloc requests), so there would be a win only
>when the program is linked with a lot of large shared libraries which are
>not all needed at the same time. This may be the case for X (:-)) but probably
>not for very many other libraries; and given the number of MiNT specific
>X implementations on the ST, this may not be much of a win :-(.

As with the example of scanf above, it could be a biggish win in programs
which only use a subset of the libraries at any one time. It would also be a
big win for programs waiting on I/O or something like that, as long as other
programs aren't using the modules (routines) then they could be purged to
load other modules in.

The memory freed by unloading modules could be used to load other modules
in, it's a sort of paging mechanism which won't need a PMMU. The modules
will all have to be position independent, however.

>
>Thanks for your input, 
>Eric
>

I'm just throwing ideas into the wind!

Steve

-- 
---------------------------------------------------------------------------
Computer Systems Administrator, Dept. of Earth Sciences, Oxford University.
E-Mail: steve@uk.ac.ox.earth (JANET) steve@earth.ox.ac.uk (Internet).
Tel:- Oxford (0865) 282110 (UK) or +44 865 282110 (International).