[MiNT] USERDEF stack size

[Vincent Rivière <vincent.riviere@freesbee.fr>]

Hello.

I have discovered a very sad thing.

Long time ago, I tried to compile the QED and the cflib with GCC 4.x. I
was annoyed because the resulting binary was very easy to crash: QED >
About > Info. I thunk it was just another GCC related bug, but
unfortunately, it was not.

The problem is actually about USERDEFs. They seem to be nice AES objects
which can be drawn themselves through a callback function. It is well
known that the AES must be called from user mode. As Henk pointed to me
some time ago, the AES calls the USERDEFs callbacks in supervisor mode.
It is dirty and unsafe, but not problematic in practice. The biggest
problem is that the USERDEFs callbacks are called on the AES own
supervisor stack, and that one is ridiculously small. As a result, it is
 very easy to get a stack overflow in USERDEFs.

Also, QED uses the cflib. That library supports advanced resource
objects (MagiC ?), such as group boxes. When such objects are not
supported by the underlying AES, the cflib automatically translate them
into USERDEFs, so they can display themselves correctly.

For example, the QED Info/Thanks box contains a group box. With simple
TOS (I tested 1.62) and EmuTOS, the group box is not supported, so it is
translated to USERDEF. Inside the callback, the cflib calls v_gtext() to
draw the label.

Now let's go into the GemLib. The v_gtext() implementation is like this:

void
v_gtext (short handle, short x, short y, const char *str)
{
	short vdi_control[VDI_CNTRLMAX];
	short vdi_intin[VDI_INTINMAX];
	short vdi_ptsin[2];
	register short n = vdi_str2array (str, vdi_intin);
...

VDI_INTINMAX is actually 1024, so we have here more than 2 KB on the
stack, which is already too much for the small AES stack.
If I put this array to static (actually global), and also in
vqt_extent() which has the same problem, the stack usage becomes small
enough and there is no more crash. However, there are potential
reentrency problems.

This problem does not occur with XaAES. First, it supports extended
resources, then it has probably a cleaner usage of USERDEFs.
It does not occur either with my old QED binary (from EasyMiNT ?), it
had probably been compiled with a different GemLib having fewer stack usage.

So the problem only occurs with plain TOS/EmuTOS and QED recompiled with
a recent GemLib.

I don't know what to do, so comments are welcome.

--
Vincent Rivière