Re: [MiNT] UTF-8 support

["Guido Flohr" <gufl0000@stud.uni-sb.de>]

On Tue, Apr 25, 2000 at 12:21:05PM +0300, Martin-Éric Racine wrote:
> Also, is either vcons or TosWinII being upgraded to allow
> remapping the whole Atari system font into valid Unicode
> entities?

I had already started to introduce wide character and multy-byte support
into the MiNTLib but I had to give that up.  Otherwise the thread "large
binaries with MiNTLib 0.55.2" never would have ended.  This features
has to be delayed until MiNT can dynamically load library modules.

Some people may know that I am very fond of i18n and l10n, so here is my
opinion on the topic:

The most annoying thing (after backslashes in path names and the
colon/drive letter stuff) in MiNT to me is the MS-DOS codeset that we
still use.  Even Microsoft have migrated to ISO 8859-1 when they
introduced Windows, we never did something alike.

Next problem is the GUI elements in the codeset (closer, fuller, scroll
arrows).  I think that future AES versions should overcome this nuisance
and automagically translate control characters < ASCII SPACE in BOXCHAR
AES objects (maybe in all strings) into the corresponding window elements
instead of taking them verbatim from the system font.  This would allow to
use arbitrary fonts as system fonts.

The VDI is not so bad.  It is often forgotten that text strings for the
VDI are already arrays of short and not of char (this is hidden in the
library bindings that automatically translate char[] to short[]).  Since
the data type wchar_t is equivalent to short int the MiNTLib it would be
no problem to offer an advanced string API in the VDI library bindings (as
far as I can see it all implementations of wchar_t silently assume that
wide character strings are UTF-8).  A while ago I have experimented a
little with GDOS bitmap fonts that exceed the 256-character-limit and it
mostly worked.  The font editors all had problems with them but the VDI
itself handled them correctly (I had converted them from X BDF fonts to
GDOS).  The fonts I had converted are lost but I have hand-made Monaco
fonts (default size only) for ISO-8859-1, ISO-8859-2, ISO-8859-3 and
ISO-8859-5 (plus KOIR-8 because Russians seem to prefer that encoding to
ISO-8859-5).  If somebody is interested in drawing some more of the
required glyphs so that we can glue them together to a partial UNICODE
font I can provide my current results.

Yet, the GDOS bitmap format won't be sufficient.  Although it can
theoratically hold 65536 glyphs it has unacceptable limitations.  The
glyph data is limited to 32767 bytes (maybe 65536 but that would
require the VDI to interpret offsets into the graphic data as unsigned
short and not short, at least undocumented).  Furthermore the GDOS format
doesn't like gaps in the font data.  Every font has to cover a contiguous
range of characters, otherwise (at least with monospaced fonts) you will
waste a lot of space.

The alternative to GDOS bitmap fonts are the scalable formats supported by
recent (N)VDI implementations.  But IMHO the appearance - at least on the
screen - of scalable fonts is currently very poor.  I think the text
handling of GEM (i. e. AES and VDI) is simply outdated and should be
replaced.  I really like the idea of using the X Font Server
protocol.  For example xfstt is based on the FreeType library and renders
TrueType fonts excellently and porting a font server to MiNT
should be feasible.  Since Apple's/Microsoft's TrueType format
seems to be 1st choice for scalable fonts this approach looks quite
attractive to me.  An alternative to a real font server would be an auto
folder program that hooks into the VDI trap and replaces the VDI text
functions.  Hm, could this work?

It's a pity that so few people are concerned about internationalization
for MiNT.  A few years ago i18n support was a very good chance to
compete with other systems because the computer world was completely
focussed on English.  Today we can only try to keep up at least a little
with the standards set up elsewehere.

Uhm, sorry, after re-reading my posting I must admit that my thoughts are 
a little unordered. ;-)  It's my favorite topic and I was trying to give
an overview of all the fantastic plans I have about it that will never be
accomplished.  Welcome to Atari land. :-/

Ciao

Guido
-- 
http://stud.uni-saarland.de/
Send your spam to president@whitehouse.gov and your replies to
mailto:guido at freemint dot de