Do not mess build, host and target options.
We probably don't understand each other. CFLAGS changes compiler flags globally, i.e. for gcc binary, for libgcc targets, libstdc++ targets etc. So when libgcc should be build for 5475 and we set CFLAGS with m68020-60, there's a clash because commandline looks like:
m68k-atari-mint-gcc -mcpu=5475 -o conftest -g -m68020-60 -O2 -fomit-frame-pointer conftest.c
(confirmed by test) so I was asking for some solution where the right things get CFLAGS with m68020-60 and the multilib things get the original CFLAGS and I thought --with-arch=XYZ could help but it's not supported.