Maybe extending the script functions (if that is even necessary) and generally implementing the setup as an AGS game would be a good idea. The problem I see with having a dedicated windowed application is that it either requires a different GUI framework for different platforms or you have to add huge library dependencies. Having the setup run inside the engine would make it instantly cross-platform (ScummVM works like this) and configurable by the average AGS developer (no need to compile anything).
If the engine in setup mode starts with sane defaults (windowed, software rendering, 640x480, 32 bit game, sound disabled maybe) it should work as well as the current winsetup.
If the engine in setup mode starts with sane defaults (windowed, software rendering, 640x480, 32 bit game, sound disabled maybe) it should work as well as the current winsetup.