ENGINE PLUGIN: AGSBMFontRenderer - UTF-8 Text Renderer

Started by birthggd, Mon 25/05/2015 14:55:36

Previous topic - Next topic

birthggd

Hi all.

I'm writing utf-8 text renderer for non-ascii users(japanese, chinese, korean, etc).

more info on Github repogitory here => https://github.com/bggd/AGSBMFontRenderer

I'm not sure it works any PC. Feedback is welcome :)

Thanks.

Monsieur OUXX

Please keep posting for any updates. I'll add your project to the wiki page
 


daneeca

Hi guys!

We are working on an adventure game and the investors really want to have a chinese translation. They already started to translate it in a Word document. Is it possible to use this method to create a chinese translation?

Any ideas and advice would be greatly appreciated. Thank you!

Monsieur OUXX

#4
Quote from: daneeca on Tue 11/02/2020 10:11:26
Hi guys!

We are working on an adventure game and the investors really want to have a chinese translation. They already started to translate it in a Word document. Is it possible to use this method to create a chinese translation?

Any ideas and advice would be greatly appreciated. Thank you!
I'm guessing you'd be the first person ever to translate an AGS game to Chinese. If you succeed, then please post your method. Do you speak Chinese yourself?
Food for thought : https://www.localizedirect.com/posts/chinese-game-localization-factsheet
 

Gilbert

I never knew about this plug-in, and by how inactive this thread was it seems few (if any) people actually tried it, which is a pity as it seems to work well according to the test video clip. (Though I won't use it as I am against plug-ins myself.)

Supporting UTF-8 text is a much needed feature of AGS, it'd be great if the functionality of this plug-in can be incorporated into the engine itself.

Monsieur OUXX

I mentionned the plugin on the Fonts wiki page but only now am I realizing how powerful it could be for pushing AGS' limits, especially in regards to translations.
 

fernewelten

#7
Quote from: Gilbert on Wed 12/02/2020 05:27:48
Supporting UTF-8 text is a much needed feature of AGS, it'd be great if the functionality of this plug-in can be incorporated into the engine itself.

Speaking of which, the Editor should accept and write source code in UTF-8, and the compiler should process such source code.

  • The definition of identifiers could remain the same (1st char a-z, A-Z, _; 2nd and following char a-z, A-Z, _, 0-9)
  • single-quoted  characters, e.g., 'ä', would be converted to integers in the range 0 … 65536
  • single-quoted characters could contain \x and \u sequences with high numbers, those would be converted into the corresponding integers
  • char would continue to be 0 … 255, and strings would continue to be char sequences, but could now contain UTF-8 sequences
  • double-quoted strings "…" could contain Unicode, which would be coded as UTF-8 sequences in the source file, e.g., "Frühstück"
  • double-quoted strings could contain \x and \u sequences with high numbers, those would expand into UTF-8 encoded Unicode

The nice thing about UTF-8 sequences is that they are guaranteed not to contain zeroes, so these changes would be sufficient as far as the compiler is concerned.

However, if an Engine function is passed a string that contains UTF-8 and if that Engine function isn't Unicode aware (yet) then it might garble the string or produce illegal UTF-8.

Pre-existing AGS bytecode that is only given strict ASCII as input will continue to work and ASCII source code can be directly "interpreted as" UTF-8, so those changes would be completely backward compatible.

The String class should probably need helper functions so that it is possible to iterate through its characters  even if some of those are UTF-8 sequences and need to be processed as one. For each String function we'd probably also need an additional function that works like the original but treats whole UTF-8 sequences as 1 character. This doesn't concern the compiler.

eri0o

I am not sure this thread is the adequate place to discuss engine/editor limits. This is an already open issue in the AGS repository: https://github.com/adventuregamestudio/ags/issues/711

Note: the issue above isn't exactly this, but a reduced scope functionality that would work for translations (not original AGS Script code as far as I can tell). For a general utf-8 support, it's useful to open a new issue.

Alternatively, opening a new thread for the editor on Editor Development Board.

Anyway, I quickly built this plug-in: AGSBMFontRenderer.dll

I recommend reading the README since one needs to generate the font before using this.

Crimson Wizard

#9
Quote from: eri0o on Wed 12/02/2020 11:54:26
I am not sure this thread is the adequate place to discuss engine/editor limits.

It definitely isn't.

Quote from: eri0o on Wed 12/02/2020 11:54:26
I am not sure this thread is the adequate place to discuss engine/editor limits. This is an already open issue in the AGS repository: https://github.com/adventuregamestudio/ags/issues/711

This is a wrong ticket also, it was open merely for reference on one hacky solution, and not meant to discuss full transition to UTF-8. There was also a series of utf-related comments made in another ticket recently (https://github.com/adventuregamestudio/ags/issues/990) but it was also a non-convenient place to do so.
I don't think we have a proper "main ticket" for discussing Unicode problem yet.

daneeca

Hi! Thank you for all the answers.
Unfortunately I don't speak chinese at all but we have a person for this job  :smiley: I didn't have time to try this plugin but I definitely will, and then I'll tell you my experiences.

daneeca

I tried everything I can but it's not working :( I have no idea what I'm doing wrong. I copied the DLL into the AGS folder, then I made a bitmap font from the UTF-8 text file. When I try to import this font I get an error message:
"Unable to import the font. Not a valid SCI font file"
Maybe I don't need to import it, because the script does it when I run the game, but when I run the game I get another error message:
"Error: Unable to load font 8, no renderer could load a matching file."

I tried to use the font from the Demo, the same happened. Did I miss something? I read the readme file many times, and I can't figure out what I'm doing wrong :confused:
Should I do something with these files maybe? -> src folder, thirdparty folder, appveyor.yml

Edit: I tried to remake the font in AGS and now I got this message:


"Illegal exception
An exception 0xC0000005 occurred in ACWIN.EXE at EIP = 0x578B2AF7; program pointer is +6, ACI version 3.5.0.22, gtags (7,1)
AGS cannot continue, this exception was fatal. Please note down the numbers above, remember what you were doing at the time and post the details on the AGS Technical Forum.

in "room2.asc", line 18"

daneeca

After many attempts we figured out how to use this plugin. It works with the latest AGS too (3.5.0.24). Basically you just have to follow the steps in the readme file, but in some cases you have to pay attention to a few little things. We used it to create a Chinese translation so I can show you this method:
1. First of all generate the translation file. Then open the trs file in Notepad++ (or something similar software) and save it with UTF-8 encoding (without BOM). This way you can write chinese characters in the file.
2. Translate the texts into Chinese. It's important to finish all texts because you'll need all of the chinese characters that you want to use in the game.
3. Save a new file from the final translation (keep the original as it is). It could be a simple txt file, but the most important thing is to save it with UTF-8 BOM encoding.
3. Now download the Bitmap Font Generator from the link on GitHub.
4. Set up things in the software based on the GitHub readme file (enable Match char height, disable Font smoothing, 8bit font texture only, text format .fnt) and then select a chinese font in Font settings (You can find many free chinese fonts on the internet).
5. Now go to "Edit/Select chars from file" and open the file you last saved with UTF-8 BOM encoding. This way the software selects only the characters you need.
6. Export the bitmap font as a 'fontname.fnt' file ('fontname' can be anything you want). I used Targa texture files without compression.
7. Download 'AGSBMFontRenderer.dll' and put it into the AGS's installation folder. Now you can enable the plugin in the Editor.
8. Copy the bitmap fonts into the game's compiled folder. In my case into the 'Compiled/Windows' folder.
9. Now create a new font in the Editor. Change 'OutlineStyle' to 'Automatic' and write the font's ID to 'OutlineFont'.
10. Add this code to the 'game_start' function (3 is the font's ID):
Code: ags
if (Game.TranslationFilename == "Chinese")
{
  SetBMFont("fontname.fnt", 3);
  Game.SpeechFont = 3;
}


Probably you have much more texts in your game (like dialog options, labels on GUIs etc.) with different fonts, so you have to change them too. You can do it manually, or just use these functions:
Code: ags
function ReplaceFontOnGUI(GUI *g, int old_font, int new_font)
{
  int i = 0;
  while (i < g.ControlCount)
  {
    Button *b = g.Controls[i].AsButton;
    if (b != null)
    {
      if (b.Font == old_font)
        b.Font = new_font;
    }
    Label *l = g.Controls[i].AsLabel;
    if (l != null)
    {
      if (l.Font == old_font)
        l.Font = new_font;
    }
    i++;
  }
}

function ReplaceFontOnAllGUIs(int old_font, int new_font)
{
  int i = 0;
  while (i < Game.GUICount)
  {
    ReplaceFontOnGUI(gui[i], old_font, new_font);
    i++;
  }
}

I found them somewhere in the forum and they are incredibly useful.

11. If it's possible to change the language while playing your game, then you have to add these things there too. With an 'else' part to change everything back when you select English of course.
12. Now you can Build your game and then click on "Build/Run" option. You got an error message, right? Yeah, you can't use this "Run" button anymore :D You have to run the game with "Run game setup" or just run the .exe file in the compiled folder. But it should work this way.

Suggestion: The easiest way to do this if your game's default language is English. Because English characters are in the same "place" in ANSI and UTF-8 encoded files too (and AGS can't handle UTF-8 files natively). If your game's default language is for example Hungarian, then it won't work, because the special Hungarian characters are in different "places" in ANSI and UTF-8 encoded files. So the engine can't recognize the same sentences.

I think I've covered every detail, but if you have any questions, feel free to ask!


shaun9991

Hi guys,

I've been trying out this plugin as I'm trying to see if we are able to translate our next game into Chinese characters (Simplified & Traditional).

I've hit a big issue - text that is supposed to be displayed over two lines or more is simply not displayed at all. There seems to be no problem displaying the Chinese characters if the text is short enough to display in one line across the screen - but if the amount of Chinese characters that should be displayed exceeds the width of the screen, no text is displayed.

Does anyone know how this could be happening?

Many thanks,
Shaun


Support Cloak and Dagger Games on Patreon: https://www.patreon.com/user?u=460039

Crimson Wizard

Quote from: shaun9991 on Mon 21/12/2020 22:38:39
I've hit a big issue - text that is supposed to be displayed over two lines or more is simply not displayed at all. There seems to be no problem displaying the Chinese characters if the text is short enough to display in one line across the screen - but if the amount of Chinese characters that should be displayed exceeds the width of the screen, no text is displayed.

Does anyone know how this could be happening?

Where exactly do you display this text, Say command, GUI label, ...?

shaun9991

Hi Crimson Wizard!

Sorry, should have said that. It's any text - .Say command, Gui labels, Button text etc.

I'm also using AGS 3.5.0

Thanks,
Shaun
Support Cloak and Dagger Games on Patreon: https://www.patreon.com/user?u=460039

Crimson Wizard

#16
I don't know how the plugin works, maybe it reports text metrics wrong.

What happens if you make GUI label half-screen wide and with larger height?

Have you tried using explicit linebreak characters in text?

shaun9991

Thanks for the suggestions Crimson Wizard. Unfortunately changing GUI label size doesn't help. I haven't tried using linebreak characters, if that did work I think it would just create more problems with the translation side of things. A bit of a pickle! I've contacted daneeca to see if he faced the same issue when he made Falcon City.
Support Cloak and Dagger Games on Patreon: https://www.patreon.com/user?u=460039

Crimson Wizard

Could you PM me the game, unless its too huge? I could maybe debug it to see if the problem could be spotted quickly enough.

shaun9991

Thanks Crimson Wizard! Would you need the source files or just a build of the game?
Support Cloak and Dagger Games on Patreon: https://www.patreon.com/user?u=460039

SMF spam blocked by CleanTalk