Unicode support

[rbaleksandar]

Is there still no support of Cyrillic?

[Babar]

Request! Request! Request!


This is probably the wrong place to do this, but I also wish for unicode support. It'll help in game translations into languages with other scripts, right?

[rbaleksandar]

Is it really so much work to make it with unicode support? Because if it is, I'll understand why there's no such in AGS (because of so few requests).

PS: I would love to make a game in my own language. But I can't...Also there's a great limitation in the translation possibilities...Or not?

[AdamM]

Well spotted, thanks for reporting this.

Yay! My first bug report! Thanks for the fix CJ.

PS: I would love to make a game in my own language. But I can't...

You can use Cyrillic, as long as you import a font (which you would probably have to create yourself) where standard characters (i.e. A-Z) are replaced with Cyrillic characters. Did you not understand the answer given to you in this topic? It's obviously irritating not being able to use Unicode but this workaround does exist.

[Dualnames]

Great improvements. It looks quite stable for me.

EDIT: Some note, in case you were inaware of this, if you create a section (not type it but just click on the interaction editor of the room), like Room_Load, then go into the script and delete it you get an error (quite normal so no worries but just saying)

[rbaleksandar]

PS: I would love to make a game in my own language. But I can't...

You can use Cyrillic, as long as you import a font (which you would probably have to create yourself) where standard characters (i.e. A-Z) are replaced with Cyrillic characters. Did you not understand the answer given to you in this topic? It's obviously irritating not being able to use Unicode but this workaround does exist.

Sure I did. But you have to create a whole new font for that That's why I'm still asking if AGS supports NORMAL Cyrillic font. Obviously not.


btw Here's something that you should know (the numbers are only the # of the letters):

Latin has 26 big and small characters.
Cyrillic (Bulgarian) has 29 big characters and 30 small.
Cyrillic (Russian) has 32 big and small characters.

So in order to replace a Latin font, I'll have to replace some of the non-letter characters with Cyrillic! What if I need these...Do I have to import another font so that I'll be able to use them?


Not to mention Chinese. There will be a total disaster... As we all know, even to read a stupid newspaper, you need over 3 000 symbols! Educated people know up to 6-7 000. In some dictionaries you can find more than 60 000 characters. If a Chinese guy wants to create an AG with even 3-4 000 characters, how the hell will he be able to replace them with this minor pile of 256 characters?! Next time pls be more polite and think. As I mentioned before: if it's soooo hard to make it, I'll shut my mouth. But no one answered my question.

[Snarky]

ASCII has 128 basic characters (though the bottom 32 are used for control symbols), so even with a Russian alphabet you should have plenty of slots. In game dialogue, you're unlikely to need characters like @, $, %, #, ^, \, [, ], {, }, <, >, |, and _. You can probably even get rid of the number digits. That still leaves you with all of the regular punctuation.

AGS doesn't support Chinese. It wasn't made for making games in Chinese. So when you ask "how the hell" a Chinese guy will be able to make an AGS game in Chinese: he won't. Please be more polite and think.

That said, yeah, the workarounds and the limitations are tedious. Unicode support would be a major improvement to AGS.

I've had an idea for a game set on a sailboat with crew from around the world, each speaking their own language (English, Portuguese, Spanish, German, Russian, Norwegian, French, Turkish...)--like in The Last Express. There'd be no way to support all the different special characters for each language in one 256-character set, and currently I would probably need a different font for each crew member. Unicode support would make stuff like this much easier.

[rbaleksandar]

you're unlikely to need characters like @, $, %, #, ^, \, [, ], {, }, <, >, |, and _. You can probably even get rid of the number digits.

Sorry, got little carried away
But (unlikely != never).
And my question is still without an answer

[Pumaman]

Q: Why doesn't AGS support Unicode?

A: Because over the last 10 years I think I've only seen it requested twice. Therefore it's way down the priority list.

Is it really so much work to make it with unicode support?

It would be a significant amount of work, yes. All of the parts of AGS that handle strings would have to be updated to recognise multi-byte characters, so it would take quite a bit of work to add it.

[Babar]

I've run into the problem once or twice, which is why I brought it up. Translating is easy enough to do if the language you are translating to is of the latin alphabet, but if it is of another script, then it is simply not possible. And (as far as I can see) there really isn't any workaround for this, unless you knew from beforehand that you might want to make a translation, and there is enough space to put 2 alphabets in the font.

I suppose it isn't really top priority, but just to let you know that there is desire for this to be implemented. Now it is 5 times in the last ten years!

[rbaleksandar]

Is it really so much work to make it with unicode support?

It would be a significant amount of work, yes. All of the parts of AGS that handle strings would have to be updated to recognise multi-byte characters, so it would take quite a bit of work to add it.

THANK YOU! Finally an answer Okay, my mouth will remain closed (as long as I don't bump into a problem that I can't solve and need more than just replacing ) Hope this won't happen soon

PS: Thanks to the moderator or whoever made this a thread.

[subspark]

Perhaps then in a future release. The maturitry of AGS is a continuous effort thanks entirely to Chris Jones. While we would all love to see this feature, we respect his priorities.

[Vince Twelve]

In the meantime, you could just make the game with voices, and no text.  I tend to prefer having subtitles in in games, even if they're not very wordy, like a shooter or something.  However, having a game that was spoken in my language with no text would be better than not having a game in my language at all...

[Charity]

I've taken a bit of Japanese, and occasionally I've considered using AGS to make a game or application that uses it.  For example a flashcard or quizzing program.  I don't have any ideas in the works, currently, but I'd like to add my voice to the group that supports and might make use of this functionality.

[Vince Twelve]

Alternatively, if your game doesn't have a huge amount of text, you could make each line an image. 

I did that for Linus Bruckman.  All the Japanese subtitles (about 1000 of them) were sprites that I called by number.  I wrote out the whole game script in a spreadsheet, then numbered each line, then translated each line, then imported the lines as an array of strings with index corresponding to sprite slots to which I imported graphical versions of the text.  Lines and sprites were consecutively numbered and then in the game I used functions like playLines(51, 55, 832, 836) to play a sequence of lines (lines 51 through 55 on the top and 832 through 836 on the bottom).  The downside is that it bloats your filesize and it takes you a long ass time to get all those images made and imported.

Or use SSH's module that lets you use sprites for fonts and then write a conversion program that converts your text from unicode to the sprite slots.  Still time consuming.

If your game is short and light on text, the way I did it for Linus is not a terribly hard work around.  You just need to keep your numbers straight.  If your game is really long, it might be worth it to use SSH's module and take the time to write the code to convert your unicode to sprite slots.

[Sledgy]

I already created Russian + English font support for AGS.




It's easy. Just put your font in to game folder. But for Russian need for some remake standard Cyrillic font. I made some famous, get it here

Rename font to agsfnt0.ttf, agsfnt1.ttf, ... (0, 1, ... - game font number) and put it near game *.exe.

For better view of font - use, via winsetup, 640x400 resolution.

[rbaleksandar]

Making a game with voices is bigger and much more difficult to make than a game without voicepack and only subtitles.

@Sledgy, I'll try that out.

[Sledgy]

For example,
see Russian version of 1213: Episode 1 (by Ben 'Yahtzee' Croshaw), it's not via *.TRA.

Change resolution to 640x400 (in winsetup).

[rbaleksandar]

Great! Works perfect!

[.rhavin]

Ok, just to rise the requests/year quotient: I'd be happy to have unicode support, too. Today, i'ts hard to find *any* font that doesnt at least support a few thousand chars. I'm currently just starting with AGS, and it was quite a surprise that unicode wasnt in there considering the fact that it is included in all major languages for years. I can accept that variables and functions are limited to lower ascii, but text to the user has to be UTF-8 today. On most systems, this is completely handled by the build in functions and .NET already provides the functions to manipulate strings in a way that you dont even have to know how much physical bytes your actual chars need.

Considering the various Interpunktation i usually use to say something ›special‹, „quote“, get lost…, be wise®, and some couple of regular german vocals like äÃ,,öÖüÜ, àŸáºž and some derivates to emphasis language dialects (Š“Š’æÆ), im pretty shure i cant write that with just 255 chars. and â€" of course, it would be a pain because every char i used in this post i simply have on the keyboard.

So, please, UTF-8.

[Monsieur OUXX]

Hello .rhavin.

Yes, AGS is a bit limited on the non-Latin front. But there is much more hope than in 2009, when this thread started!

- About UTF-8 : The Editor does support Unicode, because it's coded in .Net. If I'm not mistaken, you should be able to type stuff like String s = "РÃ'Æ'́Ã'Ã'ÐºÐ¸Ð¹"; in the Editor.
- In the engine, it's different : your game does not rely in any way on .Net. It relies on low-level graphical libraries (I can't remember if it's Allegro or SDL or both). They don't support unicode They do support Unicode but sadly AGS didn't take advantage of these features when it got implemented. BUT since the recent versions of AGS, AGS has been supporting 256-characters fonts. That's good news: Some Cyrillic fonts do exist in 256-chars sets (as opposed to the archaic 128-chars basic ascii).

=> Not too much hassle for you. Read this article, and it shouldn't be too hard for you to generate a cyrillic font that will work. Maybe you could start from a TTF font as explained in the article. I found some here.

Why not try that out and be our first 100% successful cyrillic game maker? 1. Download the font from here. 2. Use the tool recommended in the Wiki article ("Rulaman Wfn Editor") to turn the TTF font into a .wfn font, 3. Still using Rulaman's editor, fix the few rogue pixels that might have appeared in the process. 4. Success, fame, glory, money, men/women.

[Crimson Wizard]

- In the engine, it's different : your game does not rely in any way on .Net. It relies on low-level graphical libraries (I can't remember if it's Allegro or SDL or both). They don't support unicode

Well, this is not correct. Both Allegro and SDL support unicode (AGS uses Allegro), its just that AGS stores all strings as ANSI ones.
Changing that would require rewriting parts of the engine, and also (probably) changing some data formats.

[.rhavin]

Well, i speak german and english and a bit of español, so the're now way i'll do a cyrillic game.

Yes, i can write something like:

Code: ags

 player.Say("Höchstens 200â,¬!");

but on screen it gets:

H?chstens 200?

and writing "Hoechstens 200 EUR" just looks bad. And SDL seems to support unicode, as does allegro.

AGS stores all strings as ANSI ones.
Changing that would require rewriting parts of the engine, and also (probably) changing some data formats.

Is the source public? would that be a big issue?

Edit: ok, found it on github. string.ccp seems to be uncaring wheter it processes unicode or not, it just manipulates bytes. so, perhaps it would be enough to just update the functions actually *writing* something on screen? I noticed you write text as bitmaps on screen and use an outline font / plain font combination. well, a bit outdated, dont you think? why not just use the systems ttf dawing routines and write the text in black on screen, then offset by 2px in both directions and write it in the actual color? i do this all the time when i write text in java and it looks pretty well on all backgronds.

[Crimson Wizard]

string.ccp seems to be uncaring wheter it processes unicode or not, it just manipulates bytes. so, perhaps it would be enough to just update the functions actually *writing* something on screen?

No. The length of the string is an issue. UTF-8 has its length calculated differently.
Also, its not only string.cpp. The String class is not used everywhere in the engine. I've just started replacing C-strings with String class, but there are tonns of C-string occurances in the program.

I noticed you write text as bitmaps on screen and use an outline font / plain font combination. well, a bit outdated, dont you think? why not just use the systems ttf dawing routines and write the text in black on screen, then offset by 2px in both directions and write it in the actual color? i do this all the time when i write text in java and it looks pretty well on all backgronds.

AGS supports both TTF and WFN. People use the way they prefer.

Anyways, if you have a particular code change proposal, you can make a pull request, or place an example here.

[Monsieur OUXX]

@.rhavin  : your latest post doesn't change anything to the problem and how to solve it: it's still as easy. 
- In the article to which I gave you a link, it tells you where to get font that manage accentuated characters (all European accentuated characters you could dream of, including German, Spanish, etc.).
- And since the Editor stores fonts as ANSI, it means they are stored as 256-character tables.

That means you should have no issue in the Editor, and no issue in-game. AGS doesn't support Unicode, but it doesn't need it to manage accentuated fonts that use the standard Western-European latin table, as you plan on doing. It only means that your game cannot be Latin and Cyrillic and Chinese at the same time. 

If your goal by posting here is to get concrete solutions on how to make a "European" game, then by now you have all the solutions. If your goal is to help enhancing AGS on the long term, then I will stop stepping in and let you guys having clever conversations together