[Feature suggestion] (DONE) Translation Editor (Browser tool)

Started by Cassiebsg, Sat 11/01/2020 00:39:32

Previous topic - Next topic

Crimson Wizard

#40
Quote from: tzachs on Wed 27/01/2021 07:02:16
It seems that AGS uses whatever encoding is set in your machine, which is actually not great, as it means that if you export translation on one computer and import it on another you might have these types of issues as well (or even on the same computer if you switched the encoding). The best thing would probably to always use the same encoding (i.e UTF-8) but doing that now will break current exported translations for people.

Anyway, I tried auto-detecting the encoding of the file but the scripts I found for that were unreliable, so I'm going to add a drop-down for selecting the encoding which should then fix it. I'll update when it's ready.

This is a long known problem, but paradoxically at the same time this is the reason it is possible to easily have extended characters for other languages in AGS, as users only have to set their current locale to certain language and use game font for matching ANSI codepage.

The encoding (codepage) selection is what I was thinking to add to AGS editor as well for some time, as it would make it possible to control result rather than rely on system's locale.
Guess what stopped me is that no one reported problems with this yet (then again, maybe they had problems but did not tell), and simply consideration that it's better to focus on making ags unicode aware.

tzachs

@Baguettator, I uploaded a new version which has a drop-down to select the encoding for your file. If you select "windows-1252" then it should show correctly for your file. What I'm not sure about is whether saving the file will work as expected with AGS, let me know if you're still having issues.

Baguettator

That's perfectly working, many thanks for that !!!

I think your web app is very useful :)

Baguettator

I was asking if the encoding of the translation file should be always UTF-8.

I can select windows 1252 to have a normal display for letters like àéè etc...

But maybe I should select UTF-8 just before saving the file ? I read somewhere that the file should be in UTF-8, but I don't know if it is an obligation.

Anyway, it seems I didn't encounter any problems with that, but I encountered problems with the translation editor of Speech Center, because of the \\\\\\\\ bug.... I have deleted all the \ symbols, but I forgot that they were needed in some cases...

Crimson Wizard

#44
At the moment AGS does not recognize utf-8, so if you save it in utf-8 and it contains any extended unicode characters then they wont look correct in the game (they may be printed like two or more random characters, or text will be just cut in the middle).

If you save as utf-8 with BOM, then it may wont be able to parse translation file at all, due to unrecognized header.

Baguettator

Thanks Crimson Wizard, so I don't know what I should do. Which format should I use ?

It seems that AGS creates translation files in UTF-8, as @tzachs said before about my file. Or I don't understand anything :p

Crimson Wizard

Quote from: Baguettator on Thu 04/03/2021 09:23:08
Thanks Crimson Wizard, so I don't know what I should do. Which format should I use ?

It seems that AGS creates translation files in UTF-8, as @tzachs said before about my file. Or I don't understand anything :p

No tzachs said different thing, he said that his program expected UTF-8 before, which was a mistake. Was not Windows-1252 already working for you, or there are other problems remaining?

Baguettator

Ah ok, I understand. Windows 1252 works very well for me, so I can save the translation file in Windows 1252 ?

Baguettator

OK, I have tested a few things, and I have to report a bug.

I restart my translation file from a new one. So I translated the first lines with windows 1252 format. I compile and launch the game.

In game, original texts that were using special characters like àéè etc... weren't translated. Whereas I have translated them in the .trs file.

I tried a new translation file, I openned it with notepad program, translated the same first lines, saved it and launched the game. The texts displayed perfectly, event with such characters as àéè etc...

So I think there is a problem with AGS that don't recognize some format. But windows 1252 probably doesn't work.

I have to say that if I choose windows 1252 format, in tzachs's application, I can see the special characters àéè correctly. But when I save the file, the translation is not working in game.

For the moment, I will go on with notepad until a solution is found. But anyone knows what happens ? tzachs's browser app is very useful, but unfortunately it doesn't work again :(

EDIT : sorry that I insist on this thread, but it took me a long time of research to solve a problem linked with the translation, I thought it was my code but in fact it was the format I presume...

Crimson Wizard

#49
Quote from: Baguettator on Thu 04/03/2021 10:27:34
So I think there is a problem with AGS that don't recognize some format. But windows 1252 probably doesn't work.

To clarify, Windows 1252 is not a format, it's codepage. Format is ANSI. AGS uses ANSI (1 byte per character).

How is your file saved when you work in notepad? Also, what is your system locale for non-unicode programs, is it actually windows-1252 or something else? If you don't know for sure this is how you may find out: https://www.java.com/en/download/help/locale.html

It's important that text codepage and font match each other. Everything else is only a matter of convenience.

But, I never used tzach's application or seen its code, so can only speak about how AGS works. Maybe there's still some problem in how tzach's program saves it.

Baguettator

Thanks Crimson for the help.

Well, my system locale for non-unicode programs is called "Français (France)", so I don't know if it is "windows 1252" ?

When I use notepad, I simply open the trs file with notepad, add the translations and then save it. And it works perfect. I think tzachs is the man of the situation, since the problem seems to come from his great app ! :)

Cassiebsg

If you open notepad, and then do "Save as..." you can see the encoding it's using: ANSI.

So, when you save from Tzachs's app, save it as ANSI.
There are those who believe that life here began out there...

tzachs

Hey, so I think I misunderstood the web api for saving blobs, I gave it the charset of windows-1252 and assumed it's going to encode it with that charset but it seems as though it still saves it as utf-8. There doesn't seem to be a simple built in way in javascript to convert encoding from utf-8, just the other way around.

Baguettator, a workaround can be just to take the output file and then use a different tool to convert the encoding. I found this website which seemed to work fine when I tested it with an output file.
Can you try to take the file you get from my website and feed it to that website and see if it works for you? i.e copy/paste the text to that textbox, select windows-1252, click "Calculate" and then the "text.txt" link? Let me know if that works for you.

Quote from: Crimson Wizard on Thu 04/03/2021 10:35:41
To clarify, Windows 1252 is not a format, it's codepage. Format is ANSI.
Sorry, can you expand on this? It confused me, looking here it seems Ansi and Windows-1252 is pretty much the same thing?

Quote from: Crimson Wizard on Thu 04/03/2021 10:35:41
But, I never used tzach's application or seen its code
If you're interested, code is here.

Crimson Wizard

#53
Quote from: tzachs on Sat 06/03/2021 05:36:35
Quote from: Crimson Wizard on Thu 04/03/2021 10:35:41
To clarify, Windows 1252 is not a format, it's codepage. Format is ANSI.
Sorry, can you expand on this? It confused me, looking here it seems Ansi and Windows-1252 is pretty much the same thing?

I may be not 100% correct in using this term either, but I used to think ANSI is an umbrella term for 1-byte windows encodings, Windows-1252 is one of the several codepages that may be used to encode.
Cyrillic encoding is Windows-1251 for example, and it uses same 1-byte format (with numbers 0-255).

In practice, text editors that save files "as ANSI" usually save in default locale from system settings (they don't forcefully reencode in Windows-1252, as that would quickly become a headache for users outside of the western europe/usa).

Cassiebsg

Hi.

Just a note about using the editor.
The other day I was happily translating when I decided to "undo". For some odd reason, I can't explain, I hit the "back" button on the browser.  8-0 And thought "oh no! Now I've lost everything I've done so far!"  :~( I hit the "forward" and was happy to see the text was still there!  :-D So I translated a few more lines, and then being tired, I decided to save and go to bed... only to find to my horror that the save button didn't work!  8-0  :~( I tried saving the page, in the hopes that I could grab the text from the html page later on, but no dice either.

Probably nothing you can do about it, but just thought I would mention it. So anyone using this editor will save often and don't continue translating after such a fail.  (roll) I missed 50 lines.

I'm not sure having the encoding as UTF-8 is a good thing, but since CW has now made UTF-8 sorta working, then it probably won't matter anymore.  :)
There are those who believe that life here began out there...

Crimson Wizard

Quote from: Cassiebsg on Sat 26/06/2021 14:26:39
I'm not sure having the encoding as UTF-8 is a good thing, but since CW has now made UTF-8 sorta working, then it probably won't matter anymore.  :)

By the way, I will use this opportunity to say that will be very interested to have more testers for utf-8 translations. Right now only LauraHunt agreed to try it out.
https://www.adventuregamestudio.co.uk/forums/index.php?topic=59240.0

And yes, ideally after this feature is added there will be no reason to NOT use UTF-8 in translation files anymore.

SMF spam blocked by CleanTalk