Adventure Game Studio

AGS Development => Engine Development => Topic started by: fernewelten on Wed 26/05/2021 12:44:10

Title: AGS High-Priority Improvement Needs
Post by: fernewelten on Wed 26/05/2021 12:44:10
Quote from: FormosaFalanster on Tue 25/05/2021 07:17:49
It still means that, besides Unity, then AGS is the only one which was used by more than 2 games. Not trying to trumpet AGS here, just trying to get a picture of the situation. In short, AGS is dwarfed by Unity, but it's the only one else that gathered more than a couple users.

In Thinker downloads, 6-7 % were for Linux so far (converting to a proportion of 1 for every 15 downloads). All the others were for Windows. A large proportion of the Facebook viewers came from mobile Facebook (m.facebook.com). So I suspect that there might also be quite a lot of mobile users among the non-Facebook viewers, and all those mobile users might have been very interested in a mobile Thinker but were foiled. But I compiled with 3.5.1 which doesn't offer Android out-of-the-box.

Modern IT distribution being what it is (Windows retreating, Android and iOS coming), game writers might possibly prefer Unity over AGS if it is easier to provide mobile builds with Unity than with AGS.

Crimson Wizard is implementing Object turning functionality, which would let AGS do roughly the same animations easily that you could do with PowerPoint (in combination with Object resizing and the tweening module). That's great to have. I suspect that other adventure building systems have already been able to do this and we are only catching up in this respect. This might also be a reason that coders prefer Unity over AGS.

I think I heard the sentiment that AGS was “good enough” for hobbyist projects but if you want to go professional you need to switch sooner or later (I think from the developers of “Kathy Rain”). That would put AGS into a bad position because then many coders would think it wise to bypass AGS and start with an “adult” system right from the get-go.

AGS doesn't need to lose this particular competition, however: Almost everything within the AGS universe has always been free for professional use ever since Chris Jones' time. You do not automatically lock yourself into remaining a hobbyist if you start with AGS as a hobbyist. But I suspect that over the decades, some very specific issues might have come up that professionals can't do without and if AGS doesn't provide those specific things, then coders will shun all of AGS even if they only tentatively think of turning professional one day.

In any case, being the #2 is still a comfortable position. It is said that in every area with competition only the #1 and #2 survive in the long run. That would still garner AGS a place on the rostrum.
Title: Re: AGS High-Priority Improvement Needs
Post by: Snarky on Thu 27/05/2021 14:42:37
Off topic, but…

Quote from: fernewelten on Wed 26/05/2021 12:44:10
Crimson Wizard is implementing Object turning functionality, which would let AGS do roughly the same animations easily that you could do with PowerPoint (in combination with Object resizing and the tweening module). That's great to have. I suspect that other adventure building systems have already been able to do this and we are only catching up in this respect. This might also be a reason that coders prefer Unity over AGS.

I think I heard the sentiment that AGS was “good enough” for hobbyist projects but if you want to go professional you need to switch sooner or later (I think from the developers of “Kathy Rain”). That would put AGS into a bad position because then many coders would think it wise to bypass AGS and start with an “adult” system right from the get-go.

AGS doesn't need to lose this particular competition, however: Almost everything within the AGS universe has always been free for professional use ever since Chris Jones' time. You do not automatically lock yourself into remaining a hobbyist if you start with AGS as a hobbyist. But I suspect that over the decades, some very specific issues might have come up that professionals can't do without and if AGS doesn't provide those specific things, then coders will shun all of AGS even if they only tentatively think of turning professional one day.

Yeah, AGS has some significant holes and weaknesses as an engine (some of which are currently in the process of being addressedâ€"in particular ScummVM support and eri0o's web engine will really change the game when it comes to AGS's platform support problem), but I don't believe Object rotation is one of them. You already had rotation capability via DynamicSprite.Rotate() (though it was limited to 1-deegree increments), and you still won't be able to do the kind of animation you're talking about at high quality because you'd need things like sub-pixel positioning. (Personally I think a more powerful sprite manipulation API via DynamicSprite and DrawingSurface â€" or perhaps support for shaders â€" would provide more value than implementing a completely separate set of transforms for Objects and Characters and Cameras, but that's just me.)

I would say the main things that need to be resolved to qualify as a "serious" engine are:

-Better platform support (particularly on mobile, including things like touch gestures)
-Increased compatibility with professional dev processes (source control, automated builds, division of responsibilities, tests, etc.)
-Better performance for certain graphical operations (and any support for others)
-Better support for saves that don't break when a game is updated
-Unicode support

I know at least some elements of the first two are being actively worked on. I think the last two are currently backlogged.
Title: Re: AGS High-Priority Improvement Needs
Post by: Laura Hunt on Thu 27/05/2021 16:31:48
Quote from: Snarky on Thu 27/05/2021 14:42:37
I would say the main things that need to be resolved to qualify as a "serious" engine are:

-Better platform support (particularly on mobile, including things like touch gestures)
-Increased compatibility with professional dev processes (source control, automated builds, division of responsibilities, tests, etc.)
-Better performance for certain graphical operations (and any support for others)
-Better support for saves that don't break when a game is updated
-Unicode support

+ Removing (or at least increasing) the 8 audiochannel limit.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Thu 27/05/2021 16:58:02
Quote from: Snarky on Thu 27/05/2021 14:42:37
-Better performance for certain graphical operations (and any support for others)

This IS what object rotation was for. DynamicSprite and DrawingSurface operations are using software functions over a raw bitmap, but object rotation is hardware accelerated, and also allows subpixels in certain display mode.
Besides that, you get rotation on all the object's visuals regardless of how many graphical parts or frames there are. With DynamicSprite.Rotate you would have to rotate, save in memory and apply each single frame in animation. That is not the equivalent to Rotate propery in object.

Not sure what do you mean by "completely separate set of transforms". They are one set of transforms, it's just that script API does not have a base class to allow sharing them between all objects. And internal implementation is similarly disjointed. But principally this is all same thing.

What other graphical operations, or sprite manipulation API are you refering to?

Unicode support is among the priorities for ags4, it's just that we don't have a proper ticket for this yet, but we will.

The support for non-breaking saves is only possible if we remove reliance on global object arrays and enforce a globally unique name for each object. Then it will be possible to sync the loaded data with objects correctly.
But in a longer run better support would require to make game objects dynamically created rather than rely on static data read from the game file.
Title: Re: AGS High-Priority Improvement Needs
Post by: Snarky on Thu 27/05/2021 20:04:41
Quote from: Crimson Wizard on Thu 27/05/2021 16:58:02
Not sure what do you mean by "completely separate set of transforms". They are one set of transforms, it's just that script API does not have a base class to allow sharing them between all objects. And internal implementation is similarly disjointed. But principally this is all same thing.

I mean that I think we're heading towards a situation where some visual effects can only be done via DrawingSurface/DynamicSprite, while others can only be done via settings on game objects. It makes things a lot more complicated if you want to combine different operations, and probably means that some things that should be possible aren't (take, for example, the case of rotating something and then cropping it to a certain bounding box; I don't see how you would do that via the Object rotation properties). Would it not have been possible to instead get improved, hardware accelerated versions of the DS/DS API?

Quote from: Crimson Wizard on Thu 27/05/2021 16:58:02
What other graphical operations, or sprite manipulation API are you refering to?

Scaling and other transforms (e.g. perspective transforms) and distortions, with control over interpolation method; subpixel accuracy in transforms and positioning; filters (e.g. blur, desaturate); composing sprites in various blend modes (e.g. diff); better control over alpha masks; proper 32-bit color handling for RawDrawing; ability to read actual RGB values from bitmap…

There are a lot of things that would be useful to have. We could also ask Dualnames about the various effects he implemented via plugin for Strangeland.

Quote from: Crimson Wizard on Thu 27/05/2021 16:58:02
The support for non-breaking saves is only possible if we remove reliance on global object arrays and enforce a globally unique name for each object. Then it will be possible to sync the loaded data with objects correctly.
But in a longer run better support would require to make game objects dynamically created rather than rely on static data read from the game file.

I thought this (https://www.adventuregamestudio.co.uk/forums/index.php?topic=53753.0) was a really promising approach, and even if it still had limitations it would be an improvement. The goal shouldn't be to handle absolutely every case (that's obviously impossible in principle), but to provide some support for typical, simple changes. (And Dave Gilbert's idea of a way to prevent or at least flag changes that will break saves would also be helpful.)
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Thu 27/05/2021 20:25:11
Quote from: Snarky on Thu 27/05/2021 20:04:41
I mean that I think we're heading towards a situation where some visual effects can only be done via DrawingSurface/DynamicSprite, while others can only be done via settings on game objects. It makes things a lot more complicated if you want to combine different operations, and probably means that some things that should be possible aren't (take, for example, the case of rotating something and then cropping it to a certain bounding box; I don't see how you would do that via the Object rotation properties). Would it not have been possible to instead get improved, hardware accelerated versions of the DS/DS API?

I believe that it is practically possible to feature cropping for the whole object. Of course this cannot be done with "Rotation" property alone, but same is with sprite operations, one "rotation" parameter does not achieve that, it's a combination of parameters and operations. E.g. you define a "canvas" size and each operation will be clipped within that canvas. Maybe we could define "canvas size" for a whole object, and clip every operation within likewise.

Hardware accelerated variants of dynamic sprite and drawing surfaces are theoretically possible, except in current state of the engine that would require significantly larger effort and rewrite. Likely there will be two separate implementations for software and ha modes. Or two separate classes for that even: a class that contains a real bitmap and a class that describes a stack of operations over the image.
Of course not only the dynamic sprites themselves should be rewritten, but the way sprites are applied when preparing a render - all that should be changed.

But even having that, individual sprite editing and object transformation are meant for different use cases. Each of these have their purposes.
Like I mentioned above, whole object rotation is for when you want to rotate their whole view without dealing with each separate frame. Or rotate a composite object like GUI which have also child objects on it.

Uniform set of operations in theory may be the final goal. Where same operations may be applied to any component of an object in a hierarchy, complementing and overiding each other in a certain order.
For example: sprite -> frame -> loop -> view -> game object.
Iin principle there should be no difference between rotating and cropping a sprite and rotating and cropping a whole object.

What blocks this is mainly that AGS engine has each object and thing implemented individually, so you have to remake same operation multiple times differently (more or less) for different types. I don't think a good progress is possible without first remaking much of the drawing (and not only drawing) logic, that all game objects would share.

To give an example, GUI currently are drawn using software methods, they have all their controls painted on a bitmap first, then converted to a video texture. This means that it would be impossible to apply a hardware accelerated version of DynamicSprite to GUI or one of its controls without first converting it to a raw bitmap. There may be other similar cases.


EDIT: fixed spelling....


Quote from: Snarky on Thu 27/05/2021 20:04:41
I thought this (https://www.adventuregamestudio.co.uk/forums/index.php?topic=53753.0) was a really promising approach, and even if it still had limitations it would be an improvement. The goal shouldn't be to handle absolutely every case (that's obviously impossible in principle), but to provide some support for typical, simple changes. (And Dave Gilbert's idea of a way to prevent or at least flag changes that will break saves would also be helpful.)

I am not going to return to that anymore. The approach was not promising at all, it had very narrow use case and alot of potential for giving false expectations and then screwing things up.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 28/05/2021 00:21:36
I'd also add that it's theoretically possible to add shader support in a naive way:

Code (ags) Select

builtin managed struct Shader
{
    /// Create pixel shader
    static Shader *Create(const string shader);
};


And then let attach that to an object, or whole screen.

This way the shader will have to be compiled though, and also depend on renderer (DirectX shader, OpenGL shader, and so on will have to have separate shader scripts).
Because compiling a shader is not supported by all the systems, we may have to additionally support loading precompiled shaders from resources.

Anyway, I just realized this all is very much offtopic here. AGS development have long suffered from random ideas thrown in then implemented sporadically or forgotten. That would help if this all would be discussed thorougly to determine a roadmap for these and similar changes.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 28/05/2021 19:38:02
In regards to Unicode support: https://github.com/adventuregamestudio/ags/issues/1297
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Fri 28/05/2021 21:08:12
About unicode, do you have an idea on the order to tackle? It's a ton of areas to cover... I guess text display has to come first so we see the results? This could be tested by hardcoding a utf-8 string somewhere near the gfx drivers and seeing what appears on screen... But I mean, we have to start somewhere, it's not possible to do everything at once.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 28/05/2021 22:29:45
We do not have to do everything at same time of course, some if not all of the mentioned parts could be done separately. It's just that certain texts will get broken until everything is completed.

But if you like to test by seeing fonts on screen, then definitely font renderers is the first thing to do. These are classes TTFFontRenderer and WFNFontRenderer, and any utility code they use (like alfont & freetype library).

Graphic drivers do not deal with the text at the moment, they receive it ready drawn on bitmap.
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Fri 28/05/2021 22:44:51
Ah, very cool, everything is in Common then, which is a smaller surface to work with than the engine.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Sat 29/05/2021 00:37:49
Also opened a proposal list for 4.0.0 milestone: https://github.com/adventuregamestudio/ags/issues/1298

This is a draft for now, but my intent was to emphasize which tasks should be top priority.
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Sat 29/05/2021 02:37:05
Accelerated GUIs would still draw sprites of buttons and other elements and then manipulate them in an accelerated HW accelerated I think, at least to work with what's available.

There's a very interesting PR that's cooking for SDL 2.0.16, that adds vertex support for SDL Renderer, it would allow drawing triangles with it. So in the future it may be possible to use this to draw things directly, but there would be some effort to convert things. The API is being done so it would be possible to support imgui in the SDL Renderer which currently isn't possible and is a pain point of many game devs that wants to use both.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Sat 29/05/2021 02:57:58
Quote from: eri0o on Sat 29/05/2021 02:37:05
Accelerated GUIs would still draw sprites of buttons and other elements and then manipulate them in an accelerated HW accelerated I think, at least to work with what's available.

Sorry, but could you elaborate, because I don't think I understand what you meant here?
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Sat 29/05/2021 03:45:59
When no sprite is assigned to a button, AGS draws a gray button Instead. Same for slider. If you resize those in script, they get adjusted too. I imagine if the hardware accelerated part needs to work with the sprites when they are assigned, to make things easier, when there's no sprite assigned the gray version of these GUI Controls would be generated as bitmaps still.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Sat 29/05/2021 04:06:32
Quote from: eri0o on Sat 29/05/2021 03:45:59
When no sprite is assigned to a button, AGS draws a gray button Instead. Same for slider. If you resize those in script, they get adjusted too. I imagine if the hardware accelerated part needs to work with the sprites when they are assigned, to make things easier, when there's no sprite assigned the gray version of these GUI Controls would be generated as bitmaps still.

Yes of course, in the current system there has to be a raw bitmap created first, so, similar to the text overlays, these controls would have to generate some bitmap to be converted to texture by a renderer.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 11/06/2021 05:48:04
So, I looked into how Allegro lib supports Unicode, and apparently it has a set of functions that switch their meaning depending on mode. This is not the fastest (perfomance-wise) solution, but this may allow to have a switching mechanism to support both ASCII and UTF-8 modes in one engine with minimal changes to the code.

I did a small experiment to see how much changes would it take to just draw UTF-8 texts loaded from translation file with properly split lines, and basically this is it:
https://github.com/ivan-mogilko/ags-refactoring/commits/ags4--unicode1
(with this test build I was able to display utf-8 encoded TRS with a proper unicode font)

So, theory is, if we could add some data field to the game files to tell their encoding, engine could switch between ascii and utf-8 even in realtime, and thus support both kind of data (thus retaining backward compatibility).

Full unicode support would of course require more changes. For example, any char-by-char string manipulation in scripts will not work correctly with unicode ones - unless we add new API for getting unicode characters. And text input requires different handling.

Still I feel like complete idiot for not trying this years ago...
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Fri 11/06/2021 06:15:51
It's really great news CW.
If you add it I can write Persian font easily in AGS. I can help you for any test if you want

Title: Re: AGS High-Priority Improvement Needs
Post by: Cassiebsg on Fri 11/06/2021 15:32:14
Awesome, great work CW!  (nod)
Title: Re: AGS High-Priority Improvement Needs
Post by: FormosaFalanster on Sat 12/06/2021 01:26:56
Wait, do I understand correctly I'll be able to use traditional Chinese characters?
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Sun 13/06/2021 07:44:14
Here's an experimental engine build based on AGS 3.6.0 to let anyone willing to test it:
https://www.dropbox.com/s/cnlkmjtz9ckcodv/acwin--unicodetra.zip?dl=0

(it got to come with SDL2.dll for now)


VERY IMPORTANT:
The build is in experimental state and does not cover all potential issues. Noteably your game itself (script, etc) should not use any extended characters (chars 127-255), or text will be broken. This is a temporary problem (or so I hope).
Unicode text works only in translations (not script, or property fields).

HOW TO USE:
Create your game's translation as usual, and save TRS as UTF-8 without BOM. Then you can type any unicode texts as a translation there.
Note that Editor does not care if it's UTF so long as there's no BOM (BOM is basically a small binary header in the file, and it breaks TRA compilation).
Remember to provide a proper Unicode font for this translation (not the hacky kind of font you usually use for AGS translations).

WHAT IS SUPPOSED TO WORK so far:
* Texts displayed with TTF fonts
* Both Left-to-right and Right-to-left text modes.
* Using most String functions over translated unicode strings (appending, formatting, comparing, etc).
* TextBox control should accept unicode characters and properly work with them (like deleting with Backspace).

TODO:
* Some String functions only accept or return byte-chars, which makes them incompatible with unicode. Such as: AppendChar and Chars[]. We'd need to either change these or add alternatives.
Also, not sure yet how to correctly support individual unicode characters in String.Format (unicode strings work with %s, just like normal strings).
* Text parser
* Saving and loading strings using File* (or maybe it already works, I have not tested it).

Example of a test project (AGS 3.5.1) with russian translation:
https://github.com/adventuregamestudio/ags/files/6643382/test--unicode.zip


Quote from: FormosaFalanster on Sat 12/06/2021 01:26:56
Wait, do I understand correctly I'll be able to use traditional Chinese characters?

Hopefully yes. But really, you tell me if it works, because I don't know how to type these.
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Sun 13/06/2021 16:51:07
Thanks a lot, CW for the fast feature.
Is it only works on the translation option?
I didn't use translation yet. I used 
Code (ags) Select
player.Say(" something "); and doesn't work for the Persian language

Where is my wrong and what can I do?
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Sun 13/06/2021 21:29:38
Quote from: Mehrdad on Sun 13/06/2021 16:51:07
Thanks a lot, CW for the fast feature.
Is it only works on the translation option?

It's not a feature yet, it's only an experiment. Yes, for now it only works if you put language in translation (TRS), because that was easiest to do.

So, if you do not want to bother with real english language, I may only suggest to write simple placeholders in the script (like "Character tells his name") and then provide real text in proper language in translation.

For now this is made only for a test, and also maybe to allow translate existing games to new languages.


EDIT: You may download the ready project I posted above: https://github.com/adventuregamestudio/ags/files/6643382/test--unicode.zip
And create a more translations for it, or replace existing one.
To test string operations press Space key in the game.


UPDATE: Uploaded new version (https://www.dropbox.com/s/cnlkmjtz9ckcodv/acwin--unicodetra.zip?dl=0), now supports TextBox input so you can type unicode characters there.

!!! Just don't forget to assign a proper unicode font to GUI controls where you display these texts, otherwise you'll see garbage.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Tue 15/06/2021 02:53:13
I made a test with Persian characters, and found that Right-to-Left text direction is not working properly at the moment (it breaks some characters), so this got to be fixed.
It works left-to-right though, but obviously it means that text is inverted.
UPDATE: updated build: https://www.dropbox.com/s/cnlkmjtz9ckcodv/acwin--unicodetra.zip?dl=0
Spoiler

Idk if it's a correct text or not, I used Google translate, and maybe not the best font for this language:
(https://i.imgur.com/Kz6TAOs.png)
[close]

I was not able to make Chinese work for some reason, could be I use wrong font or something else, I need to investigate this further.
UPDATE2:
Ok here you go Chinese:
Spoiler

(https://i.imgur.com/e6bGl0U.png)
[close]

I used this free GNU font: https://unifoundry.com/unifont/index.html
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Tue 15/06/2021 06:06:36
Thanks a lot for try Persian font. It seems couldn't stick letters together. I typed improve your text in this picture

(http://www.upsara.com/images/c178904_.png) (http://www.upsara.com/)

Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Tue 15/06/2021 06:23:12
Quote from: Mehrdad on Tue 15/06/2021 06:06:36
Thanks a lot for try Persian font. It seems couldn't stick letters together. I typed improve your text in this picture

Sorry, to clarify, did it work for you in game, or you painted over the screenshot?

The letters not joining may be either a problem of a font, font's size, or font library that draws it.

I found some technical description of this problem here: https://stackoverflow.com/questions/49110006/arabic-joined-up-text-in-freetype
It sounds like this may be complicated with these kinds of languages.

We may also see if SDL2 library has better TTF support for this than our current library.
Some more technical info found: https://discourse.libsdl.org/t/sdl-for-farsi-or-persian-text/25292/4
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Tue 15/06/2021 07:02:12
I painted on the picture. I want you see right typing. Your example shows wrong for Persian letters because letters don't stick together.
It doesn't relate to fonts

I really accept that Persian letters are complicated. Sometimes they are stick and sometimes not.

There is a great tools for type Persian in engines. Maybe help you
https://leomoon.com/downloads/desktop-apps/leomoon-parsinegar/
Title: Re: AGS High-Priority Improvement Needs
Post by: Laura Hunt on Tue 15/06/2021 07:15:26
This is amazing, CW! The Chinese market is huge, and the idea that the "big languages" to go for are ES, IT, FR, DE, PT-BR and RU is quickly becoming obsolete, so being able to localize to Chinese without using weird hacks is going to be a massive game-changer.
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Tue 15/06/2021 14:44:29
Edit: nevermind...cw, check the issue in GH, I think I got something
Title: Re: AGS High-Priority Improvement Needs
Post by: Manu on Tue 15/06/2021 22:40:38
Probably it's not a high priority, but talking about the translations, have you ever thought about a feature like "Move Default Language to Translation File"?
This action will move all the strings of the game from the code to a file such as English.trs, replacing the text with placeholders. Placeholders could use a few words of the text (just to make it more readable), followed by a random number to make them unique (something like "%HIMARYHOWAREYOU-157563%", or maybe something smarter I don't know).

In English.trs you will have 2 lines, the first one is the placeholder:
Code (ags) Select

%HIMARYHOWAREYOU-157563%
Hi Mary, how are you?


The other languages will be the same, but when you create a new translation, instead of 2 lines for each text, the system could generate 3:

Code (ags) Select

%HIMARYHOWAREYOU-157563%
// Hi Mary, how are you?


The second row is the full text, as a reference for the translator, but commented. I guess the compiler will remove first the comments, so it will end always with 2 lines.

This way, when you want to change the main language text, you just change the trs and you don't need to adjust the other languages, which is a pain now.

When you add a new text, eg. Say("Whatever"), you can just add it to the code, as always, this won't break the game. The engine will detect if it's a placeholder or not, and if it's not a placeholder it will show the hardcoded text. But when you do an "Update", the string will be moved to the translation file.


Maybe it's a crazy idea, but I thought it was worth suggesting it.



Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Tue 15/06/2021 22:52:31
@emabolo, similar thing was proposed before, and afaik this is how they often script translation systems in other engines - by using keywords and line identifiers in script instead of a literal text.

One of the alternatives is to have strings like "%ID% TEXT" where you can change TEXT part as much as you like, as only %ID% will be important for translations.

In theory you can already do that, except you have to do everything by hand and there will be "default language" selection in the engine and setup which should not be chosen by player.
Or script translation system that works similar to described alternative (I think some people already did this in their AGS games)

If implemented as a feature, transition to such system should be done with careful planning, and guess we may keep support of an old method of having texts in script in case someone wants to do it old way.
Title: Re: AGS High-Priority Improvement Needs
Post by: Manu on Wed 16/06/2021 07:19:16
Quote from: Crimson Wizard on Tue 15/06/2021 22:52:31
In theory you can already do that, except you have to do everything by hand and there will be "default language" selection in the engine and setup which should not be chosen by player.
Or script translation system that works similar to described alternative (I think some people already did this in their AGS games)

Right, in fact, more or less it's all already supported. It's more a utility to make the transition from hardcoded to external translation easier. And once you have migrated, AGS should just avoid running the game with placeholders instead of real text.
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Wed 16/06/2021 11:19:38
@emabolo, Have you ever used Speech Center plugin?

I think switching to a token based dialog in the current editor will be confusing, in IDEs that have support for this you usually can edit the text of the token directly where it's used - it parses the AST and replaces the element directly in it, so you can read how things are, even with the token underneath. This sort of thing is not possible (or not possible easily) with Scintilla (specially our aging one), which is really guided towards text editors and not AST editors. So once you get a dialog with many sentences you will be alt+tabbing screens like no tomorrow. At minimum it would have to support a split view where you have the text and the text with the replaced tokens in other panel so you can see the context of sentences - faster than running the game.

I also get a feeling this auto token generation can reuse code from the speech numbering - and it's possibly not that hard to parse the sentence as is generated today and get just the speech line number and then join this with the character and handmade your own string token replacement - you now load this character+line_number from a dictionary and replace the string.
Title: Re: AGS High-Priority Improvement Needs
Post by: Manu on Wed 16/06/2021 13:53:25
Quote from: eri0o on Wed 16/06/2021 11:19:38
@emabolo, Have you ever used Speech Center plugin?

No, but if you think it's useful to manage translations I'll try it. Thanks


Quote from: eri0o on Wed 16/06/2021 11:19:38
I think switching to a token based dialog in the current editor will be confusing, in IDEs that have support for this you usually can edit the text of the token directly where it's used

In my idea this should be an option, I wasn't thinking about switching to tokens by default. Also, I think this is an option that should be used when the game is complete or almost complete.

I can think of another advantage: imagine your game doesn't have translations, it's in English only, but you want to send all the text to a professional proofreader. In "hardcoded mode", when you receive the corrections, you will have to copy and paste everything, line by line, into the code. Argh. While if you activate the "tokens mode", it's just a matter of copying the file to the game folder. Much easier.

Title: Re: AGS High-Priority Improvement Needs
Post by: Laura Hunt on Wed 16/06/2021 14:36:53
Quote from: emabolo on Wed 16/06/2021 13:53:25
I can think of another advantage: imagine your game doesn't have translations, it's in English only, but you want to send all the text to a professional proofreader. In "hardcoded mode", when you receive the corrections, you will have to copy and paste everything, line by line, into the code. Argh. While if you activate the "tokens mode", it's just a matter of copying the file to the game folder. Much easier.

The "proper", or at least most convenient, way to go about this would be in fact to generate a .trs file, call it something like "EN_corrected", send it to the proofreader and tell them to add their corrections below each original line, as if they were translating the file. Then you just use that file as the game's default "translation": https://www.adventuregamestudio.co.uk/forums/index.php?topic=58812
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Wed 16/06/2021 15:38:45
IMO identifying lines with tokens in the strings is more convenient and safe than relying on actual text, which may be changed anytime, which in current system will break any existing translations.
There are other reasons to not rely on the text itself: long known issue is that one line in one language could be translated with different lines in another language depending on the context. You won't have this problem with the unique tokens.

Naturally this token may be merged with the voice-over ID and then we have one token for everything (translation and voice-over).

Of course, the token does not have to replace the actual text, it may simply precede it as an optional method to identify the line. Then you also have the text which may be used either as a default language or just for the reference.
Title: Re: AGS High-Priority Improvement Needs
Post by: Manu on Wed 16/06/2021 16:43:16
Quote from: Laura Hunt on Wed 16/06/2021 14:36:53
The "proper", or at least most convenient, way to go about this would be in fact to generate a .trs file, call it something like "EN_corrected", send it to the proofreader and tell them to add their corrections below each original line, as if they were translating the file. Then you just use that file as the game's default "translation": https://www.adventuregamestudio.co.uk/forums/index.php?topic=58812

Sure, but then you end up with a hardcoded English version (where the English text is used as tokens basically), and a "real" English version in the file. This is what I was going to do when I realized it was not a clean solution and a token version would have been nice :)
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Thu 17/06/2021 07:22:54
@Crimson Wizard
Can you send me your Persian font ? I tested many fonts and AGS doesn't show all symbols on the Font page
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Thu 17/06/2021 07:50:25
Quote from: Mehrdad on Thu 17/06/2021 07:22:54
@Crimson Wizard
Can you send me your Persian font ? I tested many fonts and AGS doesn't show all symbols on the Font page

I linked it above: https://unifoundry.com/unifont/index.html
Here's direct download link: https://unifoundry.com/pub/unifont/unifont-13.0.06/font-builds/unifont-13.0.06.ttf

EDIT: I now remembered that for Persian test I simply used Arial.ttf from Windows fonts.

But AGS Editor cannot show all unicode symbols on the font preview, because it shows only first 256 indexes, while unicode font may contain thousands. I only changed the engine, not the editor so far.
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Thu 17/06/2021 15:22:25
@Mehrdad, if you want to try, this is a build of the engine of ags3 with only support for translated Arabian languages: ags_full_persian.zip (https://github.com/ericoporto/ags/files/6664406/ags_full_persian.zip)

It doesn't have bidirectional text support, so you need to set it in the Editor for the specific game. Below was using DejaVu Sans. It does breaks lots of things so may not be possible to roll it in ags3. It's using CW changes and a different font renderer for ttf fonts. It still has some bugs but it comes with text shaping. Try with GNU Unifont too, I couldn't tell if it was correctly shaped or not on non-anti aliased fonts.

Spoiler
(https://user-images.githubusercontent.com/2244442/122256147-2627cb80-cea5-11eb-94dd-fccd32087862.png)
[close]
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Thu 17/06/2021 16:22:08
Hi @eriOo

Sorry for this question, How can use this file? Do I have to replace in a folder of the editor? Is it work with 3.6 too?
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Thu 17/06/2021 17:11:36
@Mehrdad, you use can rename it as acwin.exe and place in the directory with the Editor (AGS 3 Editor), or just use the Editor normally and run it in the directory that the game is built. Follow CW's instructions regarding the unicode translation file, using his test project.

Link to post with instructions: https://www.adventuregamestudio.co.uk/forums/index.php?topic=59170.msg636636892#msg636636892

It's built on top of CW's ags 3.6 utf-8 updates, so should work with ags 3.6 too!

Feel free to ask more questions :)
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Fri 18/06/2021 08:21:29
Sorry but I didn't success yet. I got this pic
(http://www.upsara.com/images/r393202_.png) (http://www.upsara.com/)

I use UTF-8 without BOM too. Is it related to "Windows System Locale"? It adjusted on Persian for me
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 18/06/2021 08:49:08
Quote from: Mehrdad on Fri 18/06/2021 08:21:29
I use UTF-8 without BOM too. Is it related to "Windows System Locale"? It adjusted on Persian for me

No, it should not be related to system locale.

Can you explain what did you do, or maybe upload your project?
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Fri 18/06/2021 08:56:49
It's the same as your project and I just added Persian.trs
Here you are
https://www.dropbox.com/s/9bvt8e9fibgf1wg/test--unicode_Persian.rar?dl=0

Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 18/06/2021 09:15:15
Quote from: Mehrdad on Fri 18/06/2021 08:56:49
It's the same as your project and I just added Persian.trs
Here you are
https://www.dropbox.com/s/9bvt8e9fibgf1wg/test--unicode_Persian.rar?dl=0

Your Persian.trs is saved as UTF-8 with BOM. You need to save it simply as UTF-8.

PS. That said, when this is done properly we may adjust translation compiler to check for BOM too, that would solve this inconvenience.
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Fri 18/06/2021 12:30:35
Great. It works now!.
CW,  for the Persian language I need to turn on Right to Left in General Settings, and all of my words change like save and load the menu and so on. Is any way to change it only for talks?
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Fri 18/06/2021 12:47:24
Quote from: Mehrdad on Fri 18/06/2021 12:30:35
CW,  for the Persian language I need to turn on Right to Left in General Settings, and all of my words change like save and load the menu and so on. Is any way to change it only for talks?

I know that you can turn it on for particular translation in TRS (find line //#TextDirection= ).
But unfortunately I do not think AGS supports setting this only for chosen lines.

I think eri0o suggested to be able to set this for individual fonts, which may be a good idea for the future.
Title: Re: AGS High-Priority Improvement Needs
Post by: Mehrdad on Fri 18/06/2021 14:37:16

Quote

I know that you can turn it on for particular translation in TRS (find line //#TextDirection= ).


Ok, thanks. It works but it affects on GUI texts again like the same turn on Right to Left in General Settings.
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Sun 20/06/2021 05:20:29
Ok, since this solution seem to have a potential, I opened the pull request:
https://github.com/adventuregamestudio/ags/pull/1321

If this works, unicode translations may be supported in AGS 3.6.0 already (fingers crossed).
Title: Re: AGS High-Priority Improvement Needs
Post by: eri0o on Mon 21/06/2021 01:10:13
Here's where the Continuous Integration builds this PR if someone wants to pick up this for tests

https://cirrus-ci.com/github/adventuregamestudio/ags/pull/1321
Title: Re: AGS High-Priority Improvement Needs
Post by: Crimson Wizard on Mon 21/06/2021 01:20:17
Quote from: eri0o on Mon 21/06/2021 01:10:13
Here's where the Continuous Integration builds this PR if someone wants to pick up this for tests

https://cirrus-ci.com/github/adventuregamestudio/ags/pull/1321

Oh right. "Editor_packaging" is the item that contains fully built editor + engine installer and zip.