Resolving current situation

[Crimson Wizard]

This is what I have to say.

I think that the arised problem is primarily a result of organizational failure. About 1.5 years ago we were discussing how to organize the work, yet, as time passed, it appeared that only JJS and me are working on the engine (and Tzachs was making few changes to the editor). Somewhere in early 2013 JJS became mostly inactive (IIRC he told me he is being busy with real life issues) and I suddenly found myself alone. Feeling pretty much baffled by this fact, at first I made attempt to develop engine further, without much understanding where I am going (the result is now known as AGS 3.4.0 alpha). When people started to actually using 3.3.0 beta, they began to find bugs and ask for more features, so I switched back to that branch, completely focusing on improving the release. Meanwhile, numerous organizational issues were stacking, without being resolved. At one point I made a promise to myself to deal with them after final 3.3.0 release, being afraid of getting lost working on multiple tasks at once. The monkey_05_06's commits to master branch caught me by surprise, because prior to that he had mentioned about his plans to work on updating Anroid port, but never about redesigning anything in the branch, which was being prepared for release.
(I'll put this just in case: this is not an excuse, but explanation.)

I believe that we need to make at least two immediate organizational decisions.

First, we need to activate the strict rule on branch usage. There was already a discussion on how to do this here: http://www.adventuregamestudio.co.uk/forums/index.php?topic=46296.msg636445912#msg636445912 ;
if someone wishes to (re)state his opinion (whether accepting or declining) about this, I think this will be a good time to do so.
In brief, the idea was to not use master branch for anything but immediate hotfixes in case something is found completely broken there. The preparation for release should be done in a separate branch (e.g. release_3_3_0), further long-term development in "develop", separate features, which require serious changes or global redesign of code, - in their respected branches (preferrably created in personal repositories, rather than central one).

Second, there should be an agreement made on what is going to go to the 3.3.0 release.
My personal (but strong) opinion is that its development came to the point when it needs only careful polishing in order to make it as stable as possible, therefore no big changes should be applied in there, nor changes that do not explicitly fix incorrect behavior, but serve better program structure, class design etc; nor compatibility breaking changes - absolutely not.
Again, if someone has different opinion, please state it now.


Additionally, it would be very nice to decide a place to upload the binary dependencies, like AGS.Native.dll, which needs to be updated often. Git repository is not the best place for such files: compiled binaries compress badly, causing repository to bloat and people to get annoyed.

Then, after this is decided, we need to carefully fix the current master (so much as it requires) and proceed as planned.



On other issues.


I suppose that people involved in development should divide the parts of code/groups of tasks between themselves. I am speaking about timed division, not permanent. Such division should not prevent developers from discussing things others doing, or putting their own problems for public discussion. This should be done to avoid duplicating or contradictory changes, and to keep related changes and code more organized.
What is certain: if one at least suspects that his change will break what other is doing, the change should be discussed first, at least personally. Or, to put that other way: developer should at least warn about what he is willing to change, so that this change won't come as a total surprise.
I do not want to become paranoid from thinking that while I am working on a feature or a fix, someone else is doing the same (similar or other way).
Again, any possible tension may be easened by simply doing a work in a separate branch and/or personal repository prior to merging it with main development branch.
It also might be useful to make smaller commits, each related to one change or a number of strictly related changes. This will make it easier to pick out parts of code in conflicting situations.


If a developer is unsure about his change, and feels he is unable to come to decision himself, it is better to once again use personal repository to temporary keep the code for public review.
My suggestion is to keep central repository clean (without fanatism, ofc).


On breaking compatibility with anything.
This is not a simple decision, and there's usually a lot to consider. What is certain, if ever, this must not happen at random without preliminary discussion.
For example, here I started discussing breaking compatibility with old engine plugins: http://www.adventuregamestudio.co.uk/forums/index.php?topic=45810.msg636451744#msg636451744;
Compatibility with older games is primarily important for ports, because you cannot run original Windows executables on other platforms. If there will ever be a version which breaks compatibility with previous versions of AGS, a need will arise to maintain both old and new branches for ports, because the compatible branch may still require fixes and updates.


On changing/removing project settings, or any previously existing features.
There are several important points that must be tested when such change is done:
1. Test that the games created with new editor/engine build do work.
2. Test that old game projects could be loaded to editor, and still work with none or little updates to settings/scripts.
3. Test that older projects (pre 3.0) could be still imported and work "out of the box", with none or only little updates to settings/scripts. This may require converting old settings into new ones in such a way that'll allow to simulate old behavior with new features.
4. Test that the games created with previous versions of AGS still work as intended when ran with new engine. Similarily, some runtime data conversion may be needed.


On other branches.

The "develop" branch contains a lot of changes to the engine code, including several refactored classes and few more or less cleaned up routines (namely the evil dialog routine, that ran a "goto" controlled loop). Currently this branch is in a way separated from all other development, and as time passes, the situation becomes worse.
Hence this is my honest belief that "develop" branch should be considered as a place for any further code redesign and big updates.
The only big problem (AFAIC) about this branch is my ugly temporary solution for setting dynamic number of regions in the rooms. There was some discussion about how to solve this by introducing new UI component: http://www.adventuregamestudio.co.uk/forums/index.php?topic=48513.0
If that (or any other solution) would be implemented, the "develop" will become practically ready for normal public use, if I may say so. (Still, as alpha, but at least people could use it without being confused about how to change areas count in the room).

The custom resolutions feature is currently floating somewhere in the middle (although logically more close to 3.3.0); it still requires improvement. The question is whether to continue developing it as a first successor to 3.3.0, or merge straight to "develop" (which version number will that result in - is something not of a real big concern). If the aforementioned "develop" issue would be properly fixed, there will be nothing that would prevent us from just merging everything in there, and have a combined "no limits"/"custom resolutions" build.

[Joseph DiPerla]

I have a lot I wanted to add to this, but am at work using a cell phone to type... My suggestion is that we use a bug tracker to track bugs such ad Mantis. When a bug is received, a developer would assign to himself or someone else and work on it. More to follow as soo  as I leave work.

[Alan v.Drake]

I have always wondered why people stubbornly kept at commiting to the main branch, but I kept my mouth shut because I didn't dare to overstep my station

I'm sorry I haven't been much helpful this year, sadly I'm basically stuck at working at one of our clients for the whole year (it was supposed to be only 6 months) and when I get back home I have little left to give.

Anyway, I agree with all your points, the branch model we had picked seemed solid and, yeah, I guess it'd be best to stop adding stuff to the current release, I've heard people on IRC complaining about magical blinking and cursor moving issues with the script editor.

About the custom resolution I think it would be best to cram it directly into develop.


@Joseph we already have an issue tracker, and it seems to me it's being put to good use!

- Alan

[bicilotti]

This is what I have to say.

OT: you are doing a great job Mr. Wizard, if we as a community (and me as an individual) can help in any way (dealing with administrative tasks, etc.), just ask!

Your proposed solutions for problems 1 and 2 are sensible, I don't see why anyone would disagree. I think it would be better for everyone to reach a quick consensus on this, git "backtracking patches" is no fun. Regarding a the second point, were you thinking about a "freeze policy"?

[monkey0506]

As to the commits I made (the validity of the contents aside, because it has already been discussed elsewhere), I pushed them to the branch that I was explicitly told to. This still comes down to an organizational issue.

In terms of organizing this project, I refuse to take any part in it. I will not participate in discussions about such things.

[Calin Leafshade]

In terms of organizing this project, I refuse to take any part in it. I will not participate in discussions about such things.

Am I permitted a lol?

lol.

As to point 2, I agree. 3.3.0 needs to have a feature freeze starting now.

We don't want to fall prey to feature creep before this version is stable.

[Crimson Wizard]

In terms of organizing this project, I refuse to take any part in it. I will not participate in discussions about such things.

Well, can you at least give a hint what are you planning to work on in the nearest future?
I just want to have a trust that we understand each other.

[Adeel]

Like many others, I agree with your 2nd point that there shouldn't be further developments in 3.3.0. I believe it would be much better if it's polished enough to be released as a stable version. Once 3.3.0 becomes sufficiently stable, smaller changes can be added to it and after testing them, you can release it as 3.3.1 and so on.

As for the following part:

I do not want to become paranoid from thinking that while I am working on a feature or a fix, someone else is doing the same (similar or other way).

I can totally understand that no one would like investing their time on something which is also being developed by another. As a solution to this problem, I think that there should be a thread created for that, titled: I'm working on..... This thread should be kept public. It'll give two advantages:

1. Engine/Editor Developers will know that they shouldn't work on a specific feature unless the developer who intended to work on that particular feature specifically asked them to do so.

2. Game Developers, upon reading that their wishes are going to be fulfilled, can sit back and relax whilst wishing Engine/Editor Developers a long and happy life.

Just my two rupees...

[Knox]

2. Game Developers, upon reading that their wishes are going to be fulfilled, can sit back and relax whilst wishing Engine/Editor Developers a long and happy life.

I like that one, hehe

[TheBitPriest]

I just want to jump in and say that I'm watching this thread (not that this information matters much, but I'm posting to keep it alive).  I did a bit of work on the path-finding problems, but I did everything locally, unsure about the organization of the project.  I had assumed that CW was the default project lead on the engine at this point.  Is this correct?  Besides CW and Monkey, is anyone else working on the engine?

-TBP

[Crimson Wizard]

If I am the lead, then I am the lead de-facto, not de-jure (simply because I commit more) .

JJS was working on making ports & improving compatibility with older games, but he is not commiting much lately (probably being busy?).

Regarding path-finding, the 3.3.0 is feature-freezed now (only fixing broken things and very minor additions), but we may start preparing improvements for the next version.
Since "develop" is separated too much by tonns of changes and will require more corrections and testing, it might be better to aim for 3.3.1, I think.
If you want to publish your changes for public test, consindering merging it with "official" branch later, I'd suggest you make a personal git repository (cloning from https://github.com/adventuregamestudio/ags), create a new branch, like "feature_pathfinding" or "fix_pathfinding" (etc), forking from current "release-3.3.0", and make commits there.
Then, we will discuss how to manage this further.



UPD: I keep forgetting, there's a much more simplier way: if you are editing the local copy of the repository, then you can create a patch. This way you won't have to create your personal repository on github, nor new branch.

[Crimson Wizard]

I must write couple of normal posts with suggestions regarding further development (something I have in mind for a while), but for now just a couple of smaller notes.

The custom resolutions can't go in 3.3.1, because it's a larger change, and will take more time to polish, so normally I'd like to see it in "limits removing" 3.4.0. On other hand, it will require some time to make 3.4.0 cleaner and usable (also lots of testing), and some people already want to use custom resolutions, or, at least, custom display resolutions (if not the game ones), because players demand the games to be run with their native screen size. And at least one game dev wants to upgrade his recently released game to have this feature.
This is a dilemma... we might have an intermediate release between 3.3.0 and "limits removing" version. Or keep some "temporary" version with just the strictly features we need for running game on any res.


Other thing, there are few games made with Draconian Edition, and people cannot use engine ports to run these, because there are incompatible script functions. Since most (or all) of these functions are good ones, we may consider plan a 3.3.1 release which will not only have fixes to 3.3.0 (if necessary), but also improves compatibility with Draconian Edition.

[Radiant]

I think the simple solution is to give a 3.4.0 beta version that allows for custom resolutions, or alternatively put custom resolutions in 3.4.0 and rename the 'limits removing' to 3.5.0. I agree that these are major changes that don't belong in a 3.3.1 build. However, I should also point out that although removing limits is a good development, few designers urgently need it because the limits are pretty high already. $.02

[Crimson Wizard]

Perhaps I am not giving a clear picture on this situation... the version number is not a big deal. More importantly, the branch that contains limits removal also has more code cleaned up and some things improved, and I'd hate to not use it any longer.

[Radiant]

Then I suggest to work from that. I'm curious how many people urgently need a custom resolution build, anyway.

[BigMc]

My impression is that custom resolution is the hardest problem and potentially most distracting so I would put that into a feature branch and everything else (limit removal and cleanup, Draconian changes) in the develop branch. The current custom resolution build based on 3.3 is usable right? So people could use that or the custom resolution feature branch (that could be regularly rebased on develop).

[Knox]

My impression is that custom resolution is the hardest problem and potentially most distracting so I would put that into a feature branch and everything else (limit removal and cleanup, Draconian changes) in the develop branch. The current custom resolution build based on 3.3 is usable right? So people could use that or the custom resolution feature branch (that could be regularly rebased on develop).

Speaking just for myself, I'm currently using the custom res build (1600x900). It would be cool if we could get the limit removal+cleanup+draconian with a very basic version of the custom res in a very "temp build" for now and I'd be real happy...is that even possible?

If not, I guess I could always revert all the changes Ive made since using the new res build back to fit my old res of 1024x768 temporarily.

[Joseph DiPerla]

JJS was working on making ports & improving compatibility with older games, but he is not commiting much lately (probably being busy?).

As for the ports to iOS and Android... The interface which interacts with the OS and the engine only needs some minor work done which M.M. and Wadjet Eyes seems to have achieved already with their version of games. In fact M.M. has sent me his Android code and hopefully I can make heads or tales of how to make it work officially in AGS. With iOS, I am not very familiar with that. Perhaps Janet C can shed some light on that area as they have with Mac and Linux. The bigger issue with the ports are the C++ side of things. I think JJS set the groundwork up for that and with beta testers, ports are very much possible to do. With Android, with every release I can test a game or feature in particular and report back to you if you would like and see if we can trace the bugs together on that end. And I am sure you will have more than a few willing to do the same with the other OS. But unless JJS decides to come back to working on the ports and adding more OS to the list, or if someone else is willing to work on the ports, this would be the best we can do with that.

Also CW, I remember hearing you were working privately on porting AGS to Allegro 5, not sure if that will break things rather than improve them, however, Allegro 5.1 does support Mac, Windows, Linux, iOS and Android and may break portability to PSP(Which I am not really sure if this benefits us much). The only other major port I would like to see is Windows Phone and support for Windows 8(I haven't tried a game on this yet, so there may be no issues anyway).

The custom resolutions can't go in 3.3.1, because it's a larger change, and will take more time to polish, so normally I'd like to see it in "limits removing" 3.4.0. On other hand, it will require some time to make 3.4.0 cleaner and usable (also lots of testing), and some people already want to use custom resolutions, or, at least, custom display resolutions (if not the game ones), because players demand the games to be run with their native screen size. And at least one game dev wants to upgrade his recently released game to have this feature.
This is a dilemma... we might have an intermediate release between 3.3.0 and "limits removing" version. Or keep some "temporary" version with just the strictly features we need for running game on any res.


Other thing, there are few games made with Draconian Edition, and people cannot use engine ports to run these, because there are incompatible script functions. Since most (or all) of these functions are good ones, we may consider plan a 3.3.1 release which will not only have fixes to 3.3.0 (if necessary), but also improves compatibility with Draconian Edition.

I personally would love to see what ever is in the Draconian edition merged with the current public release. That would be great. In my opinion, as to what comes next after 3.3.1 has all its bugs fixed is to start working in the custom resolutions. For me, the bigger thing would be the limits removed, but it seems that most suggestions are moving towards custom resolutions. But for some reason I feel that trying to get the custom resolutions implemented would cause AGS development to be prolonged further. Would it be easier to add more resolutions rather than a field for custom resolutions for the meantime? For instance, Knox is using 1600x900. Can we add that resolution to the list of resolutions instead of allowing a user to manually set custom resolution numbers? Would that make it less buggy and as a good temporary solution?

Also, are there any limits that can be removed or increased in the meantime as well that wont break anything in AGS(Before removing all the limits completely)?

Personally I feel the priority at this stage is in this order:
1. Bugs.
2. Portability
3. Custom/more resolutions
4. fewer limits
5. Internal implementations of AGS plugins (LUA, RainSnow, etc..) for further backwards compatibility.
6. New Features
7. Forward save game compatibility
8. Further backward game compatibility.

Just my thoughts.

[David Ostman]

I don't know if it's been done, but perhaps put together a survey where you poll the current userbase on what we want next. Highlight it properly as a big red bar at the top of the forums so no one misses it and make it very easy and fast to click through. Should give you a rough idea of what the masses would like the most. Not saying a majority vote always gives the correct path to take, but perhaps a general indication.

As long as I have a decently stable build that I can use at a 640x360 resolution (which I pretty much have now) I'll be happy

[Crimson Wizard]

I don't know if it's been done, but perhaps put together a survey where you poll the current userbase on what we want next. Highlight it properly as a big red bar at the top of the forums so no one misses it and make it very easy and fast to click through. Should give you a rough idea of what the masses would like the most. Not saying a majority vote always gives the correct path to take, but perhaps a general indication.

This is something that is not related to the problem I am trying to solve right now. There's a bunch of barely connected features which are 50-75% ready, and I want to put them all together. As time passes and more things are added, it becomes more and more complicated, and also stupid, because I have some "advanced" code in "develop" branch that I wan't to use with new features, or I'd have to start copying same code around and make things even more tedious and ugly.
I don't think we should add big features before everything is joined, that's it.