Adventure Game Studio | Forums

AGS Development => Editor Development => Topic started by: Crimson Wizard on 19 Aug 2015, 09:49

Title: AGS Build Server
Post by: Crimson Wizard on 19 Aug 2015, 09:49
EDIT: build server is no longer maintained at the moment.

Nick Sonneveld set up an automatic build server for AGS project.

http://teamcity.bigbluecup.org/

Server is integrated with Github, and starts new build every time new changes are made to the active branches.

It also builds pull requests to simplify testing them. Pull requests are built in two ways: a) on their own, b) merged with the latest main branch (to which they are supposed to be applied). This is to make sure that they are still valid after something was changed in main branches.

To download completed build results, click "Artifacts" button to the right of the build, and download the zip file.

Maybe there is something else; I let Nick tell about it. :)
Title: Re: AGS Build Server
Post by: Wyz on 19 Aug 2015, 10:38
Very nice, very professional. AGS dev can now play with the big boys, good show! :D
Title: Re: AGS Build Server
Post by: sonneveld on 19 Aug 2015, 12:44
Hah oh hey that's me.  Mr Crimson Wizard posted most of the relevant details.

A few things on my todo list for the build server:
- setup linux, android, ios, osx, and psp builds and os builds
- setup osx builds
- implement tests then automatically run them as part of a build job <-- in progress
- build the installer (I have something working on jenkins but I think TeamCity is much nicer)
- support user forks of ags repository
- build plugins
- get a domain name for the build server (the ec2 address is temporary) http://teamcity.bigbluecup.org/
- register for TeamCity's open source pricing so I can support more users/build agents
- build games! (would probably require changes to AGS)
Title: Re: AGS Build Server
Post by: BigMc on 02 Sep 2015, 00:16
What about Android builds? Wouldn't it be nice anyway to offer an apk of stable releases?
Title: Re: AGS Build Server
Post by: sonneveld on 02 Sep 2015, 04:03
What about Android builds? Wouldn't it be nice anyway to offer an apk of stable releases?
hah sure, I just forgot to put it down.
Title: Re: AGS Build Server
Post by: Monsieur OUXX on 07 Sep 2015, 09:11
So let this be clear : this is only for the people who are working on the active branches, right? For example, if one of them works on Windows, then once his code works, he can then just push it to the main branch and get his code compiled for every platform without having to worry about all the dependencies for the other OS's. Am I correct?
Title: Re: AGS Build Server
Post by: Crimson Wizard on 07 Sep 2015, 09:35
So let this be clear : this is only for the people who are working on the active branches, right? For example, if one of them works on Windows, then once his code works, he can then just push it to the main branch and get his code compiled for every platform without having to worry about all the dependencies for the other OS's. Am I correct?
Yes, also
1. It builds not only main branches, but all the pull requests; this means that you may make pull request to our repository, even if in sake of test, and you get your version compiled.
2. Can be used by end-users who want to test new features right away, without waiting for final release.
Title: Re: AGS Build Server
Post by: sonneveld on 07 Sep 2015, 09:41
1. It builds not only main branches, but all the pull requests; this means that you may make pull request to our repository, even if in sake of test, and you get your version compiled.
Also it means that we can find out if your modifications have broken any of the ports without having to manually build them all ourselves and before we merge into master
Title: Re: AGS Build Server
Post by: Monsieur OUXX on 07 Sep 2015, 10:33
OK! Thanks
Title: Re: AGS Build Server
Post by: Crimson Wizard on 13 May 2018, 20:44
Not meaning to say a lot right now, but something has to be done with this.

Since last year the automatic building was disabled. While I was not sure if I ever make any release again, this did not bother me much, but now I did that again, and it began to.

Making a release all by hand currently takes about 2-3 hours of my time, and even more before that to prepare and check everything:
- templates (if they changed)
- build for Windows, build for Linux, build for Android.
- package everything, and upload.
- make a post, edit download links.
- tag a release on github.

The process is extremely tedious and prone to mistakes. What is worse, if there is a critical bug in release, I will have to redo everything again soon.

This is also why I do not like to make new releases often.

Few years ago I had plans to learn making and using automatic building, but then Nick Sonneveld appeared and made one, so I forgot about that and focused on other tasks. That was a mistake of course.

That would be so good to have one for AGS, even if it does not run all the time (i.e. with nightly builds), but only started to make a new release.
Title: Re: AGS Build Server
Post by: morganw on 13 May 2018, 21:03
I think the biggest issue is probably the Windows licence for a server, which wouldn't be cheap to purchase outright, and still isn't cheap to get on a VM that you pay as you use. It might be worth looking at something like Vagrant, so the pre-requisites are automatically setup, anyone can download it to build, and then have somewhere to upload the build artefacts back. It wouldn't be continuous testing/building, but it would it would be server-less (and so free).
Title: Re: AGS Build Server
Post by: tzachs on 14 May 2018, 00:47
Both VSTS (https://docs.microsoft.com/en-us/vsts/build-release/apps/windows/cpp?view=vsts&tabs=vsts) and Appveyor (https://www.appveyor.com/docs/lang/cpp/) allow you to configure CI/CD on their hosted windows servers for free.
Title: Re: AGS Build Server
Post by: morganw on 10 Jun 2018, 16:15
I've been dabbling with a script to download everything and build it, but when trying to compile the first library it looks like I need an older set of build tools.
Quote
Build started 10/06/2018 16:03:16.
Project "C:\Libs\Xiph\libogg-1.3.3\win32\VS2015\libogg_dynamic.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Win32".
Project "C:\Libs\Xiph\libogg-1.3.3\win32\VS2015\libogg_dynamic.sln" (1) is building "C:\Libs\Xiph\libogg-1.3.3\win32\VS2015\libogg_dynamic.vcxproj" (2) on node 1
 (default targets).
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cpp.Platform.targets(57,5): error MSB8020: The build tools for v120 (Platform Toolset = 'v120')
cannot be found. To build using the v120 build tools, please install v120 build tools.  Alternatively, you may upgrade to the current Visual Studio tools by sele
cting the Project menu or right-click the solution, and then selecting "Retarget solution". [C:\Libs\Xiph\libogg-1.3.3\win32\VS2015\libogg_dynamic.vcxproj]
Done Building Project "C:\Libs\Xiph\libogg-1.3.3\win32\VS2015\libogg_dynamic.vcxproj" (default targets) -- FAILED.
Done Building Project "C:\Libs\Xiph\libogg-1.3.3\win32\VS2015\libogg_dynamic.sln" (default targets) -- FAILED.

I haven't got Visual Studio installed, so does anyone know of a way to do the equivalent of "Retarget Solution" from the command line?
Title: Re: AGS Build Server
Post by: Alan v.Drake on 10 Jun 2018, 17:21
I haven't got Visual Studio installed, so does anyone know of a way to do the equivalent of "Retarget Solution" from the command line?

According to google it should be this parameter
  /p:PlatformToolset=v140/v130/v120/etc...


- Alan
Title: Re: AGS Build Server
Post by: morganw on 10 Jun 2018, 19:20
Thanks, got it going for libogg, libtheora has no solution file (so I skipped that for the moment), for libvorbis it seems to keep references to the old Windows SDK (so no include for standard libraries that are installed).

I've now found this:
https://developercommunity.visualstudio.com/content/problem/198082/upgrading-a-solution-from-the-command-line-with-vi.html (https://developercommunity.visualstudio.com/content/problem/198082/upgrading-a-solution-from-the-command-line-with-vi.html)
...so it looks like if you install Visual Studio you get a tool to do the upgrade, but you can't specify a change in the Windows SDK as part of that.

Quote from: Antoine ALLARD
The whole point of using the CLI tools like devenv or msbuild is to automate stuff and not require any user intervention. This is mandatory when trying to build a CI/CD pipeline, having to manually open a project/solution in the IDE to re-target it totally kills this usage.
Indeed.

What I have was designed to be serverless, but it looks like this is going to need modified projects to exist somewhere, so this isn't going to work.
Title: Re: AGS Build Server
Post by: morganw on 11 Jun 2018, 21:57
I used vcpkg to get the ogg, theora, and vorbis libraries, and that has let me script an install of all the requirements and run a build. Unfortunately my built game engine seems to have an input problem, where hotpots aren't registering when the cursor goes over them and room clicks are not processed (clicking on GUIs and characters is fine, but you can't walk anywhere). I've got the same problem on both the 3.4.1 and the master branches.

Anyone have any ideas?

Quote from: msbuild.exe
Build succeeded.

"C:\Libs\ags\Solutions\Engine.sln" (default target) (1) ->
"C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj" (default target) (2) ->
(ClCompile target) ->
  ..\..\Common\ac\gamesetupstruct.cpp(297): warning C4101: 'bb': unreferenced local variable [C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj]
  C:\Libs\ags\Common\font/wfnfont.h(78): warning C4267: 'return': conversion from 'size_t' to 'uint16_t', possible loss of data [C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj]
  C:\Libs\ags\Common\font/wfnfont.h(78): warning C4267: 'return': conversion from 'size_t' to 'uint16_t', possible loss of data [C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj]
  ..\..\Common\font\wfnfont.cpp(38): warning C4267: '=': conversion from 'size_t' to 'uint16_t', possible loss of data[C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj]
  C:\Libs\ags\Common\font/wfnfont.h(78): warning C4267: 'return': conversion from 'size_t' to 'uint16_t', possible loss of data [C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj]
  ..\..\Common\game\main_game_file.cpp(268): warning C4101: 'buffer': unreferenced local variable [C:\Libs\ags\Solutions\Common.Lib\Common.Lib.vcxproj]


"C:\Libs\ags\Solutions\Engine.sln" (default target) (1) ->
"C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj.metaproj" (default target) (3) ->
"C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj" (default target) (4) ->
  ..\..\Engine\ac\datetime.cpp(22): warning C4244: '=': conversion from 'time_t' to 'int', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\global_datetime.cpp(38): warning C4244: 'return': conversion from 'time_t' to 'int', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(67): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(75): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(83): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(91): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(99): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(107): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(116): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(124): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(132): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(140): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(148): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(156): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(164): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(173): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\ac\math.cpp(206): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\debug\debug.cpp(263): warning C4477: 'sprintf' : format string '%d' requires an argument of type 'int',but variadic argument 1 has type 'HWND' [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\gfx\ali3dogl.cpp(503): warning C4838: conversion from 'const int32_t' to 'BYTE' requires a narrowing conversion [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\main\engine.cpp(1628): warning C4477: 'sprintf' : format string '%08X' requires an argument of type 'unsigned int', but variadic argument 2 has type 'PVOID' [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\platform\windows\gfx\ali3dd3d.cpp(765): warning C4838: conversion from 'double' to 'float' requires a narrowing conversion [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\alogg\alogg.c(395): warning C4101: 'data_array': unreferenced local variable [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\alogg\alogg.c(396): warning C4101: 'x': unreferenced local variable [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\alogg\alogg.c(579): warning C4101: 'data_array': unreferenced local variable [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\alogg\alogg.c(580): warning C4101: 'x': unreferenced local variable [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\alogg\alogg.c(955): warning C4101: 'data_array': unreferenced local variable [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\alogg\alogg.c(956): warning C4101: 'x': unreferenced local variable [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\dumb-0.9.2\IT\ITREAD.C(244): warning C4244: '=': conversion from 'long' to 'byte', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\dumb-0.9.2\IT\ITREAD.C(253): warning C4244: '=': conversion from 'long' to 'byte', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\dumb-0.9.2\IT\ITRENDER.C(484): warning C4244: '=': conversion from 'sample_t' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\dumb-0.9.2\IT\ITRENDER.C(485): warning C4244: '=': conversion from 'sample_t' to 'float', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\dumb-0.9.2\IT\ITRENDER.C(2971): warning C4244: 'function': conversion from 'float' to 'int', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  ..\..\Engine\libsrc\dumb-0.9.2\IT\ITRENDER.C(2974): warning C4244: 'function': conversion from 'float' to 'int', possible loss of data [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]

"C:\Libs\ags\Solutions\Engine.sln" (default target) (1) ->
"C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj.metaproj" (default target) (3) ->
"C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj" (default target) (4) ->
(Link target) ->
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _uvszprintf imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _set_blender_mode imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _makecol imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _getr imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _getg imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _getb imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _drawing_mode imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _solid_mode imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]
  alfont_mt.lib(alfont.obj) : warning LNK4049: locally defined symbol _ugetxc imported [C:\Libs\ags\Solutions\Engine.App\Engine.App.vcxproj]

    48 Warning(s)
    0 Error(s)
Title: Re: AGS Build Server
Post by: Crimson Wizard on 11 Jun 2018, 22:27
hotpots aren't registering when the cursor goes over them and room clicks are not processed (clicking on GUIs and characters is fine, but you can't walk anywhere). I've got the same problem on both the 3.4.1 and the master branches.

Anyone have any ideas?

Only thing that crosses the mind is that this is related to 8-bit masks, because this is what hotspots and walkable areas are.
Maybe allegro library was compiled without 8-bit support?
Title: Re: AGS Build Server
Post by: morganw on 11 Jun 2018, 23:20
I wasn't entirely sure if I had to run make any changes if using the patched repository and the existing VS2015 solution.
Would I have to run cmake or configure anything before building it? The build instructions seem to imply that the solution just needs to be built with no configuration.

Potentially, could I have knocked out the 8-bit support by building with the Windows 10 SDK and not specifying any sort of backwards compatibility flags?
Title: Re: AGS Build Server
Post by: morganw on 15 Jun 2018, 18:06
Tried the same process on Windows 7 instead of Windows 10, and ended up with the same result.
Is it definitely the "allegro-4.4.2-agspatch" branch which I should be building?
Title: Re: AGS Build Server
Post by: Crimson Wizard on 15 Jun 2018, 19:10
Is it definitely the "allegro-4.4.2-agspatch" branch which I should be building?

What happens if you use prebuilt libraries (https://www.dropbox.com/s/4p6nw6waqwat6co/ags-prebuilt-libs.zip?dl=0)? They are built from that branch.
Also, I recall rebuilding them recently on my local machine (and constantly use these to build AGS).

May I get the build scripts somewhere to try and see these issues?
Title: Re: AGS Build Server
Post by: morganw on 15 Jun 2018, 23:38
Is it definitely the "allegro-4.4.2-agspatch" branch which I should be building?
What happens if you use prebuilt libraries (https://www.dropbox.com/s/4p6nw6waqwat6co/ags-prebuilt-libs.zip?dl=0)? They are built from that branch.
Thanks for uploading those, I'll try it again tomorrow.

May I get the build scripts somewhere to try and see these issues?
Yes, sure. Which version of Windows are you using?
I did find that some of the string handling in older versions of Powershell caused some problems, so I probably need to make some changes so it works in more places (possibly by not using Powershell).
Title: Re: AGS Build Server
Post by: Crimson Wizard on 16 Jun 2018, 01:06
Thanks for uploading those, I'll try it again tomorrow.

I could swear they are linked in the readme.md, but apparently not, need to do then.

Yes, sure. Which version of Windows are you using?

Windows 7.
Title: Re: AGS Build Server
Post by: Crimson Wizard on 16 Jun 2018, 18:15
I was testing master branch today, and found that walkable areas are broken: https://github.com/adventuregamestudio/ags/issues/457

Earlier you said that you were trying 3.4.1 also, could you double check that you actually built using old code?
Title: Re: AGS Build Server
Post by: morganw on 16 Jun 2018, 23:28
Yes, sure, I'll take another look.
Hotspots were also broken (@HOTSPOT@ wasn't getting updated and I couldn't interact with them), if it is the same issue.
Title: Re: AGS Build Server
Post by: morganw on 17 Jun 2018, 21:21
I don't think I did build the 3.4.1 code, so it is probably the same issue.
Here is a (rough) script that will build the master branch: https://www.dropbox.com/s/ub9zjh1ip2vx0on/bootstrap.cmd?dl=1 (https://www.dropbox.com/s/ub9zjh1ip2vx0on/bootstrap.cmd?dl=1)

Best to only use it in an empty VM, but it works on Windows 7 64bit and Windows 10 64bit.
First run needs elevated privileges, second run needs to be started from a Visual Studio command prompt.
Title: Re: AGS Build Server
Post by: Crimson Wizard on 17 Jun 2018, 23:44
Could you try latest master, Alan pushed a temp fix: https://github.com/adventuregamestudio/ags/issues/457
Title: Re: AGS Build Server
Post by: morganw on 19 Jun 2018, 17:56
This fixes the walking around and the hotspot detection, but character speed must have also been scaled somewhere because movement has slowed down (game speed is fine).
Title: Re: AGS Build Server
Post by: Crimson Wizard on 19 Jun 2018, 18:19
This fixes the walking around and the hotspot detection, but character speed must have also been scaled somewhere because movement has slowed down (game speed is fine).

Well, I guess that means that there was something wrong with the master branch, not build scripts.

Master branch is obviously in a non stable state right now, after merging "ags4" into it. It needs to be tested and fixed.

Perhaps, it will be more convenient to test build scripts against "ags3" branch for a while?