Topic: PLUGIN: Direct3D v1.0 (Sprite and Theora video rendering) [2012-02-08]

[Cassiebsg]

Okay, guess I'll have to confine to having to compile and upload two versions of the game, one with video and one without. 

Ps - I've now added some #define code, that Gurok suggested for this effect (creating 2 versions of the game)...

[Gurok]

Here are the instructions I sent Cassie:

--

Make a new script file and put it at the top of your script list.
Put this in the header:

Code: [Select]

#define VIDEO_VERSION     
Change your code to
     

Code: [Select]

#ifdef VIDEO_VERSION
D3D_Video* video; // defined at the top of the script
#endif

function room_Load()
{
#ifdef VIDEO_VERSION
    // load the video code
#endif
#ifndef VIDEO_VERSION
    Display("Couldn't load the plugin");
#endif
Use #ifdef VIDEO_VERSION anywhere D3D_Video stuff is referenced.
When you want to make the non-video version, comment out the #define line.

--

I'm posting this for future people who stumble across this thread, in the hope it helps others with the problem CW outlined above.

[Crimson Wizard]

In the past I wrote a feature which let you to configure stubs for plugin functions. I gave it out for testing, but no one seemed to be interested, and I later forgot about it myself thus it never ended up into the official version.

[Gurok]

Ah, I was just suggesting something like this to Cassie on IRC.
This would be great. Is the branch still around? The link to the branch didn't seem to work for me.

[Crimson Wizard]

It should be working, still in my repository. But the branch is very outdated and has to be rebased.
Branch name is "improve-scapimap", you could try fetching it in git.


Although, today I'd rather try some different approach which would do similar thing automatically, if compiler could add a note on imports that belong to plugins to let engine distinct these.

[Cassiebsg]

So, is that something I could use/implement or does it require more than what I can? 
Cause that sounds just like what I need. 

I can either offer my project for testing, or just make a snippet of it, for the purpose, if you like.

[Cassiebsg]

Okay, quick question/problem.

I decided to use the Credits Module by SSH to scroll the end credits, and all was going pretty good, until I finally was done rendering the movie and added it to the project...  Apparently it suffers from the same problem that CWs TypedText suffered. But no matter what I try, I just can't get the video running on the BG. 
Now this isn't that big a deal, since the video is mostly the ships just moving from the right to the left, so AGS can do that easy with a few sprites, but thought I would mentioned just in case you want/can fix this.

If you figure out what is causing this, or how to solve it, I'll be all ears. 

[AJA]

Are you using the correct render stage?

Code: [Select]

video.renderStage = eD3D_StageBackground;
That should render it immediately after the game background finishes rendering and before anything else.

[Cassiebsg]

Erm... yes I had that... but now I'm unsure what happened, cause now it's working. 

Last time I had tried it was running the video, and then the credits after the video was finished. 

[eri0o]

AJA, are you still around? I would reaaaally like to take a look at your code to see what you needed to initialize of Directx in it and what you didn't because AGS had already done it for you.

[AJA]

Sure, here's the source code. Do whatever you want with it. 

[eri0o]

Oh, this is amazing, thank you 

[rmonic79]

Hi Guys anyone solved scaling and aspect ratio issues with 3.5 and also the crash on Fullscreen/windowed switch?

[nightmarer]

Thank you very much for this useful plugin

 

[nightmarer]

Hello everybody.

My videos does't load if I don't set the property video.isLooping to true.
But it is strange that if I enter a Display("") to pop-up a message pausing the game after calling the function, the video is Displayed while the action is paused.


   

Code: Adventure Game Studio

1.     videofile="Rosefall.ogv";    //here I set the videofile that must be load
2.     videoloop=false;       //this is a global bool to set as value in video.islooping
3.     playbackground (videofile); //this is the call to the global function
4.     Display("Playing"); //If I enter this I see the video running, but if not the video is not shown
5.     stopplay();  //this is how I call the function to set video as null, because it didn't work in the original function in the else statement
6.     videofile="Nebula2.ogv"; //setting another video
7.     videoloop=true; //this is the way to set that the loop works agaoin
8.     playbackground (videofile); //and this oe works fine

This is how I modified this functions.

Code: Adventure Game Studio

1. function playbackground (String videofile) {
2.  if (Game.IsPluginLoaded("ags_d3d")) {
3.   // Required for autoplay
4.   D3D.SetLoopsPerSecond( GetGameSpeed() );
5.   // Open video file
6.   video = D3D.OpenVideo( videofile );
7.   if ( video ){
8.    // Use room coordinates
9.    video.relativeTo = eD3D_RelativeToRoom;
10.        
11.    // Anchor point to top left corner
12.    video.SetAnchor( -0.5, -0.5 );
13.        
14.    // Play!
15.    video.Autoplay();
16.    video.isLooping=videoloop;
17.   }
18.   else {
19.    Display( "Couldn't open video." );
20.    return;
21.   }
22.  }
23.  else{
24.   Display("Couldn't load ags_d3d plugin");
25.  }
26. }
27.  
28. function stopplay() {
29. video=null;
30. }

Do I need to do anything to let the video finish?
How can I load a video to be played just once without doing weird things?

Regards.

[Crimson Wizard]

As an experiment, I took this plugin and removed D3D part keeping only Theora playing, and then remade it to write video frames into the given sprite (number) instead of plugin's texture.
This is relatively slower, as video frame goes first to the sprite, and then from sprite to the texture inside engine itself. But this way it should work with any renderer, also one can put the sprite anywhere you usually put sprites in AGS. It's hard to tell if it may be optimized further with existing plugin api and theoraplayer library this plugin uses.

This is roughly how I'd attempt to implement playing videos on objects in AGS, but it's just easier to do with the existing plugin (with engine one would have to rewrite number of things to make it good, and also write a whole new playback API).

Did not have time today to further tidy the code etc yet, so uploaded the dirty version of the project here:
https://www.dropbox.com/s/k33h1fi7xe7mc4c/ags-spritevideo.zip?dl=0
and compiled plugin:
https://www.dropbox.com/s/0cai4gtd9uuk9df/ags-spritevideo-dll.zip?dl=0
note file is still called "ags_d3d", forgot to rename it...


One known problem is that AGS does not know that the sprite was updated, and so does not update objects on screen. I need to look more into whether plugin API allows to notify engine about that. For now I use the dirty hack by "poking" it with drawing surface ---

Example of script

Code: Adventure Game Studio

1. DynamicSprite *DSPR;
2. D3D_Video *video;
3.  
4. function room_AfterFadeIn()
5. {
6.         DSPR = DynamicSprite.Create(200, 200, false);
7.         video = D3D.OpenVideo("test.ogv", DSPR.Graphic);
8.         if (video != null)
9.         {
10.                 video.Autoplay();
11.                 oVideo.Graphic = DSPR.Graphic;
12.         }
13. }
14.  
15. function repeatedly_execute_always()
16. {
17.         if (DSPR != null)
18.         {
19.                 DrawingSurface* ds = DSPR.GetDrawingSurface();
20.                 ds.DrawPixel(-1, -1);
21.                 ds.Release();
22.         }
23. }
24.  
25. function room_Leave()
26. {
27.         video = null;
28.         DSPR.Delete();
29. }
30.  

[Crimson Wizard]

In regards to improving Direct3D plugin itself (or making an alternate version of such plugin), here are few thoughts about what's necessary to make it work well with the newer versions of AGS.

Multiple renderer support

Don't think there's much other choice than to implement all the same renderer types that engine supports: Direct3D, OpenGL, and software drawing too (if really wanted).
Plugin gets current driver ID, so may switch to necessary implementation on start.
Direct3D and OpenGL renderers would just apply quads to the scene according to their own rules similar to how plugin already does, and software renderer (if necessary) would call IAGSEngine::GetVirtualScreen to receive a stage backbuffer.

Support switching gfx modes

I do not know if plugin receives any events when game's gfx mode changes. If not, then this is a problem, as plugin won't be able to react and change its screen settings. Maybe it's still possible to adjust these at initial render stage callback.
There's still one important thing: Direct3D 9 gets reset when gfx mode changes, which may actually require to recreate video-memory resources. If plugin uses these and don't react to such reset, this may lead to crashes.

Viewport matrixes

It seems that Direct3D was made with an assumption that game image takes whole window, plus it obviously was made in the time when there was only 1 room camera and room viewport was also always covering whole game image.
This may still be usable while you have only 1 standard room viewport, or simply draw outside of room, but there still will be issues with aspect ratio (probably this was what rmonic79 noticed above).
Things will become more complicated if we add more transformation to the room viewport, or more different layers on screen.

The proper solution, I think, would be to actually pass viewport matrix into plugin on each render stage. I.e. on AGSE_PRESCREENDRAW plugin would receive corresponding room viewport matrix, on AGSE_PREGUIDRAW it would receive GUI layer matrix, and so forth.
In such case plugin will not have to make any assumptions, but simply use this parent matrix that engine provides.

Plugin callback already has an argument, which is unused for these stages, and which may be turned into pointer to a struct with necessary data. Such struct could be expanded further when necessary.
The only problem there is that this argument is currently declared as 32-bit integer and cannot work as 64-bit pointer, so this has to be adjusted. Maybe it's safe to do so, as pointer size will still be 32-bit in 32-bit engine/plugins. If it's not (as this argument is already used for other purposes), then there might be other paths to achieve this, such as expanding callback function prototype, adding second callback for new plugin API version etc. Anyway, this is all technical details.

[Crimson Wizard]

Another heads-up, few things changed in AGS 3.5.1 regarding path handling, and this plugin has a mistake by not allocating enough buffer for video path, so it has high chance to crash when trying to open video in AGS 3.5.1.
Crash itself is a consequence of unsafe plugin API (specifically GetPathToFileInCompiledFolder function) where it does not let plugin pass buffer length into engine, so engine literally does not know how much is it allowed to write there...

This is trivial to fix in plugin, but you got to rebuild it from source of course. The buffer should normally not be smaller than MAX_PATH constant (260 on Windows, may be much higher on Linux afaik).

PS. This is found in D3D_OpenSpriteFile and D3D_OpenVideo functions.

[Crimson Wizard]

Ok, I actually found out why this plugin scaling broke after 3.4.0. It happened after following engine change:
https://github.com/adventuregamestudio/ags/commit/e1cc278aa38cf66de38cb2bda2e45267e0391fc4

Or more precisely, Orthographic projection has changed from

Code: C++

1. (2.0 / (float)_mode.Width), 0.0, 0.0, 0.0,
2. 0.0, (2.0 / (float)_mode.Height), 0.0, 0.0,
3.  

to

Code: C++

1. (2.0 / (float)_srcRect.GetWidth()), 0.0, 0.0, 0.0,
2. 0.0, (2.0 / (float)_srcRect.GetHeight()), 0.0, 0.0,
3.  

Which means projection is no longer in full window resolution, but in native game resolution.
Which is questionable, considering we might want to render something outside game's frame and this projection makes that inconvenient probably (not impossible), but this was convenient for rendering at native pixel resolution mode afaik. Anyway this is what we have currently, unless this changes in the future once more...



If hardcoded fix is acceptable, all you have to do is to fix the scaling for post-3.4.0 engine is these 2 lines in D3DObject.cpp:

Code: C++

1. float screenScaleX = static_cast<float>(screen->backBufferWidth) / screen->width;
2. float screenScaleY = static_cast<float>(screen->backBufferHeight) / screen->height;
3.  

they should be just

Code: C++

1. float screenScaleX = 1.0;
2. float screenScaleY = 1.0;
3.  

because, if I understand correctly, plugin assumes it has to rescale object from native resolution to projection (which was correct with the old engine), but since now projection equals native resolution, no scale is necessary.




For non-hardcoded fix, I've been doing an experiment with passing engine matrixes into the plugin.
Unfortunately, passing through event hook arguments appeared to be too complicated as multiple changes need to be done in the engine to pass these through.
So I favor another approach: introduce new engine interface function which looks roughly like:

Code: C++

1. AGSIFUNC(void)  GetRenderStageDesc(AGSRenderStageDesc* desc);
2.  

where AGSRenderStageDesc is an extendable struct which may contain all 3 engine matrixes for the current render stage (screen, room, gui, etc):

Code: C++

1. struct AGSRenderStageDesc {
2.   // Which version of plugin the struct corresponds to;
3.   // this field must be filled by plugin before passing the struct into engine!
4.   int     Version;
5.   BITMAP *StageSurface;
6.   float   ProjectionMatrix[16];
7.   float   WorldMatrix[16];
8.   float   ViewMatrix[16];
9. };
10.  

These matrixes would be filled by each gfx renderer in their format (Direct3D and OpenGL), and thus could be used by any existing renderer and any renderer AGS might have in the future, in theory.

I'd like to try adding this new function into 3.5.1, only need to test more if it's viable to read necessary screen scaling from project matrix in the plugin itself, and if these matrixes actually let to work with custom room viewports.

[Crimson Wizard]

Sorry for all my rambling here...

Here's a rebuilt version of Direct3D plugin fixed for AGS 3.5.0/3.5.1:
https://www.dropbox.com/s/u23fvlar4q45kvd/ags_d3d_for351.zip?dl=0
It should have fixed scaling, and also fixed few more errors found during research.
Source project: https://www.dropbox.com/s/4rtfeh7gzxqs1hh/ags_d3d_for351_source.zip?dl=0

This version works in 2 modes, first is for ags 3.5.0 where it does not fully support custom room viewports, because it cannot do that correctly with existing plugin API. D3D objects will have correct viewport offset, but won't have correct camera scaling (and maybe something else will be wrong). It should work fine if you don't change default room viewport/camera.
Second mode is experimental and it only activates if engine plugin interface has certain number which is not in official release yet. But if everything is fine and we'll add this new interface into AGS 3.5.1, then D3D plugin (and others) will be able to receive full information on how to render things with Direct3D/OpenGL in modern AGS (there's a function that returns 3 transformation matrixes for each given render stage).


Couple of notes on the source project. It's made in Visual Studio 2010, because it uses libtheoraplayer library built for MSVS 2010. It will not work if you compile it with later MSVS, unless you also rebuild libtheoraplayer from source. Lib source may be found here: https://www.cateia.com/libtheoraplayer/wiki/index.php?title=Releases . Which may in turn involve more issues you have to fix; maybe I missed something, but it took me some time to make it actually work and compile, and I decided to stick to old project for now (plus I have MSVS 2010 on my pc). Just saying as a warning. Ideally someone got to gather it all up into a proper contemporary MSVS project and host on github etc. Even better - provide OpenGL support within same plugin - that would be awesome and make this plugin crossplatform.