Lighting using normal maps in AGS

[ruslan128]

Hi everyone!

I want to share with the community the results of creative experiments, where i had applied 3D sprite lighting using normal maps in AGS project. Gouraud shading is used, but with some simplifications and optimizations for the implementation of a specific task: сreatе realistic lighting with acceptable performance given that everything is done by the software processor, standard AGS API works only. The system is designed for a game we're working on
https://www.adventuregamestudio.co.uk/forums/index.php?topic=60224.0
I came up with an idea to add realistic real-time lighting to the game screen, where player move through the cave system, which makes it more lively to navigate. A small trifle, but pleasant and unexpected for AGS game, right?

Here are some more examples of the results that can be achieved by adjusting the lighting in different ways and changing the degree of reflection (metal, patina, gold, glass).

[ruslan128]

So, 5 main data buffers are used.
float N[]; //normals map
float BMf[]; // color map in float values
float SM[]; // specular map. Reading from the red channel
int Stencil[]; // marks pixels marks pixels to be calculated with values above 128 in the blue channel
float Gloss[]; // Reading from the green channel

The color map is loaded into a dynamic array as red, green, and blue float values. The normal map is loaded as a float array with the x,y,z values of
the normal vector for each pixel. The method from the Godot 4 engine is used here, with inverted green channel values.



In the specular light map each pixel has a single float value that specifies the specular strength in the range from 0.0 to 255.0.

In the gloss map each pixel is assigned a certain measure of gloss. Defined as float values from  0.0 to infinitude. But reasonable values usually chosen between 2.0, 4.0, 8.0, 16.0, 64.0. This parameter is used as an exponent for raising the final coefficient of the degree of collinearity of the reflected light ray vector and the camera vector. Simply put, it defines the size of the specular on the surface, while specular defines its overall brightness. The stencil map is integer values, zero and non-zero. It was used to optimize calculations, allowing you to cut off those pixels that do not need to be highlighted.

-----------------------------------------
For simplicity, all code is placed in the room script. At the top the parameters of the light source, the height of the source above the screen surface (in pixels) and the maximum and minimum radius of the illuminated area are set. Also, the light source can be animated to create a throbbing effect, such as the glow of a candle. But since the speed of animation is very important for good perception, you need to make sure that the FPS on the developer's and user's computer will match.


float lightAmbientR = 0.99;
float lightAmbientG = 0.99;
float lightAmbientB = 0.99;

float lightDiffuseR = 1.9;
float lightDiffuseG = 1.9;
float lightDiffuseB = 1.49;

float lightSpecularR = 0.5;
float lightSpecularG = 0.5;
float lightSpecularB = 0.5;

float lightX = 0.0;
float lightY = 0.0;
float lightZ = 7.0;

float lightRadiusMax = 30.0;
float lightRadiusMin = 0.0;


float lmin=0.98, lmax=0.99; // light animation. Amplitude limits
float ld=0.014; // increase factor
float lk=0.98;      // light factor. This is used for entry level installation


-----------------------------------------
Next, before using the functions, we load the data into buffers by specifying the slot number of the sprite from the regular set.  Need I say that the sizes of the sprites must match?)

  readNormals(35);
  readBM(32);
  readSpecularMap(36);
  readGlossMap(34);
  readStencilMap(33);

The normals are read from the rgb channels with inversion of the green chanel. Reflection map from red channel, gloss from green, stencil from blue (because of known problems with pure blue colors, stencil value is set as 0 if the value of blue is less than 128. And 1 if the value is higher).

-------------------------------
Then, from room_RepExec we call first draw_back() every cycle.  This function is separated for the sake of a possible better combination with other effects. It draws the original, unlightened image.


------------------------------------

Over this image, the PointLight(x,y) function draws pixels that have changed their luminosity due to the light source. It does all the math based on the data from the buffers and the relative positions of the illuminated picture and the light source.

Code: ags

//--------------------------------------detailed version---------------------------------------------------------------------
function PointLight1(int x,int y){
  int r, g, b;         // RGB values for result
  float rf,  gf, bf;   // RGB values as floating point numbers for intermediate calculations.
  float nx, ny, nz;    // Values of the vector of the current normal
  float lx, ly, lz;    //  The values of the light ray vector from the source to each pixel.
  float cx, cy,  cz;   //   Values of the camera view vector.  Always (0,0,1.0)
  float rad2, radmin2;  // // Squares of minimum and maximum illumination radius. They are used for optimization to take them out of the body of the main loop.
  
  // We get access to the background surface to draw on.
  DrawingSurface *surface = Room.GetDrawingSurfaceForBackground();
  
  
  /*
  Light animation. Currently, the coefficient increases and returns to the minimum value after crossing the maximum. 
This algorithm can be changed or excluded at will.
  */
  lk = lk+ld;
  if(lk >= lmax){lk = lmin;}
  
  int rad = FloatToInt(lightRadiusMax);   
  rad2 = lightRadiusMax * lightRadiusMax; //  Calculation of the squares of the min and max radii.
  radmin2 = lightRadiusMin * lightRadiusMin;
  lightX = IntToFloat(x);  // The coordinates of the light source
  lightY = IntToFloat(y);  // In the area of our applications there is no point in playing with these parameters.
  //lightZ = 30.0;
  lz = lightZ;  //  The z-coordinate of the light source enters the local variable
  
  /*
  Declaration and initialization of light components values. 
At once they are multiplied by the coefficient of overall light intensity at the current moment, obtained in
the calculation of the light intensity animation above
  */
  float lambR = lightAmbientR * lk;
  float lambG = lightAmbientG * lk;
  float lambB = lightAmbientB * lk;
  float ldifR = lightDiffuseR * lk;
  float ldifG = lightDiffuseG * lk;
  float ldifB = lightDiffuseB * lk;
  float lspcR = lightSpecularR * lk;
  float lspcG = lightSpecularG * lk;
  float lspcB = lightSpecularB * lk;
  
  cx = 0.0;  //  Camera view vector values.  Always (0,0,1.0)
  cy = 0.0;  // 
  cz = 1.0;  //Generally, the camera vector has a real value of -1, but for the sake of optimizing the calculations it is immediately written rotated.
  
  /*
  Calculation of minimum and maximum values of x,y for the enumeration
of all pixels that fall within the range of our light.
  */
  int minX, minY, maxX, maxY;
  minX = x - rad;// - offset_x;
  minY = y - rad;// - offset_y;
  maxX = x + rad;// - offset_x;
  maxY = y + rad;// - offset_y;
  
  /*
  Check to make sure that there is no exceeding of the boundaries of the illuminated picture.
  */
  if(minX<0)minX = 0;
  if(minY<0)minY = 0;
  if(maxX>=BMw)maxX = BMw-1;
  if(maxY>=BMh)maxY = BMh-1;
  
  //  Start of the main cycle.
  for(int y1=minY; y1<maxY; y1++){
    for(int x1=minX; x1<maxX; x1++){
      
      // Calculation of the light ray vector from the source to the pixel with the current coordinates (x1,y1).
      lx = lightX-IntToFloat(x1) ;
      ly = lightY-IntToFloat(y1) ;
      lz = lightZ;
      
      // Reading the rgb values from the array of pixel color values.
      rf = BMf[(y1*BMw + x1)*3 + 0];
      gf = BMf[(y1*BMw + x1)*3 + 1];
      bf = BMf[(y1*BMw + x1)*3 + 2];
      
      //  Check whether the pixel falls within the illumination area.  If it does not, move to the next pixel.
      if(Stencil[y1*StencilW + x1]==0) continue;
      
      //  Calculation of the square of the distance of the current pixel from the central pixel.
      float d2 = IntToFloat((x1-x)*(x1-x)+(y1-y)*(y1-y));
      // If this square surpassess the square of the maximum radius - no further calculations and move on to the next pixel.
      if(d2<radmin2)continue;
      // The same for comparison with the minimum radius
      if(d2>=rad2)continue;
      
      // Coefficient of light fading from the center to the edge. 
      //Reduces calculated color values from the maximum illumination values.
      // To the original pixel value (not to darkness!)
      float k = 1.0 - d2/rad2;
      
      // Read value of the normal vector of the current pixel.
      nx = N[(y1*Nw + x1)*3 + 0];
      ny = N[(y1*Nw + x1)*3 + 1];
      nz = N[(y1*Nw + x1)*3 + 2];
      
      //  Calculates the cosine of the angle between the normal vector and the light vector.
      float a = Maths.Sqrt(nx*nx + ny*ny + nz*nz) * Maths.Sqrt(lx*lx + ly*ly + lz*lz);
      if (a==0.0) a = 0.001;
      a = ((nx*lx + ny*ly + nz*lz)/a);
      
      a = a*k; // Adding the action of the fade coefficient.
      
      //Adding the calculated values of the ambient and diffuse.
      rf = rf*a*ldifR + rf*lambR;
      gf = gf*a*ldifG + gf*lambG;
      bf = bf*a*ldifB + bf*lambB;
      
      // Reading the reflection strength value for the current pixel.
      float specular = SM[y1*SMw+x1];
      
      // Calculation of the length of the light vector from the source to the current pixel.
      float n = Maths.Sqrt(lx*lx + ly*ly + lz*lz); // light vector length
      if(n==0.0) n=1.0;
      lx = lx/n;   // unit light vector
      ly = ly/n;
      lz = lz/n;
      
      //  Addition the light ray and camera vectors to get a vector of the reflected light ray.
      float bx, by, bz; 
      bx = cx + lx;
      by = cy + ly;
      bz = cz + lz;
      
      //  Calculation of the cosine of the angle between the normal vector and the reflected ray vector.

      a = Maths.Sqrt(nx*nx + ny*ny + nz*nz) * Maths.Sqrt(bx*bx + by*by + bz*bz); //CosAngleBetweenV A, Blick, N
      if (a==0.0) a = 0.001;
      a = (nx*bx + ny*by + nz*bz)/a;
      //if (a==0.0) a = 0.001;
      
      // Raise the result to the power of "gloss".
      a = Maths.RaiseToPower(a, Gloss[y1*SMw+x1]);
      
      /* Applying all obtained coefficients to a pixel to calculate its gloss and adding to previously obtained rgb values.  
      Conversion to int for further drawing of the pixel.
      */
      r = FloatToInt(specular * a *  k * lspcR + rf);
      g = FloatToInt(specular * a *  k * lspcG + gf);
      b = FloatToInt(specular * a *  k * lspcB + bf);
        
      
      /* 
      Checking for output values above 255 or below 8. 
Below eight is to prevent the problem of blue colors in AGS
      */
      if(r>255)r=255;
      if(r<8)r=8;
      
      if(g>255)g=255;
      if(g<8)g=8;
      
      if(b>255)b=255;
      if(b<8)b=8;
      
      // Finally, we draw the pixel on the surface.
      surface.DrawingColor = Game.GetColorFromRGB(r, g, b);
      surface.DrawPixel(x1, y1);
      
    }
    
  }
  //  Output to the screen.
  surface.Release();
}

---------------------------
An example of how the code works with helmet lighting.
Source data:










The result! 

[ruslan128]

This code can be significantly optimized for performance, which has been done. It doubled fps, but the result is more difficult as expected and is not included in this article. This leaves room for creativity for those who want to work with this topic)

In short, the accelerated algorithm uses four additional buffers, which are created previously, after loading the 5 main ones. They contain data that can be calculated before applying the light function and take these calculations out of the main loop. We spend more memory reducing the load of the processor.

float Nl[];
Contains lengths of the pixel`s normals. The size corresponds to the main buffers. It is created and calculated
automatically when loading the normal map.

float K[]; //coefficient of distance of each pixel from the center.
Map of distance coefficients from the center for each illuminated pixel. It has the size (lightRadiusMax * 2) * (lightRadiusMax * 2).
That is, it contains coefficients for each pixel in the illuminated area. Covers a rectangular region. Filled with values of float k = 1.0 - d2/rad2
If the value is negative, the pixel does not fall into the illumination area and is ignored.
This buffer as well as the following ones in the current version is created programmatically as it should be according to the logic of the point light source, but it can also be formed manually to create various special effects.

float LRl[]; // light ray lengths
A map of the lengths of the light ray from the light source to each pixel of the illuminated surface. It has the size (lightRadiusMax * 2) * (lightRadiusMax * 2). It is used only to optimize the speed of calculations in the main loop.
Can be filled manually to create special effects.

float LCRl[]; // light+camera  orth vectors length
Map of the sums of unit vectors of the light ray from the light source to each pixel of the illuminated surface and the inverted camera view vector. It has the size (lightRadiusMax * 2) * (lightRadiusMax * 2). It is used only to optimize the speed of calculations in the main loop. Can be filled manually to create special effects.

Buffers K[], LRl[], LCRl[] are created with function create_accelerating_maps(). This must always be started before PointLight() is called for the first time, otherwise an error will occur.  In this case, the parameters lightZ, lightRadiusMax and lightRadiusMin must be set before it is called. Their values will be used when creating the accelerating buffers and must not be changed.

Example of initialization:

Code: ags

function init(){
  lightZ = 12.0;
  lightRadiusMin = 0.0;
  lightRadiusMax = 40.0;

  readBM(14);
  readNormals(16);
  readSpecularMap(18);
  readGlossMap(15);
  readStencilMap(30);
  create_accelerating_maps();
}

Function call:

Code: ags

function room_RepExec(){
  draw_back(14);
  PointLight(mouse.x, mouse.y);
}

The destroy() function assigns null to all the pointers to the buffers. It is called after finish using  the lighting to free memory

Link to download the project
https://drive.google.com/file/d/12eg5sHYC_MhTI484wMT8R5Jrv2bUBLIi/view?usp=sharing

[ruslan128]

Now I am working on the design of this system in the form of a module. I hope to finish soon.

[Mandle]

AMAZING!

[Crimson Wizard]

Please note that if the perfomance is the issue, you may consider writing a C/C++ plugin instead. Plugins may request bitmap surfaces for dynamic sprites (and room backgrounds too, iirc), and work with them directly, free from the script API overhead.

https://adventuregamestudio.github.io/ags-manual/EnginePlugins.html
https://adventuregamestudio.github.io/ags-manual/EnginePluginRun-timeAPI.html

[ruslan128]

Please note that if the perfomance is the issue, you may consider writing a C/C++ plugin instead. Plugins may request bitmap surfaces for dynamic sprites (and room backgrounds too, iirc), and work with them directly, free from the script API overhead.

Thank you, it's a good idea!)
For  the current project the performance is quite satisfactory. If radius of lighting < 30 px, fps doesn't drop below  for about 20 on old  machines

I'd like to improve this  system in future.  Also, I found out that drawing operations do not make a decisive contribution to the total amount of computation.
This is clearly shown by the "quick lighting" function without normal maps, which I have already developed, but have not yet demonstrated here.
I'm preparing the module for release - it will be there

[newwaveburritos]

Oh wow this is really impressive work!  I had actually considered something like this for the EGA palette but super super simplified where you're just swapping one pixel for a higher brightness value but this looks very cool.  I'll look forward to the module!

[glurex]

Nice! Great work, Ruslan! I'll looking forward to the module too.

[ruslan128]

I've finished the module
https://www.adventuregamestudio.co.uk/forums/modules-plugins-tools/module-agslighting-v1-00-normal-maps-in-ags/

It can contain more functions, but now  it works alright)