Float calculation problem (SOLVED)

Started by GarageGothic, Tue 16/05/2006 22:58:28

Previous topic - Next topic

GarageGothic

Maybe I'm overlooking something obvious, but how can either of the two calculations below possibly end up with a "Error: Sqrt: cannot perform square root of negative number" error message? Even with rounding up/down I see no way that the division could end up negative?

Code: ags
if (wyt != cy) ratiocbct = Maths.Sqrt(IntToFloat(((wyt-wyb)*(wyt-wyb)) + ((wxt-wxb)*(wxt-wxb)))/IntToFloat(((wyt-cy)*(wyt-cy)) + ((wxt-cx)*(wxt-cx))));


Code: ags
if (wy != wyb) ratiosbst = Maths.Sqrt(IntToFloat((wyt - wyb)*(wyt-wyb) + (wxt - wxb)*(wxt-wxb))/IntToFloat((wy-wyb)*(wy-wyb) + (wx-wxb)*(wx-wxb)));


Edit: Added the second line after also getting an error with that.

Khris

Try this:

Code: ags
float a=IntToFloat(((wyt-wyb)*(wyt-wyb)) + ((wxt-wxb)*(wxt-wxb)));
float b=IntToFloat(((wyt-cy)*(wyt-cy)) + ((wxt-cx)*(wxt-cx)));
display("a: %.4f, b: %.4f", a, b);
if (wyt != cy) {
Ã,  float ab=a/b;
Ã,  if (ab>=0) ratiocbct=Maths.Sqrt(ab);
Ã,  else Display("ab < 0!");
}


My only explanation is that the values are extremely small and a/b ends up being negative due to float glitches.

GarageGothic

I think you're right that it was a matter of a decimal inaccuracy turning 0.000000 into -0.000001 or similar. I fixed it by adding a conditional between the float division and the square root calculation. It hasn't shown up again so far. Thanks for the help.

strazer

Yeah, it possibly has to do with this:

Quote from: AGS Manual
WARNING: When using the float data type, you may find that the == and != operators don't seem to work properly. For example:

float result = 2.0 * 3.0;
if (result == 6.0) {
  Display("Result is 6!");
}

may not always work. This is due to the nature of floating point variables, and the solution is to code like this:

float result = 2.0 * 3.0;
if ((result > 5.99) && (result < 6.01)) {
  Display("Result is 6!");
}

The way floating point numbers are stored means that 6 might actually be stored as 6.000001 or 5.999999; this is a common gotcha to all programming languages so just be aware of it if you use any floating point arithmetic.

SMF spam blocked by CleanTalk