Problem: Calculator isn't giving correct result

Here’s my code:
*set WeaponRating (((BaseWeaponDamage)x(LWModifier))x(1+(CombatLvlx0.01)))

BaseWeaponDamage: 5
LWModifier: 1.8
CombatLvl: 30

What it’s supposed to show: WeaponRating: 11.7
What is shows: WeaponRating: 11.700000000000001

Is there something wrong with my code? If not, are there any fixes to this problem?
(I changed the multiply symbols to “x” because it won’t post here.)

To write code here, you have to use the pre-formatted code block. You can do this by clicking the code icon on the edit menu or by tying the backtick `.

Like this:
pre code

Or like this:


```
   # code comes here
```

About your problem. I copied your code and it does seem to return a weird float. Don’t know why that’s happening. It may be a good idea to report to CoG.

Also, it shouldn’t really matter, unless you’re testing the result against a specific value. Right? If you’re checking if it’s above or bellow a threshold it shouldn’t matter that much. But yeah, I wish I could help more.

4 Likes

It’s all good, thanks for the reply. I’ll report it when I get a chance.

I don’t know why it gives that strange value but a simple way of patching it (until it has a proper fix) would be using the “round” command. You can find the explanation it here.

I think it should look like this.
*set WeaponRating round (((BaseWeaponDamage)x(LWModifier))x(1+(CombatLvlx0.01)))

4 Likes

This is because JavaScript (which ChoiceScript is implemented in) represents all numbers as floats (or rather, doubles), double precision decimal numbers and has no concept of a real integer. Because of the way these numbers are represented in memory, certain values can’t be accurately represented. This can cause small (but annoying, and potentially significant) accuracy errors in certain circumstances.

9 * 1.3 in a browser console will give you the same result.

tl;dr Loudbeat’s suggestion is your answer. I don’t think this is really something you’d “fix” in ChoiceScript, it’s just an accepted quirk of the environment.

5 Likes

It’s not a bug. It’s a feature! :joy:

Not sure how much good integers would be in this case anyway, 11.7 is a float.

Not like that was the crux of the argument, or the point of the post, but sure, well spotted.