Rounding mode

General discussion about LÖVE, Lua, game development, puns, and unicorns.
User avatar
wolfboyft
Prole
Posts: 17
Joined: Sat Oct 21, 2017 5:28 pm
Location: London
Contact:

Re: Rounding mode

Post by wolfboyft » Fri Jan 10, 2020 6:44 pm

So provided you use stock LÖVE or compile it yourself without changing that setting (and I can't think of any reason to), we're ok? You could just detect the mode and say "oi, recompile or just download" if it isn't the right one.

To test: https://gist.githubusercontent.com/wolf ... 3/main.lua
Tachytaenius

User avatar
pgimeno
Party member
Posts: 2313
Joined: Sun Oct 18, 2015 2:58 pm

Re: Rounding mode

Post by pgimeno » Fri Jan 10, 2020 6:52 pm

In the Stack Overflow post I linked earlier, I had this snippet which included unit testing:

Code: Select all

-- Round to nearest or even - Guaranteed to work for inputs between -2^50 and 2^50
-- if the FPU is using this rounding mode.
local function round(num)
  return num + (2^52 + 2^51) - (2^52 + 2^51)
end

local function testnum(num, expected)
  if round(num) ~= expected then
    error(("Failure rounding %.17g, expected %.17g, actual %.17g")
          :format(num+0, expected+0, round(num)+0))
  end
end

local function test(num, expected)
  testnum(num, expected)
  testnum(-num, -expected)
end

test(0, 0)
test(0.2, 0)
test(0.4, 0)
-- Most rounding algorithms you find on the net fail this one:
test(0.49999999999999994, 0)
-- Ties are rounded to the nearest even number, rather than always up:
test(0.5, 0)
test(0.5000000000000001, 1)
test(1.4999999999999998, 1)
test(1.5, 2)
test(2.5, 2)
test(3.5, 4)
test(2^50-0.5, 2^50)
test(2^50-1.5, 2^50-2)
print("All tests passed")

Post Reply

Who is online

Users browsing this forum: No registered users and 27 guests