Precision issue with lg.getColor()

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
grump
Party member
Posts: 947
Joined: Sat Jul 22, 2017 7:43 pm

Precision issue with lg.getColor()

Post by grump »

Code: Select all

local inp = 1 / 42
love.graphics.setColor(inp, inp, inp)
local outp = love.graphics.getColor()
print(inp, outp)
Output:

Code: Select all

0.023809523809524	0.023809524253011
I'm not sure if this is considered a bug, but I would expect getColor() to return the same values that were set with setColor(). It's caused by the Colorf type on the C++ side, because of the precision loss that occurs (double vs. float).
User avatar
zorg
Party member
Posts: 3436
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: Precision issue with lg.getColor()

Post by zorg »

Yes, the internal color precision of any texture will never be as precise as a double float, unless there'd be such a format supported by OpenGL. In short, it's not a bug. Similarly you can't set SoundData samplepoints to (normalized [-1,1] instead of [0,1]) values it couldn't represent with the chosen bit depth; it gets converted to the nearest possible value... although i'm assuming that since i usually only ever assigned correct values to them...

Still, for colors, the code will coerce the values to the nearest representable ones, so if you really want exact values (which are still probably os-dependent, but functionally generating them should work) then you could make tables:

Code: Select all

local int8 = {}
for i=0,255 do int8[i] = i/255 end
if you want to use 16 bit per color channel then the constant to divide by is 65535; basically it's (2^n)-1.

Also, you don't need to pre-generate those values, of course; you can use such a thing to check inputs whether they can be exactly represented or not, if you prefer errors/asserts in these areas. I think i already posted a snippet to the 11.0 thread, or a thread similar to that.

Edit: i realize that color values may be floats on the shader end but again, that's even less of a real concern, in my opinion; that said, i'm not too knowledgeable on that side so it's possible i'm wrong.
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.
User avatar
raidho36
Party member
Posts: 2063
Joined: Mon Jun 17, 2013 12:00 pm

Re: Precision issue with lg.getColor()

Post by raidho36 »

Colors were always floats on the shader end, integers have never been used for the purpose. Mainly because the output would look extremely ugly.
grump
Party member
Posts: 947
Joined: Sat Jul 22, 2017 7:43 pm

Re: Precision issue with lg.getColor()

Post by grump »

I get what you're saying, but it's not an OpenGL, Texture or Shader issue. Graphics::setColor on the C++ side of things tracks the color independently from OpenGL. It does it with 32 bit precision, while Lua uses 64 bit precision.

It's not a huge issue, I was just surprised by the precision loss. It came up when I wrote code to convert between the [0-255] and [0-1] color range and my unit tests failed with some values.
User avatar
zorg
Party member
Posts: 3436
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: Precision issue with lg.getColor()

Post by zorg »

Okay i misunderstood; that's actually a legit thing... i wonder why it does it like that.
My best mostly uneducated guess would be that löve also has a 32bit version, and 64bit values are slower on that? Not sure how much sense this makes though, probably not much. :crazy: In which case it should be turned into a double.
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.
egorcod
Prole
Posts: 17
Joined: Sun Jul 02, 2017 9:41 am

Re: Precision issue with lg.getColor()

Post by egorcod »

You can use simple patch:

Code: Select all

local lg = love.graphics

local oldSetColor = lg.setColor
local oldGetColor = lg.getColor
local oldReset = lg.reset

local def = {1, 1, 1, 1}
local color = def

lg.setColor = function(...)
  color = {...}
  oldSetColor(...)
end

lg.getColor = function()
  return unpack(color)
end

lg.reset = function()
  color = def
  oldReset()
end

Code: Select all

require 'colors'
local inp = 1 / 42
love.graphics.setColor(inp, inp, inp)
local outp = love.graphics.getColor()
print(inp, outp)
Result:

Code: Select all

0.023809523809524	0.023809523809524
User avatar
ivan
Party member
Posts: 1911
Joined: Fri Mar 07, 2008 1:39 pm
Contact:

Re: Precision issue with lg.getColor()

Post by ivan »

I'm with Grumps on this one.
getColor returns junk after the 7th decimal REGARDLESS of the "actual" input precision.
So for example:

Code: Select all

local lg = love.graphics
local c1 = 0.1
lg.setColor(c1,c1,c1)
local c2,_,_,_ = lg.getColor()
assert(c1 == c2) -- assertion failed
One "fix" could be to format/truncate the output of getColor.

Code: Select all

local function d2f(n)
  return n - n%0.0000001 -- positive numbers only
end
function love.graphics.getColor2()
  local r,g,b,a = love.graphics.getColor()
  return d2f(r),d2f(g),d2f(b),d2f(a)
end
Not very pretty but ensures that the test above is passed.
getColor is used rarely compared to setColor so it's not a big deal if it's a little bit slower.

PS. Of course, input must be floating point so anything after the 7th decimal or so will be truncated!
PPS. egorcod, that won't work for things like imageData:getPixel()
User avatar
pgimeno
Party member
Posts: 3544
Joined: Sun Oct 18, 2015 2:58 pm

Re: Precision issue with lg.getColor()

Post by pgimeno »

You can also adjust your unit tests with FFI:

Code: Select all

local ffi = require 'ffi'

print(tonumber(ffi.cast('float', 0.1))) -- prints 0.10000000149012
Edit: Alternatively, you can use math.ldexp and math.frexp, but getting the rounding right is costly.
User avatar
ivan
Party member
Posts: 1911
Joined: Fri Mar 07, 2008 1:39 pm
Contact:

Re: Precision issue with lg.getColor()

Post by ivan »

It's not really a big issue, but it should be noted somewhere.
Oh yea, did I mention that box2d/love.physics also works with floats? :)
User avatar
slime
Solid Snayke
Posts: 3131
Joined: Mon Aug 23, 2010 6:45 am
Location: Nova Scotia, Canada
Contact:

Re: Precision issue with lg.getColor()

Post by slime »

Indeed, almost every love API that accepts a number stores the number in something less precise than a 64 bit float. That's a lot of APIs.
Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot], Amazon [Bot], Semrush [Bot] and 44 guests