## Floating Point color

General discussion about LÖVE, Lua, game development, puns, and unicorns.
grump
Party member
Posts: 609
Joined: Sat Jul 22, 2017 7:43 pm

### Re: Floating Point color

pgimeno wrote:
Tue Jan 29, 2019 3:37 pm
A uniform distribution can be obtained by multiplying the random number by 1.003921568627451 (256/255), otherwise the maximum value wont ever be produced. To keep the result in [0, 1] the formula can be: math.min(1.0, love.math.random()*1.003921568627451)
I may be doing something wrong here, but:

Code: Select all

local random, histogram = love.math.random, {}
for i = 0, 255 do histogram[i] = 0 end

for i = 1, 1e8 do
local r = math.min(1, random() * 1.003921568627451)
local n = math.floor(255 * r + .5)
histogram[n] = histogram[n] + 1
end

for i = 0, 255 do print(i, histogram[i]) end
Results:

Code: Select all

0	194956
1	389922
2	389875
3	391098
... (251 pretty uniformly distributed values)
255	585194

That is far from uniform. 0 is underrepresented, 255 is overrepresented.

If you use

Code: Select all

local n = random(0, 255)
it becomes uniformly distributed over the entire range.

No conclusion here, just an observation.

EDIT:
pgimeno wrote:
Tue Jan 29, 2019 3:37 pm
Edit: Sorry, that's wrong, I get you now. You're right, given the rounding in the internal conversion, you need to compensate with this formula:
math.floor(love.math.random()*256)/255
Yep, that works fine.

MachineCode
Citizen
Posts: 70
Joined: Fri Jun 20, 2014 1:33 pm

### Re: Floating Point color

Let me just explain why I was looking at this and became confused. I have been working on a drawing engine that uses 16 bit instructions to do basic primitive things - like set pixel, draw block. Because of the 16 bit format, 4:4:4 color (12bit) is the native format, so to encode an image you first need to clip it back to 4:4:4 color space. The instructions also support a 10 bit color 3:4:3 space (green always gets the most bits).

Why restrict color spaces? Well, apart from data size, another reason is that restricting color space changes the "look" of the result and can give a retro quality. The biggest drawback is that images with gradual color changes suffer from banding, which can be annoying.

It turns out that you can actually apply a type of "dither" to 4:4:4 or 3:4:3 images to restore them to very close to the original 8:8:8 representation. Effectively you steal a bit of the color bandwidth and move it into the intensity bandwidth. For each color component of a pixel, you split the fractional integer up into an int.frac and use the fractional part as a statistical weight to decide whether that int part is bumped up to the next value. So, in a 4:4:4 encode from 8:8:8 space, each pixel is encoded as a weighted random value from the 8 possible values r = d,d+1 g = e,e+1 b = f,f+1

This encoding imposes a bit of grain on the image, however on most lcd displays, the eye just integrates this and it is invisible. I even tried it with 256 color 3:3:2 and you get visible noise, but the full color range is effectively restored. A nice feature of this scheme is that it can be applied pixel by pixel, so you could use 3:4:3 mark out regions of the image with faces or skintone for enhancement and leave the rest as is.

So, here is the encoding function to do this dithering. This is the old pre 11 code that uses integer color value. To fix these for 11.x, needs a conversion from fp to integer, so that led me to think about how that works.

Code: Select all

local function clip_10( x, y, r, g, b, a )
local rr = band(r, 0x1f)/32
local gg = band(g, 0x0f)/16
local bb = band(b, 0x1f)/32
r = band(r,  0xe0)
g = band(g,  0xf0)
b = band(b,  0xe0)
if rr > rand() then r = r+31 end
if gg > rand() then g = g+15 end
if bb > rand() then b = b+31 end
return r,g,b,a
end

local function clip_12( x, y, r, g, b, a )
local rr = band(r, 0x0f)/16
local gg = band(g, 0x0f)/16
local bb = band(b, 0x0f)/16
r = band(r,  0xf0)
g = band(g,  0xf0)
b = band(b,  0xf0)
if rr > rand() then r = r+15 end
if gg > rand() then g = g+15 end
if bb > rand() then b = b+15 end
return r,g,b,a
end

You can do this in the world of fp, but thinking about color spaces as integers and int.fracs just seems a lot more precise and natural to me.

pgimeno
Party member
Posts: 2134
Joined: Sun Oct 18, 2015 2:58 pm

### Re: Floating Point color

Just wait for 11.3 and use colorToBytes (or make your own, or use cindy).

MachineCode
Citizen
Posts: 70
Joined: Fri Jun 20, 2014 1:33 pm

### Re: Floating Point color

Currently I am using

Code: Select all

 col_int = floor(255.9 * col_fp /256) 
That effectively divides the color space up into 256 regions and excludes 1.0 from giving the out of range 9bit value of 256.

MachineCode
Citizen
Posts: 70
Joined: Fri Jun 20, 2014 1:33 pm

### Re: Floating Point color

Brain snap when I typed that above. i was mixing up the forward and reverse conversions.

col_int_8 = floor(255.9 * col_fp)

col_fp_4bit = floor(15.9 * col_fp) / 15 -- that gives 0 --> 1 in the floating point range, clipped to a 4 bit space.

It just goes to show that the continuous representation of color is not at all as simple as it seems.

MachineCode
Citizen
Posts: 70
Joined: Fri Jun 20, 2014 1:33 pm

### Re: Floating Point color

This is what the color space transforms look like. Compared to the 8:8:8, the 4:4:4 has banding. In the third image, it is still 4:4:4, but weighted dither applied to the lsb's of the color components. The banding is gone but there is some graininess visible.

sky-set.png (141.06 KiB) Viewed 1919 times

### Who is online

Users browsing this forum: No registered users and 45 guests