## Is there something like Canvas:overwriteOldImageData() ?

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

### Is there something like Canvas:overwriteOldImageData() ?

Canvas size=screen size
If I call Canvas:newImageData() each frame, it just goes over 2GB and crashes after several seconds.
If I follow it with collectgarbage(), it doesn't crash, but it's extremely slow.
How can I access canvas contents each frame without creating huge imagedata each frame?

raidho36
Party member
Posts: 1685
Joined: Mon Jun 17, 2013 12:00 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

Short answer: you can't. The image data is on the GPU, and to access it on the CPU you have to duplicate it from GPU RAM to system RAM, which is slow, for obvious reasons. Also I suspect that it's slow both with and without "collectgarbage", a single object creation and deletion times are negligible, so is performance impact of having a single extra Lua instance to be processed by a GC.

You can however do pretty much anything as long as the data never has to leave the GPU (or enter it from the outside) for no performance penalty on top of normal computational costs. Now, you can't draw a canvas to itself. You can however use two canvases as front and back buffer for this purpose.

PGUp
Citizen
Posts: 67
Joined: Fri Apr 21, 2017 9:17 am

### Re: Is there something like Canvas:overwriteOldImageData() ?

Canvas:clear()
Then set the canvas and draw things to it again
-

no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

Fortunately, I realized that now I need only 1 px and not every frame, so I can go with newImageData without huge overhead.

However I did some tests to see what is slow and how slow it is:

Reading 1px
----------------------
glReadPixels 8ms
Canvas:newImageData 8ms

Reading whole screen of pixels
----------------------------------
glReadPixels 12ms
Canvas:newImageData 15ms
Canvas:newImageData + collectgarbage 53ms
direct call to ImageData:__gc causes unexpected errors in other functions, probably corrupts something?

So we can clearly see that getting data from canvas is slow, but memory issues are causing +300% more delay for big images when default gc is not enough to collect them until out-of-memory crash.

Also tried to implement requesting data to PBO, but unfortunately adding related functions to ffi is much more complicated than just adding glReadPixels. Tried to use glewInit() and it returns success, but there's still no functions in any namespace. Probably that requires much more knowledge of related libs.
By the way, forum engine insists that opengl abbreviations like PBO or FBO are 'too common' and refuses to search for them.

zorg
Party member
Posts: 2368
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

### Re: Is there something like Canvas:overwriteOldImageData() ?

The forum can only search for keywords with lengths of at least 4 characters, which is indeed a bummer, but i do recall some talk about this before, and i believe there would be issues if they enable the 3 character minimum limit.
Me and my stuff True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.

raidho36
Party member
Posts: 1685
Joined: Mon Jun 17, 2013 12:00 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

Calling collectgarbage triggers the GC to do a full round of garbage collection - it is a slow operation and is slower if you have more live Lua objects.

I suggest you try accomplish your task without reading pixels.

no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

I was able to get pointers to missing opengl functions with wglGetProcAddress on win.

Now it takes 10^-5 seconds to get pixel from PBO (not sure how inaccurate such small values are), with glReadPixels as soon as drawing is complete and reading from PBO only when needed.

Probably it can be optimized more by manually creating framebuffer with params optimized for reading as opposed to 'for render', but at the same time I know that nvidia drivers are good at overriding hints when the usage doesn't match them, so it's also possible that the canvas was automatically optimized for reading.

Tried if I have the same issue with missing functions on linux, and there was no issue: just ffi.load("GL") is enough. It ran 19 fps without requesting pixels and 17fps with requesting pixels. But since it was on VM, I can't really make any conclusions.
Tried it for android, and it says there's no "libGL.so". Made it fallback to Canvas:newImageData if opengl loading wasn't successful. Anyways, android is to slow to run the game, so I tried it just out of curiosity.

Not sure if it's good enough, there's still a chance that I'll be forced to use the solution which doesn't involve reading pixels from framebuffers.

-- creating pbo

Code: Select all

      ffi.GLL.glGenBuffers(1, PBO_id)
ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
ffi.GLL.glBufferData(ffi.GL.GL_PIXEL_PACK_BUFFER, memsize, nil, ffi.GL.GL_STREAM_READ)
ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)

-- requesting data from canvas

Code: Select all

    love.graphics.setCanvas(zBuffer)
ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
ffi.GL.glReadPixels(love.mouse.getX(),love.mouse.getY(), 1,1, ffi.GL.GL_RGBA, ffi.GL.GL_UNSIGNED_BYTE, nil)
ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
love.graphics.setCanvas()

--getting data from pixel buffer object

Code: Select all

    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
zImage2 = ffi.cast("unsigned char *", ffi.GLL.glMapBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER,ffi.GL.GL_READ_ONLY))
ffi.GLL.glUnmapBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER)
ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
id = zImage2[0]

-- getting missing functions

Code: Select all


if ffi.GL and ffi.os == "Windows" then
ffi.cdef[[
typedef int (__stdcall *PROC)();
typedef char* LPCSTR;
PROC wglGetProcAddress(LPCSTR name);
]]

local nameHolder = ffi.new("char[100]")
ffi.GLL = {}
local function loadProc(definition, name)
ffi.copy(nameHolder, name)

ffi.GLL[name] = ffi.cast(definition, ffi.GL.wglGetProcAddress(nameHolder))

local asInt = ffi.cast("int",ffi.GLL[name])
if asInt >=-1 and asInt <=3  then
print("wglGetProcAddress returned nothing for "..name)
io.stdout:flush()
error("wglGetProcAddress returned nothing")
end
end

loadProc("void (*)(GLsizei, GLuint *)", "glGenBuffers")
loadProc("void (*)(GLenum, GLuint)", "glBindBuffer")
loadProc("void (*)(GLenum target, GLsizeiptr size, GLvoid* data, GLenum usage)", "glBufferData")
loadProc("void* (*)(GLenum target,  GLenum access)", "glMapBuffer")
loadProc("GLboolean (*)(GLenum target)", "glUnmapBuffer")
else
ffi.GLL = ffi.GL
end

raidho36
Party member
Posts: 1685
Joined: Mon Jun 17, 2013 12:00 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

What are you doing with that one pixel anyway?

no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

Checking which ui element is under the mouse. Approximately like it's described here https://en.wikibooks.org/wiki/OpenGL_Pr ... _selection, but now my implementation is much more simple since I'm not using stencil buffer, just painting a rect with color=element id to another canvas. Probably it will be more useful when/if I have overlapping non-rect shapes with holes.

grump
Party member
Posts: 378
Joined: Sat Jul 22, 2017 7:43 pm

### Re: Is there something like Canvas:overwriteOldImageData() ?

no_login_found wrote:
Thu Mar 01, 2018 4:13 pm
I'm not using stencil buffer, just painting a rect with color=element id to another canvas. Probably it will be more useful when/if I have overlapping non-rect shapes with holes.
That's a straightforward and obvious way to implement it, but beware: reading pixels from the color buffer stalls the rendering pipeline and can be a very expensive operation, depending on the rendering hardware and the complexity of your scene. If at all possible, it's usually faster to have a simple representation of your UI shapes instead, e. g. by using a region data structure or polygon approximations.

### Who is online

Users browsing this forum: SSG_DEV and 6 guests