Is there something like Canvas:overwriteOldImageData() ?

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

Is there something like Canvas:overwriteOldImageData() ?

Post by no_login_found » Sun Feb 25, 2018 9:27 am

Canvas size=screen size
If I call Canvas:newImageData() each frame, it just goes over 2GB and crashes after several seconds.
If I follow it with collectgarbage(), it doesn't crash, but it's extremely slow.
How can I access canvas contents each frame without creating huge imagedata each frame?

User avatar
raidho36
Party member
Posts: 1674
Joined: Mon Jun 17, 2013 12:00 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by raidho36 » Sun Feb 25, 2018 9:56 am

Short answer: you can't. The image data is on the GPU, and to access it on the CPU you have to duplicate it from GPU RAM to system RAM, which is slow, for obvious reasons. Also I suspect that it's slow both with and without "collectgarbage", a single object creation and deletion times are negligible, so is performance impact of having a single extra Lua instance to be processed by a GC.

You can however do pretty much anything as long as the data never has to leave the GPU (or enter it from the outside) for no performance penalty on top of normal computational costs. Now, you can't draw a canvas to itself. You can however use two canvases as front and back buffer for this purpose.

PGUp
Citizen
Posts: 66
Joined: Fri Apr 21, 2017 9:17 am

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by PGUp » Sun Feb 25, 2018 12:42 pm

Canvas:clear()
Then set the canvas and draw things to it again
really want to post on code doodles thread :/

no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by no_login_found » Mon Feb 26, 2018 9:51 pm

Fortunately, I realized that now I need only 1 px and not every frame, so I can go with newImageData without huge overhead.

However I did some tests to see what is slow and how slow it is:

Reading 1px
----------------------
glReadPixels 8ms
Canvas:newImageData 8ms

Reading whole screen of pixels
----------------------------------
glReadPixels 12ms
Canvas:newImageData 15ms
Canvas:newImageData + collectgarbage 53ms
direct call to ImageData:__gc causes unexpected errors in other functions, probably corrupts something?

So we can clearly see that getting data from canvas is slow, but memory issues are causing +300% more delay for big images when default gc is not enough to collect them until out-of-memory crash.

Also tried to implement requesting data to PBO, but unfortunately adding related functions to ffi is much more complicated than just adding glReadPixels. Tried to use glewInit() and it returns success, but there's still no functions in any namespace. Probably that requires much more knowledge of related libs.
By the way, forum engine insists that opengl abbreviations like PBO or FBO are 'too common' and refuses to search for them.

User avatar
zorg
Party member
Posts: 2284
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by zorg » Mon Feb 26, 2018 10:09 pm

The forum can only search for keywords with lengths of at least 4 characters, which is indeed a bummer, but i do recall some talk about this before, and i believe there would be issues if they enable the 3 character minimum limit.
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.

User avatar
raidho36
Party member
Posts: 1674
Joined: Mon Jun 17, 2013 12:00 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by raidho36 » Tue Feb 27, 2018 2:03 am

Calling collectgarbage triggers the GC to do a full round of garbage collection - it is a slow operation and is slower if you have more live Lua objects.

I suggest you try accomplish your task without reading pixels.

no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by no_login_found » Thu Mar 01, 2018 1:21 pm

I was able to get pointers to missing opengl functions with wglGetProcAddress on win.

Now it takes 10^-5 seconds to get pixel from PBO (not sure how inaccurate such small values are), with glReadPixels as soon as drawing is complete and reading from PBO only when needed.

Probably it can be optimized more by manually creating framebuffer with params optimized for reading as opposed to 'for render', but at the same time I know that nvidia drivers are good at overriding hints when the usage doesn't match them, so it's also possible that the canvas was automatically optimized for reading.

Tried if I have the same issue with missing functions on linux, and there was no issue: just ffi.load("GL") is enough. It ran 19 fps without requesting pixels and 17fps with requesting pixels. But since it was on VM, I can't really make any conclusions.
Tried it for android, and it says there's no "libGL.so". Made it fallback to Canvas:newImageData if opengl loading wasn't successful. Anyways, android is to slow to run the game, so I tried it just out of curiosity.

Not sure if it's good enough, there's still a chance that I'll be forced to use the solution which doesn't involve reading pixels from framebuffers.

-- creating pbo

Code: Select all

      ffi.GLL.glGenBuffers(1, PBO_id)
      ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
      ffi.GLL.glBufferData(ffi.GL.GL_PIXEL_PACK_BUFFER, memsize, nil, ffi.GL.GL_STREAM_READ)
      ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
-- requesting data from canvas

Code: Select all

    love.graphics.setCanvas(zBuffer)
    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
    ffi.GL.glReadPixels(love.mouse.getX(),love.mouse.getY(), 1,1, ffi.GL.GL_RGBA, ffi.GL.GL_UNSIGNED_BYTE, nil)
    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
    love.graphics.setCanvas() 
--getting data from pixel buffer object

Code: Select all

    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
    zImage2 = ffi.cast("unsigned char *", ffi.GLL.glMapBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER,ffi.GL.GL_READ_ONLY))
    ffi.GLL.glUnmapBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER)
    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
    id = zImage2[0]
-- getting missing functions

Code: Select all

    
    if ffi.GL and ffi.os == "Windows" then
      ffi.cdef[[
          typedef int (__stdcall *PROC)();
          typedef char* LPCSTR;
          PROC wglGetProcAddress(LPCSTR name);
      ]]

      local nameHolder = ffi.new("char[100]")
      ffi.GLL = {}
      local function loadProc(definition, name)
        ffi.copy(nameHolder, name)

        ffi.GLL[name] = ffi.cast(definition, ffi.GL.wglGetProcAddress(nameHolder))

        local asInt = ffi.cast("int",ffi.GLL[name])
        if asInt >=-1 and asInt <=3  then
          print("wglGetProcAddress returned nothing for "..name)
          io.stdout:flush()
          error("wglGetProcAddress returned nothing")
        end
      end


      loadProc("void (*)(GLsizei, GLuint *)", "glGenBuffers")
      loadProc("void (*)(GLenum, GLuint)", "glBindBuffer")
      loadProc("void (*)(GLenum target, GLsizeiptr size, GLvoid* data, GLenum usage)", "glBufferData")
      loadProc("void* (*)(GLenum target,  GLenum access)", "glMapBuffer")
      loadProc("GLboolean (*)(GLenum target)", "glUnmapBuffer")
    else
      ffi.GLL = ffi.GL
    end

User avatar
raidho36
Party member
Posts: 1674
Joined: Mon Jun 17, 2013 12:00 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by raidho36 » Thu Mar 01, 2018 3:50 pm

What are you doing with that one pixel anyway?

no_login_found
Prole
Posts: 18
Joined: Sun Dec 31, 2017 4:04 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by no_login_found » Thu Mar 01, 2018 4:13 pm

Checking which ui element is under the mouse. Approximately like it's described here https://en.wikibooks.org/wiki/OpenGL_Pr ... _selection, but now my implementation is much more simple since I'm not using stencil buffer, just painting a rect with color=element id to another canvas. Probably it will be more useful when/if I have overlapping non-rect shapes with holes.

grump
Party member
Posts: 353
Joined: Sat Jul 22, 2017 7:43 pm

Re: Is there something like Canvas:overwriteOldImageData() ?

Post by grump » Thu Mar 01, 2018 5:00 pm

no_login_found wrote:
Thu Mar 01, 2018 4:13 pm
I'm not using stencil buffer, just painting a rect with color=element id to another canvas. Probably it will be more useful when/if I have overlapping non-rect shapes with holes.
That's a straightforward and obvious way to implement it, but beware: reading pixels from the color buffer stalls the rendering pipeline and can be a very expensive operation, depending on the rendering hardware and the complexity of your scene. If at all possible, it's usually faster to have a simple representation of your UI shapes instead, e. g. by using a region data structure or polygon approximations.

Post Reply

Who is online

Users browsing this forum: grump, pgimeno and 8 guests