Bug in pixel shader [Solved]

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
Post Reply
User avatar
drunken_munki
Party member
Posts: 133
Joined: Tue Mar 29, 2011 11:05 pm

Bug in pixel shader [Solved]

Post by drunken_munki » Sun Mar 18, 2018 10:26 pm

Hey folks, this should be easy but I'm getting strange results (0.10.2):

I want to replace the pixels inside an image (or plain box) with those from an extern image passed to a shader:

Code: Select all

extern Image mask;

vec4 effect(vec4 color, Image tex, vec2 tc, vec2 sc) {  
  // tc = sc / love_ScreenSize.xy;
  vec4 pixel = Texel(mask, tc);
  return pixel * color;
}
But the shader draws just black i.e. nothing comes through with the co-ordinates kept as the original texture_coords.

If you uncomment the test line, the shader works fine -- here the extern image will be scaled to the current window. If you move the wasd about you can move the render box and see the test image in parts. This is not what I want.

I want the target box to contain the entire texture (even if you move it with wasd)... but my original example doesn't work.

What the bloody feck is going on lads?

Cheers.
Attachments
shadermask.love
(1.58 KiB) Downloaded 12 times
Last edited by drunken_munki on Mon Mar 19, 2018 4:07 pm, edited 1 time in total.

User avatar
pgimeno
Party member
Posts: 1150
Joined: Sun Oct 18, 2015 2:58 pm

Re: Bug in pixel shader

Post by pgimeno » Sun Mar 18, 2018 10:49 pm

I think that the problem is that there are no texture coordinates for rectangles. Try drawing something else. For example, it works for me if I create a dummy texture and draw it instead:

Code: Select all

local dummy = love.graphics.newImage(love.image.newImageData(1,1, '\0\0\0\0'))
...
  love.graphics.draw(dummy, x, y, 0, w, h)
Thrust II Reloaded - GifLoad for Löve - GSpöt GUI - My NotABug.org repositories
The Microsoft Github repositories I had have been closed after the acquisition announcement and will be removed in the near future.

User avatar
drunken_munki
Party member
Posts: 133
Joined: Tue Mar 29, 2011 11:05 pm

Re: Bug in pixel shader

Post by drunken_munki » Sun Mar 18, 2018 11:07 pm

That seems to work, how weird.

In my proper version I use an image but have the same problem.

I wonder If there is some sort of shader caching or something causing me problems?

User avatar
pgimeno
Party member
Posts: 1150
Joined: Sun Oct 18, 2015 2:58 pm

Re: Bug in pixel shader

Post by pgimeno » Sun Mar 18, 2018 11:43 pm

Without an example that doesn't work, I can't say much. Maybe you can trim down your program so that it still exhibits the problem, and it's something you can attach as a .love file?
Thrust II Reloaded - GifLoad for Löve - GSpöt GUI - My NotABug.org repositories
The Microsoft Github repositories I had have been closed after the acquisition announcement and will be removed in the near future.

User avatar
drunken_munki
Party member
Posts: 133
Joined: Tue Mar 29, 2011 11:05 pm

Re: Bug in pixel shader

Post by drunken_munki » Mon Mar 19, 2018 12:30 pm

Might be tricky, the game is now over 100 thousand lines.

Anyway, I'm using images in a spritebatch. So it seems that this is causing the same issue as drawing with a shape?

The spritebatch must be setting it's own width and height to normalise the co-ords of a texture? If that's the case then I should be able to rescale them to correct for this.

I'll have to test it to be sure but this seems to be the last two things to check.

Cheers for your help.


EDIT

I can confirm this works. My problem is that the spritebatch is normalising the texture co-ords to the size of the texture atlas it is linked to, and not the dimensions of the quad it is rendering -- I wonder if this is as intended or a limitation in openGL (or a bug?).

By rescaling the co-ordinates everything works grand.

I'll mock up an example later today in case anyone is curious.

User avatar
pgimeno
Party member
Posts: 1150
Joined: Sun Oct 18, 2015 2:58 pm

Re: Bug in pixel shader

Post by pgimeno » Mon Mar 19, 2018 1:26 pm

Ahh, so you're drawing quads. It makes sense then: when drawing a quad, the texture coordinates that the shader receives are the texture coordinates within the original texture to draw from. Unfortunately, Löve doesn't pass the vertex coordinates from vertex to fragment shader, which is what you would need here.

One way around that would be to pass the coordinates you're drawing on to the shader, and let the shader do the mapping necessary to convert them to texture coordinates within the mask, using the screen coordinates parameter of the effect() function.
Thrust II Reloaded - GifLoad for Löve - GSpöt GUI - My NotABug.org repositories
The Microsoft Github repositories I had have been closed after the acquisition announcement and will be removed in the near future.

User avatar
drunken_munki
Party member
Posts: 133
Joined: Tue Mar 29, 2011 11:05 pm

Re: Bug in pixel shader

Post by drunken_munki » Mon Mar 19, 2018 4:06 pm

Ok I whipped up a test program that has all the cases mentioned so far in this thread, WASD moves the render boxes about.

Here is a sample output:
Image2.png
Image2.png (22.15 KiB) Viewed 462 times

Case 1 is the obvious shape problem, so we expect nothing to come out. Case 2 and 3 work as intended.

Case 4 was my problem last night, drawing a quad gives a mixed up co-ordinate space that is normalised (internally by LOVE?) by the full atlas size and not the quad size, so it comes out wrong.

Case 6 is the fix for 4, where I rescale the texture co-ordinates back to the atlas size and then normalise it back to the texture size.

So there we are. Thank you pgimeno for your ideas.
Attachments
shader_bug_test.love
(8.84 KiB) Downloaded 10 times

User avatar
pgimeno
Party member
Posts: 1150
Joined: Sun Oct 18, 2015 2:58 pm

Re: Bug in pixel shader [Solved]

Post by pgimeno » Mon Mar 19, 2018 5:35 pm

You're drawing a spritebatch with just one quad in the example. If that's your real use case, you don't need a spritebatch for that; just love.graphics.draw(image, quad, ...) will do.

It works in this case because of the conditions you're drawing under in this example, but it won't work in all cases. If your texture wrap mode is "repeat" in both dimensions, AND your quads are all sized as an integer fraction of the texture width, and all the same size, then yes, it will work, mostly by chance, due to modular arithmetic.

I've changed the quad to the top right; the trick didn't work unless I also uncommented the setWrap line.
Thrust II Reloaded - GifLoad for Löve - GSpöt GUI - My NotABug.org repositories
The Microsoft Github repositories I had have been closed after the acquisition announcement and will be removed in the near future.

Post Reply

Who is online

Users browsing this forum: No registered users and 10 guests