LÖVE

I'm trying to create a masking effect for a character select screen, and I can't seem to make heads or tails of how stencils work.

It really doesn't help, that the only examples I found on the wiki, seem to be strictly vector-based. Is it even possible to mask an image, using another image, as a guide? If so, could somebody please post an example? I'm quite stumped.

Since my art is NSFW, I'll post a simplified diagram, showing what I'm going for:


The basic idea is that there's a spotlight in the middle of the screen, and the character sprites are displayed as negative outlines outside of the spotlight, and fully colored art, inside the spotlight.

Maybe something like this in reverse?
(Very old code so don't know how good it is)

Welcome to the forums. By default, alpha is not taken into account when drawing to the stencil, only the geometry. That can be changed with a shader. For example, this shader makes every pixel with alpha < 0.5 to not be drawn to the stencil:
See attachment for an example using your graphics.

Ref wrote: ↑Sun Aug 18, 2019 1:42 am Maybe something like this in reverse?
(Very old code so don't know how good it is)

Oh, that's a neat effect! It's not quite what I'm going for (a bit too noisy for my liking), but it looks great!
My art is strictly pixel-based, so I want really sharp lines.

pgimeno wrote: ↑Sun Aug 18, 2019 2:33 am Welcome to the forums. By default, alpha is not taken into account when drawing to the stencil, only the geometry. That can be changed with a shader. For example, this shader makes every pixel with alpha < 0.5 to not be drawn to the stencil:

Code: Select all

vec4 effect(vec4 colour, Image texture, vec2 texpos, vec2 scrpos)
{
    vec4 pixel = Texel(texture, texpos) * colour;
    if (pixel.a < 0.5) discard;
    return pixel;
}

See attachment for an example using your graphics.

Ah! Thank you! That's EXACTLY what I was going for! So, you can use the alpha for stencils? Huh, good to know! I was under the assumption that stencils were strictly bitmask-based.

So, "pixel.a" is the value of the alpha channel for a given pixel, correct? Just making sure I understand what the code is doing.

For what it's worth, attempts were made, based on information from the wiki, but the closest I was able to get, was inverting the bit mask (black to white, and white to black), and I don't even understand how THAT worked!

I've attached my previous attempts, and the result, based on the code you provided. (They're NSFW, so fair warning.)
"stencil2 + stencil3" are my attempts, but I couldn't quite get it right, after a few hours of struggling with it.
"spotlight10" is the result I was aiming for, based on your example. The curtains are a placeholder from Mario 2, for the moment.

Could the example code you provided be posted to the wiki? It would be VERY helpful for others, I imagine!

EDIT: The forums dropped my attachments, argh.

Cool, and nice graphics!

Alp wrote: ↑Sun Aug 18, 2019 1:18 pm Ah! Thank you! That's EXACTLY what I was going for! So, you can use the alpha for stencils? Huh, good to know! I was under the assumption that stencils were strictly bitmask-based.

A stencil is a buffer that holds numbers. It's not a bitmask, but you can apply a comparison method to it and the result is always a boolean which tells whether each corresponding pixel should be drawn or ignored. By default the stencil function sets it to 1, so I used "equal to 1" or "not equal to 1" as comparison functions.

Alp wrote: ↑Sun Aug 18, 2019 1:18 pmSo, "pixel.a" is the value of the alpha channel for a given pixel, correct? Just making sure I understand what the code is doing.

Yes, in GLSL you can address the elements of a vec4 with .xyzw or with .rgba (.x is the same as .r and so on).

If you don't know about shaders, there's a good introduction to the basics in the LÖVE blog: http://blogs.love2d.org/content/beginners-guide-shaders and the default shader used by LÖVE (very similar to the one I used) is here: https://love2d.org/wiki/love.graphics.newShader#Notes_2

Alp wrote: ↑Sun Aug 18, 2019 1:18 pm Could the example code you provided be posted to the wiki? It would be VERY helpful for others, I imagine!

There's one example already which uses black for transparent instead of an alpha threshold: https://www.love2d.org/wiki/love.graphi ... encil_mask