Question about shader and camera

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Question about shader and camera

Post by ChicoGameDev »

Hello,

This is my first question here and I hope to find someone that have good knowledge about Love2D graphics and shaders.

I'm actually working on a minimalistic light system for my game but actually I'm building it for the Love2D community, but not ready for release yet. That's why I'm asking here this question...

So I have this shader :

Code: Select all

#define NUM_LIGHTS 32

    struct Light {
        vec2 position;
        vec3 diffuse;
        float power;
        bool enabled;
    };

    extern Light lights[NUM_LIGHTS];

    extern vec2 screen;
    extern vec3 ambientLightColor = vec3(0);

    const float constant = 1.0;
    const float linear = 0.09;
    const float quadratic = 0.032;

    vec4 effect(vec4 color, Image image, vec2 uvs, vec2 screen_coords){
        vec4 pixel = Texel(image, uvs);
        pixel *= color;

        vec2 norm_screen = screen_coords / screen;
        vec3 diffuse = ambientLightColor;

        for (int i = 0; i < NUM_LIGHTS; i++) {
            if (lights[i].enabled) {
                Light light = lights[i];
                vec2 norm_pos = light.position / screen;
                
                float distance = length(norm_pos - norm_screen) / (light.power / 1000);
                float attenuation = 1.0 / (constant + linear * distance + quadratic * (distance * distance));
                diffuse += light.diffuse * attenuation;
            }
        }

        diffuse = clamp(diffuse, 0.0, 1.0);

        return pixel * vec4(diffuse, 1.0);
    }
So it is working wonderfully in a static screen (no camera at all).

In my game I have a camera like this :

Code: Select all

local Camera = {}

    Camera.x = 0
    Camera.y = 0
    Camera.scaleX = 1
    Camera.scaleY = 1
    Camera.rotation = 0

    Camera.shakeDuration = 0
    Camera.shakeMagnitude = 0

    function Camera:set()
        love.graphics.push()
        love.graphics.rotate(-self.rotation)
        love.graphics.scale(1 / self.scaleX, 1 / self.scaleY)
        love.graphics.translate(-self.x, -self.y)
    end

    function Camera:update(dt)
        if (self.shakeDuration > 0) then
            self.shakeDuration = self.shakeDuration - dt
        end -- if
    end -- function

    function Camera:draw()
        if (self.shakeDuration > 0) then
            local dx = love.math.random(-self.shakeMagnitude, self.shakeMagnitude)
            local dy = love.math.random(-self.shakeMagnitude, self.shakeMagnitude)
            love.graphics.translate(dx, dy)
        end -- if
    end -- function

    function Camera:unset()
        love.graphics.pop()
    end

    function Camera:move(dx, dy)
        self.x = self.x + (dx or 0)
        self.y = self.y + (dy or 0)
    end

    function Camera:rotate(dr)
        self.rotation = self.rotation + dr
    end

    function Camera:scale(sx, sy)
        sx = sx or 1
        self.scaleX = self.scaleX * sx
        self.scaleY = self.scaleY * (sy or sx)
    end

    function Camera:setPosition(x, y)
        self.x = x or self.x
        self.y = y or self.y
    end

    function Camera:setScale(sx, sy)
        self.scaleX = sx or self.scaleX
        self.scaleY = sy or self.scaleY
    end

    function Camera:startShake(duration, magnitude)
       self.shakeDuration = duration
       self.shakeMagnitude = magnitude
    end -- function

return Camera
I use Camera:setPosition(x, y) to move my camera according to my player and EVERYTHING is working great! But ... Maybe some genius lovers out there already know what my problem is!

I define a light in my shader at the position of my player, and I never update it's position (fixed light). Sadly the light will always "follow" the camera or screen. and the light will not appear correctly on the player. So I assume there is a "coordinates system" problem.

My actual understanding of the issue is that I should do some math with the global TransformProjectionMatrix variable (https://www.love2d.org/wiki/Shader_Variables) to be able to get the coordinates based on the graphics current translate or something like that.

I really hope that make any sens for somebody :P

I don't ask somebody to solve the math for me but just to know if I'm on the correct "path" for solving my issue. I already tried a whole lot of thing in the shader but literally I've just found tons of ways of doing always the same thing... :rofl:

Don't even hesitate to answer, I need reflection about the subject so even if you're not aware of a solution, if you think of something, please let me know! :awesome:

I really hope to be able to bring my light engine to the love2D community, I love this framework and it's the last real obstacle for me to let you guys add easy lights to your games.

As a side note (not for promoting) here's the trello of the project : https://trello.com/b/4kLFwfyV


Regards
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
User avatar
pgimeno
Party member
Posts: 3550
Joined: Sun Oct 18, 2015 2:58 pm

Re: Question about shader and camera

Post by pgimeno »

Well, I haven't ever tried to use TransformProjectionMatrix for this usage. Probably you only need TransformMatrix though. You basically need to transform (i.e. multiply) the coordinates of the lights by that matrix.

PS. Sorry for not replying earlier. Sometimes some posts aren't marked as unread by phpBB. I think it has to do with the initial moderation of new posters. Hopefully that's over for you :)
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Re: Question about shader and camera

Post by ChicoGameDev »

Hello Sir,

First of all thanks for your answer!

Yes I already tried what you mention, but maybe I did it wrong... I tried so many things... Hard to tell now!

But no my problem is still here and I don't mind the wait don't worry :D

I will work on a demo.love file for you to visualize and be able to maybe try ideas you may have with your experience (I already saw some of your answer on different shader topics and you seem the man with the knowledge to help me!)

Sorry I didn't post the .love file directly with my question but I hope I can provide this this evening. (Can't simply attach my game, you know kind of a serious project...)

Regards
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Re: Question about shader and camera

Post by ChicoGameDev »

Hello again,

Here we go !

A nice simplified playing field to search for the solution. I will use it too for trying to solve my issue.

I'm very grateful to everybody who will try out to find a solution to this issue! Thanks very very much!

Have a nice and lövely weekend !


Regards
Attachments
Demo_Shader.love
Safe play zone to find out how to tame this light shader. :D
(4.46 KiB) Downloaded 97 times
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
User avatar
pgimeno
Party member
Posts: 3550
Joined: Sun Oct 18, 2015 2:58 pm

Re: Question about shader and camera

Post by pgimeno »

Thanks for the test case, that really helps! So I believe that the problem here is that you're relying on autobatching, and that makes the caveat in TransformMatrix (in Shader_Variables) to apply to your case.

Possible solution 1: Don't rely on autobatching: create a spritebatch with the tiles instead and draw that. Then you can use TransformMatrix.

Possible solution 2: Update the light using love.graphics.transformPoint.

I'm a bit confused about your units, so I can't give any better advice, sorry.

Edit: Example of the second option using your sandbox:

Code: Select all

--- main.lua.orig	2019-02-15 18:44:24.000000000 +0100
+++ main.lua	2019-02-15 19:43:13.096605531 +0100
@@ -82,10 +82,12 @@
 
     -- create a light in the position of the center of screen (just so you can see it when you lauch the main.lua file...) 
     -- actually if you put a world position here the light will always be based on screen coordinates... and what ever you do when the camera is moved around the light always follow.
+--[[
     shader:send("lights[0].position", {
         love.graphics.getWidth() / 2,
         love.graphics.getHeight() / 2
     })
+--]]
     shader:send("lights[0].diffuse", { 1.0, 1.0, 1.0 })
     shader:send("lights[0].power", 15)
     shader:send("lights[0].enabled", true) 
@@ -113,10 +115,15 @@
     Camera:move(vx, vy)
 end -- function
 
+local ofs = {0,0}
 function love.draw()
 
     Camera:set()
 
+    local mapCentre = (mapSize * tileSize) / 2 
+    ofs[1], ofs[2] = love.graphics.transformPoint(mapCentre, mapCentre)
+    shader:send('lights[0].position', ofs)
+
     love.graphics.setShader(shader)
 
     for x = 1, mapSize do
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Re: Question about shader and camera

Post by ChicoGameDev »

Oh mister I used your second suggestion because I'm not able to understand the first one. And I was so surprised to see it working like heaven!

But I need more explanation :P

And sorry about the units I'm confused myself... :? Maybe you could point out some changes you'll make or just explain why you are getting confused. (I can't even explain why I'm confused...)

One more thing is I'm actually wondering about is the performance cost of the second solution. But it seems ok, but again I don't understand why.

Why a spritebatch will help the light for not moving ? :rofl:

I don't know how much to thank you but if you don't want to spent too much time on writing maybe we should consider talking over Discord or something.


Regards
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
User avatar
pgimeno
Party member
Posts: 3550
Joined: Sun Oct 18, 2015 2:58 pm

Re: Question about shader and camera

Post by pgimeno »

My confusion came from your use of screen coordinates for the light position. I think that just needs to be ditched, and lights need to be expressed in world coordinates (the coordinates of the objects in the game's world), which in your case seem to match pixel coordinates (the coordinates of each tile's pixels). It's also confusing to me that you use a scale factor of 0.25 to zoom in, and invert in the camera. The usual interpretation of scale is pretty much the same as "zoom factor" (2x means double the apparent size, 0.5x means the size halved, etc).

The problem of the approach of using love.graphics.transformPoint on every light is that it needs to transform all lights in the CPU side, yet transforms are something that shaders are really good at. I don't think the difference will be noticeable in terms of a performance reduction actually, but it's more elegant if it can be moved to the shader.

Besides the spritebatch alternative, there's a third possible solution: to rework your camera code so that it can also provide you with a matrix. That would work as follows:
  • At creation time, you create a new Transform object with love.math.newTransform()
  • In Camera:move you use self.transform:translate(), in Camera:rotate you use self.transform:rotate and so on.
  • In Camera:set, instead of love.graphics.translate etc, you use love.graphics.applyTransform(self.transform) (but keeping love.graphics.push()).
  • (Edited: that's not correct, see next page for the corrected version)
  • You add a new Camera:getTransform() that returns self.transform:getMatrix() (see Transform:getMatrix).
You can now use the camera for drawing and also have a matrix that you can send to the shader.

ChicoGameDev wrote: Fri Feb 15, 2019 7:57 pm Why a spritebatch will help the light for not moving ? :rofl:
Hm, that requires a bit of explanation. I'm not sure how much you know. You seem to work with pixel shaders just fine, so obviously you know some stuff, but not being sure how much, I have to start from the basics.
  • LÖVE uses OpenGL. It uses it for 2D purposes, but it doesn't use anything specific to 2D. GPUs are good at drawing polygons, and LÖVE sends most of its data as polygons. When you draw an image, LÖVE is internally sending 4 vertices and 2 triangles, with the UV (the texture coordinate) of each vertex set to a corner of the texture.
  • Drawing calls are expensive. The fewer times you draw, the faster. While this sounds obvious at first sight, consider this: in most GPUs it's faster to draw 100 polygons in a single draw call, than to draw 20 images each with its own drawing call. It's even probably faster to prepare the 100 polygons on the fly and send them in one call, than to draw the 20 images.
  • For that reason, LÖVE supports spritebatches. Spritebatches are internally a collection of polygons, with the UVs adjusted to the right fraction of the texture.
  • Starting with version 11.0, LÖVE supports autobatching, i.e. automatic creation of spritebatches. This means that when you execute a draw call, LÖVE doesn't draw it immediately, but instead it checks whether the texture and some other conditions are unchanged with respect to the last draw call. If so, it adds the image to a spritebatch. If not, or if love.draw finishes, it draws the previous image or spritebatch instead. This accelerates drawing substantially when drawing quads of the same image over and over. For example, using your example and disabling the shader and vsync, I get 400 FPS with LÖVE 11.2 and 270 FPS with LÖVE 0.10.2.
  • Autobatching works transparently and with no user control, but it needs certain conditions, for example to draw the same image. Whenever you draw a different image, the current batch is drawn and a new one is started.
Now, in the Shader_Variables wiki page, the documentation for TransformMatrix says:

"The transformation matrix affected by love.graphics.translate and friends. Note that automatically batched vertices are transformed on the CPU, and their TransformMatrix will be an identity matrix."

(Emphasis added). This is because as you keep drawing stuff, LÖVE doesn't use a draw call. Instead, it applies the current transform (in the CPU side, as the text says) and adds the transformed image to the batch in its final position. If you draw another image with a different transform, it will go into the batch in the correct position too, and so on. Since the transformation happens on the CPU side, the GPU needs to avoid doing any transformation to the batch for it to be correctly drawn.

So, in order to have a valid TransformMatrix with the current transform instead of the identity matrix, you need to make the batch manually instead of letting LÖVE do it automatically.

I haven't tried this. I don't know if it will work. I presume it will. I added that text myself to the wiki after asking Slime about a problem I was having similar to yours, but I haven't looked into that part of the code myself.

The modifications to the Camera in order to get the current transform, as I've suggested above, seem quite simpler, frankly.
Last edited by pgimeno on Sat Feb 16, 2019 11:04 am, edited 1 time in total.
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Re: Question about shader and camera

Post by ChicoGameDev »

"Everyday is another chance to do it right."

Hello,

I will print your answer and keep it like a very preciousssssss treasure. Your explanation is worthy of a famous book. Amazing quality.

So actually just to make things clear, I just started to give interest about shaders and such complex topic. The shader I use was originally inspired by this video tutorial : https://www.youtube.com/watch?v=BkxN2pwwRPM (just to give the right credit)

I see that I have to change my camera because it is the source of literally all the confusion both for me and for you. And the alternative you offer me here is as you said :
pgimeno wrote: Fri Feb 15, 2019 11:40 pm The modifications to the Camera in order to get the current transform, as I've suggested above, seem quite simpler, frankly.
And I totally agree and that make so much sens to me! I was already confuse by the fact my camera wasn't able to say : Hey, here my transform matrix, convert what ever you want with it.

So here what I will do: I will try implement a new camera with the technique you pointed out. And really just hope for the best since I never used these 'Transform' functions. And just to be clear too, this camera code was not mine too, was inspired by something I found long ago, and I thought I was winning is simplicity... :rofl:

I will probably be able do create a new camera but still, the calculation (shader side) with the camera matrix, how will it need to work?
Will I still need to normalize my positions with the screen? Is it the position of the light I need to multiply with the camera matrix or is it the pixel coord?

As you can see I still lack a lot of understanding in that topic, but with this kind of answer the step forward is so huge! I think you are the best reason to work with this framework rather than with Unity or Unreal engines... Thank you !


Regards
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Re: Question about shader and camera

Post by ChicoGameDev »

Hi again,

Here is my demo_v2.love I managed to implement the new camera system with the Transform object, it is working great!

But now as I mention in my previous post, I'm not able to make the light work without your previous solution with love.graphics.transformPoint(). I will keep trying tho. But now the camera return it's matrix correctly and I send it to the shader. I'm surely missing something basic here...

Mind some of the comments in the main.lua file. I've changed the ambient light color for exemple, just to know that my shader is used.

Thanks again and have a nice day.


Regards
Attachments
demo_v2.love
Here the next demo to make the final progress toward sucess !
(4.88 KiB) Downloaded 87 times
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
User avatar
ChicoGameDev
Citizen
Posts: 70
Joined: Thu Feb 14, 2019 6:02 pm
Location: Switzerland
Contact:

Re: Question about shader and camera

Post by ChicoGameDev »

Just to keep information going and avoid wasting too much your time...

I've got "result" when I changed the formula in the shader to :

Code: Select all

float distance = length(light_pos - vec4(norm_screen, 0.0, 1.0)) / (light.power);
(Note the power being divided...)

And when I set a power of 150 in the light.. And I move straight to the top left corner of the map there are some light reaction but very strange...

Actually I'm playing with the formula in case the change of coordinates system have broken everything... need to find the good balance.



Regards
Lionel Leeser

Luven : https://github.com/chicogamedev/Luven

--

Always keep Game Dev as a passion.
Post Reply

Who is online

Users browsing this forum: No registered users and 72 guests