Local coordinates and UI elements

[Altostratus]

Suppose a primitive UI element defined like so:

Code: Select all

UI.Surface = {}
function UI.Surface:new(w, h)
    local o = {
        w = w,
        h = h,
        isHover = false,
        color = {1, 0, 0, 1},
        hoverColor = {0, 1, 0, 1}
    }
    setmetatable(o, self)
    self.__index = self

    return o
end

I create and position it inside the parent object's draw() call with:

Code: Select all

local surface = UI.Surface:new(200, 100)

lova.graphics.draw()
    love.graphics.push()
    love.graphics.translate(300, 300)
        surface:draw()
    love.graphics.pop()
end

In order to calculate hover states properly, the draw() method for the object applies a transformation based on the display scale (actual dimensions / reference resolution):

Code: Select all

function UI.Surface:draw(input)
    -- we store the screen coordinates of the surface on each frame
    self.x, self.y = love.graphics.transformPoint(0, 0)
    self.x, self.y = self.x / global.display.s, self.y / global.display.s

    if self:isHover(input) then
        love.graphics.setColor(self.hoverColor)
    else
        love.graphics.setColor(self.color)
    end

    love.graphics.rectangle('fill', 0, 0, self.w, self.h)
end

This seems a bit silly because I shouldn't need to calculate the element's screen coordinate on each frame. The parent object is positioning it and any modification in its coordinates would draw it at the correct position. But I can't use "love.graphics.transformPoint(0, 0)" inside the :new() method, because the element hasn't been positioned yet and its coordinates will always be (0,0).

I could recalculate the position in update() but I don't think it would achieve much, and would moreover split the element's reference in two places. But leaving it like that seems to be very inefficient and could potentially affect performance if using many elements.

What would be the most idiomatic way to solve this in Love2D?

[Hugues Ross]

Personally I couldn't care less about whether an approach is 'idiomatic' unless I'm being paid or the consequences of not doing so would be severe (such as security hardening for instance). So take my response with a grain of salt.

With caveat out of the way, my approach is to recalculate layout geometry when it changes--ie. if the dimensions of the canvas your UI is rendering in changes, if it's transformed, etc. If you want to minimize how often this is called, you could use a dirty flag to avoid calculating more often than needed (in the event that multiple geometry-affecting events happen on the same frame).

With that said, I'm gonna be honest--I'm much less worried about the number of calculations and more about whatever this means:

In order to calculate hover states properly, the draw() method for the object applies a transformation based on the display scale (actual dimensions / reference resolution):

Are you performing mouse input checks on every element, on every frame, inside of love.draw()? Cuz if so that seems like a much more pressing issue.

[Altostratus]

Are you performing mouse input checks on every element, on every frame, inside of love.draw()? Cuz if so that seems like a much more pressing issue.

Thanks. I am actually trying to avoid doing that, with limited success. I'm not doing that inside love.draw() in main.lua, because I'm trying to create some abstractions for UI elements which have their own draw() methods. But since the entire hierarchy is called from main.lua then, essentially, yes.

Basically I'm checking on every frame whether the mouse is within the bounding box of a UI element. When I get this to work correctly I was planning on checking deeply-nested elements only if the larger parent was already hovered (and so avoid checking other parent objects and their children).

It feels like trying to reinvent the wheel but it's an interesting exercice for someone coming from React and JS where all that stuff is done for you.

Any suggestions are of course welcome...

[RNavega]

Any suggestions are of course welcome...

Since a focus change is a consequence of the mouse moving around, it's really only necessary to re-check the hovering of UI elements from within love.mousemoved(), so it's done on demand.
In fact, I've done some tests with a modified love.run: in some cases you only need to update/redraw the program if a mouse or keyboard or some timer event happens -- outside of that, the program can go to 0% CPU use.

I was also going to suggest that spatial optimization that you're already considering, of checking parents first: for the mouse to hover on an element that's part of a larger group, the mouse must also be hovering over that larger group. So if you have like several UI groups (like different forms, dialogs etc) in your program, and the mouse does not hover the bounding box of a specific group, then the mouse definitely can't hover on any elements inside that specific group.

There's also some design inspiration to be taken from larger UI libraries. Just reading things can give you some ideas:

• https://doc.qt.io/qt-6/eventsandfilters.html

• https://docs.gtk.org/gtk4/input-handlin ... ropagation

• https://developer.android.com/topic/arc ... e-ui-state

• https://developer.apple.com/tutorials/a ... -and-state

• https://docs.oracle.com/javase/tutorial ... l#twokinds

• https://doc.qt.io/qt-6/model-view-progr ... ew-classes

• https://m2.material.io/components/dialogs#anatomy

[Altostratus]

Thank you very much for these very useful links. I'm still puzzled by this in @Hugues Ross' reply:

Are you performing mouse input checks on every element, on every frame, inside of love.draw()? Cuz if so that seems like a much more pressing issue.

...because I don't understand if it's polling on every frame that's the issue, or doing it inside love.draw().

I could solve my issue and have it run in some convoluted way, but I'm doing this for fun and trying to make it elegant and efficient is part of it. That's why the entire affair evolved into considerations on how to design a UI properly.

it's really only necessary to re-check the hovering of UI elements from within love.mousemoved()

So if I understand you correctly, since I can only define love.mousemoved() (and pressed etc) once, and can't use it in multiple places (such as from within the UI elements), I should group all UI elements inside some giant UI manager table and have it update all its members accordingly.

There's also some design inspiration to be taken from larger UI libraries

I'll have a look, thanks, reading the Qt6 help files will bring back memories... But I'm also looking at Love2D UI libraries in order to get some basic ideas about common practices.

EDIT: also, this Love2D library builds, checks and redraws the UI in love.draw(). Is it an inefficient idea? Should code be split between update() and draw() (e.g. UI.update() inside the former, UI.draw() inside the latter)?

[Hugues Ross]

...because I don't understand if it's polling on every frame that's the issue, or doing it inside love.draw().

I mean, both! But here's what I was trying to get across: In my opinion, you had your priorities wrong when it comes to improving this code--while checking the window size isn't great, it's not the biggest or most urgent issue.

I think the biggest problem is that you've got input, layout, and rendering all mixed together in a single process when these should be very well separated in a retained-mode UI. The constant polling is also an issue (and RNavega's post covers very it well imo), but with a cleaner architecture it shouldn't be too hard to swap in more event-driven inputs later down the line so I consider it secondary to fixing the former.


I will also give you a word of warning: Most Love2D UI libraries suck. If you're looking for a reference to study, I recommend sticking to well-worn frameworks outside of the ecosystem (here again, RNavega's got a decent set). The code example in the README of the library you've linked is creating 13 tables per frame, for no reason, just to draw 5 ui elements. And then there's stuff like this, which needs no introduction:

Code: Select all

function UI.draw_element(e, x, y, flow_down)
   local width, height
   if e.type == "label" then
      width, height = draw_label(e, x, y)
   elseif e.type == "button" then
      width, height = draw_button(e, x, y)
   elseif e.type == "slider" then
      width, height = draw_slider(e, x, y)
   elseif e.type == "dropdown" then
      width, height = draw_dropdown(e, x, y)
   elseif e.type == "inputbox" then
      width, height = draw_inputbox(e, x, y)

I could complain further but I think at some point it's more mean than constructive, this is more than enough to illustrate my point. If you're taking notes, just be sure you're taking them from the right place.


With all that said, I think it's also important to think about your needs and priorities when it comes to UI.

The reality is that if your UI needs are very basic, nothing that I just wrote above matters at all. You can basically do whatever dogshit approach and not only will it be fine, it'll probably be ideal due to being more efficient in terms of development resources, even if it's inefficient in terms of system resources. God knows I've got some questionable patterns in my own UI code, which I'll pull out eventually--but until those shortcomings are actually relevant to my project I have much more important things to spend my dev time on.

On the other hand, if you're specifically trying to learn the ins and outs of UI, or you have super complex UI needs? Yes, this may be something to spend extra time and energy on. It's all about figuring out where your priorities are.

[RNavega]

I could solve my issue and have it run in some convoluted way, but I'm doing this for fun and trying to make it elegant and efficient is part of it. That's why the entire affair evolved into considerations on how to design a UI properly.

It's funny, I'm in a similar position and also trying to come up with some elegant UI solution, both for practical use as well as for the learning experience.

So if I understand you correctly, since I can only define love.mousemoved() (and pressed etc) once, and can't use it in multiple places (such as from within the UI elements), I should group all UI elements inside some giant UI manager table and have it update all its members accordingly.

To be clear, 'mousemoved' and others, they are keys in the global table 'love'. You can define functions and store them in those keys, and at the time a certain event gets processed by love.run, the function that happens to be stored in the relevant key will be called with the event data. It's a great convenience, but you still have the freedom to poll for events or changes to program state in any other place of your code of course, if you prefer.
If you choose to use those event handlers, it makes sense to have some abstraction (like the UI manager you mentioned) finding an appropriate UI control and passing the event data to it so it can react.
There's some choices with how the event is actually passed to the focused control. Some of those UI libraries do it in phases, like traveling the hierarchy from the parent control (like a window or dialog) all the way down to the focused child control, asking each control on the way if they're interested in capturing that event -- this is the "capturing" phase. After that, the event is sent up again that same hierarchy in case it wasn't consumed -- it's the "bubbling" phase (a phase which I personally find a bit redundant).

What Hugues' mentioned in the last paragraphs is the most important part IMO: focusing on what you need, that is, what you want the user to experience.
This means deciding on the UI behaviors, the look-and-feel of everything, and the limited set of controls that you'll need for that particular game, before any code is written. When you do start coding, it's only to support that design.
You can refactor or add more things to your UI library later with future projects. It can improve with time.

If you find some problem areas in your code, I'd love to go over them. It might benefit me too.

[Altostratus]

Thank you both, @Hugues Ross and @RNavega. You've both hit the nail on the head.

I had decided to learn Love2D and Lua by porting a hobby project of mine (a sort of interactive audio synthesizer with many visual elements) which was written in JS and React. It took me a while to realise that the entire thing was in fact a huge UI with some code attached to it.

On a side note, I also realised that Love2D doesn't have audio oscillators like the Web Audio API and that I would need to build these from scratch. No matter.

So now I have to decide whether to go for what works for this application (which wouldn't take long but would probably be ugly and unusable in other projects) or build a relatively complex generic UI manager object (which might evolve into an entire UI framework instead of my application).

I stashed everything and went back to the basics, because I also realised that I didn't understand exactly how to simulate OOP patterns in Lua. But I'm making some progress. This is what you get when you're trying to create a polished application but are also very much interested in abstract computer science. And that's why I wrote that you've both hit the nail on the head.

Most Love2D UI libraries suck

I'm in a similar position and also trying to come up with some elegant UI solution

Can't say I wasn't baffled by the amount of posts by folks each publishing yet another UI framework for Love2D. The barebones design of both the framework and the language is elegant and flexible, but the lack of an accepted standard of doing UI things in Love2D did come as a surprise. Not very different from web technologies, to be honest, although a small number of mature UI frameworks are the norm there.

This seems like a natural candidate for a collaborative open-source effort by dedicated individuals. Unfortunately, I'm not a professional developer and I'm sure everyone has their own app requirements as a priority. I'm not sure how helpful I can be here to be frank, considering I've set Love2D's window to nil and went back to trying to understand inheritance with metatables, which is still a bit fuzzy in my mind after years of JS and Python.

Your comments are immensely helpful. Thank you for taking the time to share your thoughts. I'll post some code here when I have a basic working UI design idea, just in case it helps @RNavega or someone else in the future.