Love2d/lua screen scanning and detecting images?

General discussion about LÖVE, Lua, game development, puns, and unicorns.
Post Reply
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Love2d/lua screen scanning and detecting images?

Post by togFox »

I've always wondered if there are libraries or anyone has tried to use lua to detect images on the screen and then react to those images. I'm not talking about classification problems in neural networks or AI - I mean just dumb image detection "is there a monster that looks exactly like this monster" type image/pixel detection.

Say a lua script that runs in the background, does screen scans and then does something when it finds an image that looks just like an image in it's library. That sort of thing. Too outrageous?
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: Love2d/lua screen scanning and detecting images?

Post by MrFariator »

Are you talking a screen scanner that scans the user's monitors, or a screen scanner within your love2d application itself? If former, you'd have to implement a way to grab snapshots of the user's system, or provided video source like webcam*. Doable, but there are perhaps better ways to implement this than love2d. You'd likely want to implement something like this on the C++ side of things, rather than lua. If latter, then I don't really see a reason to implement it, because you'll know the images ahead of time - so you can just do less expensive checks if a given image is on the screen. Of course, if you intend that users can choose and "upload" images, so that you can do the screen scanning on those images, you'd likely again be better off doing the computationally heavy stuff C++ side.

*I forget if love2d has any webcam functionality.
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Re: Love2d/lua screen scanning and detecting images?

Post by togFox »

The first one. I assume lua would be the wrong tool.
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: Love2d/lua screen scanning and detecting images?

Post by MrFariator »

Lua wouldn't necessarily be the wrong tool for the job, but if processing speed or doing this scanning close to real time is a consideration? Then yeah, there are better choices. The algorithms could be implemented in just about anything, and for testing or exploration purposes it's not too uncommon for academics to use something like Python. However, for speed and crunching larger datasets, C/C++ implementations often follow suit.
Post Reply

Who is online

Users browsing this forum: No registered users and 21 guests