Running Python helper scripts in bundled game

[pgimeno]

I think an unoptimized GIF encoder would work for me here because the optimizer I was using (gifsicle) never took more than 10 percent off the user-generated GIFs made by gifMaker.py. This is a screen shot my game at the point where the user chooses to make a gif and selects the text to go into the gif:

<snip>

The user-generated gif is the little typewriter (here, at 4x speed) in the pop-up window above (or below at the very bottom. I think I messed up the formatting of this comment).

Is there a reason why this can't be generated in real time as opposed to saving a gif? Does the gif really need to be saved? Is that for the user to keep or something like that?

The vgif.love does get used in my game. It's what opens the gif in the screenshot above.

And is there a reason why you haven't integrated it into your program and are instead invoking it as an external executable, requiring Löve to be available in the path in the target machines in addition to Python? That game is going to be a hell to distribute. Running external executables is almost always a bad idea.

bookParse.py is important. It is essential, while the gifMaker is just a nice-to-have. I wrote bookParse.py in Python because I needed the help of a Python library (c-w/gutenberg on gitHub), specifically the function load_etext called on bookParse.py, line 309, here:

<snip>

This function takes a book_id-number as argument and downloads a plain text version of that book from a Project Gutenberg library at http://gutenberg.readingroo.ms. The indexing scheme of the readingroo.ms website is explained in the c-w library. So I think theoretically I could replicate the load_etext function. The problem is it's just beyond me right now. I spent a little time a month or two ago trying to figure out lua socket, and I never got any of the test cases to work.

Really? See if this works for you:

Code: Select all

local http = require('socket.http')
local mirror = "http://gutenberg.readingroo.ms"

local function download(url)
  local ret, status = http.request(url)
  return status == 200 and ret
end

local function get_gutenberg_text(id)
  local sid = tostring(id)
  local s0id = id < 10 and "0" .. sid or sid
  local url = s0id:sub(1, 1)
  for i = 2, #s0id - 1 do
    url = url .. "/" .. s0id:sub(i, i)
  end
  url = mirror .. "/" .. url .. "/" .. sid .. '/' .. sid
  local text = download(url .. "-0.txt") -- utf-8
  if text then
    return text
  end
  -- We don't know how to convert encodings, so prefer the plain text
  return download(url .. ".txt")
end

There's one thing that Python can do out of the box and Lua can't, which is to convert encodings. But for texts in UTF-8 or plain ASCII, the above works. ISO-8859-1 would be easy to support, but others not so much.

[meteopath]

Thank you very much for the socket code. It's awesome. It works for me in a test run. I'll have to rewrite the python regex and make data-flow adjustments before I can integrate it into my game. But it's huge. Means I've got all the core-game things in lua now. Also, I guess it will be a big performance boost. I'm going to have to learn about performance testing now, which, by the way, please suggest any tools or tutorials. I've just been going by what works on my machine, which I know isn't a great way to do it.

Is there a reason why this can't be generated in real time as opposed to saving a gif? Does the gif really need to be saved? Is that for the user to keep or something like that?

Yes, it's for the user to keep. After gif creation, the user is given options to view it (via vgif.love) and/or go to the save folder, which calls the function:

Code: Select all

love.system.openURL("file://"..love.filesystem.getSaveDirectory())

Speaking of which,

And is there a reason why you haven't integrated it into your program and are instead invoking it as an external executable, requiring Löve to be available in the path in the target machines in addition to Python?

No, there's no reason. Just an oversight. If I did see that in your docs, I must have said I'll figure it out later and just forgot. Thanks for the reminder. I'll definitely go back and integrate it. I do want to make this game available to people who don't have Love. Your point about Python also made me think. I thought I'd avoided that problem by including a python interpreter in the venv folder that gets copied into the Love save folder at the start of the game. But thinking again, I guess I didn't instruct bookParse.py or gifMaker.py to refer to the Python interpreter there in the folder with them. So they were probably getting the interpreter from my PATH. Which is definitely not how I wanted it to work.

And, lastly, it looks like I made a mistake about that all-inclusive zipped game. I spoke too soon. It's not working now. I moved the .love file into a new folder by itself and deleted the save folder to give it clean slate to make sure it was building everything itself. At that point when I tried running it, it didn't work. I'm going to do some more tests. I definitely had some positive results in the partial tests I did. So I'm still somewhat optimistic. But I was a little bit too out in front about this earlier. Thanks again for looking at this.

[pgimeno]

Yes, it's for the user to keep. After gif creation, the user is given options to view it (via vgif.love) and/or go to the save folder, which calls the function:
<snip>

The problem with saving a gif is not so much about how to encode it, but how to generate an adequate palette and match the colours to the palette. If I end up writing a gif encoder, I don't think I would write the palette generator and matcher, just the file format handling itself, so it would be up to the user to supply a palette and an already indexed image. Therefore, it would not really be suitable for generating gifs from Löve images.

With that in mind, how important is the gif generation functionality in your project? Is it something you could do without?

Your point about Python also made me think. I thought I'd avoided that problem by including a python interpreter in the venv folder that gets copied into the Love save folder at the start of the game. But thinking again, I guess I didn't instruct bookParse.py or gifMaker.py to refer to the Python interpreter there in the folder with them. So they were probably getting the interpreter from my PATH. Which is definitely not how I wanted it to work.

There are a couple points you seem to have missed.

The Python interpreter does not include the libraries. Including the interpreter without including each and every library that your program depends on, directly or (most importantly) indirectly, won't work in a machine where Python isn't installed. You'd need to bundle every depended on library with your project.

And then there's the problem of the OS support. If you include an executable for a specific OS and CPU, your program will only work in the OS/CPU/bit width for which it was made. You'll lose the compatibility with other OSes. Some of the Python libraries you're indirectly using also require binary libraries that are once again OS-dependent.

Did I say running external executables was almost always a bad idea? Well, if you only target power users that can install Python or any other requirements in their system, then that would be OK, but for a wider audience, it would not. So I'd advise you to try to work out a solution that doesn't use io.popen at all; doing so would also help you keep the game cross-platform, so the potential audience would be even bigger.

[meteopath]

The gifs are something I could do without. They have become important to me because I like them and spent time on them. But the priority is without question making the game as widely available as possible. I definitely do not want to limit it to users with Python and/or Love. Just to note, though, the gifs aren't generated by Love images. The user simply selects a text. They don't see the image before it gets produced and written to a gif. The text they select is the only argument passed to gifMaker.py. Image assembly happens there. gifMaker.py has access to a folder of sprites that are loaded into the game like all of the other assets before (at?) runtime. gifMaker.py takes these .png sprites and overlays them and animates them to make a series of images, basically 'n' loops of a typewriter typing a line -- where n is determined by the size of the text selected by the user -- bracketed by an intro (an envelope opening) and an outro (the typed page removed, read, then thrown away). This is a gif generated by gifMaker.py during runtime:



I was looking into optimizing the color palette a little while ago. I quit before figuring it out. But before I quit, I did redo my sprites to remove gradients. I got the whole thing down to 8 colors or less, I think. But I moved on before figuring out how to set the palette, so I didn't get any of the savings.

I looked at the lua-vips library today. I thought I might be able to combine it with ffmpeg to replace the functionality of the Python PIL library I use in gifMaker.py. I thought I could create images layering my sprites with commands like these:

Code: Select all

local image = vips.Image.new_from_file('gifMaker/sprites/casing.png',{access = "sequential"})
local overlay = vips.Image.new_from_file('gifMaker/sprites/kA.png',{access = "sequential"})

image = image:composite(overlay, "over", {x=0,y=0})

image:write_to_file('vips_test.png')

and either save them sequentially as a table or write them into a folder as a series of pngs, depending on whether ffmpeg could take a table directly as input instead of hard-written images. But unfortunately I couldn't make vips run on my machine. I'm going to take a look with fresh eyes later, but my attempts at installing both via luarocks and git clone surprisingly failed. It's so aggravating how the simplest things sometimes.

[pgimeno]

The gifs are something I could do without. They have become important to me because I like them and spent time on them. But the priority is without question making the game as widely available as possible. I definitely do not want to limit it to users with Python and/or Love. Just to note, though, the gifs aren't generated by Love images. The user simply selects a text. They don't see the image before it gets produced and written to a gif. The text they select is the only argument passed to gifMaker.py. Image assembly happens there. gifMaker.py has access to a folder of sprites that are loaded into the game like all of the other assets before (at?) runtime. gifMaker.py takes these .png sprites and overlays them and animates them to make a series of images, basically 'n' loops of a typewriter typing a line -- where n is determined by the size of the text selected by the user -- bracketed by an intro (an envelope opening) and an outro (the typed page removed, read, then thrown away). This is a gif generated by gifMaker.py during runtime:

Well, I hope I don't sound rude but are you sure there's value to the user with this? I mean, if you generate the animation within the game (which should't be too difficult), I don't think users will want to keep the gifs of each individual text being written in the typewriter; if they want to see the animation again they can always play the game once again. That might make them want to return to the game.

The requisite for that, however, is that you generate the animation using Löve, which shouldn't be hard I'd suppose.

[meteopath]

No, not rude at all. I'm grateful for the help and the attention. I know the point is to give outside perspective and get me to think, and it has. I hope I don't sound defensive. I probably will because I am attached to it now that I've done the layout and navigation and seen it work on my machine. It's not so much that the gifs are great to look at. I think that they'd look good in a promo video. I think they're inline with the game play, which is about text manipulation and giving the user control. But you could play the game over and over and never make a gif or even notice that the option is there. It's definitely peripheral. I'll take a little time to try to figure it out, but not too much and now looking at Lua-only solutions because the socket is working great. I'm getting the books directly into Lua now, so I'm definitely and very happily out on Python for this project. Sorry if anyone was having the same problems I was and reading for a fix. I see now how I got a "false positive" on my Python-to-save-folder test run a few posts earlier. Just briefly, I had my zipped game in the same folder as all my unzipped game components, so it was the unzipped venv that got copied to the save folder and not the zipped copy. When I moved the .love file and ran it from a new folder, I got this error:

Code: Select all

cp: venv: No such file or directory

and the folder didn't get copied.