LuaPreprocess

Showcase your libraries, tools and other projects that help your fellow love users.
Post Reply
User avatar
ReFreezed
Citizen
Posts: 63
Joined: Sun Oct 25, 2015 11:32 pm
Location: Sweden
Contact:

LuaPreprocess

Post by ReFreezed » Wed Jan 02, 2019 3:28 am

LuaPreprocess is a library that adds a code preprocessing phase before your program runs. It allows you to execute code that then generates the final program.

Example:

Code: Select all

-- input.lua

-- Lines starting with "!" are part of the metaprogram.
!local DEV_MODE = true

-- Here "!()" outputs a value as a literal.
local gameData = !(parseJsonFile("game.json"))

function love.load()
	initGame()

	!if DEV_MODE then
		initLevelEditor()
	!else
		setupTelemetry()
	!end
end

Code: Select all

-- output.lua

-- This table is what parseJsonFile() returned.
local gameData = {classes={"Warrior","Archer","Wizard"},stats={"Health","Mana"}}

function love.load()
	initGame()

	initLevelEditor() -- We got this line because DEV_MODE was true.
end
More examples at GitHub

Usage:

Code: Select all

local pp = require("preprocess")

local info, err = pp.processFile{
	pathIn   = "game.lua2p",    -- This is the file we want to process.
	pathMeta = "game.meta.lua", -- Temporary output file for the metaprogram.
	pathOut  = "game.lua",      -- The output path.
}

if not info then
	error(err)
end

print("Lines of code processed: "..info.lineCount)
One benefit of all this is that certain tasks can be performed only once at build time instead of every time you run the program. For example, you may load and parse a bunch of data files only to finally convert the data to Lua tables or something. With preprocessing you can do that once at build time and enjoy a program that starts up and runs faster in the end.

Another benefit is that you can make choices in your metaprogram about what code the final program should have. For example, during development you may have a bunch of developer-only functionality that should be excluded from a public build. (Compare this to #if/#ifdef directives in C/C++.)

Yet another benefit is that some kinds of code are simply easier to write and look nicer if the code can run in multiple steps. Some boilerplate code can be reduced, for example.

Some key points about LuaPreprocess:
  • It's all Lua - metaprograms are just normal Lua.
  • Simplicity - use "!" to specify what code is part of the metaprogram.
  • Robustness - processed files are parsed properly.
  • Pure Lua - no external dependencies.
  • Can be used as a library or command line program.
LuaPreprocess on GitHub
Direct link to library file

~~~

Why did I do this?

After having published a couple of games on Steam and itch.io I knew that having a "build phase" when making games was very helpful. I could automate tasks and generally reduce a bunch of unpleasant repetitive work. A while ago I started experimenting with taking things a step further. I began to process all Lua files (and eventually images and shaders etc. too) before ever running the game, even locally. This allowed me to easily include or exclude functionality that only certain versions of the game should have/not have. For example, my game may have a developer version, a normal public version and a demo version. All I have to do now is to change a boolean and my build system spits out a different program, or set another boolean and the system spits out a Zip file for every platform and OS, ready to be published. I think any serious LÖVE game could benefit from this kind of workflow.

Also, I'm aware of other preprocessing tools for Lua but none caught my interest. *shrugs*
Last edited by ReFreezed on Thu Jan 03, 2019 8:07 pm, edited 2 times in total.
Momento Temporis: Light from the Deep (metroidvania, wip)
Momento Temporis: Arena (multiplayer pvp arena, wip)
Energize! (puzzle, completed!)
"If each mistake being made is a new one, then progress is being made."

User avatar
yetneverdone
Party member
Posts: 306
Joined: Sat Sep 24, 2016 11:20 am
Contact:

Re: LuaPreprocess

Post by yetneverdone » Thu Jan 03, 2019 6:31 am

Hi, awesome work. How does one use this to preprocess many lua files?

Does it support preprocessing bunch of lua files and then output the those preprocessed files in another folder like in c/c++ turning files into obj files? What will be the command then?

Thanks

User avatar
ReFreezed
Citizen
Posts: 63
Joined: Sun Oct 25, 2015 11:32 pm
Location: Sweden
Contact:

Re: LuaPreprocess

Post by ReFreezed » Thu Jan 03, 2019 4:02 pm

In my case I have a build script that looks something like this:

Code: Select all

local pp = require("preprocess")

-- getFilesInFolderRecursively() may use for example LuaFileSystem.
local pathsRelativeToSrcFolder = getFilesInFolderRecursively("src")

for pathRelativeToSrcFolder in ipairs(pathsRelativeToSrcFolder) do
	local pathIn   = "src/"..pathRelativeToSrcFolder
	local pathMeta = "temp/"..pathRelativeToSrcFolder:gsub("/", ";")
	local pathOut  = "srcGenerated/"..pathRelativeToSrcFolder -- The game require()s files from this folder.

	local dirOut = pathOut:gsub("/[^/]+$", "")
	createDirectory(dirOut)

	pp.processFile{
		pathIn   = pathIn,
		pathMeta = pathMeta,
		pathOut  = pathOut,

		onError = function(err)
			os.exit(1)
		end,
	}
end

-- Start the game.
os.execute("love .")
I'm also using a separate Lua installation to run this, though you could theoretically use LÖVE.

If you use the library as a command line program then the output folder is currently the same folder as where the specified files are (you send it a list of files as arguments). I should maybe add a way to specify an output path for each file if the library is used this way. The library won't be able to create any folders for the output though as that's impossible with just Lua's standard libraries and I don't want to add any external dependencies.
Momento Temporis: Light from the Deep (metroidvania, wip)
Momento Temporis: Arena (multiplayer pvp arena, wip)
Energize! (puzzle, completed!)
"If each mistake being made is a new one, then progress is being made."

User avatar
yetneverdone
Party member
Posts: 306
Joined: Sat Sep 24, 2016 11:20 am
Contact:

Re: LuaPreprocess

Post by yetneverdone » Thu Jan 03, 2019 5:47 pm

Thanks.

Yes, that should be helpful. Making the folder is the user's job.

Looking forward for this

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 5 guests