How to make bitser more robust

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
Post Reply
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

How to make bitser more robust

Post by togFox »

I want to add a 'save game' feature by serialising key tables (for security reasons) and then writing that serialised string to a file. Like many people, bitser is fantastic for this:

https://github.com/gvx/bitser

Unfortunately it throws a runtime error when it serialises something called "userdata" - which turns out to be an image:

Code: Select all

	Lander.imgEngine = love.graphics.newImage("/Assets/engine.png")
	Lander.img = love.graphics.newImage("/Assets/ship.png")
Runtime error:
Error
bitser.lua:280: cannot serialize type userdata
Traceback

[C]: in function 'error'
bitser.lua:280: in function 'serialize_value'
bitser.lua:222: in function <bitser.lua:190>
bitser.lua:281: in function 'serialize_value'
bitser.lua:210: in function <bitser.lua:190>
bitser.lua:281: in function 'serialize_value'
bitser.lua:287: in function 'serialize'
bitser.lua:415: in function 'dumps'
functions.lua:87: in function 'SaveGame'
menus.lua:36: in function 'DrawMainMenu'
main.lua:184: in function 'draw'
[C]: in function 'xpcall'
I looked at bitser.lua. The way the code is written puts it above my pay grade. Can anyone help me make bitser skip over userdata and not include it in the serialise function?

I think the secret is in line 261 ->265 where the value of t is first determined

Code: Select all

	local t = type(value)
	if t ~= 'number' and t ~= 'boolean' and t ~= 'nil' and t ~= 'cdata' then
		seen[value] = seen[SEEN_LEN]
		seen[SEEN_LEN] = seen[SEEN_LEN] + 1
	end
And, yes, not including the image in the serialise and dump means I'll have to do funky stuff on the load to compensate for the missing table data.

Thanks.
Last edited by togFox on Mon Oct 18, 2021 7:50 am, edited 1 time in total.
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: How to make bitser more robust

Post by MrFariator »

I guess my first recommendation, before going and trying to modify bitser, would be to just not save any userdata in the table you want to serialize. I find it's better to keep track of any data that may be saved separate from assets (audio, images, etc). You really don't want to serialize assets into player's save data, as it'd be redundant, waste space, and make serialization/deserialization slower. Of course, this assumes you don't do any runtime-time modification on those assets (eq. use Image:replacePixels() or similar).

For the simplest case, where you don't want to save any userdata, you could just write a serializer that basically dummies it out (untested code):

Code: Select all

local function write_userdata(value,seen)
	-- treat the userdata as though it's a nil value
	-- or something similar
	Buffer_write_byte(247) -- '247' is just magic number for 'nil'
end

-- add the userdata function to the 'types' table
local types = {number = write_number, string = write_string, table = write_table, boolean = write_boolean, ["nil"] = write_nil, cdata = write_cdata, userdata = write_userdata}
And then when you're deserializing the data, you just load those images into the correct places in the table after the fact.
User avatar
pgimeno
Party member
Posts: 3541
Joined: Sun Oct 18, 2015 2:58 pm

Re: How to make bitser more robust

Post by pgimeno »

MrFariator is pretty much spot-on. I'd like to add that since a userdata type is opaque to Lua (it can come from a number of sources, usually from Löve objects but also others like files), Lua doesn't know what to do with it. It might be possible to write a Löve-specific serializer that takes into account the Löve object format, but even then, if it finds for example an image type, it can't save that image because starting with 11.0, images don't support getting the image data.

It's also a bad idea generally. Let's imagine that it's not be a problem to save a table containing certain userdata objects. If you store a reference to an object you already have elsewhere, on deserialization the object would be recreated and you would have a duplicate. If it's a love.physics body, it would possibly duplicate objects that are already in the world, and so on.

Back to the real world where it's not possible, note that on deserialization, you need to know where the userdata was when the object was serialized, in order to set the userdata fields appropriately. Therefore, it's possible on serialization to do the inverse process and remove these fields.

Bitser is robust. You are asking it to do something impossible and it gives you a proper error that helps you diagnose the problem. It would not be robust if it saved something else instead of giving an error, for example. Implying that it's not robust is like saying that love.graphics.draw is not robust because it gives an error when you tell it to draw a boolean value.
User avatar
zorg
Party member
Posts: 3435
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: How to make bitser more robust

Post by zorg »

While it is true that you can no longer create a new ImageData from an Image, you can still do it through Canvases in 11.x with Canvas:newImageData... and hopefully, if 12.0 will merge Image and Canvas together into Texture, it'll have a similar function as well, since not all options are covered with assuming one already has an ImageData to begin with...

That said, i'd personally not try to serialize images since in most cases, the data has already come from disk anyway, so i'd rather serialize either the string path to them, or better yet, an index (number), if multiple things can use the same image, and then the list of indexes mapping to specific paths; takes way less space as well.
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.
User avatar
Xii
Party member
Posts: 137
Joined: Thu Aug 13, 2020 9:09 pm
Contact:

Re: How to make bitser more robust

Post by Xii »

Use Blob instead. It recently got userdata support when I asked for it!
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: How to make bitser more robust

Post by MrFariator »

Judging by that thread, what got added was not userdata support, but cdata support. These are two very different things. I would check the github page for blob in case there was something I'm missing, but seems like it has been taken down.

Bitser already supports cdata, but you may have to extend it to implement arbitrary cdata structs, particularly if you're using luajit's FFI. There are perfectly good reasons for wanting to serialize cdata (I use cdata for level data in my game, for instance), but again, userdata is another matter entirely, as mentioned by pgimeno and zorg in posts above.
User avatar
Xii
Party member
Posts: 137
Joined: Thu Aug 13, 2020 9:09 pm
Contact:

Re: How to make bitser more robust

Post by Xii »

Oh, sorry, I got userdata cdata mixed up.
Post Reply

Who is online

Users browsing this forum: No registered users and 13 guests