Blob.lua - binary serialization library

Showcase your libraries, tools and other projects that help your fellow love users.
MissDanish
Citizen
Posts: 65
Joined: Wed Mar 07, 2018 11:21 pm

Re: Blob.lua - binary serialization library

Post by MissDanish »

grump wrote: Wed Nov 28, 2018 8:02 am ...
The save code is fine as is and should work as expected.
Thank you so much :)
User avatar
pgimeno
Party member
Posts: 3541
Joined: Sun Oct 18, 2015 2:58 pm

Re: Blob.lua - binary serialization library

Post by pgimeno »

MissDanish wrote: Wed Nov 28, 2018 6:12 am

Code: Select all

function saves_load()
  if love.filesystem.getInfo(saveListPath) ~= nil then
    saves = BlobReader(love.filesystem.read(saveListPath))
  end
end
grump wrote: Wed Nov 28, 2018 8:02 am love.filesystem.read returns data and size; you can't pass the call result directly to BlobReader(), because it expects the optional second parameter to be byte order, not size.
If that's the issue, then just adding parentheses suffices:

Code: Select all

function saves_load()
  if love.filesystem.getInfo(saveListPath) ~= nil then
    saves = BlobReader((love.filesystem.read(saveListPath)))
  end
end
grump
Party member
Posts: 947
Joined: Sat Jul 22, 2017 7:43 pm

Re: Blob.lua - binary serialization library

Post by grump »

pgimeno wrote: Wed Nov 28, 2018 11:32 am If that's the issue, then just adding parentheses suffices:
Right, thanks.

I've changed the constructor and made the second parameter a variant type. It now accepts the love.filesystem.read() return values without the parentheses.

Code: Select all

local BlobReader = require('BlobReader')
local r = BlobReader(love.filesystem.read('main.lua'))
print(r:raw(r:size()))
User avatar
yetneverdone
Party member
Posts: 446
Joined: Sat Sep 24, 2016 11:20 am
Contact:

Re: Blob.lua - binary serialization library

Post by yetneverdone »

Hi, it says support for c data type. Would this be compatible for https://github.com/novemberisms/brinevector
?
grump
Party member
Posts: 947
Joined: Sat Jul 22, 2017 7:43 pm

Re: Blob.lua - binary serialization library

Post by grump »

yetneverdone wrote: Fri Jan 25, 2019 1:50 am Hi, it says support for c data type. Would this be compatible for https://github.com/novemberisms/brinevector
?
Not fully.

This works:

Code: Select all

local v = Vector(23, 42)

-- serialize 
local w = BlobWriter()
w:number(v.x):number(v.y)

-- deserialize
local r = BlobReader(w:tostring())
v = Vector(r:number(), r:number())
Serializing tables that contain Vectors is not supported. (I'm thinking about adding support for this)

Code: Select all

local t = { Vector(23, 42) }
BlobWriter():write(t) -- error, can't serialize cdata in tables
Serializing them directly with BlobWriter:table works, but that has drawbacks. BlobReader can't automatically recreate the Vector object.

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:table(v) -- works
When you read this back, it becomes a plain table:

Code: Select all

local r = BlobReader(w:tostring())
local v = r:table()
print(v.x, v.y) -- works
print(v.length) -- does not work, v is a plain table, not a Vector

v = Vector(v.x, v.y) -- now v is a Vector again
print(v.length) -- works
EDIT: this will also work:

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:raw(v, ffi.sizeof(v))

local r = BlobReader(w:tostring())
local v2 = ffi.cast('brinevector*', r:raw(ffi.sizeof('brinevector')))
print(v2.x, v2.y, v2.length)
I will add some code to simplify this a bit.
User avatar
yetneverdone
Party member
Posts: 446
Joined: Sat Sep 24, 2016 11:20 am
Contact:

Re: Blob.lua - binary serialization library

Post by yetneverdone »

grump wrote: Fri Jan 25, 2019 8:50 am
yetneverdone wrote: Fri Jan 25, 2019 1:50 am Hi, it says support for c data type. Would this be compatible for https://github.com/novemberisms/brinevector
?
Not fully.

This works:

Code: Select all

local v = Vector(23, 42)

-- serialize 
local w = BlobWriter()
w:number(v.x):number(v.y)

-- deserialize
local r = BlobReader(w:tostring())
v = Vector(r:number(), r:number())
Serializing tables that contain Vectors is not supported. (I'm thinking about adding support for this)

Code: Select all

local t = { Vector(23, 42) }
BlobWriter():write(t) -- error, can't serialize cdata in tables
Serializing them directly with BlobWriter:table works, but that has drawbacks. BlobReader can't automatically recreate the Vector object.

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:table(v) -- works
When you read this back, it becomes a plain table:

Code: Select all

local r = BlobReader(w:tostring())
local v = r:table()
print(v.x, v.y) -- works
print(v.length) -- does not work, v is a plain table, not a Vector

v = Vector(v.x, v.y) -- now v is a Vector again
print(v.length) -- works
EDIT: this will also work:

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:raw(v, ffi.sizeof(v))

local r = BlobReader(w:tostring())
local v2 = ffi.cast('brinevector*', r:raw(ffi.sizeof('brinevector')))
print(v2.x, v2.y, v2.length)
I will add some code to simplify this a bit.
Thanks! Yeah, vector support is really needed since serialization is used for saving objects.
grump
Party member
Posts: 947
Joined: Sat Jul 22, 2017 7:43 pm

Re: Blob.lua - binary serialization library

Post by grump »

yetneverdone wrote: Wed Jan 30, 2019 4:59 am Thanks! Yeah, vector support is really needed since serialization is used for saving objects.
You can implement object saving without explicit ctype support if you use one of the methods I have shown in the previous comment.

Doing it automatically has proven somewhat difficult, so I'd need to write code to support custom types. Adding ctype (and class) support is on my list of TODOs, but the priority of the project is low, it may take some time.
User avatar
Xii
Party member
Posts: 137
Joined: Thu Aug 13, 2020 9:09 pm
Contact:

Re: Blob.lua - binary serialization library

Post by Xii »

Still no support for cdata.
grump
Party member
Posts: 947
Joined: Sat Jul 22, 2017 7:43 pm

Re: Blob.lua - binary serialization library

Post by grump »

Xii wrote: Fri Feb 12, 2021 2:31 pm Still no support for cdata.
cdata is supported. :raw can read and write cdata just fine.

All cdata has the same type ("cdata"). There is no ffi API that provides more detailled type information, which makes fully transparent support of cdata somewhat difficult to implement. Even if type information was available, serialization would break with opaque pointers, because in C there is no information about the size of data a pointer is pointing to.
User avatar
Xii
Party member
Posts: 137
Joined: Thu Aug 13, 2020 9:09 pm
Contact:

Re: Blob.lua - binary serialization library

Post by Xii »

grump wrote: Fri Feb 12, 2021 3:47 pm cdata is supported. :raw can read and write cdata just fine.
Yeah but that means I have to manually iterate over my tables doing hacky things instead of just this as per the example:

Code: Select all

local blob = BlobWriter()
blob:write(data)
grump wrote: Fri Feb 12, 2021 3:47 pm All cdata has the same type ("cdata"). There is no ffi API that provides more detailled type information, which makes fully transparent support of cdata somewhat difficult to implement.
Can I pass the ctype information to the library somehow? I have the string definition, after all.
Post Reply

Who is online

Users browsing this forum: No registered users and 16 guests