Page 2 of 3

Re: Blob.lua - binary serialization library

Posted: Wed Nov 28, 2018 8:40 am
by MissDanish
grump wrote: Wed Nov 28, 2018 8:02 am ...
The save code is fine as is and should work as expected.
Thank you so much :)

Re: Blob.lua - binary serialization library

Posted: Wed Nov 28, 2018 11:32 am
by pgimeno
MissDanish wrote: Wed Nov 28, 2018 6:12 am

Code: Select all

function saves_load()
  if love.filesystem.getInfo(saveListPath) ~= nil then
    saves = BlobReader(love.filesystem.read(saveListPath))
  end
end
grump wrote: Wed Nov 28, 2018 8:02 am love.filesystem.read returns data and size; you can't pass the call result directly to BlobReader(), because it expects the optional second parameter to be byte order, not size.
If that's the issue, then just adding parentheses suffices:

Code: Select all

function saves_load()
  if love.filesystem.getInfo(saveListPath) ~= nil then
    saves = BlobReader((love.filesystem.read(saveListPath)))
  end
end

Re: Blob.lua - binary serialization library

Posted: Wed Nov 28, 2018 1:03 pm
by grump
pgimeno wrote: Wed Nov 28, 2018 11:32 am If that's the issue, then just adding parentheses suffices:
Right, thanks.

I've changed the constructor and made the second parameter a variant type. It now accepts the love.filesystem.read() return values without the parentheses.

Code: Select all

local BlobReader = require('BlobReader')
local r = BlobReader(love.filesystem.read('main.lua'))
print(r:raw(r:size()))

Re: Blob.lua - binary serialization library

Posted: Fri Jan 25, 2019 1:50 am
by yetneverdone
Hi, it says support for c data type. Would this be compatible for https://github.com/novemberisms/brinevector
?

Re: Blob.lua - binary serialization library

Posted: Fri Jan 25, 2019 8:50 am
by grump
yetneverdone wrote: Fri Jan 25, 2019 1:50 am Hi, it says support for c data type. Would this be compatible for https://github.com/novemberisms/brinevector
?
Not fully.

This works:

Code: Select all

local v = Vector(23, 42)

-- serialize 
local w = BlobWriter()
w:number(v.x):number(v.y)

-- deserialize
local r = BlobReader(w:tostring())
v = Vector(r:number(), r:number())
Serializing tables that contain Vectors is not supported. (I'm thinking about adding support for this)

Code: Select all

local t = { Vector(23, 42) }
BlobWriter():write(t) -- error, can't serialize cdata in tables
Serializing them directly with BlobWriter:table works, but that has drawbacks. BlobReader can't automatically recreate the Vector object.

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:table(v) -- works
When you read this back, it becomes a plain table:

Code: Select all

local r = BlobReader(w:tostring())
local v = r:table()
print(v.x, v.y) -- works
print(v.length) -- does not work, v is a plain table, not a Vector

v = Vector(v.x, v.y) -- now v is a Vector again
print(v.length) -- works
EDIT: this will also work:

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:raw(v, ffi.sizeof(v))

local r = BlobReader(w:tostring())
local v2 = ffi.cast('brinevector*', r:raw(ffi.sizeof('brinevector')))
print(v2.x, v2.y, v2.length)
I will add some code to simplify this a bit.

Re: Blob.lua - binary serialization library

Posted: Wed Jan 30, 2019 4:59 am
by yetneverdone
grump wrote: Fri Jan 25, 2019 8:50 am
yetneverdone wrote: Fri Jan 25, 2019 1:50 am Hi, it says support for c data type. Would this be compatible for https://github.com/novemberisms/brinevector
?
Not fully.

This works:

Code: Select all

local v = Vector(23, 42)

-- serialize 
local w = BlobWriter()
w:number(v.x):number(v.y)

-- deserialize
local r = BlobReader(w:tostring())
v = Vector(r:number(), r:number())
Serializing tables that contain Vectors is not supported. (I'm thinking about adding support for this)

Code: Select all

local t = { Vector(23, 42) }
BlobWriter():write(t) -- error, can't serialize cdata in tables
Serializing them directly with BlobWriter:table works, but that has drawbacks. BlobReader can't automatically recreate the Vector object.

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:table(v) -- works
When you read this back, it becomes a plain table:

Code: Select all

local r = BlobReader(w:tostring())
local v = r:table()
print(v.x, v.y) -- works
print(v.length) -- does not work, v is a plain table, not a Vector

v = Vector(v.x, v.y) -- now v is a Vector again
print(v.length) -- works
EDIT: this will also work:

Code: Select all

local v = Vector(23, 42)
local w = BlobWriter()
w:raw(v, ffi.sizeof(v))

local r = BlobReader(w:tostring())
local v2 = ffi.cast('brinevector*', r:raw(ffi.sizeof('brinevector')))
print(v2.x, v2.y, v2.length)
I will add some code to simplify this a bit.
Thanks! Yeah, vector support is really needed since serialization is used for saving objects.

Re: Blob.lua - binary serialization library

Posted: Wed Jan 30, 2019 6:06 am
by grump
yetneverdone wrote: Wed Jan 30, 2019 4:59 am Thanks! Yeah, vector support is really needed since serialization is used for saving objects.
You can implement object saving without explicit ctype support if you use one of the methods I have shown in the previous comment.

Doing it automatically has proven somewhat difficult, so I'd need to write code to support custom types. Adding ctype (and class) support is on my list of TODOs, but the priority of the project is low, it may take some time.

Re: Blob.lua - binary serialization library

Posted: Fri Feb 12, 2021 2:31 pm
by Xii
Still no support for cdata.

Re: Blob.lua - binary serialization library

Posted: Fri Feb 12, 2021 3:47 pm
by grump
Xii wrote: Fri Feb 12, 2021 2:31 pm Still no support for cdata.
cdata is supported. :raw can read and write cdata just fine.

All cdata has the same type ("cdata"). There is no ffi API that provides more detailled type information, which makes fully transparent support of cdata somewhat difficult to implement. Even if type information was available, serialization would break with opaque pointers, because in C there is no information about the size of data a pointer is pointing to.

Re: Blob.lua - binary serialization library

Posted: Fri Feb 12, 2021 4:54 pm
by Xii
grump wrote: Fri Feb 12, 2021 3:47 pm cdata is supported. :raw can read and write cdata just fine.
Yeah but that means I have to manually iterate over my tables doing hacky things instead of just this as per the example:

Code: Select all

local blob = BlobWriter()
blob:write(data)
grump wrote: Fri Feb 12, 2021 3:47 pm All cdata has the same type ("cdata"). There is no ffi API that provides more detailled type information, which makes fully transparent support of cdata somewhat difficult to implement.
Can I pass the ctype information to the library somehow? I have the string definition, after all.