[10.1, SOLVED] imagedata as string UTF-8 issues

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
Post Reply
User avatar
Helvecta
Party member
Posts: 167
Joined: Wed Sep 26, 2012 6:35 pm

[10.1, SOLVED] imagedata as string UTF-8 issues

Post by Helvecta »

Hello world,

I'm trying to send an image over a network using LUBE. I thought the best way to do it would be to send the imagedata as a string then convert it back on the other end, but I'm having some issues. When I convert an image to imagedata, it's all good, but when I change the imagedata to a string, it's encoded in UTF-8, which I suppose Lua doesn't like. If I try to print the string, I get an error complaining about the encoding being invalid, and if I try to error with the string as an argument (my common way of checking variabe values at a certain point in code), the app crashes.

I'm not very knowledgeable about the subject, but I believe this can be fixed by converting the UTF-8 string to an ASCII string and sending that, but I don't know how to do it. Furthermore, I'm confused about why L2D would convert the imagedata to UTF-8 in the first place, considering that there doesn't appear to be any way to use the returned string without first converting it. I've attached a proof-of-concept .love below; it's pretty straight forward. Check it out and let me know if you have insight into how I can get the returned string into a viewable format.

Side-note: is there any better way to see variable values at a specific time without using error(x) or assert(nil, x)?

Thanks! :o
Attachments
imgDataCrash.love
(171.13 KiB) Downloaded 195 times
Last edited by Helvecta on Mon Mar 07, 2016 12:41 am, edited 1 time in total.
"Bump." -CMFIend420
User avatar
s-ol
Party member
Posts: 1077
Joined: Mon Sep 15, 2014 7:41 pm
Location: Cologne, Germany
Contact:

Re: [10.1] imagedata as string UTF-8 issues

Post by s-ol »

Imagedata data are random bytes and will be neither ASCII nor utf8 data but just binary data. Therefore decoding will fail when assuming it is either. Sendung the uncompressed image (as ImageData) is going to take considerably longer than sending the original, compressed image (the PNG for example, as FileData in LÖVE).

To transport binary data as a string you can encode it with something like base64 (which love.filesystem.newFileData supports) for example.

I'm not sure how you can encode a FileData as a base64 string without an external Lua library but maybe there is some function for that I missed.

s-ol.nu /blog  -  p.s-ol.be /st8.lua  -  g.s-ol.be /gtglg /curcur

Code: Select all

print( type(love) )
if false then
  baby:hurt(me)
end
User avatar
Helvecta
Party member
Posts: 167
Joined: Wed Sep 26, 2012 6:35 pm

Re: [10.1] imagedata as string UTF-8 issues

Post by Helvecta »

Thanks for your advice! I can now send most of the string, I believe. It seems to get truncated during transfer, resulting in a cut-off image, as seen in the two love files below (run the server first, run the client, then press any key on the client; the file will be saved in the L2D save directory). You can also feel free to click here and view a screenshot showing all of it; on the top left is the image I'm trying to transfer, on the top right is what the server receives. Any idea why that's occuring?

Also can you help me understand what the second argument is in love.filesystem.newFileData? What it's set to doesn't seem to impact the application.. :monocle:
Attachments
ImgServer.love
(5.45 KiB) Downloaded 188 times
imgClient.love
(35.02 KiB) Downloaded 191 times
"Bump." -CMFIend420
User avatar
Nixola
Inner party member
Posts: 1949
Joined: Tue Dec 06, 2011 7:11 pm
Location: Italy

Re: [10.1] imagedata as string UTF-8 issues

Post by Nixola »

Can you send embedded zeroes in luasocket/enet? That might be the issue...
lf = love.filesystem
ls = love.sound
la = love.audio
lp = love.physics
lt = love.thread
li = love.image
lg = love.graphics
User avatar
Helvecta
Party member
Posts: 167
Joined: Wed Sep 26, 2012 6:35 pm

Re: [10.1] imagedata as string UTF-8 issues

Post by Helvecta »

It is definitely a buffer issue, since sending smaller images doesn't cause them to get cut off midway. The large image used in my second post is 31387 bytes in size, but something is restricting my packet size to 8192 bytes at a time, meaning that the rest of the information is truncated. I know that I shouldn't change the buffer size and should instead respect the limit, so anything that is larger than 8192 needs to be divided into segments, thrown into a table, packed by lube.bin:pack() and sent to the other side.

I've spent all day working on this idea, and I can successfully send the table to the other side - but LUBE can't unpack it. I believe lube.bin:unpack is trying to utilize string.find, but it doesn't work well with the data in this string, so it returns nil, causing an error. The wiki suggests that using lube.bin:setseperators() may allow me to send nested tables, but there's no mention of unpacking FileData.

EDIT: No, wait, I was trying to send the packed table with > 8192 bytes thinking lube.bin:pack would split it for me; after sending manually piece by piece, I'm getting closer:
Image
"Bump." -CMFIend420
User avatar
HugoBDesigner
Party member
Posts: 403
Joined: Mon Feb 24, 2014 6:54 pm
Location: Above the Pocket Dimension
Contact:

Re: [10.1] imagedata as string UTF-8 issues

Post by HugoBDesigner »

This is probably not the most efficient way of doing this, but I tried this before and it worked: save the image as a .png file, then read it and set a variable as a tostring of that file [ img = tostring(love.filesystem.read("myImage.png") ]

Then I'd send that image over, save it as a new temporary file [ love.filesystem.write("myImage.png", myString) ], then load it as a normal image (and delete it later since it's just temporary).
@HugoBDesigner - Twitter
HugoBDesigner - Blog
User avatar
Helvecta
Party member
Posts: 167
Joined: Wed Sep 26, 2012 6:35 pm

Re: [10.1] imagedata as string UTF-8 issues

Post by Helvecta »

Aha! Got it. Using UDP, splitting the string into portions and sending it piece by piece does work correctly, I just wasn't splitting the string correctly. Thanks for the advice though, Hugo, that'd work if I were sending low amounts of data.
"Bump." -CMFIend420
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 50 guests