[Solved] Using video as sprite

[qwdqwqwffqw]

Hi. I'm now making character sprites for my isometric action game on LOVE. I've rendered every needed frames out of 3D model on Blender; and usually big sprite sheets of those frames, with quads or SpriteBatch will do the work.

But the thing is, as there are a huge number of frames(for every prime character, many action types, for every 32 facing directions, 30fps, maximum 128x128 resolution each) it is taking up much memory(about 50~100MB per characters). That's why I thought about utilizing video for sprite.

Although encoded video do have significantly less size, this raised some new problems and I need your help and advice.

1. Theora video doesn't have alpha channel for transparent pixel.
>> As there is no semi-transparent pixel, maybe this can be done by filling all transparent pixel on frame with same unused color and applying shader to video, I guess?

2. As far as I know, there is no way to speed up or slow down video in LOVE. So, I considered using video:seek(s) on videos encoded with really low fps(fps=1 or lower). I would be able to set video to the right position for needed frame while video is playing. Whenever time 'ticks' every cycle, I add speed*dt from current position, calculate next needed frame position(rounding off to closest integer) and I can set right seconds for video:seek().

Even if speed change is not a concern, being able to set video on certain precalculated frame position will help a lot in controlling sprite anyway, by freely switching angle and action type of it. That's what I thought.

Main Problem - when video:seek(s) is called, video often shows ugly glitches and sometimes noticeablely delays. The real annoyance is that, this never happens with offsets on specific cycle.
For example when I used video of 30fps in test, setting offset to every multiple of 0.4 seconds didn't cause any problem(no delay or glitch on seek). In 60fps video, every multiple of 0.2 seconds was fine. In 0.3fps video, every multiple of 200(but not 40) didn't have any problems.
The cycles must have something to do with frame rate but I cannot see consistent relation.

What should I do to avoid 'wrong' offset of video? Is this whole using video as a sprite a bad idea? Any advice on using small videos as game resource?

p.s.
When I decoded videos gain, there was no glitch in any frame. So I think video file itself is clean.
I encoded video with ffmpeg(ex. ffmpeg -f image2 -framerate 30.00 -i "a (%d).png" -vcodec libtheora -qscale:v 7 a.ogv).
In my game, it is expected to have about 5 of those chracter sprites on screen at the same time.
test code I used :

Code: Select all

vid = love.graphics.newVideo("a.ogv")
vid:play()
a = 200 -- change this number for test

function love.keypressed(key, scan)
	if key == 'escape' then love.event.quit() end
	if key == 'space' then
		if vid:isPlaying() then vid:pause();
		else vid:play() end
	end
	if key == '1' then vid:seek(1*a) end
	if key == '2' then vid:seek(2*a) end
	if key == '3' then vid:seek(3*a) end
	if key == '4' then vid:seek(4*a) end
	if key == '5' then vid:seek(5*a) end
	if key == '6' then vid:seek(6*a) end
	if key == '7' then vid:seek(7*a) end
	if key == '8' then vid:seek(8*a) end
	-- ...
end

function love.draw()
	love.graphics.draw(vid, 10, 10)
end

[darkfrei]

Use mipmaps for many objects, but hi-res for zoomed in window.

[grump]

As there is no semi-transparent pixel, maybe this can be done by filling all transparent pixel on frame with same unused color and applying shader to video, I guess?

Video compression will lead to color bleeding that will be difficult to mask.

when video:seek(s) is called, video often shows ugly glitches and sometimes noticeablely delays.

This is because compressed video isn't usually stored as full frames. Seeking to a specific frame requires constructing the frame from the previous key frame and the delta frames inbetween. This can be slow and apparently glitchy. You could try to compress the video with only key frames and no delta frames, but this will increase size considerably.

Videos are streamed from storage. If you have many of them playing at the same time, a lot of seeking will take place. This might become a bottleneck, depending on the type of storage.

You might be better off using spritesheets in some compressed texture format and stream those as required. DXT1 offers a 6:1 compression ratio with 1 bit alpha and 15 bits color. You can fit 256 128x128 frames in one 2048x2048 sheet, that's 2 MiB in DXT1 format.

[qwdqwqwffqw]

Those three things you mentioned now make me see that video won't be any better in most aspects compared to conventional sprite sheets. I'll go try using DDS images instead. Thanks for the kindly answer, grump.