## mjpeg player

utunnels
Citizen
Posts: 75
Joined: Fri Jan 06, 2012 5:20 pm

### mjpeg player

github repo: https://github.com/utunnels/mjpeg-playe ... OVE-engine

I made two different versions. One for individual frames (a series of jpg files), one for mjpeg encoded avi file.

Usage:

Code: Select all

require "mplayer"

mplayer = VideoPlayer.create("path_to_your_movie_folder")
mplayer:start()
end

function love.update(dt)
mplayer:play()
end

function love.draw()
mplayer:draw()
end
Check the comments in the lua file for details.

Notice:
Reading from zip (.love) file can be very slow!
Choose "store" in the compressor's compression level option when making the zip file.

Example:

https://www.dropbox.com/s/j2mxmn7pafaqp ... ayerv2.zip

You can also get a multi-threaded version here: http://love2d.org/forums/viewtopic.php? ... 79#p109220

====================================================
You can copy, modify, distribute and perform the work, even
for commercial purposes, all without asking permission.
====================================================
Attachments
mplayerv1.lua
For individual frames
mplayer.lua
For mjpeg avi
Last edited by utunnels on Sun Jul 21, 2013 10:58 am, edited 11 times in total.

Davidobot
Party member
Posts: 1163
Joined: Sat Mar 31, 2012 5:18 am
Location: Cambridge, UK
Contact:

### Re: mjpeg player

I haven't tested this out yet, but if it works, you are a hero!
My projects: Raycaster, love.js v11.3 port
My library: LovelyMoon

T-Bone
Inner party member
Posts: 1492
Joined: Thu Jun 09, 2011 9:03 am

### Re: mjpeg player

I was able to watch a one minute clip from Haruhi with this. Crazy! The audio goes out of sync very fast, though, no matter what bufferSize and moveFPS you pick.

Great job anyway!

Also, as a tip for others wanting to try this out: VLC can convert more or less anything to mjpeg & ogg.

EDIT: Video in 320*180 works very well!

utunnels
Citizen
Posts: 75
Joined: Fri Jan 06, 2012 5:20 pm

### Re: mjpeg player

Yeah, it it a bit tricky.

I don't know why it went out of sync on your side (the clip I tried looks fine, although it is 2:30 in length). I need to do more investigation....

The player calculates frames using audio time, so perhaps it needs some keyfaming mechanics?

Perhaps I can add an option that forces the video length and audio length to match each other, will see later.

T-Bone
Inner party member
Posts: 1492
Joined: Thu Jun 09, 2011 9:03 am

### Re: mjpeg player

utunnels wrote:Yeah, it it a bit tricky.

I don't know why it went out of sync on your side (the clip I tried looks fine, although it is 2:30 in length). I need to do more investigation....

The player calculates frames using audio time, so perhaps it needs some keyfaming mechanics?

Perhaps I can add an option that force the video length and audio length to match each other, will see later.
It seems it got out of sync because it couldn't keep up with the framerate. It's a 30 FPS clip, and it ran in 20 FPS. When I lowered the resolution, it worked just fine.

Davidobot
Party member
Posts: 1163
Joined: Sat Mar 31, 2012 5:18 am
Location: Cambridge, UK
Contact:

### Re: mjpeg player

utunnels wrote:Yeah, it it a bit tricky.

I don't know why it went out of sync on your side (the clip I tried looks fine, although it is 2:30 in length). I need to do more investigation....

The player calculates frames using audio time, so perhaps it needs some keyfaming mechanics?

Perhaps I can add an option that force the video length and audio length to match each other, will see later.
Do you use dt? It makes games run all at the same pace on all computers.
My projects: Raycaster, love.js v11.3 port
My library: LovelyMoon

T-Bone
Inner party member
Posts: 1492
Joined: Thu Jun 09, 2011 9:03 am

### Re: mjpeg player

Davidobot wrote:
utunnels wrote:Yeah, it it a bit tricky.

I don't know why it went out of sync on your side (the clip I tried looks fine, although it is 2:30 in length). I need to do more investigation....

The player calculates frames using audio time, so perhaps it needs some keyfaming mechanics?

Perhaps I can add an option that force the video length and audio length to match each other, will see later.
Do you use dt? It makes games run all at the same pace on all computers.
It doesn't use dt, but using it doesn't really make sense in this context...

The video and audio are exactly equally long too, so I suspect that using audio time to calculate the frame doesn't work, it seems to just play the video as fast as it can rather than skipping frames to catch up.

utunnels
Citizen
Posts: 75
Joined: Fri Jan 06, 2012 5:20 pm

### Re: mjpeg player

T-Bone wrote:It seems it got out of sync because it couldn't keep up with the framerate. It's a 30 FPS clip, and it ran in 20 FPS. When I lowered the resolution, it worked just fine.

Yeah, that is still strange. Because current frame is calculted in this way:

currentFrame = math.floor(audioTime*movieFps)

It could go out of sync, but should be too much (because it should skip those frames). Also the frame buffer seems a bit pointless, especially for high resolution.

-------------------

Oh snap, I just found the bug:

if not audioSource or self.audioSource:isStopped() then

LOL it should be self.audioSource so the audio time is not used at all.

T-Bone
Inner party member
Posts: 1492
Joined: Thu Jun 09, 2011 9:03 am

### Re: mjpeg player

utunnels wrote:
T-Bone wrote:It seems it got out of sync because it couldn't keep up with the framerate. It's a 30 FPS clip, and it ran in 20 FPS. When I lowered the resolution, it worked just fine.

Yeah, that is still strange. Because current frame is calculted in this way:

currentFrame = math.floor(audioTime*movieFps)

It could go out of sync, but should be too much (because it should skip those frames). Also the frame buffer seems a bit pointless, especially for high resolution.

-------------------

Oh snap, I just found the bug:

if not audioSource or self.audioSource:isStopped() then

LOL it should be self.audioSource so the audio time is not used at all.
Yep, that fixed it! It's actually quite usable now. Of course, CPU usage shoots throught the roof. Multithreading would really help, if it's possible to do. It might not be since love.graphics can't be used in other threads...

EDIT: But love.image can be used, so you can actually decode frames in other threads to ImageData. Then all the main thread has to do is to convert ImageData to Images, which is fairly fast.

utunnels
Citizen
Posts: 75
Joined: Fri Jan 06, 2012 5:20 pm

### Re: mjpeg player

I'm not quite familiar with thread. I thought single threaded forces the decoding process so it should be faster.
I've tried threaded way, the fps jumps to 300 (screen refresh) but the decoder thread is too lazy when the main thread seems to have higher priority.

-------------------

Edit*

OH it seems that works. My CPU usage showed 40 or so before, but now it is over 70%.
Attachments
decoder.lua