Page 1 of 1

Measuring Lua performance

Posted: Fri Sep 09, 2011 5:17 pm
by kikito
I'd like to compare two different implementations of an algorithm and I'm trying to come out with a satisfactory way of timing pieces of Lua code easily.

This is what I've got so far:

Code: Select all

-- time.lua
return function(title, f)

  collectgarbage()

  local startTime = os.clock()

  for i=0,10000 do f() end

  local endTime = os.clock()

  print( title, endTime - startTime )

end
It basically uses os.clock for measuring time in milliseconds, while repeating the same piece of code 10k times. This is how you use it:

Code: Select all

local time = require 'time'

time('first implementation', function()
  -- do stuff here
end)

time('second implementation, function()
  -- do other stuff here
end)
Output:

Code: Select all

first implementation     0.1
second implementation 0.01
I'd like to know your opinion; do you see any obvious problem with this implementation? What would you have done differently?

Thanks!

Re: Measuring Lua performance

Posted: Fri Sep 09, 2011 5:29 pm
by GijsB
I think that that is the best way to do it.

Re: Measuring Lua performance

Posted: Fri Sep 09, 2011 7:37 pm
by Rad3k
One drawback of this approach is that it only tells you how much time passed between starting and finishing a certain task, but this can depend on what the other programs (and OS) are doing (edit: ignore this - not true). On Linux (maybe Mac too) you can use time command. It measures only the time CPU spent executing the command you specified.

Re: Measuring Lua performance

Posted: Fri Sep 09, 2011 10:46 pm
by benloran
Depending on what you're measuring, you may want to consider disabling the garbage collector completely with collectgarbage('stop') at the beginning. Otherwise, that's exactly how I do it.

You could also look into using something like luaProfiler or pepperfish (the latter is much easier to set up).

Re: Measuring Lua performance

Posted: Sat Sep 10, 2011 5:27 am
by Xgoff
kikito wrote:I'd like to compare two different implementations of an algorithm and I'm trying to come out with a satisfactory way of timing pieces of Lua code easily.

This is what I've got so far:

Code: Select all

-- time.lua
return function(title, f)

  collectgarbage()

  local startTime = os.clock()

  for i=0,10000 do f() end

  local endTime = os.clock()

  print( title, endTime - startTime )

end
It basically uses os.clock for measuring time in milliseconds, while repeating the same piece of code 10k times. This is how you use it:

Code: Select all

local time = require 'time'

time('first implementation', function()
  -- do stuff here
end)

time('second implementation, function()
  -- do other stuff here
end)
Output:

Code: Select all

first implementation     0.1
second implementation 0.01
I'd like to know your opinion; do you see any obvious problem with this implementation? What would you have done differently?

Thanks!
it probably wouldn't hurt to subtract from those the runtime of a loop calling a (localized) empty function with the same number of iterations. for some things you might be timing the function call overhead could be fairly significant

Re: Measuring Lua performance

Posted: Sat Sep 10, 2011 1:43 pm
by vrld
Rad3k wrote:One drawback of this approach is that it only tells you how much time passed between starting and finishing a certain task
os.clock() measures CPU time, so it's equivalent to the time command.
For simple profiling this seems to be the best approach. However, I would add a relative comparison of the algorithms.

Re: Measuring Lua performance

Posted: Sat Sep 10, 2011 2:47 pm
by Rad3k
vrld wrote: os.clock() measures CPU time, so it's equivalent to the time command.
I didn't know about that - my bad, I should have checked it myself.

Re: Measuring Lua performance

Posted: Mon Sep 12, 2011 9:35 pm
by kikito
I must have forgotten to press the "Submit" button.

Thanks for all the feedback. I still need to do some more tests, but it's good to know that the path I'm walking looks correct.

Regards!