Timing analysis of many test cases in Python

I am learning about timing analysis in Python. I used time module before it was pointed out that timeit module would be better for timing analysis.

My main focus on timing analysis is for conducting many test cases on 2 variations of the same function. In this example code I used 2 variations of function for reversing a number.

Can the timing analysis be done in a better way? Is the method scalable for many test cases?

def reverse_num(num):
    '''returns the reverse of an integer '''
    if num  0:
        rev *= 10
        rev += num % 10
        num /= 10
    return rev

if __name__ == "__main__":
    from timeit import Timer
    def test(f):
        for i in xrange(1000):
            f(i)

    print Timer(lambda: test(reverse_num)).timeit(number = 100)
    print Timer(lambda: test(reverse_num2)).timeit(number = 100)

I am posting sample test runs for my usage of timeit module for timing for the following code. I wrote it for this question.

```
def mysort(words):
mylist1 = sorted([i for i in words if i[:1] == "s"])
mylist2 = sorted([i for i in words if i[:1] != "s"])
list = mylist1 + mylist2
return list

def mysort3(words):
ans = []
p = ans.append
q = words.remove
words.sort()
for i in words[:]:
if i[0] == 's':
p(i)
q(i)
return ans + words

def mysort4(words):
ans1 = []
ans2 = []
p = ans1.append
q = ans2.append
for i in words:
if i[0] == 's':
p(i)
else:
q(i)
ans1.sort()
ans2.sort()
return ans1 + ans2

def mysort6(words):
return ( sorted([i for i in words if i[:1] == "s"]) +
sorted([i for i in words if i[:1] != "s"])
)

if __name__ == "__main__":
from timeit import Timer
def test(f):
f(['a','b','c','abcd','s','se', 'ee', 'as'])

print Timer(lambda: test(mysort)).timeit(number = 10000)
print Timer(lambda: test(mysort3)).timeit(number = 10000)
print

Software timing benchmarks are notoriously noisy because your computer isn't a very good controlled experimental platform.

• Some algorithms are nondeterministic. For instance, some quicksort implementations choose a random pivot element.

• While you're running your benchmarks, your python process may get preempted by other running applications, operating system threads, etc.

• Even if your process got the same amount of CPU during every run, the timing may be different because a different mix of applications is running so your processor's memory caches have different hit/miss patterns.

• Some runs may get better branch prediction than others (again, due to the mix of other processes running). This will cause more pipeline stalls.

I would suspect that the effects I've given are given in order of decreasing strength (except that I believe Python's sort algorithm is deterministic, so the first effect is not a factor here). I also suspect that I've just scratched the surface.

EDIT:

OK, for reversing a number, suppose that your typical input will be 5 digits then you can use this.

if __name__ == "__main__":
    from timeit import Timer
    import random
    def test(f):
        f(random.randint(10000,99999))

    print Timer(lambda: test(reverse_num)).timeit(number = 10000000)
    print Timer(lambda: test(reverse_num2)).timeit(number = 10000000)

StackExchange Code Review Q#28373, answer score: 6