How to test static method call - Benchmarks

This is a continuation of my last post.

For a long time I rejected to even test Delegates as a method to make static method calls "virtual" because of performance. My mind doesn't allow me to accept the fact that they can be viable alternative.

But real numbers was a little strange.

First, test console program

class Program : IProgram
    {
        private static Action TestStaticDelegate = TestStatic;
        private static Action TestStaticNoInlineDelegate = TestStaticNoInline;

        static void Main(string[] args)
        {
            TestStatic();
            TestStaticNoInline();
            IProgram p = new Program();
            Action TestInterfaceDelegate = p.TestInterface;
            p.TestInterface();

            int number = 1000000000;
            
            DateTime start = DateTime.Now;
            for (int i = 0; i < number; i++ )
                TestStatic();
            Console.WriteLine("TestStatic: " + (DateTime.Now - start).Ticks);

            start = DateTime.Now;
            for (int i = 0; i < number; i++)
                TestStaticNoInline();
            Console.WriteLine("TestStaticNoInline: " + (DateTime.Now - start).Ticks);

            start = DateTime.Now;
            for (int i = 0; i < number; i++)
                p.TestInterface();
            Console.WriteLine("TestInterface: " + (DateTime.Now - start).Ticks);

            start = DateTime.Now;
            for (int i = 0; i < number; i++)
                TestStaticDelegate();
            Console.WriteLine("TestStaticDelegate: " + (DateTime.Now - start).Ticks);

            start = DateTime.Now;
            for (int i = 0; i < number; i++)
                TestStaticNoInlineDelegate();
            Console.WriteLine("TestStaticNoInlineDelegate: " + (DateTime.Now - start).Ticks);

            start = DateTime.Now;
            for (int i = 0; i < number; i++)
                TestInterfaceDelegate();
            Console.WriteLine("TestInterfaceDelegate: " + (DateTime.Now - start).Ticks);
        }

        private static int x = 0;

        static void TestStatic()
        {
            x++;
        }

        private static int y = 0;
        [MethodImpl(MethodImplOptions.NoInlining)]
        static void TestStaticNoInline()
        {
            y++;
        }

        private int z = 0;
        public void TestInterface()
        {
            z++;
        }        
    }

    public interface IProgram
    {
        void TestInterface();
    }

As you can see I want to compare static method calls with delegates and (the most important) with virtual call.
I've run this program on my AMD Athlon XP 2400+ 2GHz machine (it was Release version of course :-). The results were following:

TestStatic: 18326352
TestStaticNoInline: 39556880
TestInterface: 56280928
TestStaticDelegate: 67697344
TestStaticNoInlineDelegate: 61989136
TestInterfaceDelegate: 56180784

For me the most intresting numbers are TestInterface and TestStaticDelegate, because it compares Dependency Injection using interface with DI using Delegates. As you can see delegates are around 17% slower then interface method call. That was predictable but I've suspected much higher difference. I've run this test few times and the results was always similiar.

According to the power this method gives me, it's acceptable performance penalty, but for some strange reason I've thought that it will be interesting to test it on Intel platform also, so I've run it on my work machine with Intel Pentium 4 3 Ghz. The results were little strange:

TestStatic: 17031359
TestStaticNoInline: 61250392
TestInterface: 55000352
TestStaticDelegate: 51250328
TestStaticNoInlineDelegate: 49844069
TestInterfaceDelegate: 34218969

Woow, it seems that on Intel, Delegates are something around 7% faster then interface call. What's more, delegate instance call is 40% faster then interface one.

I've tried to find some bug in this code, but everything looks, fine for me. Event tried to change (DateTime.Now - start).Ticks to (DateTime.Now - start).ToString() but results were similiar:

TestStatic: 00:00:01.6875108
TestStaticNoInline: 00:00:06.0625388
TestInterface: 00:00:05.5312854
TestStaticDelegate: 00:00:05.0625324
TestStaticNoInlineDelegate: 00:00:05.2187834
TestInterfaceDelegate: 00:00:03.4687722

Seeing this, I saw also that even on AMD, delegate instance call (TestInterfaceDelegate) is faster then normal interface call (TestInterface).

 

My Conclusions

• If you need performance use static methods and TypeMock for testing



• If you need to inject some code at runtime you can freely use Delegates without any significant performance loose. Even better - it looks like .NET 3.5 JIT promotes functional programing (static methods and delegates) by making it more performant than object oriented (interface calls) one - at least on Intel platform.



• If you don't wan't to use TypeMock you can safely use Delegates based Dependency Injection.