A hacked-up testing framework

Inspired by this post, I wanted to be able, in any vba project I could be working on, to create a test class and write test methods. Like this:


ThisWorkbook Workbook | class module (client code)

Option Explicit

Public Sub TestAreEqual()
    Assert.AreEqual 12, 34, "Values should be equal."
End Sub

Public Sub TestAreNotEqual()
    Assert.AreNotEqual 12, 34, "Values should not be equal."
End Sub

Public Sub TestAreSame()
    Assert.AreSame New Collection, New Collection, "Objects should be same reference."
End Sub

Public Sub TestAreNotSame()
    Assert.AreNotSame New Collection, New Collection, "Objects should not be the same reference."
End Sub

Public Sub TestFail()
    Assert.Fail "This wasn't meant to be."
End Sub

Public Sub TestInconclusive()
    Assert.Inconclusive "No idea."
End Sub

Public Sub TestIsFalse()
    Assert.IsFalse False, "True should be False."
End Sub

Public Sub TestIsNothing()
    Dim foo As Object
    Assert.IsNothing foo, "Foo should be nothing."
End Sub

Public Sub TestIsNotNothing()
    Dim foo As New Collection
    Assert.IsNotNothing foo, "Foo shouldn't be nothing."
End Sub

Public Sub TestIsTrue()
    Assert.IsTrue True, "False should be True."
End Sub

Public Sub TestBlowUp()
    Debug.Print 1 / 0
    Assert.Fail "Test should have failed by now."
End Sub

Public Sub Test()

    Dim methods As List
    Set methods = List.Create

    methods.Add "TestAreEqual", _
                "TestAreNotEqual", _
                "TestAreSame", _
                "TestAreNotSame", _
                "TestFail", _
                "TestInconclusive", _
                "TestIsFalse", _
                "TestIsNothing", _
                "TestIsNotNothing", _
                "TestIsTrue", _
                "TestBlowUp"

    TestClass.RegisterTestClass Me, methods
    TestClass.RunAllTests

End Sub

Test() method output:

`2014-09-14 23:00:54 TestIsTrue: [PASS]
2014-09-14 23:00:54 TestIsFalse: [PASS]
2014-09-14 23:00:54 TestAreEqual: [FAIL] - Ar

I really like that Test methods are this simple.

Public Sub TestAreEqual()
    Assert.AreEqual 12, 34, "Values should be equal."
End Sub

And message doesn't print if the test passes. This is good, and what people will expect after working with the .Net framework.

What I don't like is the boiler plate code.

Public Sub Test()

   Dim methods As List
    Set methods = List.Create

   methods.Add "TestAreEqual", _
                "TestAreNotEqual", _
                "TestAreSame", _
                "TestAreNotSame", _

         .....    

   TestClass.RegisterTestClass Me, methods
    TestClass.RunAllTests

End Sub

My problem with it isn't that it's boiler plate, my problem is that there's no way to automatically generate it for the person writing the test. Manually writing these registrations would be time consuming and error prone. Neither of which you would want in a Unit Testing frame work.

I really think you need to leverage the VBA Extensibility Library to insert Public Sub Test() into the class. My VBEX project on GitHub has a getProcedures method that would be easy to leverage in a GetTestMethods() function.

Public Function GetTestMethods(CodeMod As CodeModule) As vbeProcedures
    Dim procs As New vbeProcedures
    Dim proc As vbeProcedure

    If mVbeProcedures.Count = 0 Then
        getProcedures
    End If

    For Each proc In mVbeProcedures
        If InStr(0, proc.Lines, "@TestMethod") Then
            procs.Add proc
        End If
    Next proc

    Set GetTestMethods = procs

End Function

Note that I took the approach of tagging the test procedures with '@TestMethod. I suppose you could key on the word Public instead, but this feels safer to me and more in line with the .Net framework. (There could be private code that shouldn't be executed in your TestClass. You wouldn't want to register those.)

Do you see the repetition in the Assertion events? I think you had it right when you suggested a Completed event to me. Instead of passing a message in the assert event, pass a TestResult and let the output decide how to print the result. The client code doesn't ever need to know about the concept of a TestResult class.

Public Sub IsTrue(ByVal condition As Boolean, Optional ByVal message As String)
    dim result as New TestResult

    result.Outcome = IIf(condition, Succeeded, Failed)
    result.Message = message

    RaiseEvent Completed(result)
End Sub

Of course, this means we need to add a message property to TestResult.

Private Type TTestResult
    outcome As TestOutcome
    output As String
    message As String
End Type

Private this As TTestResult

Public Property Get Message() As String
    Message = this.Message
End Property

Friend Property Let Message(ByVal value As String)
    this.message = value
End Property

So now TestClass has only one event, Completed, so there's no longer a reason to make sure no other events have fired. It becomes this easy.

Private Sub assertion_Completed(ByVal result As TestResult)

    testOutput.WriteResult currentTest, result

End Sub

But what about Inconclusive results? As it is, we'll only ever get success or failure. Well, that's easy enough to deal with in your existing error handler by directly passing a result to the output.

CleanFail:
    If Err.Number = AssertError Then
        testOutput.WriteResult TestResult.Create(Failed, Err.Description)
        Resume CleanExit

    ElseIf Err.Number = InconclusiveAssertError Then
        testOutput.WriteResult TestResult.Create(Inconclusive, Err.Description)
        Resume CleanExit

    Else
        testOutput.WriteResult TestResult.Create(Inconclusive, "Test raised an error: " & Err.Description)
        Resume CleanExit
    End If
End Sub

There's one other that took me way too long to notice: the lack of a ITestResult and IAssert for injection purposes. You probably don't need it, but it would make it easier to change and build IOutput classes and TestClasses if you have complete control over their dependencies.

StackExchange Code Review Q#62925, answer score: 3