|
|
| Zeile 1: |
Zeile 1: |
| | [[en:NewTestRunner]] | | [[en:NewTestRunner]] |
| - | This is a first draft for an AccUnit TestRunner.
| |
| - |
| |
| - | [[Datei:AccUnit_TestRunner_Draft.jpg|480px]]
| |
| - |
| |
| - |
| |
| - | == Details ==
| |
| - |
| |
| - | (Gathered mainly as results of the discussion between Josef and Paul on Jan 13th, 2012)
| |
| - |
| |
| - | TO BE ORDERED AND REVIEWED
| |
| - |
| |
| - | === Reponsibilites and Input/Output ===
| |
| - |
| |
| - | * '''TestManager'''
| |
| - | ** Responsibility: Orchestrates the parts described below
| |
| - | ** Responsibilites in particular:
| |
| - | *** A2: listening to TestCollector and maintaing the list of test related info in the repository
| |
| - | *** B0: instantiates the TestRunner for each new test run
| |
| - | *** B1: providing a list of tests to be run by the TestRunner
| |
| - | **** some sources: all tests from the repository, selection from a test tree, test according to the current selection of the IDE, a prioritizer component, selection by tags, ...)
| |
| - | *** B3: creating an instance for the current TestRun (actually before starting)
| |
| - | *** B2: starting the TestRunner (as reaction to some user action)
| |
| - | *** B4: listening to the outputs of the TestRunner and maintaining the current TestRun object
| |
| - | *** B5: adding the current TestRun object to the list of test runs in the repository
| |
| - | ** TO BE CLARIFIED: Should this work as a facade?
| |
| - | ** Events:
| |
| - | *** BeforeCollectingTests (maybe better: BeforeScanningProject? - there are also the test-related modules)
| |
| - | *** AfterCollectingTests (maybe better: AfterScanningProject? - reasoning as above)
| |
| - | *** AddedTestRunToRepository(ITestRun)
| |
| - |
| |
| - | * '''TestCollector'''
| |
| - | ** Responsibility: Collects the relevant information for running tests. (rephrased: In order to run a test, no one has to go back to the VBProject to look up something)
| |
| - | ** Input: Events from a component in VbeProjectManagement or just a list of all modules in the VBProject
| |
| - | ** Output: IObservable<ITest>/Events
| |
| - | *** FoundTestMethod(ITest)
| |
| - | *** LostTestMethod(ITest)
| |
| - | *** FoundTestRelatedMethod(IAttributedMethod)
| |
| - | *** LostTestRelatedMethod(IAttributedMethod)
| |
| - | ** maybe better name: ProjectScanner
| |
| - | *** there are not just the test classes itself, but also the test-related modules
| |
| - | ** TestManager listens to the output and maintains the list of all test related info in the repository
| |
| - |
| |
| - | * '''Repository.AllTests'''
| |
| - | ** Responsibility: Represents the current structure of the VBProject with respect to tests
| |
| - | ** Flat list (no fixture-tests hierarchy), hierarchies must be derived by UI components
| |
| - | ** Input: Method calls by the TestManager
| |
| - | ** Output: Enumeration/IObservable<ITest>/Events:
| |
| - | *** AddedTest(ITest)
| |
| - | *** RemovedTest(ITest)
| |
| - | *** Cleared
| |
| - |
| |
| - | * '''Repository.TestRelated''' (NOT IN THE DRAWING)
| |
| - | ** Responsibility: Represents all modules in the VBProject that are just test-related, but not actually test fixtures.
| |
| - | ** global info needed by the TestRunner
| |
| - | ** Input: method calls by the TestManager in reaction to events from TestCollector
| |
| - | ** Output: none
| |
| - | ** The TestManager instantiates the TestRunner for each run with this information
| |
| - |
| |
| - | * '''TestRunner'''
| |
| - | ** Responsibility: Coordinates the calls to the methods of the test suite (tests and tag hook handler)
| |
| - | ** Note: Its the TestExecutor that actually invokes the methods via COM, not the TestRunner!
| |
| - | ** Input:
| |
| - | *** List of tests (subset of test in Repository.AllTests)
| |
| - | *** List of test related modules (Repository.TestRelated)
| |
| - | ** Output: IObservable<ITestResult>/Events
| |
| - | *** StartingTestRun(bool cancel)
| |
| - | *** StartedTestRun(ITestRun)
| |
| - | *** TagSetChanged(IEnumerable<ITagChangeInfo>)
| |
| - | *** FinishedTestRun(ITestRun)
| |
| - | *** ITestRun steadiliy populated during the test run
| |
| - | ** The calling of tag hook handlers (see below) is also managed by the TestRunner
| |
| - |
| |
| - | * '''TestExecutor'''
| |
| - | ** Responsibility: Is able to instantiate classes and invoke methods and understands exceptions from failed assertions
| |
| - | * Input:
| |
| - | ** name of the class to instantiate/terminate
| |
| - | ** name of the method to execute (along with parameters in the case of a call for a row test)
| |
| - | ** Don't use ITest to pass the input parameters.
| |
| - | *** That's too much (attributes not needed generally, rowdata needed without name of the row or ignored state)
| |
| - | *** DIFFERENT FROM THE DRAWING
| |
| - | * Output:
| |
| - | ** True: method call succeeded
| |
| - | ** False: an assertion failed (assertion message is provided to the TestRunner)
| |
| - | ** Non-Assert exceptions are passed to the TestRunner as is
| |
| - | *** Future extension point: special handling for exceptions
| |
| - |
| |
| - |
| |
| - | === Conceptual ===
| |
| - |
| |
| - | * No support of FixtureSetup/FixtureTeardown
| |
| - | ** Seldom used, and when, then often misused (violating independency of tests)
| |
| - | ** More global setup/cleanup (as setting up backend for test) via tag-hooks
| |
| - |
| |
| - | * Tags applied on module level inherit to all children (test methods and rows in row tests)
| |
| - |
| |
| - | * Support for tag hierarchies?
| |
| - | ** path notation: e.g. DatabaseTests/ADO/Oracle
| |
| - |
| |
| - | * No guarantee how often a test class will be instantiated during a test run
| |
| - | ** discourage to use class scope (Initialize/Terminate and Fields)
| |
| - |
| |
| - | * Tag Hooks
| |
| - | ** Every time a tag appears during a test run, a setup tag-hook is called (if available)
| |
| - | ** Every time a tag vanishes during a test run, a teardown tag-hook is called (if available)
| |
| - | ** Tag hook handler
| |
| - | *** reside in class modules annotated with AccUnit:TestRelated
| |
| - | *** are tagged with AccUnit:Setup("MyTag"), and AccUnit:Teardown("MyTag"), resp.
| |
| - |
| |
| - | * Test related modules (AccUnit:TestRelated) can be standard modules or class modules
| |
| - | ** Test related modules with tag hooks (AccUnit:Setup("MyTag"), AccUnit:Teardown("MyTag")) must reside in class modules
| |
| - | *** This helps to avoid global state enduring over the end of a test run (still could be done by copy state to fields in standard module)
| |
| - |
| |
| - |
| |
| - | === Technical ===
| |
| - |
| |
| - | * New assembly AccUnit.Framework
| |
| - | ** No References to COM (SimplyVbUnit/Access/TLI)
| |
| - |
| |
| - | * No IFixture
| |
| - | ** By now, no relevant information besides the name
| |
| - | ** The name of the fixture is a property of ITest
| |
| - |
| |
| - | * AccUnit:Rollback and AccUnit:ClickingMsgBox implemented as kind of plugins to the structure
| |
| - |
| |
| - |
| |
| - | === Data structures ===
| |
| - |
| |
| - | * IAttributedMethod
| |
| - | ** modulename
| |
| - | ** methodname
| |
| - | ** attributes
| |
| - |
| |
| - | * ITest
| |
| - | ** fixturename
| |
| - | ** methodname
| |
| - | ** attributes
| |
| - | ** tags (specialized subset of attributes)
| |
| - | ** rowdata (specialized subset of attributes)
| |
| - |
| |
| - | * ITestRun
| |
| - | ** ProjectInfo (Name of the file, version info?, ...)
| |
| - | ** List<ITestResult>
| |
| - | ** State (prepared, running, finished, canceled, aborted)
| |
| - | ** StartedAt
| |
| - | ** EndedAt
| |
| - | ** Summary
| |
| - |
| |
| - | * ITagChangeInfo
| |
| - | ** Tag (string)
| |
| - | ** Change (appeared/disappeared)
| |
| - |
| |
| - | *ITestResult
| |
| - | ** ITest
| |
| - | ** status (success, failed, error, ignored) ... error = runtime error; failed = error by asserts
| |
| - | ** message (??? rowtest: summary or null ?)
| |
| - | ** rowdata (testresults of rows)
| |
| - | ** StartedAt
| |
| - | ** EndedAt
| |
| - | ** Summary ??? ... summary of row tests?
| |
| - |
| |
| - | === Open issues ===
| |
| - | * Test Reactive Extensions in a VBE AddIn
| |