NewTestRunner
Ansichten
Aus AccUnit
(Unterschied zwischen Versionen)
(→Data structures) |
|||
| Zeile 150: | Zeile 150: | ||
** Change (appeared/disappeared) | ** Change (appeared/disappeared) | ||
| - | + | *ITestResult | |
| + | ** ITest | ||
| + | ** status (success, failed, error, ignored) ... error = runtime error; failed = error by asserts | ||
| + | ** message (??? rowtest: summary or null ?) | ||
| + | ** rowdata (testresults of rows) | ||
| + | ** StartedAt | ||
| + | ** EndedAt | ||
| + | ** Summary ??? ... summary of row tests? | ||
=== Open issues === | === Open issues === | ||
* Test Reactive Extensions in a VBE AddIn | * Test Reactive Extensions in a VBE AddIn | ||
Version vom 17:48, 14. Jan. 2012
This is a first draft for an AccUnit TestRunner.
Inhaltsverzeichnis |
Details
(Gathered mainly as results of the discussion between Josef and Paul on Jan 13th, 2012)
TO BE ORDERED AND REVIEWED
Reponsibilites and Input/Output
- TestManager
- Responsibility: Orchestrates the parts described below
- Responsibilites in particular:
- A2: listening to TestCollector and maintaing the list of test related info in the repository
- B0: instantiates the TestRunner for each new test run
- B1: providing a list of tests to be run by the TestRunner
- some sources: all tests from the repository, selection from a test tree, test according to the current selection of the IDE, a prioritizer component, selection by tags, ...)
- B3: creating an instance for the current TestRun (actually before starting)
- B2: starting the TestRunner (as reaction to some user action)
- B4: listening to the outputs of the TestRunner and maintaining the current TestRun object
- B5: adding the current TestRun object to the list of test runs in the repository
- TO BE CLARIFIED: Should this work as a facade?
- Events:
- BeforeCollectingTests (maybe better: BeforeScanningProject? - there are also the test-related modules)
- AfterCollectingTests (maybe better: AfterScanningProject? - reasoning as above)
- AddedTestRunToRepository(ITestRun)
- TestCollector
- Responsibility: Collects the relevant information for running tests. (rephrased: In order to run a test, no one has to go back to the VBProject to look up something)
- Input: Events from a component in VbeProjectManagement or just a list of all modules in the VBProject
- Output: IObservable<ITest>/Events
- FoundTestMethod(ITest)
- LostTestMethod(ITest)
- FoundTestRelatedMethod(IAttributedMethod)
- LostTestRelatedMethod(IAttributedMethod)
- maybe better name: ProjectScanner
- there are not just the test classes itself, but also the test-related modules
- TestManager listens to the output and maintains the list of all test related info in the repository
- Repository.AllTests
- Responsibility: Represents the current structure of the VBProject with respect to tests
- Flat list (no fixture-tests hierarchy), hierarchies must be derived by UI components
- Input: Method calls by the TestManager
- Output: Enumeration/IObservable<ITest>/Events:
- AddedTest(ITest)
- RemovedTest(ITest)
- Cleared
- Repository.TestRelated (NOT IN THE DRAWING)
- Responsibility: Represents all modules in the VBProject that are just test-related, but not actually test fixtures.
- global info needed by the TestRunner
- Input: method calls by the TestManager in reaction to events from TestCollector
- Output: none
- The TestManager instantiates the TestRunner for each run with this information
- TestRunner
- Responsibility: Coordinates the calls to the methods of the test suite (tests and tag hook handler)
- Note: Its the TestExecutor that actually invokes the methods via COM, not the TestRunner!
- Input:
- List of tests (subset of test in Repository.AllTests)
- List of test related modules (Repository.TestRelated)
- Output: IObservable<ITestResult>/Events
- StartingTestRun(bool cancel)
- StartedTestRun(ITestRun)
- TagSetChanged(IEnumerable<ITagChangeInfo>)
- FinishedTestRun(ITestRun)
- ITestRun steadiliy populated during the test run
- The calling of tag hook handlers (see below) is also managed by the TestRunner
- TestExecutor
- Responsibility: Is able to instantiate classes and invoke methods and understands exceptions from failed assertions
- Input:
- name of the class to instantiate/terminate
- name of the method to execute (along with parameters in the case of a call for a row test)
- Don't use ITest to pass the input parameters.
- That's too much (attributes not needed generally, rowdata needed without name of the row or ignored state)
- DIFFERENT FROM THE DRAWING
- Output:
- True: method call succeeded
- False: an assertion failed (assertion message is provided to the TestRunner)
- Non-Assert exceptions are passed to the TestRunner as is
- Future extension point: special handling for exceptions
Conceptual
- No support of FixtureSetup/FixtureTeardown
- Seldom used, and when, then often misused (violating independency of tests)
- More global setup/cleanup (as setting up backend for test) via tag-hooks
- Tags applied on module level inherit to all children (test methods and rows in row tests)
- Support for tag hierarchies?
- path notation: e.g. DatabaseTests/ADO/Oracle
- No guarantee how often a test class will be instantiated during a test run
- discourage to use class scope (Initialize/Terminate and Fields)
- Tag Hooks
- Every time a tag appears during a test run, a setup tag-hook is called (if available)
- Every time a tag vanishes during a test run, a teardown tag-hook is called (if available)
- Tag hook handler
- reside in class modules annotated with AccUnit:TestRelated
- are tagged with AccUnit:Setup("MyTag"), and AccUnit:Teardown("MyTag"), resp.
- Test related modules (AccUnit:TestRelated) can be standard modules or class modules
- Test related modules with tag hooks (AccUnit:Setup("MyTag"), AccUnit:Teardown("MyTag")) must reside in class modules
- This helps to avoid global state enduring over the end of a test run (still could be done by copy state to fields in standard module)
- Test related modules with tag hooks (AccUnit:Setup("MyTag"), AccUnit:Teardown("MyTag")) must reside in class modules
Technical
- New assembly AccUnit.Framework
- No References to COM (SimplyVbUnit/Access/TLI)
- No IFixture
- By now, no relevant information besides the name
- The name of the fixture is a property of ITest
- AccUnit:Rollback and AccUnit:ClickingMsgBox implemented as kind of plugins to the structure
Data structures
- IAttributedMethod
- modulename
- methodname
- attributes
- ITest
- fixturename
- methodname
- attributes
- tags (specialized subset of attributes)
- rowdata (specialized subset of attributes)
- ITestRun
- ProjectInfo (Name of the file, version info?, ...)
- List<ITestResult>
- State (prepared, running, finished, canceled, aborted)
- StartedAt
- EndedAt
- Summary
- ITagChangeInfo
- Tag (string)
- Change (appeared/disappeared)
- ITestResult
- ITest
- status (success, failed, error, ignored) ... error = runtime error; failed = error by asserts
- message (??? rowtest: summary or null ?)
- rowdata (testresults of rows)
- StartedAt
- EndedAt
- Summary ??? ... summary of row tests?
Open issues
- Test Reactive Extensions in a VBE AddIn