Thanks for the comments. I have some replies below.
We did receive it and i had a quick read through. Its great to see
work being done on testing. we need lots of it.
Two comments/questions (to prove that I read it!):
1. I am not entirely convinced by your replacement of TSL with Excel.
TSL is a formal grammar. It has limitations which have been
described elsewhere but I don't think that the typing of '[' has ever
really featured highly I think you may be confusing the
specification with the authoring tool when you say that your revised
TSL is to "use excel". So on the theory side I'd be a bit
uncomfortable with this aspect of your paper.
(BTW did you look at schematron as an alternative grammar for
expressing tests? I know one of the criticisms of TSL is that it does
not allow for a very literate human-readable approach. schematron
allows for a human readable assertions and - though xpath is usually
used for schema validation - tests can be implemented in any language.
Perhaps an off-the wall thought ...)
TSL in my paper is about the specification developed by Ostrand 1988.
They developed this specification within the method they called
Category Partition - a combination between Equivalence Partition and
Boundary Analysis. This specification as I feel is a bit complicated
because it required typing [ or ] character. My revision aims to save
time for typing this and also make it readable for a computer program.
I selected Excel because of its popularity and easy-to-use.
CPM is the most popular techniques in black box testing. Cause graph
effect is another method but it requires to draw graphical diagram so
it seems impossible in practice.
XPath or Schematron seems to be out of the scope of CPM.
2. I am not very clear - perhaps didn't read carefully enough - how
you generate the actual tests from the source document. It looks to
me that it requires quite a significant understanding of the
application , the classes etc. I guess this is where understanding
the grammar, if any, is important. Could you generate selenium tests
from it? This strikes me as the most black-box kind of testing.
No, CPM can not generate actual test cases and test data. It can
generate test frame (a combination of different input values). Tester
has to provide the test data, hence, to build test case. I find it
hard to have any automation tool to generate test cases (with test
data) automatically. And my tool just does a simple job: 1) read the
Excel file containing category and partition, 2) combination based on
their properties and constraints, 3) and build the test frames based
on the combination.
How easy would it be to use your tool to do wider test coverage of
dhis? Would be a good thing.
Here we come with the test coverage. Black box is different from white
box. It can not produce any code coverage (statement, block, branch,
predicate etc). Not at all. It can not be used with mutation. It seems
there is no way to assess whether a test set is adequate - good
enough. Any one have better idea can correct me on this.
But I am very confident that if we follow the CPM strictly, together
with the support from my tool, i.e. revise the category and partition
building back and forth, we can end up with a 100% test coverage.
The challenge is, for web layer of DHIS2, we could not apply white box
testing. This is partial reason why there is no unit testing for
action class. It is so difficult to mock all the session and
interceptor at least in DHIS2. We have only one choice, that is black
box testing. And so far, we have not done any systematic testing on
web modules. Our current approach is more ad-hoc.
More input needed.
On Thu, Jun 3, 2010 at 3:40 PM, Bob Jolliffe <firstname.lastname@example.org> wrote:
On 2 June 2010 12:03, Ngoc Thanh Nguyen <email@example.com> wrote:
this is a project I did for the testing course this semester. I hope
it could shed a light for testing aspect of dhis2.
Notice: the case was built based on the assumption that the system
specification requires name of orgunit, group, and group set to be
have between 2-255 characters. Though dhis2 implemented it with text
field max char = 160.
Mailing list: https://launchpad.net/~dhis2-devs
Post to : firstname.lastname@example.org
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp