All:
Here is a follow-up on the "test
assertions guideline" line of work initiated in the Quality SC. As the
TAB figured it, finalizing such a guideline would require more bandwidth than the
TAB has, and would need experts external to the TAB, hence the idea of hosting
that work under a separate TC.
This is the draft of a call
for sponsoring a new TC that would focus on that guideline.
Please check in particular
how appropriate the references to the TAB are.
Jamie:
Among the two options that
were considered:
(1) a long-lasting "Conformance
TC" that would undertake most quality-related works, with possibly
a revolving-door membership,
(2) a well-focused,
short-lived TC,
this is the second option
that is drafted below. ( option (1) is certainly more tricky from an IPR
viewpoint, and would require more consideration.)
Regards,
Jacques
(initial call for co-sponsorship)
We are considering the
creation of an OASIS TC with a charter exclusively focused on producing material
that would promote and facilitate the creation of test assertions for OASIS
specifications. The TC would be named Test Assertions Guideline (TAG) TC.
The general objective served
by this TC is the improvement of the quality of the specifications of other
OASIS TCs, as well as a faster time-to-deployment of these outputs. This is
aligned with the intent of the former OASIS Conformance TC, although the focus
is here on a particular aspect of quality and conformance. Certainly, better
support for writing test assertions is not the only way to achieve this general
objective, but for the sake of scoping the effort in a precise way, it is
preferred that other initiatives serving the same general objective be hosted
by other similar TCs with well focused charters. An outline of the charter will
be communicated separately from this announce, and those interested in
co-sponsoring this effort are invited to help review and finalize this charter.
The rationale in a focus on
a Test Assertion Guideline is:
- General guidelines on
Conformance have already been produced by the former OASIS Conformance TC.
These guidelines focus on the conformance clause, and on the proper way to
write a specification. There is a need for a complementary, detailed document
that guides TC members in enhancing their specification with test assertions so
that test suites and test-beds can be developed as soon as the specification is
finalized. This is on the critical path in shortening the lead time to testing
implementations for conformance and ultimately, to some form of certification
or badging.
- Test Assertions represent
a necessary step from the narrative of a specification to the concrete test
suite that will verify conformance to this specification. Test Assertions
should be usable by a Quality Assurance
team external to the OASIS
TC that produced the specification under test, as the main input for designing
test suites. No one is better qualified than members of a TC to write Test
Assertions relative to their specification.
When designed at the same
time a specification is written, Test Assertions also help the TC in sharpening
the specification requirements and finding missing requirements or error cases.
The need for a common
methodology for associating test assertions with OASIS specifications has been
recognized by the OASIS Technical Advisory Board (TAB). This TC would submit
its deliverables to the TAB, for recommendation to the OASIS Board. The TC
would accept feedback and recommendations from the TAB so that its output can
be considered by the OASIS Board for inclusion in a future revision of the
OASIS TC process, thus addressing the current intent of OASIS Board and Staff
to improve the quality and adoption of OASIS output.
The main deliverable of the
TC will be a Test Assertion Guideline (TAG document), for use by OASIS TCs. The
TAG document would provide the methodology and meta-data for other TCs to
express their test assertions in a consistent way across OASIS.
Optionally, the deliverables
may include a mark-up language for expressing and representing test assertions.
It is also expected that the
test assertion meta-data produced will be suitable for expressing
interoperability test assertions, in addition to conformance test assertions.
The TC will accept as
initial input:
-
the Test Assertion Guideline
working draft .07, (TAB) January 2005.
-
the Conformance Requirements
for Specifications V1.0, (Conformance TC) March 2003.
Also, as the TAG TC will
seek broad adoption of its output, it will consider relevant work done by
bodies external to OASIS, and will consider opportunities for some form of convergence
to the extent that the IP status of these external works allows it:
-
QA Framework: Specification
Guidelines (W3C working draft), November 2004.
http://www.w3.org/TR/qaframe-spec/
-
Test Assertion Documents for
WS-I profiles (WS-I), 2003-2005.
It is in the scope of this
TC work to examine current practices in other OASIS TCs or other organizations which
have already written test assertions for their specifications.
At this time we would like
to know if you are interested in (a) co-sponsoring this initiative (meaning
mostly helping finalize its charter and be involved in the preliminary phases
of TC creation) and participating in it, (b) just participating in it.
We envision that a short
timeframe for producing a committee draft of the TAG deliverable (about 6
months) is realistic, with no need for more than a modest commitment in terms
of resource and time from participants.
The ambition of this TC is
to also get the TAG adopted by other organizations than OASIS, and for this
some liaison with external contributors and groups will be considered, as well
as joint submissions.
Andy Moir