Technical Advisory Board (TAB)

 View Only
  • 1.  test assertions follow-up

    Posted 08-03-2005 23:34
     MHonArc v2.5.0b2 -->
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

    tab message

    [Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


    Subject: test assertions follow-up


    All:

    Here is a follow-up on the "test assertions guideline" line of work initiated in the Quality SC. As the TAB figured it, finalizing such a guideline would require more bandwidth than the TAB has, and would need experts external to the TAB, hence the idea of hosting that work under a separate TC.

    This is the draft of a call for sponsoring a new TC that would focus on that guideline.

    Please check in particular how appropriate the references to the TAB are.

    Jamie:

    Among the two options that were considered:

    (1) a long-lasting "Conformance TC" that would undertake most quality-related works, with possibly a revolving-door membership,

    (2) a well-focused, short-lived TC,

    this is the second option that is drafted below. ( option (1) is certainly more tricky from an IPR viewpoint, and would require more consideration.)

    Regards,

    Jacques

    (initial call for co-sponsorship)

    We are considering the creation of an OASIS TC with a charter exclusively focused on producing material that would promote and facilitate the creation of test assertions for OASIS specifications. The TC would be named Test Assertions Guideline (TAG) TC.

    The general objective served by this TC is the improvement of the quality of the specifications of other OASIS TCs, as well as a faster time-to-deployment of these outputs. This is aligned with the intent of the former OASIS Conformance TC, although the focus is here on a particular aspect of quality and conformance. Certainly, better support for writing test assertions is not the only way to achieve this general objective, but for the sake of scoping the effort in a precise way, it is preferred that other initiatives serving the same general objective be hosted by other similar TCs with well focused charters. An outline of the charter will be communicated separately from this announce, and those interested in co-sponsoring this effort are invited to help review and finalize this charter.

    The rationale in a focus on a Test Assertion Guideline is:

    - General guidelines on Conformance have already been produced by the former OASIS Conformance TC. These guidelines focus on the conformance clause, and on the proper way to write a specification. There is a need for a complementary, detailed document that guides TC members in enhancing their specification with test assertions so that test suites and test-beds can be developed as soon as the specification is finalized. This is on the critical path in shortening the lead time to testing implementations for conformance and ultimately, to some form of certification or badging.

    - Test Assertions represent a necessary step from the narrative of a specification to the concrete test suite that will verify conformance to this specification. Test Assertions should be usable by a Quality Assurance

    team external to the OASIS TC that produced the specification under test, as the main input for designing test suites. No one is better qualified than members of a TC to write Test Assertions relative to their specification.

    When designed at the same time a specification is written, Test Assertions also help the TC in sharpening the specification requirements and finding missing requirements or error cases.

    The need for a common methodology for associating test assertions with OASIS specifications has been recognized by the OASIS Technical Advisory Board (TAB). This TC would submit its deliverables to the TAB, for recommendation to the OASIS Board. The TC would accept feedback and recommendations from the TAB so that its output can be considered by the OASIS Board for inclusion in a future revision of the OASIS TC process, thus addressing the current intent of OASIS Board and Staff to improve the quality and adoption of OASIS output.

    The main deliverable of the TC will be a Test Assertion Guideline (TAG document), for use by OASIS TCs. The TAG document would provide the methodology and meta-data for other TCs to express their test assertions in a consistent way across OASIS.

    Optionally, the deliverables may include a mark-up language for expressing and representing test assertions.

    It is also expected that the test assertion meta-data produced will be suitable for expressing interoperability test assertions, in addition to conformance test assertions.

     

    The TC will accept as initial input:

    -        the Test Assertion Guideline working draft .07, (TAB) January 2005.

    -        the Conformance Requirements for Specifications V1.0, (Conformance TC) March 2003.

    Also, as the TAG TC will seek broad adoption of its output, it will consider relevant work done by bodies external to OASIS, and will consider opportunities for some form of convergence to the extent that the IP status of these external works allows it:

    -        QA Framework: Specification Guidelines (W3C working draft), November 2004.

    http://www.w3.org/TR/qaframe-spec/

    -        Test Assertion Documents for WS-I profiles (WS-I), 2003-2005.

    It is in the scope of this TC work to examine current practices in other OASIS TCs or other organizations which have already written test assertions for their specifications.

    At this time we would like to know if you are interested in (a) co-sponsoring this initiative (meaning mostly helping finalize its charter and be involved in the preliminary phases of TC creation) and participating in it, (b) just participating in it.

    We envision that a short timeframe for producing a committee draft of the TAG deliverable (about 6 months) is realistic, with no need for more than a modest commitment in terms of resource and time from participants.

    The ambition of this TC is to also get the TAG adopted by other organizations than OASIS, and for this some liaison with external contributors and groups will be considered, as well as joint submissions.

    Andy Moir



    [Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]