OASIS Open Document Format for Office Applications (OpenDocument) TC

Expand all | Collapse all

Our Position on the Conformance Proposal

  • 1.  Our Position on the Conformance Proposal

    Posted 02-26-2009 15:47
    Hello Michael/Rob and TC Members/Observers...
    
    Several TC members have asked us to clarify our vote against the most recent committee draft (CD).  Let me note at the outset that the sole reason for our "no" vote was the conformance proposal that had not yet been voted on within the TC.  I will therefore describe our general view of conformance, explain our disagreement with the current proposal, and recommend an alternative approach. 
    
    
    Conformance Principles.  We believe the following principles should guide document format standards activities in general, and ODF activities in particular:
    
    - Conformance should be defined in terms of the minimum requirements; it should not be used to establish an "upper limit" for implementations.
    
    - Extensibility mechanisms are useful for all file formats.  Extensibility is central to XML, of course, and it's useful in the context of a file format for all of the same reasons that it's useful in other XML applications and vocabularies.
    
    - The ability to add custom semantics to documents, and in particular custom semantics that are not broadly applicable (and are therefore unlikely to ever be formalized as an international standard), is critical to many document users.  This is especially true for organizations that are automating large-scale workflows and business processes.  Such capabilities should be as simple and straightforward as possible for implementers because they are typically used in rapidly changing competitive environments where needless complexity would slow the pace of innovation.  It should also be possible to validate document content against the standard without a need to understand the custom semantics that may be used in the document.  We believe these types of extension points are a "must-have" feature for modern XML-based document formats because they enforce a standardized approach to custom semantics.
    
    - We have found many interoperability problems between existing ODF implementations in our testing, and for the most part these have not been caused by the use of foreign elements.  Rather, the problems we've found are caused by other issues such as variation in interpretation of the standard, unsupported features in some implementations, product bugs, and other undocumented details of various implementations.  If the goal is to improve real-world interoperability, we feel implementers can make a big difference by documenting these sorts of details, and the removal of foreign element support from ODF won't do anything to address the vast majority of interoperability problems we've encountered.
    
    
    The Current Proposal.  Looking now at the specific conformance clause that appears in the latest CD, the aspect of it that we find most problematic is the distinction between "Conforming OpenDocument Documents" and "Conforming OpenDocument Extended" documents.  This distinction implies that it is not possible to allow for standards-based extensibility, which is not true.  ODF has historically allowed standards-based extensibility (through use of language similar to the current proposal's "extended" class), and so does IS29500 (through the semantics of the customXml element, as well as the Markup Compatibility and Extensibility mechanisms defined in Part 3 of IS29500).
    
    The distinction between "conforming" and "extended" documents would also create unrealistic expectations because of the misleading implication that "conforming" documents are more interoperable than "extended" documents.  As I mentioned above, the vast majority of real-world ODF interoperability problems we've seen have nothing to do with this distinction.
    
    
    Next Steps.  There are two possible solutions that would address our concerns:
    
    1) Eliminate the "conforming" class altogether, and use the existing definition of the "extended" class (for producers and documents) as the one and only conformance class/level.  This would mean that implementers would continue to have the same extensibility options that they have  had in prior versions of ODF.
    
    2) Add some other extensibility mechanism, such as those used in IS29500, which would allow for standards-based validation of document content that may contain custom semantics from non-standardized namespaces.  This would require some engineering work within the TC, and for that reason we are slightly inclined toward option #1 above, but we would be glad to contribute to this work if the TC would like to explore other extensibility mechanisms.
    
    
    One question that has arisen in our discussions to date is "why not use RDF for custom semantics in documents?"  Although it is theoretically possible to use RDF for document semantics, there is a great deal more complexity in that approach than in the simple straightforward use of foreign elements (as in ODF 1.1) or customXml tagging (as in IS29500).  This complexity grows rapidly if there are nested structures, which are typical in these scenarios.  If we are going to require implementers to re-design their extension points, we feel the TC should work to provide a more straightforward mechanism.
    
    
    A final point I'd like to reiterate is that our implementation of ODF 1.1 in Office 2007 SP2 does not use foreign elements at all.  We have not extended the spec in any way.  Others have extended the ODF spec, however, including many smaller implementers who have at most one vote in this process, and in some cases no votes.  We believe it is inappropriate to force all implementers of ODF to submit every potential extension to the TC for consideration in a future version of ODF.  Consider, for example, an implementer who had an idea for an extension in 2006 - they would still be waiting today, in 2009, for the next version of the spec which might contain that extension.  And worse yet, there is little chance that a small implementer could get any extension into a future version of ODF without support from  the larger implementers, as a simple analysis of last week's voting on the CD would show.
    
    
    I hope this helps clarify our voting on the CD last week.  Please let me know if you have any questions, and I look forward to working with the TC on finishing up ODF 1.2.
    
    
    Rgds,
    -Stephen
    
    


  • 2.  Re: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 17:01
    Stephen Peront 


  • 3.  Re: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 17:11

    OF480620EC.0B2488D5-ON85257569.0058F959-85257569.005D68FA@lotus.com" type="cite">
    1) Eliminate the "conforming" class altogether, and use the existing
    definition of the "extended" class (for producers and documents) as 
    the one and only conformance class/level.  This would mean that 
    implementers would continue to have the same extensibility options 
    that they have  had in prior versions of ODF.
    
        
    
    This proposal was already considered. We also considered the exact 
    opposite, having only a single conformance class, which did not allow any 
    extensions at all.  Their was consensus for neither proposal.  The present 
    proposal is a compromise that gives two conformance clauses, one for each 
    use.  This compromise had greater support than either of the single 
    conformance class proposals.
    
    I have a feeling that if we did reopen discussion on this topic, we'd be 
    more likely to end up with a single conformance class that disallowed 
    extensions altogether than a single conformance class that allowed 
    extensions.   To me it appears that allowing an "extended" conformance 
    class was a gracious concession.  "Live and let live".  Your ability to 
    have an "extended" conformance class does not interfere with someone 
    else's desire to have an unextended conformance class, and vice versa. If 
    you upset that compromise, and go for an absolutist "my way or the 
    highway" approach, the results might be even less acceptable to you. 
    That's my personal observation.
    +1

    Jomar Silva


  • 4.  RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 17:35
    Rob, your core point seems to be that allowing two conformance classes is a gracious accommodation of documents that use the extension points already defined in ODF.  And our position is that putting such documents in a separate category is not in the best interest of implementers or document users.  You are trying to make the conformance clause distinguish between two concepts that we see as one and the same: standards-conformant documents that include custom semantics from non-standardized namespaces.  I have many such DOCX documents on my laptop computer right now, and they are not divided into "truly conformant" and "extended."  I think it would be great for ODF to be equally accommodating of this sort of combination of standardization and innovation.
    
    - Doug
    
    


  • 5.  RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 17:57
    Doug Mahugh 


  • 6.  RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 18:14
    > Having two conformance classes increases choice.  It doesn't reduce it.
    
    How does it increase choice?  People already have the ability to choose between extensions or not.  This change simply adds labels to those choices, and those labels imply that there is a qualitative distinction.
    
    I think these labels will be confusing to many people, and will lead to a general sense that the "one true ODF conformance" is the one that doesn't allow for extensions.  And I suspect there will be people actively promoting such an interpretation.  If that's what you're  referring to when you imply there  is "value" in this distinction, then we'll have to agree to disagree on whether that's valuable.  As Stephen said, this distinction does nothing to address most of the actual interoperability challenges that users face today.
    
    By the way, we're not "denying others" anything at all.  We were asked for our position, and we've expressed it.  That's all that's going on here.
    
    Regards,
    Doug
    
    
    


  • 7.  Res: RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 18:43
    Doug,
    
    >How does it increase choice?  People
    >already have the ability to choose
    >between extensions or not.
    
    It's not true (at least in Brazil).
    
    Today, people choose applications and applications developers choose extensions or not. 
    
    Some people choose applications based on their extensions usage or not, and with two conformance classes, applications choices will be transparent to users and maybe we'll reach a day when people will have the ability to choose between extensions or not.
    
    I would like to register that it is very strange to me to see that kind of divergence from you guys.
    
    On the beginning of this discussion, when "no extensions" was on the table you give a lot of reasons that convinced me to accept the idea of two conformance (I was supporting a single "not extended" class). 
    
    Now that you've managed to get the extensions support, you are asking us to forget about "not extended" documents conformance... Is that game ?
    
    Best,
    
    Jomar
    
    PS.: If you said that you'll not use extensions, I really don' t understand why are you guys so worried about the proposal on the table (that allows both kinds of documents ???). 
    
    


  • 8.  RE: RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 22:10
    Hi Jomar...
    
    Doug is stuck in meetings, so I will respond. Doug can follow-up if necessary when he is free again.
    
    > On the beginning of this discussion, when "no extensions" was on the 
    > table you give a lot of reasons that convinced me to accept the idea 
    > of two conformance (I was supporting a single "not extended" class).
    
    Yes, and we do believe that extensions are a critical part of a document standard. We have done our best to be consistent in saying that we believe that there should be one conformance clause that includes an extensibility mechanism.
    
    Best Regards,
    -Stephen
    
    


  • 9.  Re: RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 23:35
    Hi Stephen
    
    One extensibility clause that includes a conformance mechanism :-)
    That's what I really want to avoid.
    
    I believe there is a balance to be struck between predictability and
    space for implementors to innovate.  Perhaps ironically, having a
    tightly constrained document format can provide the best common basis
    to innovate and be creative yet still play in tune with the other band
    members.  There's no great history to draw on in terms of
    standardized, interoperable office document formats.  Lots of not so
    good history.  So whereas I can accept that you do really believe that
    extensions are a critical part of a document standard, I think the
    jury is still out over whether the benefits outweigh the cost.
    
    You say that "extensions are a critical part of the standard", but I
    rather think of them as extensions of the standard.  Which is not
    necessarily bad (he says guardedly).  And if they use a well defined
    extension mechanism then so much the better.  That to my mind is the
    purpose of the extended conformance class.  It also seems clear that
    continued work on such a mechanism, including porting some ideas from
    OOXML, might be valuable but not this late in the cycle of v1.2.  But
    the call for requirements is now open.
    
    Regards
    Bob
    
    2009/2/26 Stephen Peront 


  • 10.  RE: RE: [office] Our Position on the Conformance Proposal

    Posted 02-27-2009 02:57
    Hi Jomar,
    
    I think Stephen answered your question, but I'd add that I never thought there was a compelling reason for two classes of conformance.  I could imagine a person wanting a particular extension, or not wanting it, but I can't imagine why a person would want to forego any and all extensions.  And even if a particular person wanted that (which I accept, since there are clearly person on the TC who seem to want that), I don't think we should add conformance classes to support all such possibilities.  Some users (working with the blind, for example, or working through extremely slow network connections) might prefer documents that don't have images in them, but I don't think we need a no-images conformance class either.
    
    Regards,
    Doug
    
    


  • 11.  Re: RE: [office] Our Position on the Conformance Proposal

    Posted 02-27-2009 16:47
    I think that there is value in knowing that a document contains no
    extensions. In fact, I think producers should aim to use as few
    extensions as possible to do what they want. If documents are produced
    without extensions, it should be easier for consumers to implement the
    functionality needed to process that document.
    
    In conclusion and even though I am not a voting member right now, I
    support the no-extension conformance class. I hope that it will
    represent the way the the vast majority of documents are produced so
    that we will have the greatest variety of consumers. The part I don't
    like of the dual conformance clauses is that it even allows something
    that has extensions to be called ODF. Defining this standard is, for
    me, about interoperability and not about allowing mass divergence from
    the standard through extensions. Otherwise, what point is there in
    having the standard?
    
    Sincerely,
    wt
    
    On Thu, Feb 26, 2009 at 18:56, Doug Mahugh 


  • 12.  RE: RE: [office] Our Position on the Conformance Proposal

    Posted 02-27-2009 21:18
    This is my elaboration on the compromise approach that Michael Brauer just sketched
    (http://lists.oasis-open.org/archives/office/200902/msg00249.html).  While each of us may attach different importance and rationale to the dual-level approach, I continue to support it.
    
     - Dennis
    
    MY THOUGHTS
    
    We need to remind ourselves that the OASIS ODF 1.0, ISO/IEC 26300:2005, and OASIS 1.1 standards all define what is proposed to be the extended document level as the only standard conformant document level.  So the status quo at this moment is that extensions via foreign elements and attributes and by other specific means are all allowed.
    
    I have no idea why that is the precedent, but it is the precedent and it was subject to all of the scrutiny and deliberations that were involved in advancing those versions of the ODF specification to their current status as (international) standards.  
    
    Whether we are satisfied with that or not, we are faced with a legacy situation that was left to us by the "founding fathers" of ODF.  It falls to us to demonstrate care in the transition to what may be a preferable level of conformance for a broad community, perhaps the broadest community, of ODF users and their chosen software products.  However, this is an open standard in the sense that many point to with great pride.  A consequence is that we have no idea what all the ways the standard has been taken up and applied, all around the globe.  There is no requirement for implementers and those who make solutions in their areas of application to consult or advise us about that.  And even if we were to withdraw earlier standards as pre-empted by 1.2, it would take some time for that to ripple through the community.  I believe that, no matter what our own navels reveal about what is safe to break in going to 1.2, that we should err on the side of caution always and make transitions, not eliminations, and from that assess appropriate future steps.
    
    We have no advance assurance that a universal preference, with procurement teeth, for the proposed pure conformance level will indeed take place, and we need to be careful.  I, for one, am personally satisfied that the dual level for conformant documents will allow us to discover what the larger community of stakeholders will find acceptable, giving us useful feedback for further transition beyond 1.2.  (I still think the namings and the potential for confusion are too great, but prefer to work out the conformance language in the remainder of 1.2 before seeing if there is some reconciliation that can be made with that.  I am not interested in debating that until we are able to see how the details in 1.2 are lined up with the conformance levels.)
    
    It is already possible to specify a procurement and deployment policy requiring limitation to the "strict" schema, although that is  not supported by a defined conformance level in the current specifications and it does not directly forbid foreign elements and attributes (which are extra-schema provisions using either schema), absent a policy.  Note that ODF 1.1 Section 1.5 does not mention the strict schema at all and OFD 1.1 section 1.4 only mentions it with regard to the use of arbitrary content within meta information elements and formatting properties elements.  Section 1.4 is very explicit that the schema that is carried in the body of the specification is "the normative XML Schema for the OpenDocument format."
    
    Now we are moving toward introduction of a normative definition of such a pure (extension-free) level, although it will take more than just the conformance section to accomplish that.  (The remaining extension mechanisms that are separate from those involving foreign elements and attributes are yet to be addressed and identified with a conformance level in the specification.)
    
    It has been said that we can, of course, make ODF 1.2 as different from ODF 1.1 as we choose, and we can change the ceiling on conformance any way we please because that won't change the status of ODF 1.1 and earlier documents.  That is technically true.  It is not a practical truth however, because it ignores the social contract, however weakly it is being handled, with regard to upward compatibility, the assurance that ODF documents will be preservable and processable into the indefinite future, and other expectations that are the context in which the work of the ODF TC will be appraised.  I have no idea how that will play out, but I do think it involves greater issues and concerns, by more stakeholders than those of us engaged in this debate.
    
    
    
    


  • 13.  RE: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 19:25
    Doug Mahugh 


  • 14.  Re: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 18:28
    2009/2/26  


  • 15.  Re: [office] Our Position on the Conformance Proposal

    Posted 02-26-2009 19:25
    Dave Pawson