OASIS Open Document Format for Office Applications (OpenDocument) TC

 View Only
Expand all | Collapse all

ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

Rob Weir

Rob Weir01-19-2009 17:17

Patrick Durusau

Patrick Durusau01-19-2009 18:06

Rob Weir

Rob Weir01-19-2009 19:10

  • 1.  ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-19-2009 16:45
    That was a very interesting discussion on conformance with regard to having a single-level conformant document.
    
    I want to reaffirm that, with the single-level approach, we need to deal with all of those places where implementation-defined features are allowed.  (Another example is in the use of alternative renditions and objects where there is currently no specification of MIME types for recognized ones.)
    
    What I think is more interesting is that having only strictly conforming 1.2 documents may have the unintended consequence of prolonging the life of ODF 1.1 processors and ODF 1.1 documents.  The major ODF 1.2 addition, OpenFormula, is perfectly admissible for usage as a 1.1 table:formula value once that part of the ODF 1.2 specification is stable and available as an OASIS Standard.  It hadn't occured to me until now that there might be an useful transition via ODF 1.0/1.1 with extensions from 1.2 (among others) rather than ODF 1.2 wholesale.  And the prospects for successful ingestion of 1.2 and interchange with implementations of 1.2 processors would remain pretty good.  Hmm. ...
    
    My conservatism around preference for the dual levels has to do with not wanting to eliminate something valuable by mistake.  
    
    (And, of course, having a strictly conformant document is no assurance of interoperability, since a conformant processor, as defined so far, need not provide for the semantics of all features and there are many places of underspecification which remain to be dealt with).  I am all in favor of stricter specification, but not so confident about having *only* (strictly-) conforming documents.
    
     - Dennis
    
    PS: I am distressed to hear the casual supposition that the RDF metadata can be used to introduce behaviors into a document and its processing and that it can be used to handle extension cases.  It is called metadata, not processing instructions, for a reason.  It might be able to express implementation assumptions or required capabilities (I need a million rows in this table, sort of thing), but there is nothing in the specification that would have a processor attend to the metadata as instructive and there are no agreements on how such concepts would be expressed, any more than with custom metadata.xml entries.  Using RDF for semantic markup (so-called) is very different than extension via elements and attributes and rules for handling unrecognized ones.
    
    Dennis E. Hamilton
    ------------------
    NuovoDoc: Design for Document System Interoperability 
    mailto:Dennis.Hamilton@acm.org | gsm:+1-206.779.9430 
    http://NuovoDoc.com http://ODMA.info/dev/ http://nfoWorks.org 
    
    


  • 2.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-19-2009 17:17
    "Dennis E. Hamilton" 


  • 3.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-19-2009 18:06


  • 4.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-19-2009 18:32
    Patrick Durusau 


  • 5.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-19-2009 18:58


  • 6.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-20-2009 07:55
    Patrick,
    
    On 01/19/09 19:57, Patrick Durusau wrote:
    > Rob,
    > 
    > robert_weir@us.ibm.com wrote:
    > 


  • 7.  RE: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-19-2009 18:11
    Thanks Rob,
    
    With regard to current allowances of implementation-defined features, the
    choice of schema matters.  The "strict" schema actually removes some areas
    where implementation-defined extensions were explicit in the schema itself.
    (And, for a different example, I assert that it is a feature, not a
    limitation, to have omission of table:protection-key-digest-algorithm mean
    that the hash function that produces the table:protection-key is
    implementation defined [and also upward-compatible from 1.1 as well].)  
    
    If we leave the non-strict schema as the only one, that is a different story
    since it is the schema against which Conforming Document is defined in 1.1.
    Also, the appearance of optional arbitrary metadata elements and
    style:*-properties is explicit in the schema and in the 1.1 conformance
    clause, which goes on to say that a conformant processor should preserve
    such material.
    
    I don't see this language in the current conformance clause.  So that is
    something we need to be clear about with regard to 1.2 and the chosen
    schema.
    
    I think this is like one of those strict-constructionism debates.  I have no
    idea why these provisions and the foreign element provisions are in ODF
    1.0/1.1, nor the problems that were being addressed.  I can see benign and
    valuable uses of them, but that is an academic point.  I just don't like
    fixing something by breaking it.  With regard to interoperability, I don't
    think the issue is even close to hinging on what the spec says a conformant
    document is, strictly or loosely.
    
    I must remember to point out that there is no advice around what it means to
    provide no semantic support for something that is part of a conformant
    document, while there is clear advice around dealing with an unrecognized
    foreign element.  It would be nice to be able to say that if no semantic
    support is provided for a feature of a conformant document, the processor
    should treat it using the rules for foreign elements and  attributes.  This
    also works for unrecognized features using ODF-specified namespaces too.
    (The office:process-content advice could be extended for that as well,
    rather than deprecated.)
    
     - Dennis
    
    


  • 8.  RE: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-19-2009 19:10
    "Dennis E. Hamilton" 


  • 9.  Re: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-19-2009 19:11
    On Monday 19 January 2009, robert_weir@us.ibm.com wrote:
    > Since current implementations do not (to my 
    > knowledge) introduce foreign attributes and elements into documents, they 
    > would not be impacted by this change.
    
    By foreign, do you mean attributes and elements not defined by ODF?
    Then I beg to differ, there are plenty of these, see below.
    
    > Arbitrary extensions are outside of any basis for interoperability, so I'd 
    > argue they should be outside of conformance.
    
    Obviously.
    
    > The point is that the ODF 1.2 metadata was designed to remove the need for 
    > foreign elements and attributes. That original extensibility feature was 
    > not intended for extending the processing model of ODF, though in theory 
    > it could be (mis)used for that purpose.  But in practice, the mechanism 
    > has not been used at all.
    
    I am very surprised by this; either I'm misunderstanding what this is about,
    or the use of extensions isn't known enough by this TC.
    
    Let me point out some examples:
    
    * KWord can save a table style that points to a frame+paragraph style, it does that by using the
    koffice:frame-style-name and koffice:paragraph-style-name properties in the style. This doesn't
    create a huge interop problem since existing documents are rendered fine by other implementations,
    those properties are only used when creating new tables.
    
    * KWord can define the behavior of a frame when a new page is created (for DTP-like usage
    where you might want a similar frame to be created on the next page at the exact same position;
    another one use case is a company logo in a page corner for instance).
    This is done with koffice:frame-behavior-on-new-page in the frame style; and here again it
    only affects further editing, not rendering of a given document.
    
    * OpenOffice has similar things, using the xmlns:ooo="http://openoffice.org/2004/office" namespace for instance.
    or xmlns:ooow="http://openoffice.org/2004/writer" or xmlns:oooc="http://openoffice.org/2004/calc".
    
    Is this what "foreign elements and attributes" was about?
    
    -- 
    David Faure, faure@kde.org, sponsored by Qt Software @ Nokia to work on KDE,
    Konqueror (http://www.konqueror.org), and KOffice (http://www.koffice.org).
    


  • 10.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-19-2009 20:53
    David Faure 


  • 11.  Re: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-20-2009 18:03
    On Monday 19 January 2009, robert_weir@us.ibm.com wrote:
    > David Faure 


  • 12.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-20-2009 18:31
    I have to agree with David here.
    
    +1
    
    
    On 20/01/09 10:02 AM, "David Faure" 


  • 13.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-20-2009 18:52
    I agree with you on almost all of your points, but I'd frame it 
    differently.
    
    First, the use of XML provides extensibility.  This is an undeniable 
    benefit of the technology.  However, this is not the same as saying the 
    ODF standard should define conformance to allow extensions beyond the 
    schema defined by the standard.  These are two different things.  One does 
    not imply the other.
    
    You properly use the example of HTML as an format where useful extensions 
    have been made.  But when you look at the XHTML Recommendation, you'll 
    find that it only defines strict conformance, and in fact states that 
    mixing XHTML with other namespaces is not conformant (section 3.1.2). 
    Certainly the fact that XHTML does not allow extensions in conformant 
    documents has not prevented innovation?
    
    We can't prevent extensions, and we shouldn't try, IMHO,  Extensions, at 
    least where the user has some control over where and when they are used, 
    are powerful tools. But the user does not have control if they think they 
    are using ODF but their word processor is putting undocumented, 
    incompatible and non-interoperable extensions into their documents. 
    
    That said, I'd be less adverse to having both strict and loose conformance 
    classes if we also required that ODF Producers have a mode of operation 
    where they would create only strictly conformant documents.  Then that 
    puts the control back into the user's hands as to what mode they want to 
    operate in.
    
    -Rob
    
    David Faure 


  • 14.  Re: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-20-2009 20:00
    On Tuesday 20 January 2009, robert_weir@us.ibm.com wrote:
    > You properly use the example of HTML as an format where useful extensions 
    > have been made.  But when you look at the XHTML Recommendation, you'll 
    > find that it only defines strict conformance, and in fact states that 
    > mixing XHTML with other namespaces is not conformant (section 3.1.2). 
    > Certainly the fact that XHTML does not allow extensions in conformant 
    > documents has not prevented innovation?
    
    I chose a rather poor example. Things are quite different with HTML and browsers, in fact,
    because the most common case is that the person or application creating the document 
    is completely separate from the application rendering the document, so indeed there is
    a need for strict compliance and only writing out things that independent renderers
    (the web browsers) will be able to render.
    And, HTML documents are mostly readonly, there are typically no editing-related settings in them.
    
    On the other hand, with OpenDocument, the most common case is that the
    application creating the document will be the one editing it again, with of course
    the possibility to view and edit the document in other applications (interoperability),
    but there is a real use case for application-specific settings, *especially* those
    that affect editing rather then rendering, like those I presented. This is more
    related to a nice user experience with the application than it is related to actual
    interoperability -- the equivalent features might be very different in the GUI anyway.
    Because of this need for editing-related settings in OpenDocument files, I see
    no reason for forbid application-specific extensions.
    
    Yes, if people store additional *content* in extensions then that content won't appear
    in other applications, this is obvious, and the people doing that will break interoperability,
    and shouldn't do it. But extrapolating from there to "all extensions are forbidden"
    breaks a valid for use case for extensions: editing-related settings
    (and really-application-specific settings).
    
    > We can't prevent extensions, and we shouldn't try, IMHO,  Extensions, at 
    > least where the user has some control over where and when they are used, 
    > are powerful tools. But the user does not have control if they think they 
    > are using ODF but their word processor is putting undocumented, 
    > incompatible and non-interoperable extensions into their documents. 
    
    The user doesn't have to care, as long as the extensions don't break interoperability.
    If they do, then indeed it's bad and affects the user.
    
    > That said, I'd be less adverse to having both strict and loose conformance 
    > classes if we also required that ODF Producers have a mode of operation 
    > where they would create only strictly conformant documents.  Then that 
    > puts the control back into the user's hands as to what mode they want to 
    > operate in.
    
    This doesn't make sense to me. "Save as ODF without koffice:foo attributes"
    would only give you the exact same document, with less features if you reopen
    it in koffice, and with the exact same features if you reopen it in other ODF apps...
    
    I know that it's not koffice you're afraid of here, but certain huge companies
    who are known "embrace and extend" and not care much for writing out
    files with "undocumented, incompatible and non-interoperable extensions". I know.
    But whether an extension is good or bad, is not something that a simple
    validation against an XML schema can tell us. It's something we'll have to
    discuss between humans.
    
    -- 
    David Faure, faure@kde.org, sponsored by Qt Software @ Nokia to work on KDE,
    Konqueror (http://www.konqueror.org), and KOffice (http://www.koffice.org).
    


  • 15.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-20-2009 20:42
    David Faure 


  • 16.  Re: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-21-2009 01:05
    On Tuesday 20 January 2009, robert_weir@us.ibm.com wrote:
    > But holding back the label of "conformant" is the primary 
    > way to bring vendors to the table to propose and document such features. 
    > If we just label everything conformant, than why would a vendor trouble 
    > with the time and effort to get their proposals accepted formally into the 
    > standard?  Why buy the cow when you can get the milk for free?
    
    I understand that. But on the other hand, why should an implementation
    be marked as "non-conformant" just because it adds one tiny attribute
    in a style somewhere for mostly internal reasons?
    
    > But your point on editing hints versus content extensions is well taken. 
    > Maybe there is some way we can formulate a conformance clause that takes 
    > account of that.  But I'd rather have an extension framework that handles 
    > things like that in a structured way than to allow any XML anywhere.
    
    Right, in fact I think that allowing extensions in style properties is a given,
    while indeed allowing any element in the middle of content elements
    would just open the door to non-standard content. I think your ideas work
    well together: by only allowing extensions inside style properties, only
    additional settings to _existing_ content elements can be added, no actual
    new content.
    Of course styles can affect rendering as well as editing, but that's the
    impossible-to-draw line.
    
    -- 
    David Faure, faure@kde.org, sponsored by Qt Software @ Nokia to work on KDE,
    Konqueror (http://www.konqueror.org), and KOffice (http://www.koffice.org).
    


  • 17.  RE: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-21-2009 05:40
    There is currently a standard (ODF 1.0/1.1) foreign-element-and-attribute
    treatment that recommends how a compliant processor could reduce what it
    processes to be equivalent to a conformant document (assuming there is one
    underneath there) by ignoring the element tags and dealing with the content,
    or by ignoring the element completely if there is an attribute that says to
    do so (or to not do so).  Those specifications also provide for processing
    elements.  The current proposal does not mention processing elements.
    
    In the current draft for 1.2 it is a little more complicated with the
    current recommended default behavior depending on whether the element is in
    text or not.  (It is not clear why that change is necessary.)
    
    The point is that someone who introduces a foreign element knowing this rule
    can be careful to ensure that the foreign-element treatment is benign.
    
    This actually comes up with regard to down-level processing of ODF 1.2
    documents by ODF 1.1 documents.  If the ODF 1.2 document uses the new
    


  • 18.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-21-2009 17:27
    I think that this does not recognize how XML is processed in general.  There are two parts - the parse and the call back actions.
    
    When XML is parsed, all that is check is that it is well formed XML (or valid in case of schema or DTD).  XML is ingested as a one dimensional stream of bytes.  Xerces and other parsers commonly use 4 standard handlers for this - error handler, DTD/Schema handler, Entity resolver, and content handler.   The XML either passes or it doesn't.  If your schema allows extensibility, then such should pass the parse.
    
    The second part of the equation is what the content handler does with the parsed XML.  In most cases, it takes the callback events and builds a DOM, the DOM then get turned into an ODF document.  If the callback handler passes something into the DOM that is not required (mandatory) to build the document and is not needed by the ODF compliant application, it should, by default, just sit there harmlessly in memory doing nothing. Like HTML , un needed elements should not harm compliant applications UNLESS there is a construct that disrupts the deterministic access of a needed element's value.
    
    Looking at the schema, as long as extensibility is not allowed in the middle of paths (example: root_element.child1.


  • 19.  RE: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-21-2009 19:05
    Duane,
    
    There are many XML-level processing models that might apply, including ad
    hoc ones that Michael suggests may be common for ODF implementations (for
    performance reasons, I presume).  For the level at which we are talking in
    regard to conformance and foreign elements and attributes, it is about what
    might be accepted as additional markup not accounted for in the strict
    schema, but related to the strict schema in a particular way.  We are
    talking about what the behavior may be, not how it is achieved.
    
    Since the provision has been in since ODF 1.0, anyone who has foreign
    elements and attributes or who encounters them without rejecting the
    document has made some sort of provision.  My experience is that these
    extensions (and those of unimplemented ODF features) tend to simply be
    ignored and, if the document is touched in any way, they disappear from the
    saved modification.  See our investigation of style:wrap-dynamic-threshold
    http://lists.oasis-open.org/archives/office/200809/msg00109.html
    
    
    Duane's comment about buzzword is interesting though.  
    
    If the XMP is in a separate item of the package, is it accounted for in
    manifest.xml?  There are security reasons why a processor that does not
    recognize that item may well strip it to avoid perpetuating unknown content
    in a document.  There are also matters of prudence in the case of documents
    having some significance as records and Easter eggs and other goodies are
    frowned upon.  I also believe we do not have a conformance requirement on
    what is in a package beyond what is essential to a given document structure,
    but I must go look.  (I will be doing that over on OIC TC anyhow.)
    
    If the XMP is injected into one of the ODF XML documents, which one?  Is it
    made an element or elements in metadata.xml or is it placed in or near the
    document root element of content.xml?  This matters for 1.0/1.1 because
    there are different provisions for metadata than for content.  For ODF 1.2,
    it's not clear to me where the TC is leaning on that one and you might want
    to express your preference in the matter.
    
     - Dennis  
    
    PS: It might be good to publish some sort of profile on your incorporation
    of XMP into ODF documents so that others could decide to recognize the
    material and even update it properly, yes?  Is this on the Adobe site
    somewhere?
    
    


  • 20.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-21-2009 19:33
    Inline:
    
    
    On 21/01/09 11:05 AM, "Dennis E. Hamilton" 


  • 21.  Re: [office] ODF 1.2 Single-Level Conformance and Law of UnintendedConsequences

    Posted 01-21-2009 08:57
    Hi David,
    
    On 01/21/09 02:05, David Faure wrote:
    > On Tuesday 20 January 2009, robert_weir@us.ibm.com wrote:
    >>  But holding back the label of "conformant" is the primary 
    >> way to bring vendors to the table to propose and document such features. 
    >> If we just label everything conformant, than why would a vendor trouble 
    >> with the time and effort to get their proposals accepted formally into the 
    >> standard?  Why buy the cow when you can get the milk for free?
    > 
    > I understand that. But on the other hand, why should an implementation
    > be marked as "non-conformant" just because it adds one tiny attribute
    > in a style somewhere for mostly internal reasons?
    > 
    >> But your point on editing hints versus content extensions is well taken. 
    >> Maybe there is some way we can formulate a conformance clause that takes 
    >> account of that.  But I'd rather have an extension framework that handles 
    >> things like that in a structured way than to allow any XML anywhere.
    > 
    > Right, in fact I think that allowing extensions in style properties is a given,
    > while indeed allowing any element in the middle of content elements
    > would just open the door to non-standard content. I think your ideas work
    > well together: by only allowing extensions inside style properties, only
    > additional settings to _existing_ content elements can be added, no actual
    > new content.
    
    ODF 1.1 has the concept of the foreign elements, and in addition, the 
    non strict schema allows arbitrary elements and attributes within styles 
    and metadata. My conformance proposal indeed treats both features the 
    same. Maybe that is a mistake.
    
    So, it seems to me that you have no objections to disallow foreign 
    elements and attributes everywhere but to still allow them within 
    styles. That's something we of cause may do.
    
    There are many options how we could achieve this. A few of them are:
    
    a) We keep the strict and the non-strict schemas, where the difference 
    is that the non-strict schema allows arbitrary content in styles (or 
    formatting perties elements, to be more precise). We then define two 
    conformance levels. A strict conforming document is one that validates 
    against the strict schema, and a conforming document, is one that 
    validates against the non-strict schema.
    b) Like a), except that we call the "strict conforming" documents 
    "conforming" in the future, and the other ones "loosely conforming".
    c) We define a single schema and a single conformance mode, which allows 
    foreign elements within formatting property elements, and only there.
    d) Like c), except that we declare the use of foreign elements within 
    formatting properties to be deprecated. This would allow a smooth 
    transition of extensions into either attributes/elements in the 
    standard, or into some other extensibility mechanism.
    
    Does any of these options make sense to you, or others?
    
    
    Best regards
    
    Michael
    
    
    > Of course styles can affect rendering as well as editing, but that's the
    > impossible-to-draw line.
    > 
    
    
    -- 
    Michael Brauer, Technical Architect Software Engineering
    StarOffice/OpenOffice.org
    Sun Microsystems GmbH             Nagelsweg 55
    D-20097 Hamburg, Germany          michael.brauer@sun.com
    http://sun.com/staroffice         +49 40 23646 500
    http://blogs.sun.com/GullFOSS
    
    Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1,
    	   D-85551 Kirchheim-Heimstetten
    Amtsgericht Muenchen: HRB 161028
    Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
    Vorsitzender des Aufsichtsrates: Martin Haering
    


  • 22.  RE: [office] ODF 1.2 Single-Level Conformance and Law of Unintended Consequences

    Posted 01-19-2009 21:30
    David,
    
    Yes, that is exactly what it is about.
    
    +1
    
     - Dennis 
    
    PS: Agreed, the nicest ones are those that when ignored as foreign elements
    have no deleterious effect on the conformant document that remains.