OASIS Open Document Format for Office Applications (OpenDocument) TC

Greetings!

in general based on a detailed feedback from Michael we made up a list 
of suggested changes.
As it does not scale to discuss every wording change in the TC, nor has 
this been done earlier for the main spec, I separated the issues in two 
lists.

a) Issues to be discussed
b) List of editorial changes

Please note both lists is not complete yet, but should give us already a 
change to start discussing.
Patrick's idea was to start a thread on each issue. The idea is to come 
to an agreement on the list and approve the outcome on the items of a) 
during next TC call.

Note: Although I have already done changes on the working draft, the 
majority of the SC was against an update, therefore I won't make the 
working document public.

Issues to be discussed:
======================
I) Exchange of occurrences of OpenOffice to OpenDocument (Resolved)
Resolution: Agreed: On TC call on 07/06/06 to add the new elements to 
the spec.

II) Change of RDF/XML describing terms
Michael suggests to that description of RDF vocabulary uses no longer 
XML terms but RDF terms using node/property instead of element/attribute.
Furthermore the XML related brackets for elements '<' and '>' should be 
removed as well.

III) Removal of definition of non persistent odf:value property
Michael suggested to remove the following paragraph, as the described 
RDF property is not stored, but only available at runtime. Due to this 
case, the property would be out of scope of our metadata format 
specification.
“For every IRI representing an OpenDocument element an RDF statement can 
be made about the element content. In this RDF statement the element IRI 
is the RDF subject, the property of odf:content is the RDF predicate and 
the content literal of the OpenDocument element is the RDF object.”

IV) OWL definition of odf:path
Property has only been defined in OWL for odf:File not for odf:Element

V) Removal of odf:mimeType
Property would be redundant to MIME information of original manifest


List of editorial changes:
======================
1) Removal of sentence with duplicate content, mentioned in the previous 
introduction
"In addition to the metadata RDF/XML files, metadata may also be 
specified in content using the in content metadata (see section 1.2)."

2) Changed wording from:
"The metadata manifest file shall be stored at “/”. The metadata 
manifest file shall be named “manifest.rdf”."
to
"The metadata manifest file shall be named “manifest.rdf”, and shall be 
stored in root of the package."

3) Changed wording from:
"The metadata manifest file is based on RDF/XML and fulfills two basic 
functions:"
to
“The metadata manifest file fulfills two basic functions”.
That it is RDF/XML is said below already, and not related to the purpose 
of the file.

4) Changed wording from:
"The package (i.e. the document) is identified by the IRI of the 
rdf:about attribute on the 

On 6/26/07, Svante Schubert 

Bruce,

Bruce D'Arcus wrote:
> On 6/26/07, Svante Schubert 

Greetings!

here an updated list of issues to be discussed and editorial changes.

New suggested of changes to be discussed
=========================================

VI) Dropping non modification requirement
"Metadata files should not be modified unless the content of the
metadata file is changed.

This sentence describes application behavior. The described behavior is
moreover not essential, nor do we have something similar for ODF
content, like keeping the XML structure, when the document is not being
changed.



VII) Dropping the following sentence as we introduced no restriction on
RDF/XML.

"The OpenDocument format does not restrict RDF/XML elements or
attributes for metadata files except as defined for the OpenDocument
format."



VIII) Adjust 'shall' requirement to 'should' for xml:id

"All implementations SHALL preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

Similar as other standards (e.g. CSS) we should not try to force
features by specification, but should let the market sort this out.
Moreover the specification could be interpreted that it is even
forbidding to delete the xml:id or its value, even when deleting the
content, therefore a 'SHOULD' is sufficient.



IX) Dropping paragraph about preserved content of text:meta-field

Asked to drop the following section:
"The generated content of a 

Svante,

I suggest these go to the main TC list. This one, in particular ...

On 6/27/07, Svante Schubert 

Hi,

first of all, it is my understanding that the issue that Svante is 
raising here regarding the usage of shall and should is not that 
applications should get the freedom to remove the xml:id and other 
metadata features whenever they like. Of course, they should keep them 
whenever this is possible and appropriate.

The issue is to find an appropriate language for the things we want to 
achieve. And independent of the issues we are discussing here, this in 
general includes the option to omit a certain language if we believe 
that what we say may be self-evident or given anyway, so that the 
language is not required at all.

Below are few suggestion that may or may not be helpful for addressing 
the metadata related should and shall issues. But before. I would like 
to make some comments:

1. We have to make sure that the language we are choosing is precise, 
and permits reasonable edit operations on documents. Related to xml:ids, 
that means that the language must permit to remove the attribute or to 
change its values if this happens as the result of a user action or a 
machine processing the document.
2. If a document is opened and saved again, we all expect that the 
paragraph content is preserved. The same applies to tables, lists, 
images. etc. Does the specification has a language that enforces
that? No, it doesn't. But we all expect that these features are 
preserved anyway.
But what's different with the xml:id (and metadata in general) that
there is the assumption that it may get removed unless there is a
language that forbids that?
I personally believe that everyone who is interested in ODF is 
interested in implementing as much of ODF as possible (though there may 
of cause be resource or technical restrictions), and this includes 
metadata, regardless what language we use for above features. I further 
believe that we encourage more vendors to implement metadata if we talk 
about the value of implementing them, than we do by making certain 
things mandatory. For that reason, and because it seems not to be too 
easy to find an appropriate language, it would be okay for me if we 
would omit that language at all.
3. The focus of ODF of course are office documents. But there always was 
the assumption that also other kind of applications should be able to 
use ODF. So, if someone develops a small text editor and wishes to 
support ODF to the extend that typical text editors can, this should be 
be possible. Our language should not prohibit that. We should also not 
forget the various ODF plug-in efforts for MS Office or similar ODF 
implementations. They have only limited control of what happens with 
certain information during complex load, edit and save operations within 
MS Office. I'm not sure if they can preserve all metadata and all 
xml:ids under all circumstances in a way that keeps the metadata 
consistent and therefore of value.

Having that said, here are my suggestions. Please do not consider them 
as a proposal. They are only suggestions, and the SC may follow them as 
a whole or partially, or may not.

1. We may move all the metadata related should/shall language into the 
general conformance section. This has the advantage that it is not 
overlooked as easy as it would be if it is in the element and attribute 
description. It further has the advantage that metadata is mentioned at 
a very prominent position.
2. We may introduce the term of a metadata-aware application (or 
something like that), and define conformance definitions along the 
following lines for it:
- A metadata aware ODF implementation *shall* not remove the xml:id 
attributes defined in sections [?] or change its values unless the 
removal or modification is the result of an edit operation caused be the 
user, or a similar action taken by some automatic processing of the 
document.
- [any other requirement that may exist]
3. We may rephrase the above statement for general ODF implementation, 
replacing the *shall* with a *should*:
- An ODF implementation *should* not remove the xml:id attributes 
defined in sections [?] or change its values unless the removal or 
modification is the result of an edit operation caused be the user, or a 
similar action taken by some automatic processing of the document.
4. Some time ago we have discussed whether the question which 
implementation should/shall support what features may be a topic for ODF 
1.3. So we may go with no or only a very limited number of metadata 
related conformance requirements for ODF 1.2, and make a deeper 
discussion part of a more general discussion for ODF 1.3.

Maybe these comments and suggestions are somehow useful.

Best regards

Michael


Bruce D'Arcus wrote:
> Svante,
> 
> I suggest these go to the main TC list. This one, in particular ...
> 
> On 6/27/07, Svante Schubert 

On Jun 29, 2007, at 10:06 AM, Michael Brauer - Sun Germany - ham02 - 
Hamburg wrote:

> 1. We have to make sure that the language we are choosing is precise, 
> and permits reasonable edit operations on documents. Related to 
> xml:ids, that means that the language must permit to remove the 
> attribute or to change its values if this happens as the result of a 
> user action or a machine processing the document.

Right.

> 2. If a document is opened and saved again, we all expect that the 
> paragraph content is preserved. The same applies to tables, lists, 
> images. etc.

Does this include attributes?

> Does the specification has a language that enforces
> that? No, it doesn't. But we all expect that these features are 
> preserved anyway.
> But what's different with the xml:id (and metadata in general) that
> there is the assumption that it may get removed unless there is a
> language that forbids that?

The bottomline is, because we move so much of the RDF logic into the 
package, the xml:id attributes become crucial anchor points. In short, 
if an application removes, say, the xml:id from a text:meta-field or 
otherwise causes the URI binding to be invalid, the field will break. 
It would be bad for interoperability for applications to do this.

...

> 3. The focus of ODF of course are office documents. But there always 
> was the assumption that also other kind of applications should be able 
> to use ODF. So, if someone develops a small text editor and wishes to 
> support ODF to the extend that typical text editors can, this should 
> be be possible. Our language should not prohibit that. We should also 
> not forget the various ODF plug-in efforts for MS Office or similar 
> ODF implementations. They have only limited control of what happens 
> with certain information during complex load, edit and save operations 
> within MS Office. I'm not sure if they can preserve all metadata and 
> all xml:ids under all circumstances in a way that keeps the metadata 
> consistent and therefore of value.

Well, let's say an application doesn't care about metadata. All they 
have to do is preserve the files in the package and the xml:ids as is. 
They need not do any kind of processing.

I don't see how this is any real burden (?).

> Having that said, here are my suggestions. Please do not consider them 
> as a proposal. They are only suggestions, and the SC may follow them 
> as a whole or partially, or may not.
>
> 1. We may move all the metadata related should/shall language into the 
> general conformance section. This has the advantage that it is not 
> overlooked as easy as it would be if it is in the element and 
> attribute description. It further has the advantage that metadata is 
> mentioned at a very prominent position.
> 2. We may introduce the term of a metadata-aware application (or 
> something like that), and define conformance definitions along the 
> following lines for it:

I think the rules should apply to all ODF 1.2 compliant applications. 
Carving out a separate category of "metadata aware" leaves a large 
loophole.

On that basis, perhaps option 1 is preferable, where the language 
remains "shall." I'd go even further, n fact, and require preservation 
of all attributes. That makes it a generic requirement that is not 
specific to metadata, but ensures xml:id preservation.

Bruce

> - A metadata aware ODF implementation *shall* not remove the xml:id 
> attributes defined in sections [?] or change its values unless the 
> removal or modification is the result of an edit operation caused be 
> the user, or a similar action taken by some automatic processing of 
> the document.
> - [any other requirement that may exist]
> 3. We may rephrase the above statement for general ODF implementation, 
> replacing the *shall* with a *should*:
> - An ODF implementation *should* not remove the xml:id attributes 
> defined in sections [?] or change its values unless the removal or 
> modification is the result of an edit operation caused be the user, or 
> a similar action taken by some automatic processing of the document.
> 4. Some time ago we have discussed whether the question which 
> implementation should/shall support what features may be a topic for 
> ODF 1.3. So we may go with no or only a very limited number of 
> metadata related conformance requirements for ODF 1.2, and make a 
> deeper discussion part of a more general discussion for ODF 1.3.
>
> Maybe these comments and suggestions are somehow useful.
>
> Best regards
>
> Michael
>
>
> Bruce D'Arcus wrote:
>> Svante,
>> I suggest these go to the main TC list. This one, in particular ...
>> On 6/27/07, Svante Schubert 

Bruce D'Arcus wrote:
...
>> 3. The focus of ODF of course are office documents. But there always 
>> was the assumption that also other kind of applications should be 
>> able to use ODF. So, if someone develops a small text editor and 
>> wishes to support ODF to the extend that typical text editors can, 
>> this should be be possible. Our language should not prohibit that. We 
>> should also not forget the various ODF plug-in efforts for MS Office 
>> or similar ODF implementations. They have only limited control of 
>> what happens with certain information during complex load, edit and 
>> save operations within MS Office. I'm not sure if they can preserve 
>> all metadata and all xml:ids under all circumstances in a way that 
>> keeps the metadata consistent and therefore of value.
>
> Well, let's say an application doesn't care about metadata. All they 
> have to do is preserve the files in the package and the xml:ids as is. 
> They need not do any kind of processing.
>
> I don't see how this is any real burden (?).
It is our goal that a wide range of applications comply to the 
OpenDocument file format.
And yes, your request would be indeed another burden for any of these 
applications to accept the OpenDocument format.
Therefore it seems best to keep these rules optional unless the 
application plan to implement the metadata feature.
And when I mentioned ODF applications, I do not meant OpenOffice.org, 
for which it won't be a problem as we desire the feature.

regards,
Svante

I'm cc-ing this to the main TC list, because I think this question 
about interoperability is critical to ODF; it's not just about the new 
metadata support.

The question is, what should the language be on preservation of xml:id 
attributes in ODF 1.2. The xml:id attributes are critical to the 
metadata proposal, but they could also be used for other purposes.

The current proposal says they "shall" be preserved. Michael and Svante 
have suggested this be changed to the weaker "should" or perhaps to 
introduce some notion of "metadata aware" ODF applications (effectively 
a loop-hole).

My position is that if we change it, we should make it a general 
requirement to preserve attributes. We cannot allow applications to 
arbitrarily throw out critical attributes.

Moving on ...

On Jun 29, 2007, at 11:12 AM, Svante Schubert wrote:

> ...

I wrote (quoting Michael):

>>> 3. The focus of ODF of course are office documents. But there always 
>>> was the assumption that also other kind of applications should be 
>>> able to use ODF. So, if someone develops a small text editor and 
>>> wishes to support ODF to the extend that typical text editors can, 
>>> this should be be possible. Our language should not prohibit that. 
>>> We should also not forget the various ODF plug-in efforts for MS 
>>> Office or similar ODF implementations. They have only limited 
>>> control of what happens with certain information during complex 
>>> load, edit and save operations within MS Office. I'm not sure if 
>>> they can preserve all metadata and all xml:ids under all 
>>> circumstances in a way that keeps the metadata consistent and 
>>> therefore of value.
>>
>> Well, let's say an application doesn't care about metadata. All they 
>> have to do is preserve the files in the package and the xml:ids as 
>> is. They need not do any kind of processing.
>>
>> I don't see how this is any real burden (?).
> It is our goal that a wide range of applications comply to the 
> OpenDocument file format.

Same with mine. I just think this argument is a red herring; some vague 
hypothetical concern measured against absolutely certain and concrete 
problems that will emerge if applications do not preserve xml:id 
attributes and associated files.

If an ODF application does not preserve them, it should not be called 
ODF 1.2 compliant.

> And yes, your request would be indeed another burden for any of these 
> applications to accept the OpenDocument format.

How?

> Therefore it seems best to keep these rules optional unless the 
> application plan to implement the metadata feature.
> And when I mentioned ODF applications, I do not meant OpenOffice.org, 
> for which it won't be a problem as we desire the feature.

I strongly disagree. Preserving files and attributes is a trivial 
requirement. Not doing so will introduce large compatibility problems.

Really, just to be clear: if applications do not preserve xml:id 
attributes, fields will break, and any metadata about document 
fragments will be made invalid. Is that really in anyone's interest? 
They need not support metadata in any explicit way to do this.

Bruce

I'm cc-ing this to the main TC list, because I think this question 
about interoperability is critical to ODF; it's not just about the new 
metadata support.

The question is, what should the language be on preservation of xml:id 
attributes in ODF 1.2. The xml:id attributes are critical to the 
metadata proposal, but they could also be used for other purposes.

The current proposal says they "shall" be preserved. Michael and Svante 
have suggested this be changed to the weaker "should" or perhaps to 
introduce some notion of "metadata aware" ODF applications (effectively 
a loop-hole).

My position is that if we change it, we should make it a general 
requirement to preserve attributes. We cannot allow applications to 
arbitrarily throw out critical attributes.

Moving on ...

On Jun 29, 2007, at 11:12 AM, Svante Schubert wrote:

> ...

I wrote (quoting Michael):

>>> 3. The focus of ODF of course are office documents. But there always 
>>> was the assumption that also other kind of applications should be 
>>> able to use ODF. So, if someone develops a small text editor and 
>>> wishes to support ODF to the extend that typical text editors can, 
>>> this should be be possible. Our language should not prohibit that. 
>>> We should also not forget the various ODF plug-in efforts for MS 
>>> Office or similar ODF implementations. They have only limited 
>>> control of what happens with certain information during complex 
>>> load, edit and save operations within MS Office. I'm not sure if 
>>> they can preserve all metadata and all xml:ids under all 
>>> circumstances in a way that keeps the metadata consistent and 
>>> therefore of value.
>>
>> Well, let's say an application doesn't care about metadata. All they 
>> have to do is preserve the files in the package and the xml:ids as 
>> is. They need not do any kind of processing.
>>
>> I don't see how this is any real burden (?).
> It is our goal that a wide range of applications comply to the 
> OpenDocument file format.

Same with mine. I just think this argument is a red herring; some vague 
hypothetical concern measured against absolutely certain and concrete 
problems that will emerge if applications do not preserve xml:id 
attributes and associated files.

If an ODF application does not preserve them, it should not be called 
ODF 1.2 compliant.

> And yes, your request would be indeed another burden for any of these 
> applications to accept the OpenDocument format.

How?

> Therefore it seems best to keep these rules optional unless the 
> application plan to implement the metadata feature.
> And when I mentioned ODF applications, I do not meant OpenOffice.org, 
> for which it won't be a problem as we desire the feature.

I strongly disagree. Preserving files and attributes is a trivial 
requirement. Not doing so will introduce large compatibility problems.

Really, just to be clear: if applications do not preserve xml:id 
attributes, fields will break, and any metadata about document 
fragments will be made invalid. Is that really in anyone's interest? 
They need not support metadata in any explicit way to do this.

Bruce

On 6/29/07, Bruce D'Arcus <bdarcus@gmail.com> wrote:

>> I don't see how this is any real burden (?).
> It is our goal that a wide range of applications comply to the
> OpenDocument file format.

Same with mine. I just think this argument is a ***red herring;*** some vague
hypothetical concern measured against absolutely certain and concrete
problems that will emerge if applications do not preserve xml:id
attributes and associated files.

Tch. Tch. and you lectured me about making accusations. "Similar to ignoratio elenchi, a red herring is an argument, given in reply, that does not address the original issue. Critically, a red herring is a deliberate attempt to change the subject or divert the argument."

<http://en.wikipedia.org/wiki/Red_herring_fallacy>. So was your admonition to me to "do as I say not as I do?" You can accuse Sun of subterfuge but I cannot? Sounds like a double standard to me. :-)


 

On 6/29/07, Bruce D'Arcus <bdarcus@gmail.com> wrote:

>> I don't see how this is any real burden (?).
> It is our goal that a wide range of applications comply to the
> OpenDocument file format.

Same with mine. I just think this argument is a ***red herring;*** some vague
hypothetical concern measured against absolutely certain and concrete
problems that will emerge if applications do not preserve xml:id
attributes and associated files.

Tch. Tch. and you lectured me about making accusations. "Similar to ignoratio elenchi, a red herring is an argument, given in reply, that does not address the original issue. Critically, a red herring is a deliberate attempt to change the subject or divert the argument."

<http://en.wikipedia.org/wiki/Red_herring_fallacy>. So was your admonition to me to "do as I say not as I do?" You can accuse Sun of subterfuge but I cannot? Sounds like a double standard to me. :-)


 

On Jun 29, 2007, at 11:49 PM, marbux wrote:

> Tch. Tch. and you lectured me about making accusations. "Similar to 
> ignoratio elenchi, a red herring is an argument, given in reply, that 
> does not address the original issue. Critically, a red herring is a 
> deliberate attempt to change the subject or divert the argument."

I used the term "red herring" perhaps incorrectly then. I drew no 
assumption about intent; I simply meant to indicate Svante was drawing 
an inappropriate conclusion from my argument about preserving 
attributes.

So apologies to Svante for my imprecise use of metaphor :-)

Bruce

On Jun 29, 2007, at 11:49 PM, marbux wrote:

> Tch. Tch. and you lectured me about making accusations. "Similar to 
> ignoratio elenchi, a red herring is an argument, given in reply, that 
> does not address the original issue. Critically, a red herring is a 
> deliberate attempt to change the subject or divert the argument."

I used the term "red herring" perhaps incorrectly then. I drew no 
assumption about intent; I simply meant to indicate Svante was drawing 
an inappropriate conclusion from my argument about preserving 
attributes.

So apologies to Svante for my imprecise use of metaphor :-)

Bruce

On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
> > Therefore it seems best to keep these rules optional unless the
> > application plan to implement the metadata feature.
> > And when I mentioned ODF applications, I do not meant OpenOffice.org,
> > for which it won't be a problem as we desire the feature.
>
> I strongly disagree. Preserving files and attributes is a trivial
> requirement. Not doing so will introduce large compatibility problems.

This, naturally, is not an isolated case. There currently are no 
applications that support all the ODF features but they still can be ODF 
compliant.  Even should one loose 90% of the features between loading and 
saving.

So the question becomes, is metadata any different. I'm inclined to say 
no.  Its not different, its just a feature that the application should 
support. If it doesn't then a better application comes along and replaces 
it.
Looking at the ODF landscape I consider this;
* OOo and KOffice both think its a good feature to support, and are both 
moving to do so.  I'm sure other big office suites agree and do so as 
well.  Meaning that ODF doesn't have to force their hands.
* Almost all professional users of office suites benefit from metadata in 
one way or another. So an application that follows user requests will 
quickly see metadata on their TODO list.
* There are not that many application-types that both load and save ODF 
files. Viewers and writers are the bigger segment. We are only concerned 
with the load+save type.

Bottom line is that while I fully agree its very important to retain this 
information that does not automatically imply that ODF has to make it 
mandatory. Market forces can do that too.
Furthermore I would find it hypocritical to make it mandatory while almost 
all features in ODF are not mandatory to be retained between load and 
saves.

> Really, just to be clear: if applications do not preserve xml:id
> attributes, fields will break, and any metadata about document
> fragments will be made invalid. Is that really in anyone's interest?
> They need not support metadata in any explicit way to do this.

I think this said is best; things will break in a very visible way if apps 
don't support it.  So users will quickly realize a problem and apps will 
be requested to support the feature of metadata.

The only reason I see to make it mandatory is for fear of people not 
supporting it, and I can easy your mind there, you guys have been doing 
an awesome job. Applications will line up to support this stuff. Don't 
worry so much! :)


ps. not sure if I have write access to the metadata ML, if this doesn't 
appear there, please consider forwarding. Thanks.
-- 
Thomas Zander

On 6/29/07, Thomas Zander <zander@kde.org> wrote:

On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
> > Therefore it seems best to keep these rules optional unless the
> > application plan to implement the metadata feature.
> > And when I mentioned ODF applications, I do not meant 
OpenOffice.org,
> > for which it won't be a problem as we desire the feature.
>
> I strongly disagree. Preserving files and attributes is a trivial
> requirement. Not doing so will introduce large compatibility problems.

This, naturally, is not an isolated case. There currently are no
applications that support all the ODF features but they still can be ODF
compliant.  Even should one loose 90% of the features between loading and
saving.

But they cannot be ODF interoperable if they do so.
 


So the question becomes, is metadata any different. I'm inclined to say
no.  Its not different, its just a feature that the application should
support. If it doesn't then a better application comes along and replaces
it.
Looking at the ODF landscape I consider this;
* OOo and KOffice both think its a good feature to support, and are both
moving to do so.  I'm sure other big office suites agree and do so as
well.  Meaning that ODF doesn't have to force their hands.
* Almost all professional users of office suites benefit from metadata in
one way or another. So an application that follows user requests will
quickly see metadata on their TODO list.
* There are not that many application-types that both load and save ODF
files. Viewers and writers are the bigger segment. 

Not in terms of usage/number of users.
 


We are only concerned
with the load+save type.

Bottom line is that while I fully agree its very important to retain this
information that does not automatically imply that ODF has to make it
mandatory. Market forces can do that too.
Furthermore I would find it hypocritical to make it mandatory while almost
all features in ODF are not mandatory to be retained between load and
saves.

That "hypocrisy" is better remedied by making other features mandatory, not by eliminating the requirement of preserving medatadata. And market forces can not produce an ecosystem of ODF documents that are portable among all ODF applications. That takes conformance requirements and validators.

> Really, just to be clear: if applications do not preserve xml:id
> attributes, fields will break, and any metadata about document
> fragments will be made invalid. Is that really in anyone's interest?
> They need not support metadata in any explicit way to do this.

I think this said is best; things will break in a very visible way if apps
don't support it.  So users will quickly realize a problem and apps will
be requested to support the feature of metadata.

The only reason I see to make it mandatory is for fear of people not
supporting it, and I can easy your mind there, you guys have been doing
an awesome job. Applications will line up to support this stuff. Don't
worry so much! :)

<sarcasm>

Sure, just like they lined up to support the foreign elements and attributes.
 
One can only wonder why the W3C chose to include conformance requirements in RDF. Obviously, they simply did not understand that no one would possibly consider writing non-conformant RDF, even to create a competitive advantage over those who wrote conformant RDF.

I've got an idea, since the market does such a wonderful job of encouraging conformance, why don't we just do away with the conformance section of the ODF specification? Oh, I forgot, an empty Zip file is a conformant ODF document, so I guess we already did away with the conformance section.

<sarcasm />

Seriously, the issue under discussion is not the support for the xml:id attribute (except for the Sun statement that it does not intend to fully implement the metadata SC work, which cuts against your analysis), the issue is whether applications should be allowed to destroy xml:id attributes. So far, no one has offered a plausible reason why conformant applications should be allowed to destroy those attributes except by user-initiated actions. And noi one has offered a use case illustrating any need for any programmatic destruction of those attributes. Do you have one?

On 6/29/07, Thomas Zander <zander@kde.org> wrote:

On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
> > Therefore it seems best to keep these rules optional unless the
> > application plan to implement the metadata feature.
> > And when I mentioned ODF applications, I do not meant 
OpenOffice.org,
> > for which it won't be a problem as we desire the feature.
>
> I strongly disagree. Preserving files and attributes is a trivial
> requirement. Not doing so will introduce large compatibility problems.

This, naturally, is not an isolated case. There currently are no
applications that support all the ODF features but they still can be ODF
compliant.  Even should one loose 90% of the features between loading and
saving.

But they cannot be ODF interoperable if they do so.
 


So the question becomes, is metadata any different. I'm inclined to say
no.  Its not different, its just a feature that the application should
support. If it doesn't then a better application comes along and replaces
it.
Looking at the ODF landscape I consider this;
* OOo and KOffice both think its a good feature to support, and are both
moving to do so.  I'm sure other big office suites agree and do so as
well.  Meaning that ODF doesn't have to force their hands.
* Almost all professional users of office suites benefit from metadata in
one way or another. So an application that follows user requests will
quickly see metadata on their TODO list.
* There are not that many application-types that both load and save ODF
files. Viewers and writers are the bigger segment. 

Not in terms of usage/number of users.
 


We are only concerned
with the load+save type.

Bottom line is that while I fully agree its very important to retain this
information that does not automatically imply that ODF has to make it
mandatory. Market forces can do that too.
Furthermore I would find it hypocritical to make it mandatory while almost
all features in ODF are not mandatory to be retained between load and
saves.

That "hypocrisy" is better remedied by making other features mandatory, not by eliminating the requirement of preserving medatadata. And market forces can not produce an ecosystem of ODF documents that are portable among all ODF applications. That takes conformance requirements and validators.

> Really, just to be clear: if applications do not preserve xml:id
> attributes, fields will break, and any metadata about document
> fragments will be made invalid. Is that really in anyone's interest?
> They need not support metadata in any explicit way to do this.

I think this said is best; things will break in a very visible way if apps
don't support it.  So users will quickly realize a problem and apps will
be requested to support the feature of metadata.

The only reason I see to make it mandatory is for fear of people not
supporting it, and I can easy your mind there, you guys have been doing
an awesome job. Applications will line up to support this stuff. Don't
worry so much! :)

<sarcasm>

Sure, just like they lined up to support the foreign elements and attributes.
 
One can only wonder why the W3C chose to include conformance requirements in RDF. Obviously, they simply did not understand that no one would possibly consider writing non-conformant RDF, even to create a competitive advantage over those who wrote conformant RDF.

I've got an idea, since the market does such a wonderful job of encouraging conformance, why don't we just do away with the conformance section of the ODF specification? Oh, I forgot, an empty Zip file is a conformant ODF document, so I guess we already did away with the conformance section.

<sarcasm />

Seriously, the issue under discussion is not the support for the xml:id attribute (except for the Sun statement that it does not intend to fully implement the metadata SC work, which cuts against your analysis), the issue is whether applications should be allowed to destroy xml:id attributes. So far, no one has offered a plausible reason why conformant applications should be allowed to destroy those attributes except by user-initiated actions. And noi one has offered a use case illustrating any need for any programmatic destruction of those attributes. Do you have one?

I don't like replying to a post as cynical as yours, its not good for the 
conversation as a whole, but I'll add my 2 cents to the below assertion.

On Saturday 30 June 2007 09:25:41 marbux wrote:
> market forces can not produce an ecosystem of ODF documents that are
> portable among all ODF applications. That takes conformance
> requirements and validators.

I have very different experiences. I suggest you look at the concept of 
RFC which is what the internet is build on. I am of the opinion over time 
all open source software implementing RFCs have grown and many, if not 
most, support a big enough chunk of those specs that, indeed, there IS an 
ecosystem.  And a pretty darn successfull one.
Its mostly proprietary apps that don't support the full standard. And the 
effect has always been that, indeed, they are not very interoperable and 
the more compliant ones take over the market.

I have a very good track record to point to on a worldwide scale, and it 
disagrees with your assertion above.

At the same time, I am hoping there will be move people that will spent 
time working on the ODF testsuite.  Its very important for the consumer 
to see which applications support ODF the best, but we (always) need more 
people writing good tests.
-- 
Thomas Zander

On 6/30/07, Thomas Zander <zander@kde.org> wrote:

I don't like replying to a post as cynical as yours, its not good for the
conversation as a whole, but I'll add my 2 cents to the below assertion.

The feeling is mutual, I guess.
 

On Saturday 30 June 2007 09:25:41 marbux wrote:
> market forces can not produce an ecosystem of ODF documents that are
> portable among all ODF applications. That takes conformance
> requirements and validators.

I have very different experiences. I suggest you look at the concept of
RFC which is what the internet is build on. I am of the opinion over time
all open source software implementing RFCs have grown and many, if not
most, support a big enough chunk of those specs that, indeed, there IS an
ecosystem.  And a pretty darn successfull one.
Its mostly proprietary apps that don't support the full standard. And the
effect has always been that, indeed, they are not very interoperable and
the more compliant ones take over the market.

I think RFC's have worked well for communications protocols but not so well for document formats. 
E.g., IIRC several browsers now for the most part pass the Acid2 test for CSS-HTML support, but the overwhelming market leader (MSIE--proprietary) still does not. This, by the time that HTML and CSS2 are on the verge of obsolescence.

But at the same time, most RFC's for document formats I have looked at have conformance requirements, so I don't see how this advances your argument anyway.


At the same time, I am hoping there will be move people that will spent
time working on the ODF testsuite.  Its very important for the consumer
to see which applications support ODF the best, but we (always) need more
people writing good tests.

Not to belittle your work on the test suite, but push come to shove I would much rather see sufficient ODF requirements that the less featureful apps can round-trip documents with the more featureful apps -- and with Microsoft Office and WordPerfect Office -- without data loss. Somehow we have to get past the TC's fixation on software as an end point rather than as routers of information. Indeed, there is no sign in the TC list archives that anyone other than Gary, Florian, and me even understand that concept.

So ODF is being forked. I hope you at least enjoy being stuck in 1995.

Thomas,

On 6/30/07, Thomas Zander 

Thomas,

On 6/30/07, Thomas Zander 

On Saturday 30 June 2007 13:00:07 Bruce D'Arcus wrote:
> > So the question becomes, is metadata any different. I'm inclined to
> > say no.  Its not different, its just a feature that the application
> > should support. If it doesn't then a better application comes along
> > and replaces it.
>
> Absolutely agreed. This is not in any way specific to metadata. So
> let's make sure to keep these issues separate.

Hmm? How do you come from agreeing two things are the same and then 
suggesting they should be treated separately?
That just doesn't make sense to me.
Two issues being the same should be handled the same, don't they?

> > Looking at the ODF landscape I consider this;
> > * OOo and KOffice both think its a good feature to support, and are
> > both moving to do so.  I'm sure other big office suites agree and do
> > so as well.  Meaning that ODF doesn't have to force their hands.
> > * Almost all professional users of office suites benefit from
> > metadata in one way or another. So an application that follows user
> > requests will quickly see metadata on their TODO list.
> > * There are not that many application-types that both load and save
> > ODF files. Viewers and writers are the bigger segment. We are only
> > concerned with the load+save type.
> >
> > Bottom line is that while I fully agree its very important to retain
> > this information that does not automatically imply that ODF has to
> > make it mandatory. Market forces can do that too.
> > Furthermore I would find it hypocritical to make it mandatory while
> > almost all features in ODF are not mandatory to be retained between
> > load and saves.
>
> As I said: I'm not in any way saying it should be mandatory ODF 1.2
> applications support metadata. That's a red herring, so can we please
> put that aside?
>
> I'm saying it should be mandatory that applications don't destroy
> crucial data!

So was I. That's what "retain this information" means in my comment you 
quoted.

> The crucial bit here is: what is it OK for an application to do with
> optional (or foreign) attributes? The new xml:id attribute we rely on
> for metadata is just one example of this, but I'm sure there are
> others. Is it OK for an application to read that data in but throw it
> out?
>
> Or files it doesn't know about: is it OK for an ODF application to dump
> them?

That is not your and not my decision to make.
We can just make very clear what the consequences are. And they are very 
obviously going to loose data and functionality. Just like not supporting 
a 'feature' like text:p looses data.

Your questions are valid, just not posted in the right context.
The questions are valid as they surely will be relevant to the end users. 
Which will loose functionality if data is lost.  Your idea to ask this in 
the ODF technical committee(s) is the wrong context. It is almost never a 
good idea to make sections mandatory in a specification. If only because 
as soon as you do the applications not implementing the mandatory part 
will still call it ODF, and you get to do police work with all the 
negative market consequences that come with that.

So, to answer your question; an application losing data is not one that 
people will like a lot.

I have to ask you;  how do you think history will unfold if you make this 
part mandatory?
If you think that suddenly everyone will do it right then you are in for a 
surprise. The world doesn't work like that.
I can point to issues where OOo is not ODF compliant, and just not 
altering their behavior since it makes sense for them. How does that 
change when you mark something mandatory?

Note the reason I mentioned the testsuite; 
http://testsuite.opendocumentfellowship.org  the reason is that this is 
an excellent tool to tell the world how good or how bad an implementation 
is. We already talked about having an extra metric that specifies how 
well data is retained between load and save.  Its unfortunately not done 
due to lack of manpower.
A second question for you; do you have doubts that the market can make 
sure ODF compliance becomes a consideration for people choosing their 
tools?
-- 
Thomas Zander

On Jun 30, 2007, at 7:20 AM, Thomas Zander wrote:

> On Saturday 30 June 2007 13:00:07 Bruce D'Arcus wrote:
>>> So the question becomes, is metadata any different. I'm inclined to
>>> say no.  Its not different, its just a feature that the application
>>> should support. If it doesn't then a better application comes along
>>> and replaces it.
>>
>> Absolutely agreed. This is not in any way specific to metadata. So
>> let's make sure to keep these issues separate.
>
> Hmm? How do you come from agreeing two things are the same and then
> suggesting they should be treated separately?
> That just doesn't make sense to me.
> Two issues being the same should be handled the same, don't they?

Sorry, by "these issues" I was meaning metadata on one hand, and the 
general issues of preservation/conformance on the other. I agree that 
while we brought up the latter in the context of the former, they are 
separate issues.

Does that make better sense?

...

>> The crucial bit here is: what is it OK for an application to do with
>> optional (or foreign) attributes? The new xml:id attribute we rely on
>> for metadata is just one example of this, but I'm sure there are
>> others. Is it OK for an application to read that data in but throw it
>> out?
>>
>> Or files it doesn't know about: is it OK for an ODF application to 
>> dump
>> them?
>
> That is not your and not my decision to make.

It is the TC's decision as a whole to make though. There are places all 
through the spec that use "shall" language, and a whole lot of others 
that use "should." It's the TC's job to make those distinctions.

> We can just make very clear what the consequences are. And they are 
> very
> obviously going to loose data and functionality. Just like not 
> supporting
> a 'feature' like text:p looses data.
>
> Your questions are valid, just not posted in the right context.
> The questions are valid as they surely will be relevant to the end 
> users.

Yes, and probably application developers as well.

> Which will loose functionality if data is lost.  Your idea to ask this 
> in
> the ODF technical committee(s) is the wrong context.

This is too strong, and frankly a little pedantic.

As I say above, we're writing a spec; discussing the language of the 
spec is absolutely part of that process. If it weren't up for 
discussion, then we'd keep the "shall" language in the proposal. So 
here we're discussing it.

> It is almost never a good idea to make sections mandatory in a 
> specification. If only because
> as soon as you do the applications not implementing the mandatory part
> will still call it ODF, and you get to do police work with all the
> negative market consequences that come with that.
>
> So, to answer your question; an application losing data is not one that
> people will like a lot.
>
> I have to ask you;  how do you think history will unfold if you make 
> this
> part mandatory?
> If you think that suddenly everyone will do it right then you are in 
> for a
> surprise. The world doesn't work like that.
> I can point to issues where OOo is not ODF compliant, and just not
> altering their behavior since it makes sense for them. How does that
> change when you mark something mandatory?

You can at least say its wrong! It is an obvious bug.

> Note the reason I mentioned the testsuite;
> http://testsuite.opendocumentfellowship.org  the reason is that this is
> an excellent tool to tell the world how good or how bad an 
> implementation
> is. We already talked about having an extra metric that specifies how
> well data is retained between load and save.  Its unfortunately not 
> done
> due to lack of manpower.
> A second question for you; do you have doubts that the market can make
> sure ODF compliance becomes a consideration for people choosing their
> tools?

Let me ask the entire group a bigger question: what does it mean for an 
application to say it is "ODF conformant"?

Does it matter -- to users and developers -- that we can answer that 
question with any kind of precision?

Maybe you're right that it's not something that should be tightly 
controlled in the spec and the schema; that we don't provide a binary 
yes/no answer to that question, but rather allow for shades of gray, 
backed up by test suites. I'm fine with that notion.

But then surely it needs to be outlined some place, with some kind of 
blessing of the TC, exactly what those shades of gray are?

I seem to recall this issue of conformance has come up before; but 
would just reiterate it's an important issue.

Bruce

On 6/30/07, Thomas Zander <zander@kde.org> wrote:

I have to ask you;  how do you think history will unfold if you make this
part mandatory?
If you think that suddenly everyone will do it right then you are in for a
surprise. The world doesn't work like that.
I can point to issues where OOo is not ODF compliant, and just not
altering their behavior since it makes sense for them. How does that
change when you mark something mandatory?

Note the reason I mentioned the testsuite;
http://testsuite.opendocumentfellowship.org  the reason is that this is
an excellent tool to tell the world how good or how bad an implementation
is. We already talked about having an extra metric that specifies how
well data is retained between load and save.  Its unfortunately not done
due to lack of manpower.
A second question for you; do you have doubts that the market can make
sure ODF compliance becomes a consideration for people choosing their
tools?

On the latter question, no doubts, although not in the manner you have been discussing.  Europe has been quite clear that it intends to enforce interoperability conformance requirements and testing through the government procurement power and that one of their fundamental requirements is lossless interoperability. They are also preparing to use the procurement power to force the convergence of ODF and MOOXML, with ODF providing the common functionality. Really, if you take the time to read the various presentations at the Berlin Conference, you would realize that the eminent demise of both OASIS ODF and Ecma OOXML in their present forms were announced at that conference,

If you don't want to take the time to read all of the presentations, accessible from this page, <http://ec.europa.eu/idabc/en/document/6474/5935
>, I recommend that you at least skim the slides at <http://ec.europa.eu/idabc/servlets/Doc?id=27865> (expectations on industry players) (4 slides); <
http://ec.europa.eu/idabc/servlets/Doc?id=27866> (How do we get from standards to interoperability?) (5 slides); and <
http://ec.europa.eu/idabc/servlets/Doc?id=27956> (Open Document Exchange Formats: Report to the plenary session) (8 slides). And from <
http://ec.europa.eu/idabc/servlets/Doc?id=28841> (PEGSCO Recommendations (9 slides) ("Think about conformance testing to safeguard interoperability").


I do not believe that Europe will accept that interoperability is simply too complicated to warrant mandatory conformance requirements.

Thomas,

I am curious about the term "support" in this context.

Does preservation of data imply that a feature is supported?

In other words, what if I have a very lite weight application that 
doesn't actually read any of the content of an ODF file or parse any of 
the existing metadata files but simply adds an entry to the metadata 
manifest to add metadata about the document as a whole (such as would 
happen in a clerk of court's office with filing date, who filed it, 
etc.) and added an appropriate metadata file. It then saves *all* of the 
file.

Granted, I haven't implemented a lot of the features offered by ODF but 
I have preserved the file, even for features I don't support.

Is there a useful distinction to make between implementation of a 
feature and preservation of a file's contents that you are given for 
processing?

Granting that I could also implement a very lite weight application that 
doesn't offer metadata or any number of other features since we don't 
require an application to implement any specific set of features.

So, is preservation separate from feature implementation? (I am asking 
because I haven't seen it raised in this context and while it seems 
evident to me the two are different, opinions may and probably do vary 
on that point.)

Hope you are having a great day!

Patrick

Thomas Zander wrote:
> On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
>   
>>> Therefore it seems best to keep these rules optional unless the
>>> application plan to implement the metadata feature.
>>> And when I mentioned ODF applications, I do not meant OpenOffice.org,
>>> for which it won't be a problem as we desire the feature.
>>>       
>> I strongly disagree. Preserving files and attributes is a trivial
>> requirement. Not doing so will introduce large compatibility problems.
>>     
>
> This, naturally, is not an isolated case. There currently are no 
> applications that support all the ODF features but they still can be ODF 
> compliant.  Even should one loose 90% of the features between loading and 
> saving.
>
> So the question becomes, is metadata any different. I'm inclined to say 
> no.  Its not different, its just a feature that the application should 
> support. If it doesn't then a better application comes along and replaces 
> it.
> Looking at the ODF landscape I consider this;
> * OOo and KOffice both think its a good feature to support, and are both 
> moving to do so.  I'm sure other big office suites agree and do so as 
> well.  Meaning that ODF doesn't have to force their hands.
> * Almost all professional users of office suites benefit from metadata in 
> one way or another. So an application that follows user requests will 
> quickly see metadata on their TODO list.
> * There are not that many application-types that both load and save ODF 
> files. Viewers and writers are the bigger segment. We are only concerned 
> with the load+save type.
>
> Bottom line is that while I fully agree its very important to retain this 
> information that does not automatically imply that ODF has to make it 
> mandatory. Market forces can do that too.
> Furthermore I would find it hypocritical to make it mandatory while almost 
> all features in ODF are not mandatory to be retained between load and 
> saves.
>
>   
>> Really, just to be clear: if applications do not preserve xml:id
>> attributes, fields will break, and any metadata about document
>> fragments will be made invalid. Is that really in anyone's interest?
>> They need not support metadata in any explicit way to do this.
>>     
>
> I think this said is best; things will break in a very visible way if apps 
> don't support it.  So users will quickly realize a problem and apps will 
> be requested to support the feature of metadata.
>
> The only reason I see to make it mandatory is for fear of people not 
> supporting it, and I can easy your mind there, you guys have been doing 
> an awesome job. Applications will line up to support this stuff. Don't 
> worry so much! :)
>
>
> ps. not sure if I have write access to the metadata ML, if this doesn't 
> appear there, please consider forwarding. Thanks.
>   

-- 
Patrick Durusau
patrick@durusau.net
Chair, V1 - US TAG to JTC 1/SC 34
Acting Convener, JTC 1/SC 34/WG 3 (Topic Maps)
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Co-Editor, OpenDocument Format (OASIS, ISO/IEC 26300)

Thomas,

I am curious about the term "support" in this context.

Does preservation of data imply that a feature is supported?

In other words, what if I have a very lite weight application that 
doesn't actually read any of the content of an ODF file or parse any of 
the existing metadata files but simply adds an entry to the metadata 
manifest to add metadata about the document as a whole (such as would 
happen in a clerk of court's office with filing date, who filed it, 
etc.) and added an appropriate metadata file. It then saves *all* of the 
file.

Granted, I haven't implemented a lot of the features offered by ODF but 
I have preserved the file, even for features I don't support.

Is there a useful distinction to make between implementation of a 
feature and preservation of a file's contents that you are given for 
processing?

Granting that I could also implement a very lite weight application that 
doesn't offer metadata or any number of other features since we don't 
require an application to implement any specific set of features.

So, is preservation separate from feature implementation? (I am asking 
because I haven't seen it raised in this context and while it seems 
evident to me the two are different, opinions may and probably do vary 
on that point.)

Hope you are having a great day!

Patrick

Thomas Zander wrote:
> On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
>   
>>> Therefore it seems best to keep these rules optional unless the
>>> application plan to implement the metadata feature.
>>> And when I mentioned ODF applications, I do not meant OpenOffice.org,
>>> for which it won't be a problem as we desire the feature.
>>>       
>> I strongly disagree. Preserving files and attributes is a trivial
>> requirement. Not doing so will introduce large compatibility problems.
>>     
>
> This, naturally, is not an isolated case. There currently are no 
> applications that support all the ODF features but they still can be ODF 
> compliant.  Even should one loose 90% of the features between loading and 
> saving.
>
> So the question becomes, is metadata any different. I'm inclined to say 
> no.  Its not different, its just a feature that the application should 
> support. If it doesn't then a better application comes along and replaces 
> it.
> Looking at the ODF landscape I consider this;
> * OOo and KOffice both think its a good feature to support, and are both 
> moving to do so.  I'm sure other big office suites agree and do so as 
> well.  Meaning that ODF doesn't have to force their hands.
> * Almost all professional users of office suites benefit from metadata in 
> one way or another. So an application that follows user requests will 
> quickly see metadata on their TODO list.
> * There are not that many application-types that both load and save ODF 
> files. Viewers and writers are the bigger segment. We are only concerned 
> with the load+save type.
>
> Bottom line is that while I fully agree its very important to retain this 
> information that does not automatically imply that ODF has to make it 
> mandatory. Market forces can do that too.
> Furthermore I would find it hypocritical to make it mandatory while almost 
> all features in ODF are not mandatory to be retained between load and 
> saves.
>
>   
>> Really, just to be clear: if applications do not preserve xml:id
>> attributes, fields will break, and any metadata about document
>> fragments will be made invalid. Is that really in anyone's interest?
>> They need not support metadata in any explicit way to do this.
>>     
>
> I think this said is best; things will break in a very visible way if apps 
> don't support it.  So users will quickly realize a problem and apps will 
> be requested to support the feature of metadata.
>
> The only reason I see to make it mandatory is for fear of people not 
> supporting it, and I can easy your mind there, you guys have been doing 
> an awesome job. Applications will line up to support this stuff. Don't 
> worry so much! :)
>
>
> ps. not sure if I have write access to the metadata ML, if this doesn't 
> appear there, please consider forwarding. Thanks.
>   

-- 
Patrick Durusau
patrick@durusau.net
Chair, V1 - US TAG to JTC 1/SC 34
Acting Convener, JTC 1/SC 34/WG 3 (Topic Maps)
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Co-Editor, OpenDocument Format (OASIS, ISO/IEC 26300)

On Saturday 30 June 2007 20:05:04 Patrick Durusau wrote:
> Thomas,
>
> I am curious about the term "support" in this context.
>
> Does preservation of data imply that a feature is supported?

In my mails I used the term "support" as a catch all; to not complicate 
matters by being too precise that it hurst readability.
In reality; to support a feature the work that an implementation has to do 
differs per feature.
A feature might be numbered lists. And that means that the application has 
to parse the list and the numbered-paragraph xml-tags. It means it will 
have to render basic lists and it means that the concept of the list is 
written back to the file on saving.
The part where the implementation shows the list is very important to some 
people, while the writing of the exact right tags is important to 
another.
I'm not the one to judge there. And IMO the TC is not one to judge either.

> In other words, what if I have a very lite weight application that
> doesn't actually read any of the content of an ODF file or parse any of
> the existing metadata files but simply adds an entry to the metadata
> manifest to add metadata about the document as a whole (such as would
> happen in a clerk of court's office with filing date, who filed it,
> etc.) and added an appropriate metadata file. It then saves *all* of
> the file.

Excellent example, this kind of example shows its very hard to define what 
is sufficient in an abstract matter, usable to say "this app is a good 
ODF 1.2 citizen".

Naturally, in your example I fully agree that it is a good citizen. It 
lacks some features (like showing the file) and the end user gets to 
decide if the tool is something he likes based on the feature set it 
supports.

> Granted, I haven't implemented a lot of the features offered by ODF but
> I have preserved the file, even for features I don't support.
>
> Is there a useful distinction to make between implementation of a
> feature and preservation of a file's contents that you are given for
> processing?
>
> Granting that I could also implement a very lite weight application
> that doesn't offer metadata or any number of other features since we
> don't require an application to implement any specific set of features.
>
> So, is preservation separate from feature implementation?

I think preservation is a unique feature, one in the vast array of 
features that any application has.

Let me explain why I say this, as its a very practical background that 
makes this view not only reasonable, but essential;

Any application that reads an XML file of any size has to make a decision. 
It either uses an internal model that is based on the XML.  This 
typically means the app keeps the DOM tree in memory and works from that.
It can also define its own internal tree. One that gets build by reading 
XML and that gets modified internally after which it gets queried to 
write the XML again.  This internal tree may have very little resemblance 
to the ODF model (and thus the XML).

If an application chooses the second model, its own internal data 
structure, we hit a problem.
What if ODF adds a set of options to tables, but this application doesn't 
know the concept of tables.  Then it becomes very hard, to impossible, 
for this application to store meta data that was tagged on that table.
Afterall, it has no place to store that information in the internal tree.

Tables are just a big example, the real problems tend to be really 
annoying and small. But very diverse.
And every time new features are added to ODF, the application must add 
fields in their datastructure «somehow».

The approach to have a datastructure specific to the application, that may 
or may not have large sections in common with ODF, is the most used one 
for any application larger than a couple hundred lines.

So, to get back to your question;
an ODF implementation does not only have to know about a feature 
like "xml:id" it has to have a data structure to store it and then write 
it back afterwards.  And this limitation means that support for a feature 
is mostly done in two steps.
a) loading + saving is done correctly.
b) it actually renders the feature on screen and allows the user to alter 
it.

So, in that context, yes, preservation is slightly separated.
-- 
Thomas Zander

On Jun 30, 2007, at 12:13 PM, Thomas Zander wrote:

> an ODF implementation does not only have to know about a feature
> like "xml:id" it has to have a data structure to store it and then 
> write
> it back afterwards.

I realize foreign child elements are another -- more complicated -- 
matter, so let's put that aside.

But attributes are just unordered key-values; why can't any application 
just store all attributes (or at least those it doesn't know about) in 
a hash such that it can write them back out without any particular 
knowledge?

Likewise, what's the practical problem with saying applications need to 
maintain package files?

To take Patrick's hypothetical (but quite reasonable) example further 
in a different direction: what happens if I decide to be malicious, and 
I write a little script that goes through an ODF file and removes all 
optional attributes from the content file and all non-essential files 
from the package. I call my script "The Super ODF File Enhancer" or 
some such.

Would it be valid to call my application an officially compliant ODF 
application? Do we even know what that means such that we can answer 
that question?

I don't think saying "let the market decide" is an adequate answer. We 
need to have real answers for these questions, because the future 
success of ODF is going to depend on reliable document exchange among 
disparate applications, with disparate feature sets.

Bruce

On Saturday 30 June 2007 20:45:09 Bruce D'Arcus wrote:
> Would it be valid to call my application an officially compliant ODF
> application? Do we even know what that means such that we can answer
> that question?

There is no 'compliant' ruleset, and I think this is just another reason 
why there should not be one either.
I'm absolutely sure that after you made a definition of what 'compliant' 
is, someone will come and write a malicious client that still is 
compliant, but advertising it as such will just make everyone cringe.

> I don't think saying "let the market decide" is an adequate answer. We
> need to have real answers for these questions, because the future
> success of ODF is going to depend on reliable document exchange among
> disparate applications, with disparate feature sets.

Well, I agree that reliable document exchange is required. But claiming 
that having some sort of compliant rubber stamp is going to help is just 
not realistic.
In fact; there is no proof at all that a "certified compliant" label has 
ever helped to further compatibility.   All the way from people taking 
tests to certify them as MS or Linux capable people to certified alarm 
systems. They don't make perfect techies and they don't ensure you are 
free from buglers. It makes you personally feel good, sure, but the real 
world doesn't really care about that.

As I stated in various mails already in this thread; making thing 
mandatory or putting an "officially compliant ODF" label on a piece of 
software will not make software more capable of interchanging with 
others. It never has, and it never will. That's just good hope without 
any proof that it ever worked for anybody.

And don't forget the concept of measuring the wrong metric! [1]

There is a conference in the US every year where producers of software 
(samba etc) come to test how well their software interacts. The really 
practically important stuff; like "does it actually work!". There is one 
planned for ODF apps in a couple of months.
It will be the first, so lets see where it goes. But coming together and 
checking if big usecases exchange correctly is a good measure of 
interoperability, as I'm sure you'll agree.

So, what I'm saying is that ODF compliance has little to do with 
interoperability.  Sure, its connected, but they have very separate 
metrics to measure their success. And you seem to think that going for 
one metric will automatically mean success in another.  Just like the 
Bürgermeister found out, that's not how things work.


1) http://www.robweir.com/blog/2007/05/legend-of-rat-farmer.html
-- 
Thomas Zander

On 6/30/07, Bruce D'Arcus <bdarcus@gmail.com> wrote:

I don't think saying "let the market decide" is an adequate answer. We
need to have real answers for these questions, because the future
success of ODF is going to depend on reliable document exchange among
disparate applications, with disparate feature sets.

+1

I'll add that procurement officials want to be able in their software procurement tenders to specify that bidders supply software that is conformant with Standard X and fully supports that standard, rather than having to write very detailed specifications of precisely what features they want supported in the software they acquire. A good example from the IDABC conference earlier this year where government IT reps from 20 European nations achieved consensus on what they want in Open Document Exchange Formats:

 No incomplete implementations, no proprietary extensions

...

 Conformance testing and document validation possibilities
  are needed -> in order to facilitate mapping / conversion;

<
http://ec.europa.eu/idabc/servlets/Doc?id=27956> (PDF, slide 7).


So I'll say again that the market has spoken and it wants conformance requirements.

I suppose I should throw in my $.02.

First, we should remember that ODF mandates
behavior at several levels.  The schema itself encodes requirements
in terms of what elements or attributes are optional or mandatory, what
nesting is permitted, what restrictions there are on data types, etc.  
And then the normative text of the standard, along with external normative
references, make additional provisions by the use of "shall"
and "shall not".  But note that in that case,the provision
is only applicable to those who implement that feature.  A "shall"
concerning the calculation of the SUM() spreadsheet function may be totally
ignored by someone who is implementing a word processor only.  Finally,
we have the conformance clause, that defines which features and additional
constraints are required for conformance with the standard.

Today our conformance clause designates
requirements for conformant documents, conformant applications that read,
conformant applications that write, and conformant applications that both
read and write.  

As you may already know, OASIS has added
a new requirement for all OASIS standards:

"A specification that is approved
by the TC at the Public Review Draft, Committee Specification or OASIS
Standard level must include a separate section, listing a set of numbered
conformance clauses, to which any implementation of the specification must
adhere in order to claim conformance to the specification (or any optional
portion thereof) "

When we make the changes required for
the new OASIS rules, I suggest we think about conformance in general, and
consider making a more substantial statement. For example, we could define
things at a more granular level:  a conformant ODF spreadsheet shall
support workbooks of at least a single sheet, with at least 100 rows and
25 columns and at least the Group 1 spreadsheet functions.  (Just
an example, not a real proposal).  So we have the opportunity to specify
multiple levels of conformance, either in the main text, or as separate
profiles.

To the specific question at hand, I
am concerned with the loose use of the word "preserve."  What
exactly does that mean?  For example, must the xml:id's of the saved
document be lexically identical to the read document?  Or are looser
version of equivalence allowed?  For example, if the id originally
is "foo" and then it is saved with the id "bar" is
that permitted, provided that the structure and referential integrity of
the id and references are maintained?   Remember, it will be common
for an application to read an XML document and convert id's and links into
internal runtime representations that are not at all similar to the XML.
  Id/references might be converted into C-language pointer references
between objects, etc.  Then when writing out the document, new unique
ID's might be generated on-the-fly, perhaps in sequential order.  This
might vary from implementation to implementation.  Beyond referential
integrity, I don't know if there is any additional value in saying that
a document created in KOffice must have identical ID labels when that document
is later saved in OpenOffice.  

We should also note that it is a feature
of some programs, such as Office 2007, to have a menu item specifically
for removing metadata from a document, for privacy and security reasons.
 I don't think we want to prevent such an application from claiming
conformance.

So we need to be need to be very careful
how we word this.  Perhaps something like "Conforming applications
that read and write documents shall be capable of "preserving"
xml:id's, etc."  With the proviso that "preserving"
needs a better definition, this ensures that conforming applications support
preservation, while also allowing that not every mode of use may actually
do so, such as when a user deletes content or metadata, etc.

-Rob
________________


Rob Weir

Software Architect

Workplace, Portal and Collaboration Software

IBM Software Group


email: robert_weir@us.ibm.com

phone: 1-978-399-7122

blog: http://www.robweir.com/blog/

On Jul 1, 2007, at 4:54 PM, robert_weir@us.ibm.com wrote:

> To the specific question at hand, I am concerned with the loose use of 
> the word "preserve."  What exactly does that mean?  For example, must 
> the xml:id's of the saved document be lexically identical to the read 
> document?  Or are looser version of equivalence allowed?  For example, 
> if the id originally is "foo" and then it is saved with the id "bar" 
> is that permitted, provided that the structure and referential 
> integrity of the id and references are maintained?  

 From the perspective of the metadata proposal, the xml:id values are 
not important because of the binding abstraction. What is important is 
that the bindings remain valid (e.g. that it referential integrity be 
maintained).

So in your example, if we have:

	

.. and then in manifest.rdf the binding:

	

On Sunday 01 July 2007 23:37:31 Bruce D'Arcus wrote:
> But if a user creates this document in application A and sends it to a
> colleague using application B: if that second application silently
> removes those metadata attributes and the first user gets it back with
> that information stripped, that's bad, and should -- I believe -- be
> discouraged by this TC.

I have seen nobody that has a problem with such a thing. I certainly agree 
silently losing data is bad and should be discouraged.
The question is what can the TC can do that will not harm real use cases 
and which actually will have the effect we are after.
So far I am unconvinced that the suggestions made to discourage this 
losing of data satisfy these requirements.

And if the suggestion does not satisfy the above two requirements, then it 
will do more harm to ODF than good.
At this point I don't have a suggestion what would be a good solution to 
the problem, except for the one about market forces. Which I understand 
is a bit of a leap of faith for some.
-- 
Thomas Zander

On 7/1/07, Thomas Zander <zander@kde.org> wrote:

On Sunday 01 July 2007 23:37:31 Bruce D'Arcus wrote:
> But if a user creates this document in application A and sends it to a
> colleague using application B: if that second application silently
> removes those metadata attributes and the first user gets it back with
> that information stripped, that's bad, and should -- I believe -- be
> discouraged by this TC.

I have seen nobody that has a problem with such a thing. 

Then let me give you a concrete example. Sun's apps destroy all foreign elements and attributes other than paragraphs and text spans. That stymied the Foundation's ability to establish non-lossy interoperability between MS Office and OOo. The Foundation now plans to use the Metadata SC's work, but Sun is in here pitching for permission to destroy xml:id attributes.

And there was this interesting exchange of private emails between Gary Edwards and Michael Brauer in May of this year:

Gary: "The change in ODF 1.2 that will matter most will be other ODF ready applications "preserving" these 
foreign elements when and if they ever come across them in a document. "

Michael: :I'm not sure if one can say that this way. What will be new in ODF 
1.2
is that metadata aware application should preserve metadata (where
possible and reasonable), where metadata is contained in some new

RDF-XML metadata streams, or some some new elements and attributes that we will add the schema. ***The specification will not make any other assumptions about 
foreign elements (in the meaning of elements not defined by the schema) than those we have already."***
 
 
So King Michael had already decreed in May that metadata only "should" be preserved and that there will be no change requiring preservation of foreign elements.

But I have yet to hear a rational explanation of why preservation of metadata should be permissive if it **should** be preserved. Why is it so important to Sun that the specification allow the destruction of xml:id attributes. Why do they want that permission if they have no intent of exercising it? I've asked for use cases exposing the need, but none have been forthcoming.


I certainly agree
silently losing data is bad and should be discouraged.

Why just discouraged? Why not prohibited? Why should an application be entitled to a conformance title if it silently destroys data? What does that do to the respect for the standard and for the quality of this committee's work?
 

The question is what can the TC can do that will not harm real use cases
and which actually will have the effect we are after.

So offer us some use cases and tell us the effect you want.
 


So far I am unconvinced that the suggestions made to discourage this
losing of data satisfy these requirements.

I can well imagine since you have identified no requirements.


And if the suggestion does not satisfy the above two requirements, then it
will do more harm to ODF than good.
At this point I don't have a suggestion what would be a good solution to
the problem, except for the one about market forces. Which I understand
is a bit of a leap of faith for some.

More than a leap of faith, a leap past common sense.  Or maybe you missed King Brauer's suggestion that we put off all interoperability work until ODF 1.3
 and make destruction of xml:id attributes optional, with nary a single use case exposing why we should do so.
 
Or maybe you just don't care about being able to round-trip documents with other ODF apps with high fidelity? Draw some guidance from ECIS, Thomas, which speaks for Sun and IBM at DG Competition:

""Interoperability is a cornerstone of the ICT industry. In today's networked ICT environments, devices do not function purely on their own, but must interact with other programs and devices. A device that cannot interoperate with the other products with which consumers expect it to interoperate is essentially worthless. It is interoperability that drives competition on the merits and innovation. The ability of different computer products to interoperate allows consumers to choose among them. Because consumers can choose among them, interoperable products must compete with one another, and it is this competition that has driven innovation in the software industry."

<http://www.ecis.eu/inter/index.html>

Of course that didn't stop Sun/IBM/ECIS from falsely claiming:

"The merits of ODF have already been established by its wide industry adoption. As noted above, numerous PPA vendors have implemented support for it in their products both on Windows and on other operating systems. Such widespread adoption is only possible ***because ODF is fully disclosed and created to allow for document interoperability by making it easy for various applications to exchange documents with full fidelity, 
i.e., without any loss of data or formatting of the document."***

<http://www.ecis.eu/inter/documents/ECIS_ODF_OOXML_comparison.pdf
>

Really easy, especially if applications are allowed to destroy foreign elements and attributes and metadata. Not.

On Monday 02 July 2007 01:27:11 marbux wrote:
> if that second application silently
>
> > > removes those metadata attributes and the first user gets it back
> > > with that information stripped, that's bad, and should -- I believe
> > > -- be discouraged by this TC.
> >
> > I have seen nobody that has a problem with such a thing.
>
> Then let me give you a concrete example.

You misread my mail, to repeat; nobody has a problem with the TC 
discouraging the data loss issue.  Or in other words (remove double 
negative). Everyone agrees the issue is real and a problem we may want to 
remedy, if possible.
Really, we are on the same side. Please stop fighting like we are not.

> Sun's apps destroy all foreign 
> elements and attributes other than paragraphs and text spans. That
> stymied the Foundation's ability to establish non-lossy
> interoperability between MS Office and OOo. The Foundation now plans to
> use the Metadata SC's work, but Sun is in here pitching for permission
> to destroy xml:id attributes.

Well, I think your personal attachment to the foundation plugin is 
coloring your judgment. Its ok to have an interrest, but you are pulling 
everything that could possibly hurt your vested interrests into the far 
negative.  It really makes your opinions hard to take seriously. After 
all, this is not a political party, this is a technical committee.

> > I certainly agree
> > silently losing data is bad and should be discouraged.
>
> Why just discouraged? Why not prohibited? 

Have you read any of my mails on this thread?  I've gone over that again 
and again...
In short; doing so will seriously hurt ODF adoption in new and unheard of 
applications.

> > The question is what can the TC can do that will not harm real use
> > cases
> > and which actually will have the effect we are after.
>
> So offer us some use cases and tell us the effect you want.

Same here, there have been enough on this thread.

-- 
Thomas Zander

On 7/1/07, Thomas Zander <zander@kde.org> wrote:

On Monday 02 July 2007 01:27:11 marbux wrote:
> if that second application silently
>
> > > removes those metadata attributes and the first user gets it back
> > > with that information stripped, that's bad, and should -- I believe
> > > -- be discouraged by this TC.
> >
> > I have seen nobody that has a problem with such a thing.
>
> Then let me give you a concrete example.

You misread my mail, to repeat; nobody has a problem with the TC
discouraging the data loss issue.  Or in other words (remove double
negative). Everyone agrees the issue is real and a problem we may want to
remedy, if possible.
Really, we are on the same side. Please stop fighting like we are not.

> Sun's apps destroy all foreign
> elements and attributes other than paragraphs and text spans. That
> stymied the Foundation's ability to establish non-lossy
> interoperability between MS Office and OOo. The Foundation now plans to
> use the Metadata SC's work, but Sun is in here pitching for permission
> to destroy xml:id attributes.

Well, I think your personal attachment to the foundation plugin is
coloring your judgment. Its ok to have an interrest, but you are pulling
everything that could possibly hurt your vested interrests into the far
negative.  


That's the second time in 24 hours I've been told I have a vested interest in the Foundation's plug-in. For the record, I have no financial interest in any software or software project whatsoever. My only source of revenue is a vested retirement stipend. I am financially beholden to no person or entity on this planet.

It really makes your opinions hard to take seriously. After
all, this is not a political party, this is a technical committee.

You have a very naive view of what goes on in this TC. It is political in the extreme.
 


> > I certainly agree
> > silently losing data is bad and should be discouraged.
>
> Why just discouraged? Why not prohibited?

Have you read any of my mails on this thread?  I've gone over that again
and again...
In short; doing so will seriously hurt ODF adoption in new and unheard of
applications.

How specifically? I've heard nothing but platitudes from you on this, not a single use case.
 

> > The question is what can the TC can do that will not harm real use

> > cases
> > and which actually will have the effect we are after.
>
> So offer us some use cases and tell us the effect you want.

Same here, there have been enough on this thread.

But none from you. I've offered the real-life use case of Sun's apps breaking high-fidelity interoperability with Microsoft Office by destroying foreign elements and attributes.  How do your "market forces" cure that problem? My "market force" would be a conformance requirement that gives Sun the choice between fixing the problem or being ineligible for government software procurement tenders because of non-conformance with the standard.


 

Thomas Zander wrote:
> On Sunday 01 July 2007 23:37:31 Bruce D'Arcus wrote:
>> But if a user creates this document in application A and sends it to a
>> colleague using application B: if that second application silently
>> removes those metadata attributes and the first user gets it back with
>> that information stripped, that's bad, and should -- I believe -- be
>> discouraged by this TC.
> 
> I have seen nobody that has a problem with such a thing. I certainly agree 
> silently losing data is bad and should be discouraged.
> The question is what can the TC can do that will not harm real use cases 
> and which actually will have the effect we are after.

I agree to what Thomas says here. Silently removing metadata is bad, and 
should be discouraged. It may be that I change my mind if this 
interesting discussion continues, but right now, I believe that the best 
solution is to state in some guidelines that metadata should not be 
silently removed.

Michael


-- 
Michael Brauer, Technical Architect Software Engineering
StarOffice/OpenOffice.org
Sun Microsystems GmbH             Nagelsweg 55
D-20097 Hamburg, Germany          michael.brauer@sun.com
http://sun.com/staroffice         +49 40 23646 500
http://blogs.sun.com/GullFOSS

Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1,
	   D-85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering

On 7/1/07, robert_weir@us.ibm.com <robert_weir@us.ibm.com> wrote:

I suppose I should throw in my $.02.

First, we should remember that ODF mandates
behavior at several levels.  The schema itself encodes requirements
in terms of what elements or attributes are optional or mandatory, what
nesting is permitted, what restrictions there are on data types, etc.  
And then the normative text of the standard, along with external normative
references, make additional provisions by the use of "shall"
and "shall not".  

But virtually all are undercut by the following sentence in the conformance section:

"There are ***no rules regarding the elements and
attributes that actually have to be supported by conforming
applications,*** except that applications should not use foreign
elements and attributes for features defined in the OpenDocument
schema."
 

But note that in that case,the provision
is only applicable to those who implement that feature.  A "shall"
concerning the calculation of the SUM() spreadsheet function may be totally
ignored by someone who is implementing a word processor only.  Finally,
we have the conformance clause, that defines which features and additional
constraints are required for conformance with the standard.

Today our conformance clause designates
requirements for conformant documents, conformant applications that read,
conformant applications that write, and conformant applications that both
read and write.  

We have very few conformance *requirements,* in the sense of mandatory requirements. Here is the sum total:

>>>

Documents that conform to the OpenDocument
specification may contain
elements and attributes not specified within the OpenDocument
schema. Such elements and attributes **must not** be part of a
namespace that is defined within this specification and are called
foreign elements and attributes.
 
...

Conforming applications either **shall** read documents that are valid
against the OpenDocument schema if all foreign elements and
attributes are removed before validation takes place, or **shall** write documents that are valid
against the OpenDocument schema if all foreign elements and
attributes are removed before validation takes place.

...

Foreign elements may have an office:process-content attribute
attached that has the value true or false. If the attribute's value is
true, or if the attribute
does not exist, the element's content should be processed by conforming
applications. Otherwise conforming applications should not process the element's
content, but may only
preserve its content. If the element's content should be processed,
the document itself ***shall***
be valid against the OpenDocument schema if the unknown element is
replaced with its content only.

Conforming applications ***shall*** read documents containing
processing instructions and should preserve them.

<<<

We should also realize that all of those "may" and "optional" requirements keywords changed their meaning between ODF 
1.0 and 1.1. In ODF 1.0, they meant:

>>>

5. MAY   This word, or the adjective "OPTIONAL", mean that an item is truly optional.  One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)

<http://www.ietf.org/rfc/rfc2119.txt>. This is the definition used by nearly all OASIS standards.

<<<

At ISO's request, that definition changed to:

>>>

The verbal forms shown in Table G.3 shall be used to indicate a course of action permissible
within the limits of the document.

Table G.3 — Permission


    Verbal form
    Equivalent expressions for use in exceptional cases
    (see 6.6.1.3)

may
    is permitted
    is allowed
    is permissible
   
need not
    it is not required that
    no … is required

Do not use "possible" or "impossible" in this context.

Do not use "can" instead of "may" in this context.

NOTE 1
"May" signifies permission expressed by the  document, whereas "can"
refers to the ability of a user of the document or to a possibility open to him/her.

NOTE 2
The French verb "pouvoir" can indicate both permission and possibility.
For clarity, the use of other expressions is advisable if otherwise there is a risk of misunderstanding.

<<<

<http://72.14.253.104/search?q=cache:DxJI76h9l8QJ:www.iec.ch/tiss/iec/Directives-Part2-Ed4.pdf+nnex+H+of+%5BISO/IEC+Directives&hl=en&ct=clnk&cd=1&gl=us
>, pg. 62.

So in ODF 1.0 the keywords "may" and "optional" imported a requirement of interoperability. In ODF 1.1, that requirement disappeared with the stroke of a pen. My reading of the ISO directives suggests that we do not have the option of going back to the RFC 2119 definitions. But nonetheless it is my understanding that the TC did not study the impact of the change in requirements keyword definitions before making the change.

For example, the use of the word "may" in the preservation of foreign elements and attributes section would at least arguably, under the RFC 2119 definition, **require** preservation of foreign elements and attributes needed for interoperability purposes whether or not an application supported foreign elements and attributes.

But I think it might fly with ISO to use the RFC 2119 definition of "may" and "optional" in the conformance section alone and that might put us further down the road toward interoperability.


As you may already know, OASIS has added
a new requirement for all OASIS standards:

"A specification that is approved
by the TC at the Public Review Draft, Committee Specification or OASIS
Standard level must include a separate section, listing a set of numbered
conformance clauses, to which any implementation of the specification must
adhere in order to claim conformance to the specification (or any optional
portion thereof) "

I think thisis particularly important because procurement officers want to be able to simply specify that a candidate application must produce conformant format X. They do not want to, in effect, have to write their own file format specifications
 

When we make the changes required for
the new OASIS rules, I suggest we think about conformance in general, and
consider making a more substantial statement. For example, we could define
things at a more granular level:  a conformant ODF spreadsheet shall
support workbooks of at least a single sheet, with at least 100 rows and
25 columns and at least the Group 1 spreadsheet functions.  (Just
an example, not a real proposal).  So we have the opportunity to specify
multiple levels of conformance, either in the main text, or as separate
profiles.

+1. I'd add that we should approach such issues with suspicion that every option is a potential interoperability breakpoint.


To the specific question at hand, I
am concerned with the loose use of the word "preserve."  What
exactly does that mean?  For example, must the xml:id's of the saved
document be lexically identical to the read document?  Or are looser
version of equivalence allowed?  For example, if the id originally
is "foo" and then it is saved with the id "bar" is
that permitted, provided that the structure and referential integrity of
the id and references are maintained?   Remember, it will be common
for an application to read an XML document and convert id's and links into
internal runtime representations that are not at all similar to the XML.
  Id/references might be converted into C-language pointer references
between objects, etc.  Then when writing out the document, new unique
ID's might be generated on-the-fly, perhaps in sequential order.  This
might vary from implementation to implementation.  Beyond referential
integrity, I don't know if there is any additional value in saying that
a document created in KOffice must have identical ID labels when that document
is later saved in OpenOffice.  

I do not have the technical knowledge to answer that question. However, I request that we approach the issue from recognition that a document may pass through many applications before wending its way back  to the originating application. From a layman's view, it would seem that a shifting vocabulary would interfere with interoperability mightily in situations where it is unknown what application will be the next to process a document.  

We should also note that it is a feature
of some programs, such as Office 2007, to have a menu item specifically
for removing metadata from a document, for privacy and security reasons.
 I don't think we want to prevent such an application from claiming
conformance.

Wouldn't an exception for user initiated actions cover this situation?


So we need to be need to be very careful
how we word this.  Perhaps something like "Conforming applications
that read and write documents shall be capable of "preserving"
xml:id's, etc."  With the proviso that "preserving"
needs a better definition, this ensures that conforming applications support
preservation, while also allowing that not every mode of use may actually
do so, such as when a user deletes content or metadata, etc.

I'm not sure that "capable" helps a lot. E.g., if an application is capable of preserving metadata but ships with that option turned off and an arcane set of keystrokes to enable the option known only to the developers, the app is still "capable" of preserving metadata. Maybe call that an Easter Egg optional setting.
 
While on the subject of the conformance section and requirements keywords, we have another problem to deal with. The Notation section currently reads: "

Within this
specification, the key words "shall",
"shall not", "
should", "should not"
and "may" are to be interpreted as described in Annex H of [ISO/IEC
Directives] ***if they appear in bold letters.*** Between ODF 1.0 and ODF 1.1, many of the keywords lost their boldfacing. I suspect that is because we tend to bat language back and forth in plain text email, which strips text attributes.
1. We could avoid much of that kind of problem in the future if we switched to keywords in all cap rather than bold face, since they will remain all caps in emails.
2. Does anyone know if their are any instances of the keywords that should ***not*** be boldfaced (or all caps)? If not, we have a simple global search and replace task. If so, we have a tedious review ahead of us.

I think you're reading too much into
the IETF's definition of MAY.  It explicitly says that a vendor is
permitted to omit the item, though it must accommodate itself and degrade
functionality as necessary.  What is not permitted is that the application
utterly crash when presented with an item it does not understand.  
At least that is the way it works for the IETF standards I'm familiar with.


Although intuitively we want to say,
"Preserve metadata unless the user explicitly intended otherwise,"
I don't see how to express this in standards terms.  We can't have
a conformance depend on "user intent".  And reference to
a user doesn't help. Documents can be processed by automation, and I think
we would equally be unhappy if metadata were arbitrarily stripped there.
 In any case, I think we need to work along the lines of "shall
be capable of" or "shall allow at least one mode of operation
where" or something like that.   That would be testable.  

You suggested that a devious implementation
might makes this mode of operation hard to find in order to hurt interoperability.
  But then I could also suggest a devious user who arbitrarily deletes
metadata in order to hurt interoperabiity.  I'm not sure a document
format standard can prevent either.  

-Rob


marbux <marbux@gmail.com> wrote on 07/01/2007
06:31:11 PM:


>


> On 7/1/07, robert_weir@us.ibm.com <robert_weir@us.ibm.com>
wrote: 
>

> I suppose I should throw in my $.02.

>

> First, we should remember that ODF mandates behavior at several

> levels.  The schema itself encodes requirements in terms of what

> elements or attributes are optional or mandatory, what nesting is

> permitted, what restrictions there are on data types, etc.  
And

> then the normative text of the standard, along with external

> normative references, make additional provisions by the use of

> "shall" and "shall not".  
>

> But virtually all are undercut by the following sentence in the

> conformance section:

>

> "There are ***no rules regarding the elements and attributes
that

> actually have to be supported by conforming applications,*** except

> that applications should not use foreign elements and attributes for

> features defined in the OpenDocument schema."

>  
>

> But note that in that case,the provision is only applicable to those

> who implement that feature.  A "shall" concerning the
calculation of

> the SUM() spreadsheet function may be totally ignored by someone who

> is implementing a word processor only.  Finally, we have the

> conformance clause, that defines which features and additional

> constraints are required for conformance with the standard.

>

> Today our conformance clause designates requirements for conformant

> documents, conformant applications that read, conformant

> applications that write, and conformant applications that both read

> and write.  
>

> We have very few conformance *requirements,* in the sense of

> mandatory requirements. Here is the sum total:

>

> >>>

>

> Documents that conform to the OpenDocument specification may contain

> elements and attributes not specified within the OpenDocument

> schema. Such elements and attributes **must not** be part of a

> namespace that is defined within this specification and are called

> foreign elements and attributes.

>  

> ...

>

> Conforming applications either **shall** read documents that are

> valid against the OpenDocument schema if all foreign elements and

> attributes are removed before validation takes place, or **shall**

> write documents that are valid against the OpenDocument schema if

> all foreign elements and attributes are removed before validation
takes place.

>

> ...

>

> Foreign elements may have an office:process-content attribute

> attached that has the value true or false. If the attribute's value
is true

> , or if the attribute does not exist, the element's content should

> be processed by conforming applications. Otherwise conforming applications

> should not process the element's content, but may only preserve its

> content. If the element's content should be processed, the document
itself ***

> shall*** be valid against the OpenDocument schema if the unknown

> element is replaced with its content only.


> Conforming applications ***shall*** read documents
containing

> processing instructions and should preserve them.

>

> <<<

>

> We should also realize that all of those "may" and "optional"

> requirements keywords changed their meaning between ODF 1.0 and 1.1.

> In ODF 1.0, they meant:

>

> >>>

>

> 5. MAY   This word, or the adjective "OPTIONAL", mean
that an item

> is truly optional.  One vendor may choose to include the item

> because a particular marketplace requires it or because the vendor

> feels that it enhances the product while another vendor may omit the

> same item. An implementation which does not include a particular

> option MUST be prepared to interoperate with another implementation

> which does include the option, though perhaps with reduced

> functionality. In the same vein an implementation which does include

> a particular option MUST be prepared to interoperate with another

> implementation which does not include the option (except, of course,

> for the feature the option provides.)

>

> <http://www.ietf.org/rfc/rfc2119.txt>. This is the definition
used

> by nearly all OASIS standards.

>

> <<<

>

> At ISO's request, that definition changed to:

>

> >>>

>

> The verbal forms shown in Table G.3 shall be used to indicate a

> course of action permissible

> within the limits of the document.

>

> Table G.3 — Permission

>

>

>     Verbal form

>     Equivalent expressions for use in exceptional cases

>     (see 6.6.1.3)

>

> may

>     is permitted

>     is allowed

>     is permissible

>    

> need not

>     it is not required that

>     no … is required

>

> Do not use "possible" or "impossible" in this
context.

>

> Do not use "can" instead of "may" in this context.

>

> NOTE 1

> "May" signifies permission expressed by the  document,
whereas "can"

> refers to the ability of a user of the document or to a possibility

> open to him/her.

>

> NOTE 2

> The French verb "pouvoir" can indicate both permission and
possibility.

> For clarity, the use of other expressions is advisable if otherwise

> there is a risk of misunderstanding.

>

> <<<

>

> <http://72.14.253.104/search?q=cache:DxJI76h9l8QJ:www.iec.

> ch/tiss/iec/Directives-Part2-Ed4.pdf+nnex+H+of+%

> 5BISO/IEC+Directives&hl=en&ct=clnk&cd=1&gl=us >,
pg. 62.

>

> So in ODF 1.0 the keywords "may" and "optional"
imported a

> requirement of interoperability. In ODF 1.1, that requirement

> disappeared with the stroke of a pen. My reading of the ISO

> directives suggests that we do not have the option of going back to

> the RFC 2119 definitions. But nonetheless it is my understanding

> that the TC did not study the impact of the change in requirements

> keyword definitions before making the change.

>

> For example, the use of the word "may" in the preservation
of

> foreign elements and attributes section would at least arguably,

> under the RFC 2119 definition, **require** preservation of foreign

> elements and attributes needed for interoperability purposes whether

> or not an application supported foreign elements and attributes.

>

> But I think it might fly with ISO to use the RFC 2119 definition of

> "may" and "optional" in the conformance section
alone and that might

> put us further down the road toward interoperability.

>


> As you may already know, OASIS has added a new
requirement for all

> OASIS standards:

>

> "A specification that is approved by the TC at the Public Review

> Draft, Committee Specification or OASIS Standard level must include

> a separate section, listing a set of numbered conformance clauses,

> to which any implementation of the specification must adhere in

> order to claim conformance to the specification (or any optional

> portion thereof) "
>

> I think thisis particularly important because procurement officers

> want to be able to simply specify that a candidate application must

> produce conformant format X. They do not want to, in effect, have
to

> write their own file format specifications

>  
>

> When we make the changes required for the new OASIS rules, I suggest

> we think about conformance in general, and consider making a more

> substantial statement. For example, we could define things at a more

> granular level:  a conformant ODF spreadsheet shall support

> workbooks of at least a single sheet, with at least 100 rows and 25

> columns and at least the Group 1 spreadsheet functions.  (Just
an

> example, not a real proposal).  So we have the opportunity to

> specify multiple levels of conformance, either in the main text, or

> as separate profiles.
>

> +1. I'd add that we should approach such issues with suspicion that

> every option is a potential interoperability breakpoint. 
>

> To the specific question at hand, I am concerned with the loose use

> of the word "preserve."  What exactly does that mean?
 For example,

> must the xml:id's of the saved document be lexically identical to

> the read document?  Or are looser version of equivalence allowed?
 

> For example, if the id originally is "foo" and then it is
saved with

> the id "bar" is that permitted, provided that the structure
and

> referential integrity of the id and references are maintained?  

> Remember, it will be common for an application to read an XML

> document and convert id's and links into internal runtime

> representations that are not at all similar to the XML.  

> Id/references might be converted into C-language pointer references

> between objects, etc.  Then when writing out the document, new

> unique ID's might be generated on-the-fly, perhaps in sequential

> order.  This might vary from implementation to implementation.
 

> Beyond referential integrity, I don't know if there is any

> additional value in saying that a document created in KOffice must

> have identical ID labels when that document is later saved in OpenOffice.
 
>

> I do not have the technical knowledge to answer that question.

> However, I request that we approach the issue from recognition that

> a document may pass through many applications before wending its way

> back  to the originating application. From a layman's view, it
would

> seem that a shifting vocabulary would interfere with

> interoperability mightily in situations where it is unknown what

> application will be the next to process a document.   
>

> We should also note that it is a feature of some programs, such as

> Office 2007, to have a menu item specifically for removing metadata

> from a document, for privacy and security reasons.  I don't think
we

> want to prevent such an application from claiming conformance.
>

> Wouldn't an exception for user initiated actions cover this situation?


> So we need to be need to be very careful how
we word this.  Perhaps

> something like "Conforming applications that read and write

> documents shall be capable of "preserving" xml:id's, etc."
 With the

> proviso that "preserving" needs a better definition, this
ensures

> that conforming applications support preservation, while also

> allowing that not every mode of use may actually do so, such as when

> a user deletes content or metadata, etc.
>

> I'm not sure that "capable" helps a lot. E.g., if an application
is

> capable of preserving metadata but ships with that option turned off

> and an arcane set of keystrokes to enable the option known only to

> the developers, the app is still "capable" of preserving
metadata.

> Maybe call that an Easter Egg optional setting.

>  
> While on the subject of the conformance section
and requirements

> keywords, we have another problem to deal with. The Notation section

> currently reads: "
> Within this specification, the key words "shall",
"shall not", " should", "

> should not" and "may" are to be interpreted as described
in Annex H

> of [ISO/IEC Directives] ***if they appear in bold letters.***

> Between ODF 1.0 and ODF 1.1, many of the keywords lost their

> boldfacing. I suspect that is because we tend to bat language back

> and forth in plain text email, which strips text attributes. 
> 1. We could avoid much of that kind of problem
in the future if we

> switched to keywords in all cap rather than bold face, since they

> will remain all caps in emails. 
> 2. Does anyone know if their are any instances
of the keywords that

> should ***not*** be boldfaced (or all caps)? If not, we have a

> simple global search and replace task. If so, we have a tedious

> review ahead of us.

>

On 7/1/07, robert_weir@us.ibm.com <robert_weir@us.ibm.com> wrote:

I think you're reading too much into
the IETF's definition of MAY.  It explicitly says that a vendor is
permitted to omit the item, though it must accommodate itself and degrade
functionality as necessary.  What is not permitted is that the application
utterly crash when presented with an item it does not understand.  
At least that is the way it works for the IETF standards I'm familiar with.

We live in a world of ambiguities. :-) I can't testify to customary practice, but what you describe is hard to square with the specific language of the definition. E.g., it's hard for me to equate "not crashing" with "interoperate."
 

Although intuitively we want to say,
"Preserve metadata unless the user explicitly intended otherwise,"
I don't see how to express this in standards terms.  We can't have
a conformance depend on "user intent".  And reference to
a user doesn't help. Documents can be processed by automation, and I think
we would equally be unhappy if metadata were arbitrarily stripped there.
 In any case, I think we need to work along the lines of "shall
be capable of" or "shall allow at least one mode of operation
where" or something like that.   That would be testable.  

The XML 1.0 standard takes this approach, referring back to the RFC 2119 definition of "may" and "optional:"
at user option

									[Definition: Conforming software
MAY or MUST (depending on the modal verb in the sentence) behave as described;
if it does, it MUST provide users a means to enable or disable the behavior
described.]
								
<http://www.w3.org/TR/REC-xml/#sec-terminology>. The "at user option" phrase is heavily used in that specification.
 

So taking a first cut at some language for the conformance section (not a proposal, just for discussio purposes), we might do something like this:

Conformant applications MAY preserve  meta information and processing instructions, including but not limited to foreign elements and attributes but SHALL do so to the extent needed for lossless round trip interoperability with all other conformant applications. However, the application must provide users with the means to disable the preservation of such information. This section shall not be interpreted as meaning that an application or routine designed expressly to strip meta information  at the users' option.


You suggested that a devious implementation
might makes this mode of operation hard to find in order to hurt interoperability.
  But then I could also suggest a devious user who arbitrarily deletes
metadata in order to hurt interoperabiity.  I'm not sure a document
format standard can prevent either.  

Me either. But a standard can deny such an app the right to be branded as conformant, which should result in ineligibilty for software procurement tenders. And my Easter Egg example was intended as an extreme example, a caricature if you will of MS Office setting MOOXML as the default file save format. How many users will ever bother to change that setting? And how many MOOXML files wind up in circulation because of it.

I'll try to give an example from the HTTP spec: 
http://www.w3.org/Protocols/rfc2616/rfc2616.html

In 14.15 Content-MD5 it is stated that servers MAY generate the 
Content-MD5 header field. (and that proxies obviously MUST NOT generate 
it). It further states that "[a]ny recipient of the entity- body, 
including gateways and proxies, MAY check that the digest value in this 
header field matches that of the entity-body as received."

Consequently the client is not under any obligation to do anything with 
this field. Whether it emits a warning if the check fails or whether it 
implements such a check at all is up to the discretion of the client's 
implementor.

Thus, as Rob already stated, the specification informs the client 
implementor that a conforming server might send such a header. Whether 
he decides to act upon the information contained in the header has no 
impact on the actual conformance of that implementation.

/Lars

marbux wrote:
> 
> 
> On 7/1/07, *robert_weir@us.ibm.com 

On 6/30/07, Patrick Durusau <patrick@durusau.net> wrote:

Thomas,

I am curious about the term "support" in this context.

Does preservation of data imply that a feature is supported?


FWIW, the RFC 2119 definition of "may" that applied to ODF 
1.0  draws a distinction between implementation and interoperability, the latter of which can obviously require  preservation of data:

"5. MAY This word, or the adjective "OPTIONAL", mean that an item is truly optional. One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.) "

<http://www.ietf.org/rfc/rfc2119.txt>


In other words, what if I have a very lite weight application that
doesn't actually read any of the content of an ODF file or parse any of
the existing metadata files but simply adds an entry to the metadata

manifest to add metadata about the document as a whole (such as would
happen in a clerk of court's office with filing date, who filed it,
etc.) and added an appropriate metadata file. It then saves *all* of the

file.

Granted, I haven't implemented a lot of the features offered by ODF but
I have preserved the file, even for features I don't support.

Is there a useful distinction to make between implementation of a
feature and preservation of a file's contents that you are given for
processing?

There certainly is at least for the Foundation's MS Office ODF plug-in. It implements no ODF features whatsoever, merely converts between one format and another. And to do so, preservation of metadata that can not be mapped is essential to its round-tripping of documents.
 
Likewise, I know of only two methods for allowing the less featureful ODF apps to round-trip documents with the more featureful ODF apps. One is to declare interoperability subsets and create corresponding compatibility modes in the apps at both ends of the round-trip. The other is to require the preservation of unsupported metadata.

Granting that I could also implement a very lite weight application that
doesn't offer metadata or any number of other features since we don't
require an application to implement any specific set of features.

So, is preservation separate from feature implementation? (I am asking
because I haven't seen it raised in this context and while it seems
evident to me the two are different, opinions may and probably do vary
on that point.)

Hope you are having a great day!

Patrick

Thomas Zander wrote:
> On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
>
>>> Therefore it seems best to keep these rules optional unless the
>>> application plan to implement the metadata feature.
>>> And when I mentioned ODF applications, I do not meant 
OpenOffice.org,
>>> for which it won't be a problem as we desire the feature.
>>>
>> I strongly disagree. Preserving files and attributes is a trivial
>> requirement. Not doing so will introduce large compatibility problems.
>>
>
> This, naturally, is not an isolated case. There currently are no
> applications that support all the ODF features but they still can be ODF
> compliant.  Even should one loose 90% of the features between loading and
> saving.
>
> So the question becomes, is metadata any different. I'm inclined to say
> no.  Its not different, its just a feature that the application should
> support. If it doesn't then a better application comes along and replaces
> it.
> Looking at the ODF landscape I consider this;
> * OOo and KOffice both think its a good feature to support, and are both
> moving to do so.  I'm sure other big office suites agree and do so as
> well.  Meaning that ODF doesn't have to force their hands.
> * Almost all professional users of office suites benefit from metadata in
> one way or another. So an application that follows user requests will
> quickly see metadata on their TODO list.
> * There are not that many application-types that both load and save ODF
> files. Viewers and writers are the bigger segment. We are only concerned
> with the load+save type.
>
> Bottom line is that while I fully agree its very important to retain this
> information that does not automatically imply that ODF has to make it
> mandatory. Market forces can do that too.

> Furthermore I would find it hypocritical to make it mandatory while almost
> all features in ODF are not mandatory to be retained between load and
> saves.
>
>
>> Really, just to be clear: if applications do not preserve xml:id
>> attributes, fields will break, and any metadata about document
>> fragments will be made invalid. Is that really in anyone's interest?
>> They need not support metadata in any explicit way to do this.
>>
>
> I think this said is best; things will break in a very visible way if apps
> don't support it.  So users will quickly realize a problem and apps will
> be requested to support the feature of metadata.
>
> The only reason I see to make it mandatory is for fear of people not
> supporting it, and I can easy your mind there, you guys have been doing
> an awesome job. Applications will line up to support this stuff. Don't
> worry so much! :)
>
>
> ps. not sure if I have write access to the metadata ML, if this doesn't
> appear there, please consider forwarding. Thanks.
>

--
Patrick Durusau
patrick@durusau.net
Chair, V1 - US TAG to JTC 1/SC 34
Acting Convener, JTC 1/SC 34/WG 3 (Topic Maps)
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Co-Editor, OpenDocument Format (OASIS, ISO/IEC 26300)

On 6/30/07, Patrick Durusau <patrick@durusau.net> wrote:

Thomas,

I am curious about the term "support" in this context.

Does preservation of data imply that a feature is supported?


FWIW, the RFC 2119 definition of "may" that applied to ODF 
1.0  draws a distinction between implementation and interoperability, the latter of which can obviously require  preservation of data:

"5. MAY This word, or the adjective "OPTIONAL", mean that an item is truly optional. One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.) "

<http://www.ietf.org/rfc/rfc2119.txt>


In other words, what if I have a very lite weight application that
doesn't actually read any of the content of an ODF file or parse any of
the existing metadata files but simply adds an entry to the metadata

manifest to add metadata about the document as a whole (such as would
happen in a clerk of court's office with filing date, who filed it,
etc.) and added an appropriate metadata file. It then saves *all* of the

file.

Granted, I haven't implemented a lot of the features offered by ODF but
I have preserved the file, even for features I don't support.

Is there a useful distinction to make between implementation of a
feature and preservation of a file's contents that you are given for
processing?

There certainly is at least for the Foundation's MS Office ODF plug-in. It implements no ODF features whatsoever, merely converts between one format and another. And to do so, preservation of metadata that can not be mapped is essential to its round-tripping of documents.
 
Likewise, I know of only two methods for allowing the less featureful ODF apps to round-trip documents with the more featureful ODF apps. One is to declare interoperability subsets and create corresponding compatibility modes in the apps at both ends of the round-trip. The other is to require the preservation of unsupported metadata.

Granting that I could also implement a very lite weight application that
doesn't offer metadata or any number of other features since we don't
require an application to implement any specific set of features.

So, is preservation separate from feature implementation? (I am asking
because I haven't seen it raised in this context and while it seems
evident to me the two are different, opinions may and probably do vary
on that point.)

Hope you are having a great day!

Patrick

Thomas Zander wrote:
> On Friday 29 June 2007 17:30:38 Bruce D'Arcus wrote:
>
>>> Therefore it seems best to keep these rules optional unless the
>>> application plan to implement the metadata feature.
>>> And when I mentioned ODF applications, I do not meant 
OpenOffice.org,
>>> for which it won't be a problem as we desire the feature.
>>>
>> I strongly disagree. Preserving files and attributes is a trivial
>> requirement. Not doing so will introduce large compatibility problems.
>>
>
> This, naturally, is not an isolated case. There currently are no
> applications that support all the ODF features but they still can be ODF
> compliant.  Even should one loose 90% of the features between loading and
> saving.
>
> So the question becomes, is metadata any different. I'm inclined to say
> no.  Its not different, its just a feature that the application should
> support. If it doesn't then a better application comes along and replaces
> it.
> Looking at the ODF landscape I consider this;
> * OOo and KOffice both think its a good feature to support, and are both
> moving to do so.  I'm sure other big office suites agree and do so as
> well.  Meaning that ODF doesn't have to force their hands.
> * Almost all professional users of office suites benefit from metadata in
> one way or another. So an application that follows user requests will
> quickly see metadata on their TODO list.
> * There are not that many application-types that both load and save ODF
> files. Viewers and writers are the bigger segment. We are only concerned
> with the load+save type.
>
> Bottom line is that while I fully agree its very important to retain this
> information that does not automatically imply that ODF has to make it
> mandatory. Market forces can do that too.

> Furthermore I would find it hypocritical to make it mandatory while almost
> all features in ODF are not mandatory to be retained between load and
> saves.
>
>
>> Really, just to be clear: if applications do not preserve xml:id
>> attributes, fields will break, and any metadata about document
>> fragments will be made invalid. Is that really in anyone's interest?
>> They need not support metadata in any explicit way to do this.
>>
>
> I think this said is best; things will break in a very visible way if apps
> don't support it.  So users will quickly realize a problem and apps will
> be requested to support the feature of metadata.
>
> The only reason I see to make it mandatory is for fear of people not
> supporting it, and I can easy your mind there, you guys have been doing
> an awesome job. Applications will line up to support this stuff. Don't
> worry so much! :)
>
>
> ps. not sure if I have write access to the metadata ML, if this doesn't
> appear there, please consider forwarding. Thanks.
>

--
Patrick Durusau
patrick@durusau.net
Chair, V1 - US TAG to JTC 1/SC 34
Acting Convener, JTC 1/SC 34/WG 3 (Topic Maps)
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Co-Editor, OpenDocument Format (OASIS, ISO/IEC 26300)

On 6/29/07, Michael Brauer - Sun Germany - ham02 - Hamburg <Michael.Brauer@sun.com> wrote:

Hi,

first of all, it is my understanding that the issue that Svante is
raising here regarding the usage of shall and should is not that
applications should get the freedom to remove the xml:id and other
metadata features whenever they like. Of course, they should keep them
whenever this is possible and appropriate.

When is it impossible and inappropriate? Svante already announced during a Metadata SC call that Sun does not intend to support the entire Metadata proposal on the rationale that the market should decide which metadata features should be supported. That is the same excuse he gave for removing the requirement of xml:id attribute preservation. However, the market has already spoken on this subject. "No incomplete implementations, no proprietary extensions[.]" <
http://ec.europa.eu/idabc/servlets/Doc?id=27956> (PDF, slide 7).

If you believe that there are valid reasons for destroying xml:id attributes, please present your use cases and draft a "SHALL preserve *except* when ...." replacement sentence for consideration.

The issue is to find an appropriate language for the things we want to
achieve. And independent of the issues we are discussing here, this in
general includes the option to omit a certain language if we believe
that what we say may be self-evident or given anyway, so that the
language is not required at all.

I disagree. If ODF is to become an interoperable format standard we must have conformance requirements for interoperability. For example, I am told that Sun's applications destroy all foreign elements and attributes other than  paragraphs and text spans. That despite it being "self-evident" from an interoperability standpoint that all foreign elements and attributes necessary for interoperability purposes should be preserved. And also despite the fact that in ODF 
1.0 the RFC 2119 definition of MAY made it mandatory that Sun do so. Sun's destruction of foreign elements and attributes has posed incredible difficulties in the Foundation's efforts to establish non-lossy interoperability with Microsoft Office. Difficulties that the Foundation had planned to work around using the Metadata SC's work. But now Sun seeks permission to destroy xml:id attributes as well. Why should it matter to Sun if it plans to preserve xml:id attributes anyway? The only justifications offered are flimsy beyond any credibility.

So please tell us unambiguously why it is so important to Sun that applications be allowed to destroy xml:id attributes, so important that we must be bothered with this request. And please do tell us straightforwardly whether Sun intends to implement the entire Metadata SC proposal. Was Svante telling us the truth?


Below are few suggestion that may or may not be helpful for addressing

the metadata related should and shall issues. But before. I would like
to make some comments:

1. We have to make sure that the language we are choosing is precise,
and permits reasonable edit operations on documents. Related to xml:ids,
that means that the language must permit to remove the attribute or to
change its values if this happens as the result of a user action or a
machine processing the document.

Agreed on user-initiated actions. But I disagree on removal of the attribute by a "machine processing the document." That is so ambiguous as to permit the wholesale stripping of the attributes for no reason at all, much as Sun's applications strip most of the foreign elements and attributes.  

2. If a document is opened and saved again, we all expect that the
paragraph content is preserved. The same applies to tables, lists,
images. etc. Does the specification has a language that enforces
that? No, it doesn't. But we all expect that these features are
preserved anyway.

But so long as we do not require their preservation, we leave the door open for a developer to destroy them and still claim conformance, as Sun does despite  its destruction of foreign elements and attributes. Europe has this to say, "Conformance testing and document validation possibilities
are needed -> in order to facilitate mapping / conversion." See slides linked above.

Right now, an empty Zip file is a conformant ODF document. I guess that makes the conformance testing easy, right?

 
But what's different with the xml:id (and metadata in general) that
there is the assumption that it may get removed unless there is a
language that forbids that?

Those who do not learn from history are doomed to repeat it. The Foundation has learned the hard way that Sun will destroy whatever metadata is necessary to block the Foundation's plugin from providing high fidelity interoperability with MS Office.  We will be proposing several other requirements to prevent such misbehavior. And if they are not implemented in ODF, they will be implemented in our fork of ODF and will have center stage in our opposition to ODF 
1.2 at ISO, in support of our request that this TC be required to fix the specification before ISO adoption.


I personally believe that everyone who is interested in ODF is
interested in implementing as much of ODF as possible (though there may
of cause be resource or technical restrictions), and this includes
metadata, regardless what language we use for above features. 

Then please explain why: [i] Sun destroys foreign elements and attributes; and [ii] Svante said Sun does not intend to fully implement the Metadata SC proposal. 


I further
believe that we encourage more vendors to implement metadata if we talk
about the value of implementing them, than we do by making certain
things mandatory. For that reason, and because it seems not to be too
easy to find an appropriate language, it would be okay for me if we
would omit that language at all.

May I suggest that you begin by talking to your own staff about fully implementing the Metadata SC proposal? And about preserving foreign elements and attributes? In the meantime, it is unacceptable that Sun retain the discretion to break interoperability with other apps, so we will press for conformance requirements that Sun so richly deserves.
 

3. The focus of ODF of course are office documents. But there always was
the assumption that also other kind of applications should be able to
use ODF. So, if someone develops a small text editor and wishes to
support ODF to the extend that typical text editors can, this should be
be possible. Our language should not prohibit that. 


Unless we add conformance requirements for the preservation of metadata and processing instructions, the less featureful apps will never  be able to round-trip documents with the more featureful apps. Our language should require that. Personally, I believe that the software-as-an-end-point client-side office suites are dinosaurs at the end of their era. They are being finished off by a thousand cuts as users spend less and less time using them and more and more time using other apps, such as web apps. ODF either develops methods for interoperability among all apps or it will die along with the office suites. 
E.g., Microsoft knows this and is busily migrating its Office development budget across the Sharepoint/Exchange server hubs to the network. Meanwhile, this TC fiddles with preserving the 1995 software-as-an-endpoint vision.
 
 We should also not
forget the various ODF plug-in efforts for MS Office or similar ODF
implementations. They have only limited control of what happens with
certain information during complex load, edit and save operations within
MS Office. I'm not sure if they can preserve all metadata and all
xml:ids under all circumstances in a way that keeps the metadata
consistent and therefore of value.

And this speculation is a reason for allowing the destruction of metadata and xml:id attributes? Why don't you ask their developers?
 
   Having that said, here are my suggestions. Please do not consider them
as a proposal. They are only suggestions, and the SC may follow them as
a whole or partially, or may not.

1. We may move all the metadata related should/shall language into the
general conformance section. This has the advantage that it is not
overlooked as easy as it would be if it is in the element and attribute
description. It further has the advantage that metadata is mentioned at
a very prominent position.

+1. That is precisely what I advocated on the Metadata SC. But there are far more areas of the specification requiring similar treatment.
 

2. We may introduce the term of a metadata-aware application (or
something like that), and define conformance definitions along the
following lines for it:
- A metadata aware ODF implementation *shall* not remove the xml:id
attributes defined in sections [?] or change its values unless the
removal or modification is the result of an edit operation caused be the
user, or a similar action taken by some automatic processing of the
document.
- [any other requirement that may exist]

We have been discussing something similar for some time. We see a need for two conformance classes: [i] fully integratable software qualified as routers of information in business processes; [ii] software that qualifies only as an end point solution. The latter would be for legacy ODF apps and ODF apps whose developers are satisfied  with one-way interop with applications that have different feature sets and would have minimal conformance requirements and most likely would not qualify for government software procurement tenders. The routers of information class would have strict conformance requirements as to interoperability features such as preservation of metadata and processing instructions. This class would attempt  to satisfy, as closely as is feasible, the European Open Document Exchange Formats requirements, which incidentally calls for convergence of the Microsoft and ODF formats, with ODF providing the common functionality.

But again, please be less ambiguous about "a similar action taken by some automatic processing of the document." That reads like a signed cheque with the amount to be filled in left blank.
 


3. We may rephrase the above statement for general ODF implementation,
replacing the *shall* with a *should*:

Disagree.
 

- An ODF implementation *should* not remove the xml:id attributes
defined in sections [?] or change its values unless the removal or
modification is the result of an edit operation caused be the user, 

Change that "should" to "shall" and we might find agreement. "Should" is meaningless in the ISO requirements keyword definitions. It is a license for abuse.

or a
similar action taken by some automatic processing of the document.

Again, this needs more detail. We can not support language so ambiguous.
 

4. Some time ago we have discussed whether the question which
implementation should/shall support what features may be a topic for ODF
1.3. So we may go with no or only a very limited number of metadata
related conformance requirements for ODF 
1.2, and make a deeper
discussion part of a more general discussion for ODF 1.3.

From our perspective it would be better to aim for doing the job in ODF 1.2, even if that requires delay. We will oppose ODF 
1.2 at ISO unless the interoperability warts are cleaned up. What the market requires is no longer in doubt. See the slides linked above and further presentations linked from this page, <
http://ec.europa.eu/idabc/en/document/6474/5935>. Substantial progress toward those goals would seem to be mandatory to maintain Europe's preference for a harmonized set of file formats that uses ODF to provide the common functionality. Delaying commencement of such work enhances the likelihood that governments will tire of waiting for ODF to become interoperable with MS Office and simply go with MOOXML.  We may not be able to force Microsoft to participate in the harmonization work, but we will be in a far better position if we have done everything we can in aid of that interoperability without Microsoft's assistance.

As the situation stands, we have what is known in the U.S. as a "Mexican stand-off," where neither side has taken a solitary step toward what Europe has requested. We have decided to do that work via a fork of ODF; it is up to this TC whether it wishes to cooperate in that effort.

On Jun 29, 2007, at 11:25 PM, marbux wrote:

> From our perspective it would be better to aim for doing the job in 
> ODF 1.2, even if that requires delay. We will oppose ODF 1.2 at ISO 
> unless the interoperability warts are cleaned up.... We have decided 
> to do that work via a fork of ODF; it is up to this TC whether it 
> wishes to cooperate in that effort.

Who is "we"? Let's use proper names please, and take some 
responsibility.

In any case, I really think these vague threats are pretty 
reprehensible marbux. Even if I agree with one or two of your points on 
conformance, I certainly don't think the TC should be moved by your 
tactics.

And not only will I not have anything to do with a fork of ODF, I will 
publicly call it what it is: a self-serving and petty effort at 
extortion.

Bruce

Marbux,

marbux wrote:
>
> Svante already announced during a Metadata SC call that Sun does not 
> intend to support the entire Metadata proposal on the rationale that 
> the market should decide which metadata features should be supported.

I am sorry if there had been some misunderstanding.  Be sure we intend 
to support the complete Metadata proposal, never hesitated. What part 
should we drop for what reasons?
Did I really said this? Is there any taken minutes you have read, that I 
need to comment?

regards,
Svante

The issue raised by Sun (see below) is whether application interoperability will be required or be only optional.

On 6/27/07, Svante Schubert <Svante.Schubert@sun.com">
Svante.Schubert@sun.com> wrote:
Greetings!

here an updated list of issues to be discussed and editorial changes.

New suggested of changes to be discussed
=========================================

VI) Dropping non modification requirement
"Metadata files should not be modified unless the content of the
metadata file is changed.

This sentence describes application behavior. The described behavior is
moreover not essential, nor do we have something similar for ODF
content, like keeping the XML structure, when the document is not being
changed.

Virtually every tag description in the ODF specification describes application behavior. I would vote for changing the "should" to "must." The described behavior is not essential, unless you believe interoperability of ODF applications should be a req
 

VII) Dropping the following sentence as we introduced no restriction on
RDF/XML.

"The OpenDocument format does not restrict RDF/XML elements or
attributes for metadata files except as defined for the OpenDocument
format."



VIII) Adjust 'shall' requirement to 'should' for xml:id

"All implementations SHALL preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

Similar as other standards (e.g. CSS) we should not try to force

features by specification, but should let the market sort this out.
Moreover the specification could be interpreted that it is even
forbidding to delete the xml:id or its value, even when deleting the
content, therefore a 'SHOULD' is sufficient.



IX) Dropping paragraph about preserved content of text:meta-field

Asked to drop the following section:
"The generated content of a <text:meta-field> element SHALL be preserved
upon each "save" operation to facilitate use of a document with an
application that does not support generation of content from metadata."

As the text:meta-field chapter will be part of the field chapter of the
main spec, there exist already some default behavior for fields.
e.g. "The content of the element is the textual representation of the
current field value as it would be displayed or printed."

It questionable we need a special preservation application behavior for
text:meta-field.



X) New 'pkg' prefix and namespace to make metadata model reusable even
for non ODF applications

We would have differentiate for the metadata manifest the existing
"odf:" prefixed RDF vocabulary into two vocabularies.
One representing the vocabularies necessary for all packages (e.g.
prefixed by "pkg:") and a second for the ODF relevant part (still
prefixed odf:).
All form odf: property/nodes will become pkg: property/nodes with the
exception of the ODF related elements, which are:

odf:ContentFile - the OpenDocument content.xml
odf:StylesFile - the OpenDocument styles.xml
odf:Element - an OpenDocument XML element




New added editorial changes:
======================

15) Changed wording from:
"A metadata file in an OpenDocument package is represented by a node of
class odf:File or by one of its subclasses odf:ContentFile,
odf:StylesFile or odf:MetadataFile. The relationship between a metadata
file and its package is expressed using the property  odf:hasPart."
to
"The resource of a file in an OpenDocument package is represented by an
element of class odf:File or by one of its subclasses odf:ContentFile,
odf:StylesFile or odf:MetadataFile. The relationship between a file and
its package is expressed using the property odf:hasPart."

16) Changed wording from:
"An rdf:about attribute defines the <odf:File> or one of its subclasses
by an IRI. In case of a metadata RDF/XML file this IRI represents a
named RDF graph."
to
"The identifier (IRI) of an odf:MetadataFile RDF node represents its
named RDF graph"

17) Changed wording from:
"The relationship between a content.xml or styles.xml file and the
<odf:Element> node is expressed using the property <odf:hasPart>."
to
"Whether an element is contained in the content.xml
 or styles.xml is
described by the odf:hasPart property."

18)Exchange 'must' to 'shall'
"All implementations must preserve any xml:id attribute and its value
when present on any of the elements listed in 
1.4.3.."
to
"All implementations shall preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

19)Exchanged the order of two list elements, as one referred to the next
* m:data-value: The RDF object, if it appears. Otherwise, the literal
content of the OpenDocument element is the RDF object.
* m:data-type: Data type of m:data-value

Regards,
Svante

Svante, you might draw some guidance here from the following statement by Michael Brauer in a private email to Gary Edwards dated March 5, 2007:

>>>

I'm not sure if one can say that this way. What will be new in ODF 
1.2 is that metadata aware application *should preserve metadata* (where possible and reasonable), where metadata is contained in some new RDF-XML metadata streams, or some some new elements and attributes that we will add the schema. The specification will not make any other assumptions about foreign elements (in the meaning of elements not defined by the schema) than those we have already.

<<<

Why the change in position on preservation of metadata?

Best regards,

Marbux

On 6/27/07, Svante Schubert <
Svante.Schubert@sun.com> wrote:
Greetings!

here an updated list of issues to be discussed and editorial changes.

New suggested of changes to be discussed
=========================================

VI) Dropping non modification requirement

"Metadata files should not be modified unless the content of the
metadata file is changed.

This sentence describes application behavior. The described behavior is
moreover not essential, nor do we have something similar for ODF
content, like keeping the XML structure, when the document is not being
changed.



VII) Dropping the following sentence as we introduced no restriction on
RDF/XML.

"The OpenDocument format does not restrict RDF/XML elements or
attributes for metadata files except as defined for the OpenDocument
format."



VIII) Adjust 'shall' requirement to 'should' for xml:id

"All implementations SHALL preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

Similar as other standards (e.g. CSS) we should not try to force
features by specification, but should let the market sort this out.
Moreover the specification could be interpreted that it is even
forbidding to delete the xml:id or its value, even when deleting the
content, therefore a 'SHOULD' is sufficient.



IX) Dropping paragraph about preserved content of text:meta-field

Asked to drop the following section:
"The generated content of a <text:meta-field> element SHALL be preserved
upon each "save" operation to facilitate use of a document with an
application that does not support generation of content from metadata."

As the text:meta-field chapter will be part of the field chapter of the
main spec, there exist already some default behavior for fields.
e.g. "The content of the element is the textual representation of the
current field value as it would be displayed or printed."

It questionable we need a special preservation application behavior for
text:meta-field.



X) New 'pkg' prefix and namespace to make metadata model reusable even
for non ODF applications

We would have differentiate for the metadata manifest the existing
"odf:" prefixed RDF vocabulary into two vocabularies.
One representing the vocabularies necessary for all packages (
e.g.
prefixed by "pkg:") and a second for the ODF relevant part (still
prefixed odf:).
All form odf: property/nodes will become pkg: property/nodes with the
exception of the ODF related elements, which are:

odf:ContentFile - the OpenDocument content.xml
odf:StylesFile - the OpenDocument styles.xml
odf:Element - an OpenDocument XML element




New added editorial changes:
======================

15) Changed wording from:
"A metadata file in an OpenDocument package is represented by a node of
class odf:File or by one of its subclasses odf:ContentFile,
odf:StylesFile or odf:MetadataFile. The relationship between a metadata
file and its package is expressed using the property  odf:hasPart."
to
"The resource of a file in an OpenDocument package is represented by an
element of class odf:File or by one of its subclasses odf:ContentFile,
odf:StylesFile or odf:MetadataFile. The relationship between a file and
its package is expressed using the property odf:hasPart."

16) Changed wording from:
"An rdf:about attribute defines the <odf:File> or one of its subclasses
by an IRI. In case of a metadata RDF/XML file this IRI represents a
named RDF graph."
to
"The identifier (IRI) of an odf:MetadataFile RDF node represents its
named RDF graph"

17) Changed wording from:
"The relationship between a content.xml or styles.xml file and the
<odf:Element> node is expressed using the property <odf:hasPart>."
to
"Whether an element is contained in the content.xml
 or styles.xml is
described by the odf:hasPart property."

18)Exchange 'must' to 'shall'
"All implementations must preserve any xml:id attribute and its value
when present on any of the elements listed in 
1.4.3.."
to
"All implementations shall preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

19)Exchanged the order of two list elements, as one referred to the next
* m:data-value: The RDF object, if it appears. Otherwise, the literal
content of the OpenDocument element is the RDF object.
* m:data-type: Data type of m:data-value

Regards,
Svante

On 6/27/07, Svante Schubert <Svante.Schubert@sun.com> wrote:

VIII) Adjust 'shall' requirement to 'should' for xml:id

"All implementations SHALL preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

Similar as other standards (e.g. CSS) we should not try to **force
features by specification,** but should let the market sort this out.
Moreover the specification could be interpreted that it is even
forbidding to delete the xml:id or its value, even when deleting the
content, therefore a 'SHOULD' is sufficient.

Please identify which specific feature(s) you believe the language attempts to force. The only feature I can think of that you might be referring to is application round-trip interoperability. Is that a feature Sun does not intend to implement? If so, why not and why should the rest of the TC members support maintenance of an interoperability barrier?

Your last sentence could be addressed by adding a sentence to the language under discussion, stating, "This does not mean that an attribute's element may not be deleted if initiated by user action."

 

On 6/27/07, Svante Schubert <Svante.Schubert@sun.com> wrote:

Greetings!

here an updated list of issues to be discussed and editorial changes.

New suggested of changes to be discussed
=========================================

VI) Dropping non modification requirement
"Metadata files should not be modified unless the content of the
metadata file is changed.

This sentence describes application behavior. The described behavior is
moreover not essential, nor do we have something similar for ODF
content, like keeping the XML structure, when the document is not being
changed.

Virtually every section of the specification describes application behavior. That is an irrelevant factor. Moreover, the described behavior is essential for round-trip intoperability purposes. And the fact that we have no equivalent requirement for maintaining the XML structure counsels that we create such a requirement; it is not a justification for breaking interoperability in another area.

This suggestion should be withdrawn unless you can provide a more compelling justification for it.

marbux wrote:
>
>
> On 6/27/07, *Svante Schubert* 

On 6/27/07, Svante Schubert <Svante.Schubert@sun.com> wrote:

Greetings!

IX) Dropping paragraph about preserved content of text:meta-field

Asked to drop the following section:
"The generated content of a <text:meta-field> element SHALL be preserved
upon each "save" operation to facilitate use of a document with an
application that does not support generation of content from metadata."

As the text:meta-field chapter will be part of the field chapter of the
main spec, there exist already some default behavior for fields.
e.g. "The content of the element is the textual representation of the
current field value as it would be displayed or printed."

It questionable we need a special preservation application behavior for
text:meta-field.


The language you suggest dropping includes a mandatory requirement of metadata preservation. But the language you quote from the text:meta-field chapter includes no such mandatory requirement. Doing away with metadata preservation requirements seems to be the unifying theme of your post.

The effect, of course, is to allow Sun even more discretion to break interoperability with other applications, as it has already done with its applications' in regard to destroying foreign elements and attributes, which were expressly designed for interoperability purposes, and by parking a host of Sun eXtensions to ODF in the OOo namespace rather than submitting them to the TC for specification and standardization.

Svante, I can well imagine that you were ordered to submit these proposed changes and that doing so made you nauseus. Nonetheless, I thank you for bringing Sun's hostility to interoperability out into the open.

On 6/28/07, marbux 

To all those who said we were being unreasonable to decide that we were no longer willing to place our faith in Sun's leadership of the OpenDocument Technical Committee, we give you Exhibit A. Sun Microsystems is the enemy of ODF interoperability. It will be interesting to see how those who eschew confrontation deal with these proposals, which will -- if adopted -- eviscerate the Metadata SC's work, allowing Sun to break interoperability with other applications.

On 6/27/07, Svante Schubert <Svante.Schubert@sun.com> wrote:

Greetings!

here an updated list of issues to be discussed and editorial changes.

New suggested of changes to be discussed
=========================================

VI) Dropping non modification requirement
"Metadata files should not be modified unless the content of the
metadata file is changed.

This sentence describes application behavior. The described behavior is
moreover not essential, nor do we have something similar for ODF
content, like keeping the XML structure, when the document is not being
changed.



VII) Dropping the following sentence as we introduced no restriction on
RDF/XML.

"The OpenDocument format does not restrict RDF/XML elements or
attributes for metadata files except as defined for the OpenDocument
format."



VIII) Adjust 'shall' requirement to 'should' for xml:id

"All implementations SHALL preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

Similar as other standards (e.g. CSS) we should not try to force
features by specification, but should let the market sort this out.
Moreover the specification could be interpreted that it is even
forbidding to delete the xml:id or its value, even when deleting the
content, therefore a 'SHOULD' is sufficient.



IX) Dropping paragraph about preserved content of text:meta-field

Asked to drop the following section:
"The generated content of a <text:meta-field> element SHALL be preserved
upon each "save" operation to facilitate use of a document with an
application that does not support generation of content from metadata."

As the text:meta-field chapter will be part of the field chapter of the
main spec, there exist already some default behavior for fields.
e.g. "The content of the element is the textual representation of the
current field value as it would be displayed or printed."

It questionable we need a special preservation application behavior for
text:meta-field.



X) New 'pkg' prefix and namespace to make metadata model reusable even
for non ODF applications

We would have differentiate for the metadata manifest the existing
"odf:" prefixed RDF vocabulary into two vocabularies.
One representing the vocabularies necessary for all packages (
e.g.
prefixed by "pkg:") and a second for the ODF relevant part (still
prefixed odf:).
All form odf: property/nodes will become pkg: property/nodes with the
exception of the ODF related elements, which are:

odf:ContentFile - the OpenDocument content.xml
odf:StylesFile - the OpenDocument styles.xml
odf:Element - an OpenDocument XML element




New added editorial changes:
======================

15) Changed wording from:
"A metadata file in an OpenDocument package is represented by a node of
class odf:File or by one of its subclasses odf:ContentFile,
odf:StylesFile or odf:MetadataFile. The relationship between a metadata
file and its package is expressed using the property  odf:hasPart."
to
"The resource of a file in an OpenDocument package is represented by an
element of class odf:File or by one of its subclasses odf:ContentFile,
odf:StylesFile or odf:MetadataFile. The relationship between a file and
its package is expressed using the property odf:hasPart."

16) Changed wording from:
"An rdf:about attribute defines the <odf:File> or one of its subclasses
by an IRI. In case of a metadata RDF/XML file this IRI represents a
named RDF graph."
to
"The identifier (IRI) of an odf:MetadataFile RDF node represents its
named RDF graph"

17) Changed wording from:
"The relationship between a content.xml or styles.xml file and the
<odf:Element> node is expressed using the property <odf:hasPart>."
to
"Whether an element is contained in the content.xml
 or styles.xml is
described by the odf:hasPart property."

18)Exchange 'must' to 'shall'
"All implementations must preserve any xml:id attribute and its value
when present on any of the elements listed in 
1.4.3.."
to
"All implementations shall preserve any xml:id attribute and its value
when present on any of the elements listed in 1.4.3."

19)Exchanged the order of two list elements, as one referred to the next
* m:data-value: The RDF object, if it appears. Otherwise, the literal
content of the OpenDocument element is the RDF object.
* m:data-type: Data type of m:data-value

Regards,
Svante

Come on now.  Svante giving a list
of suggested changes for discussion.  No war crimes have been committed.
 Everyone is entitled to make proposals and have them discussed.  We
don't need to agree with the proposals, but I don't want to have an environment
where we can't have an open & public airing of technical proposals
without fear of intimidation.

Put another way, Marbux.  I don't
mind if your dial goes up to 11, but can we at least turn up the volume
more gradually? ;-)

-Rob
___________________________


Rob Weir

Software Architect

Workplace, Portal and Collaboration Software

IBM Software Group


email: robert_weir@us.ibm.com

phone: 1-978-399-7122

blog: http://www.robweir.com/blog/

marbux <marbux@gmail.com> wrote on 06/28/2007
10:12:11 PM:


> To all those who said we were being unreasonable to decide that we

> were no longer willing to place our faith in Sun's leadership of the

> OpenDocument Technical Committee, we give you Exhibit A. Sun

> Microsystems is the enemy of ODF interoperability. It will be

> interesting to see how those who eschew confrontation deal with

> these proposals, which will -- if adopted -- eviscerate the Metadata

> SC's work, allowing Sun to break interoperability with other applications.


> On 6/27/07, Svante Schubert <Svante.Schubert@sun.com>
wrote:
> Greetings!

>

> here an updated list of issues to be discussed and editorial changes.

>

> New suggested of changes to be discussed

> =========================================

>

> VI) Dropping non modification requirement

> "Metadata files should not be modified unless the content of
the

> metadata file is changed.

>

> This sentence describes application behavior. The described behavior
is

> moreover not essential, nor do we have something similar for ODF

> content, like keeping the XML structure, when the document is not
being

> changed.

>

>

>

> VII) Dropping the following sentence as we introduced no restriction
on

> RDF/XML.

>

> "The OpenDocument format does not restrict RDF/XML elements or

> attributes for metadata files except as defined for the OpenDocument

> format."

>

>

>

> VIII) Adjust 'shall' requirement to 'should' for xml:id

>

> "All implementations SHALL preserve any xml:id attribute and
its value

> when present on any of the elements listed in 1.4.3."

>

> Similar as other standards (e.g. CSS) we should not try to force

> features by specification, but should let the market sort this out.

> Moreover the specification could be interpreted that it is even

> forbidding to delete the xml:id or its value, even when deleting the

> content, therefore a 'SHOULD' is sufficient.

>

>

>

> IX) Dropping paragraph about preserved content of text:meta-field

>

> Asked to drop the following section:

> "The generated content of a <text:meta-field> element SHALL
be preserved

> upon each "save" operation to facilitate use of a document
with an

> application that does not support generation of content from metadata."

>

> As the text:meta-field chapter will be part of the field chapter of
the

> main spec, there exist already some default behavior for fields.

> e.g. "The content of the element is the textual representation
of the

> current field value as it would be displayed or printed."

>

> It questionable we need a special preservation application behavior
for

> text:meta-field.

>

>

>

> X) New 'pkg' prefix and namespace to make metadata model reusable
even

> for non ODF applications

>

> We would have differentiate for the metadata manifest the existing

> "odf:" prefixed RDF vocabulary into two vocabularies.

> One representing the vocabularies necessary for all packages ( e.g.

> prefixed by "pkg:") and a second for the ODF relevant part
(still

> prefixed odf:).

> All form odf: property/nodes will become pkg: property/nodes with
the

> exception of the ODF related elements, which are:

>

> odf:ContentFile - the OpenDocument content.xml

> odf:StylesFile - the OpenDocument styles.xml

> odf:Element - an OpenDocument XML element

>

>

>

>

> New added editorial changes:

> ======================

>

> 15) Changed wording from:

> "A metadata file in an OpenDocument package is represented by
a node of

> class odf:File or by one of its subclasses odf:ContentFile,

> odf:StylesFile or odf:MetadataFile. The relationship between a metadata

> file and its package is expressed using the property  odf:hasPart."

> to

> "The resource of a file in an OpenDocument package is represented
by an

> element of class odf:File or by one of its subclasses odf:ContentFile,

> odf:StylesFile or odf:MetadataFile. The relationship between a file
and

> its package is expressed using the property odf:hasPart."

>

> 16) Changed wording from:

> "An rdf:about attribute defines the <odf:File> or one of
its subclasses

> by an IRI. In case of a metadata RDF/XML file this IRI represents
a

> named RDF graph."

> to

> "The identifier (IRI) of an odf:MetadataFile RDF node represents
its

> named RDF graph"

>

> 17) Changed wording from:

> "The relationship between a content.xml or styles.xml file and
the

> <odf:Element> node is expressed using the property <odf:hasPart>."

> to

> "Whether an element is contained in the content.xml or styles.xml
is

> described by the odf:hasPart property."

>

> 18)Exchange 'must' to 'shall'

> "All implementations must preserve any xml:id attribute and its
value

> when present on any of the elements listed in 1.4.3.."

> to

> "All implementations shall preserve any xml:id attribute and
its value

> when present on any of the elements listed in 1.4.3."

>

> 19)Exchanged the order of two list elements, as one referred to the
next

> * m:data-value: The RDF object, if it appears. Otherwise, the literal

> content of the OpenDocument element is the RDF object.

> * m:data-type: Data type of m:data-value

>

> Regards,

> Svante

On 6/28/07, robert_weir@us.ibm.com <robert_weir@us.ibm.com> wrote:

Come on now.  Svante giving a list
of suggested changes for discussion.  No war crimes have been committed.
 Everyone is entitled to make proposals and have them discussed.  We
don't need to agree with the proposals, but I don't want to have an environment
where we can't have an open & public airing of technical proposals
without fear of intimidation.

Put another way, Marbux.  I don't
mind if your dial goes up to 11, but can we at least turn up the volume
more gradually? ;-)

No.

Marbux,

I would like to remind you that this is the mailing list of a technical
committee. It's for the technical discussion of proposals. That means,
technical feedback on proposals is of cause very welcome, but we should
abstain from makings assumptions and accusations.

marbux wrote:
> 
> 
> On 6/28/07, *robert_weir@us.ibm.com 

On 6/29/07, Michael Brauer - Sun Germany - ham02 - Hamburg <Michael.Brauer@sun.com> wrote:

Marbux,

I would like to remind you that this is the mailing list of a technical
committee. It's for the technical discussion of proposals. That means,
technical feedback on proposals is of cause very welcome, but we should
abstain from makings assumptions and accusations.

This mailing list is most emphatically not only for technical discussions. We work here on a proposal to be submitted to ISO as a draft international standard. The Agreement on Technical Barriers to Trade places only a single requirement on our deliberations:

"Members shall ensure that technical regulations are not **prepared,**
        adopted or applied with a view to or with the effect of creating
        unnecessary obstacles to international trade."
 
<http://www.wto.org/english/res_e/booksp_e/analytic_index_e/tbt_01_e.htm#article2A
>.

The treaty does not require us to prepare technically excellent standards or to work in a collegial manner. It tells us only that we must be vigilant in protecting the draft standard from efforts to tilt the competitive playing field. I have no intention of standing quiet when I see such abuse.


We should all be interested in keeping the OpenDocument TC a place where

proposals are discussed in a fair manner on a technical level. I hope
that you agree to that, and that your "no" above does not mean you are
not interested in this.

As stated above, I reject your suggestion that only technical issues are relevant.
 

On Jun 29, 2007, at 1:37 AM, marbux wrote:

rob wrote:

>> Put another way, Marbux.  I don't mind if your dial goes up to 11, 
>> but can we at least turn up the volume more gradually? ;-)
>
> No.

Then you're effectively shutting yourself out of any productive 
conversation. The constant uninformed impulse to conspiracy theory is 
annoying and unproductive.

Do you really think those of us who have spent the past 18 months 
working on this proposal (as opposed to taking pot shots from the 
side-lines) don't care enough about the details to get this right?

Bruce

Marbux.

marbux wrote:
> To all those who said we were being unreasonable to decide that we 
> were no longer willing to place our faith in Sun's leadership of the 
> OpenDocument Technical Committee, we give you Exhibit A. Sun 
> Microsystems is the enemy of ODF interoperability. It will be 
> interesting to see how those who eschew confrontation deal with these 
> proposals, which will -- if adopted -- eviscerate the Metadata SC's 
> work, allowing Sun to break interoperability with other applications.
>

As you may or may not know, Svante's first language is not English but 
he has done a hell of a lot better job than I would trying to write a 
standard in German.

Asking for more information before simply jumping to the conclusion that 
the proposals are the enemy of interoperability would be a more 
productive course.

Take the first proposal for instance:

> VI) Dropping non modification requirement
> "Metadata files should not be modified unless the content of the
> metadata file is changed.
>
> This sentence describes application behavior. The described behavior is
> moreover not essential, nor do we have something similar for ODF
> content, like keeping the XML structure, when the document is not being
> changed.
What is at issue is not modification of the metadata file but  
preservation of the metadata file.

In other words, an application might simply keep the metadata file but 
on the other hand, it might also read the metadata file and save it in 
another RDF syntax that it prefers for the file. Is that modification? 
Yes.  Is that preservation? Yes.

It may be that our wording was inelegant in the proposal in that the 
intent was to have the metadata preserved but not to restrict how that 
was done.

Since we don't constrain the RDF format in the first place, requiring it 
to not be modified doesn't really advance interoperability. One can 
expect to encounter RDF in a finite number of formats as specified by 
the W3C as the proposal is currently written. So taken at face value, if 
we preserve the metadata, we have no more and certainly no less 
interoperability than we did before.

You can argue that we should require some particular RDF format (I 
personally don't have strong feelings one way or the other) and that 
such format be preserved but that is a separate issue.

The key is understanding the technical aspects of what is being proposed 
and not simply assuming that every change is a change for the worse.

I have worked for a very long time on the metadata proposal and I have 
every intention of ODF having an interoperable metadata mechanism. I 
think I know enough about both to make sure that happens.

Hope you are having a great day!

Patrick

-- 
Patrick Durusau
patrick@durusau.net
Chair, V1 - US TAG to JTC 1/SC 34
Acting Convener, JTC 1/SC 34/WG 3 (Topic Maps)
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Co-Editor, OpenDocument Format (OASIS, ISO/IEC 26300)

On Jun 29, 2007, at 9:23 AM, Patrick Durusau wrote:

> It may be that our wording was inelegant in the proposal in that the 
> intent was to have the metadata preserved but not to restrict how that 
> was done.

Maybe given Svante's hunt for more precise RDF language, we should use 
that to resolve the language here?

E.g.:

"ODF applications shall not remove metadata statements unless ..."?

So focus on the triples, rather than where or how they're serialized.

Bruce

On 6/29/07, Patrick Durusau <patrick@durusau.net> wrote:

Marbux.

marbux wrote:
> To all those who said we were being unreasonable to decide that we
> were no longer willing to place our faith in Sun's leadership of the
> OpenDocument Technical Committee, we give you Exhibit A. Sun
> Microsystems is the enemy of ODF interoperability. It will be
> interesting to see how those who eschew confrontation deal with these
> proposals, which will -- if adopted -- eviscerate the Metadata SC's
> work, allowing Sun to break interoperability with other applications.
>

As you may or may not know, Svante's first language is not English but
he has done a hell of a lot better job than I would trying to write a
standard in German.

Asking for more information before simply jumping to the conclusion that
the proposals are the enemy of interoperability would be a more
productive course.

Did you not see my posts that asked for more information?


Take the first proposal for instance:

> VI) Dropping non modification requirement
> "Metadata files should not be modified unless the content of the
> metadata file is changed.
>
> This sentence describes application behavior. The described behavior is
> moreover not essential, nor do we have something similar for ODF
> content, like keeping the XML structure, when the document is not being
> changed.
What is at issue is not modification of the metadata file but
preservation of the metadata file.

In other words, an application might simply keep the metadata file but
on the other hand, it might also read the metadata file and save it in
another RDF syntax that it prefers for the file. Is that modification?
Yes.  Is that preservation? Yes.

It may be that our wording was inelegant in the proposal in that the
intent was to have the metadata preserved but not to restrict how that
was done.

Since we don't constrain the RDF format in the first place, requiring it
to not be modified doesn't really advance interoperability. One can
expect to encounter RDF in a finite number of formats as specified by
the W3C as the proposal is currently written. So taken at face value, if
we preserve the metadata, we have no more and certainly no less
interoperability than we did before.

You can argue that we should require some particular RDF format (I
personally don't have strong feelings one way or the other) and that
such format be preserved but that is a separate issue.

I think we are having a confusion of terms. By "interoperability" I mean round-tripping of documents amongst a multitude of applications without loss of data, i.e., fully integratable applications acting in the role of routers of information rather than as end points.  If Application A sends a document to Application B which sends it to Application C which sends it to Application A, will Application A still find the data it needs to properly render the file? Right now, ODF is only for end point solutions and because StarOffice/OpenOffice.org are the most featureful, they are the only applications that can handle everything another ODF app may throw at another.

Do we need to preserve Application A's unique flavor of RDF in order for the data to survive a trip through many applications before returning? If so, then we should be concerned about specifying "some particular RDF format" and the preservation of such metadata.


The key is understanding the technical aspects of what is being proposed

and not simply assuming that every change is a change for the worse.

I am not making assumptions. I am dealing with what I understand to be factual. If I get it wrong, I get it wrong. But I am not dealing with assumptions.
 

I have worked for a very long time on the metadata proposal and I have
every intention of ODF having an interoperable metadata mechanism. I
think I know enough about both to make sure that happens.

As with Bruce, I question whether you have sufficient votes to make sure that happens.
 

On 6/29/07, marbux