OASIS Darwin Information Typing Architecture (DITA) TC

Hi, After Kris found the DTD issue with the (map mapgroup-d) token last week, I realized that the copy of DITA 1.2 checked in to SVN did not match the copy of DITA 1.2 shipped by OASIS. I did some comparison of the shipped DITA 1.2 and SVN 1.2, and found many inconsistencies. Everything I've listed below is also an issue for the DITA 1.3 DTDs. I found all of these issues by comparing 1.2 level DTDs - not comparing the RNGs - so this presumably mixes RNG issues with conversion tool issues, and also means that this testing did not cover anything new to 1.3. 1. All embedded files are missing public IDs. These have been present since 1.0, and in 1.2 we explicitly made all internal public IDs use version-specific values. This was done by TC request, and they should be added back. 2. Several entity declarations have been removed from the DTD. Some are unlikely to cause problems but all could potentially be referenced in specialized DTDs (I know of DTDs that reference the first one): - %rel-atts in topic.mod: deprecated in 1.2 but left in for backwards compatibility - %hazard.cnt used to set content models in hazard domain - In the tblDecl.mod, unlikely to cause problems but for completeness: -- Entity %bodyatt; removed from table attribute list - was empty, present as part of OASIS table model -- Dropped entity %tbl.table.name; (value was "table") -- previously used to define the element name -- Dropped entity %titles; (value was "title?") -- it was declared but unused in OASIS DITA 1.2 3. The error Kris found in the mapgroup domain token is still present in the glossref domain and learningMapDomain tokens (both claim to be based on topic). 4. The domain token for learningAssessment has changed - not sure if this is intentional - from " (topic learningBase+learningInteractionBase-d learningAssessment) " to " (topic learningAssessment) " 5. In ditabase.dtd, the declaration of <dita> has dropped several attributes. The following are present in DITA 1.2 but missing in DITA 1.3: %arch-atts -- @ditaarch:DITAArchVersion, @xmlns:ditaarch, %debug-atts -- @xtrc, @xtrc. Related: in 1.3 we've added @translate and @dir to the existing @xml:lang -- I think these should be added using the common localization-atts group rather than declared independently? 6. The task domain entity (topic task) has been dropped from ditabase.dtd, task.dtd, and machineryTask.dtd 7. glossary.dtd is broken: it is supposed to redirect to glossentry.dtd but actually does nothing 8. The topic containment entities are broken in learningContent.dtd. The learningContent element previously allowed lots of children, currently 1.3 changed this to only allow <no-topic-nesting>: 1.2 as shipped: < !ENTITY  % learningContent-info-types "((concept task reference topic)*, (learningAssessment)?, (learningSummary)?)" > 1.2 and 1.3 in SVN: < !ENTITY  % learningContent-info-types "no-topic-nesting" > 9. Related - the content model for <topic> inside the learningContent doctype has also changed, pretty sure this was not intentional. DITA 1.2 shipped with this declaration in learningContent.dtd: < !ENTITY  % topic-info-types "(no-topic-nesting)?" > The SVN copies do not declare this entity, and as a result the default is picked up, and <topic> now allows nesting of <topic> 10. The strictTaskbodyConstraint domain entity name is invalid. The name of the entity name has changed from "taskbody-constraints" to "strictTaskbody-constraints". This will break any user of the constraint, and also violates the spec (spec requires the entity use the literal element name). I believe this error was introduced recently, it was not in my copy of SVN until I updated on Friday. 11. The map.mod module drops the default empty declaration of included-domains (it is still in topic.mod). 12. Every element has its attribute entity (such as alt.attributes) declared twice. Strictly speaking this does not introduce errors but really needs to be fixed (it's confusing and looks bad). 13. Many of the DTD modules have bad headers (every ENT file I spot checked and several of the MOD files). The ENT files appear to grab their headers (including public ID info) from the equivalent MOD file. Nit-picks that we would fix if creating these by hand: - In topic.mod, the entities "info-types" and "topic-info-types" are each declared twice - In machinery taskbody constraint, %prelreqs and %closereqs are each declared twice - DITA 1.2 /1.3 in SVN has added an entity idElements; it's empty and unused, so not sure of the purpose Robert D Anderson IBM Authoring Tools Development Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ )

I'll take a look at all of these. I have documented variances from the original 1.2 declarations that are an unavoidable consequence of generating the DTDs and XSDs, but most of these do no fall into that category, I don't think. On issue 13, headers for .ent files, that's a problem: since the RNG does not have separate mod and ent files, there's only one header comment in the RNG source, so not obvious how to produce different headers for .mod and .ent files unless we include both headers in the RNG source (which we can do). When you say "bad headers" do you simply mean that the headers don't match the original 1.2 headers for the .ent files? The idElements pattern in the RNG is required in order to handle the RNG-specific rules for patterns involving elements with ID-type attributes. I can ensure that those patterns don't get reflected in the generated DTDs and XSDs. I'm still working out details of constraint module generation: there's some complexity there. The current code is not always generating correct DTD declarations. It may be necessary to special case all constraint module generation, I'm not sure yet. I'll add these issues as cards in the Trello board I set up for the remaining grammar generation issues: https://trello.com/b/y42J0sEq/rng-to-dtd-and-xsd-conversion Cheers, E. ————— Eliot Kimber, Owner Contrext, LLC http://contrext.com On 10/20/14, 9:27 AM, "Robert D Anderson" <robander@us.ibm.com> wrote: >Hi, > >After Kris found the DTD issue with the (map mapgroup-d) token last week, >I realized that the copy of DITA 1.2 checked in to SVN did not match the >copy of DITA 1.2 shipped by OASIS. I did some comparison of the shipped >DITA 1.2 and SVN 1.2, and found many inconsistencies. Everything I've >listed below is also an issue for the DITA 1.3 DTDs. I found all of these >issues by comparing 1.2 level DTDs - not comparing the RNGs - so this >presumably mixes RNG issues with conversion tool issues, and also means >that this testing did not cover anything new to 1.3. > >1. All embedded files are missing public IDs. These have been present >since 1.0, and in 1.2 we explicitly made all internal public IDs use >version-specific values. This was done by TC request, and they should be >added back. > >2. Several entity declarations have been removed from the DTD. Some are >unlikely to cause problems but all could potentially be referenced in >specialized DTDs (I know of DTDs that reference the first one): >- %rel-atts in topic.mod: deprecated in 1.2 but left in for backwards >compatibility >- %hazard.cnt used to set content models in hazard domain >- In the tblDecl.mod, unlikely to cause problems but for completeness: >-- Entity %bodyatt; removed from table attribute list - was empty, >present as part of OASIS table model >-- Dropped entity %tbl.table.name; (value was "table") -- previously used >to define the element name >-- Dropped entity %titles; (value was "title?") -- it was declared but >unused in OASIS DITA 1.2 > >3. The error Kris found in the mapgroup domain token is still present in >the glossref domain and learningMapDomain tokens (both claim to be based >on topic). > >4. The domain token for learningAssessment has changed - not sure if this >is intentional - from "(topic learningBase+learningInteractionBase-d >learningAssessment)" to "(topic learningAssessment)" > >5. In ditabase.dtd, the declaration of <dita> has dropped several >attributes. The following are present in DITA 1.2 but missing in DITA 1.3: >%arch-atts -- @ditaarch:DITAArchVersion, @xmlns:ditaarch, >%debug-atts -- @xtrc, @xtrc. >Related: in 1.3 we've added @translate and @dir to the existing @xml:lang >-- I think these should be added using the common localization-atts group >rather than declared independently? > >6. The task domain entity (topic task) has been dropped from >ditabase.dtd, task.dtd, and machineryTask.dtd > >7. glossary.dtd is broken: it is supposed to redirect to glossentry.dtd >but actually does nothing > >8. The topic containment entities are broken in learningContent.dtd. The >learningContent element previously allowed lots of children, currently >1.3 changed this to only allow <no-topic-nesting>: >1.2 as shipped: <!ENTITY % learningContent-info-types "((concept task >reference topic)*, (learningAssessment)?, (learningSummary)?)"> >1.2 and 1.3 in SVN: <!ENTITY % learningContent-info-types >"no-topic-nesting" >> > >9. Related - the content model for <topic> inside the learningContent >doctype has also changed, pretty sure this was not intentional. DITA 1.2 >shipped with this declaration in learningContent.dtd: ><!ENTITY % topic-info-types "(no-topic-nesting)?"> >The SVN copies do not declare this entity, and as a result the default is >picked up, and <topic> now allows nesting of <topic> > >10. The strictTaskbodyConstraint domain entity name is invalid. The name >of the entity name has changed from "taskbody-constraints" to >"strictTaskbody-constraints". This will break any user of the constraint, >and also violates the spec (spec requires the entity use the literal >element name). I believe this error was introduced recently, it was not >in my copy of SVN until I updated on Friday. > >11. The map.mod module drops the default empty declaration of >included-domains (it is still in topic.mod). > >12. Every element has its attribute entity (such as alt.attributes) >declared twice. Strictly speaking this does not introduce errors but >really needs to be fixed (it's confusing and looks bad). > >13. Many of the DTD modules have bad headers (every ENT file I spot >checked and several of the MOD files). The ENT files appear to grab their >headers (including public ID info) from the equivalent MOD file. > >Nit-picks that we would fix if creating these by hand: >- In topic.mod, the entities "info-types" and "topic-info-types" are each >declared twice >- In machinery taskbody constraint, %prelreqs and %closereqs are each >declared twice >- DITA 1.2 /1.3 in SVN has added an entity idElements; it's empty and >unused, so not sure of the purpose > >Robert D Anderson >IBM Authoring Tools Development >Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ )

> can do). When you say "bad headers" do you simply mean that the headers
> don't match the original 1.2 headers for the .ent files?

I spot-checked the ones in the base directory, and they all seemed to be literal copies of the header from the .mod file. As a result, the comments (such as Updated X on date Y) all reflect changes made in another file, and they all list the public ID / system ID for the mod file. For example, in the file hazardstatementDomain.ent, we have this sequence of comment lines:
<!--  Refer to this file by the following public identifier or an  -->
<!--       appropriate system identifier                           -->
<!-- PUBLIC "-//OASIS//ELEMENTS DITA Hazard Statement Domain//EN"  -->
<!--       Delivered as file "hazardstatementDomain.mod"                -->

The odd idElements entity really isn't a problem - we've had more annoying entities in the past - I'm mostly just calling that out as a difference between the actual 1.2 and SVN 1.2. I think most of the other items are more urgent for fixing.

Thanks -

Robert D Anderson
IBM Authoring Tools Development
Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ )

Eliot Kimber <ekimber@contrext.com> wrote on 10/20/2014 09:46:54:

> From: Eliot Kimber <ekimber@contrext.com>
> To: Robert D Anderson/Rochester/IBM@IBMUS, DITA TC <dita@lists.oasis-open.org>
> Date: 10/20/2014 09:47
> Subject: Re: [dita] Issues in DTDs checked into SVN
>
> I'll take a look at all of these. I have documented variances from the
> original 1.2 declarations that are an unavoidable consequence of
> generating the DTDs and XSDs, but most of these do no fall into that
> category, I don't think.
>
> On issue 13, headers for .ent files, that's a problem: since the RNG does
> not have separate mod and ent files, there's only one header comment in
> the RNG source, so not obvious how to produce different headers for .mod
> and .ent files unless we include both headers in the RNG source (which we
> can do). When you say "bad headers" do you simply mean that the headers
> don't match the original 1.2 headers for the .ent files?
>
> The idElements pattern in the RNG is required in order to handle the
> RNG-specific rules for patterns involving elements with ID-type
> attributes. I can ensure that those patterns don't get reflected in the
> generated DTDs and XSDs.
>
> I'm still working out details of constraint module generation: there's
> some complexity there. The current code is not always generating correct
> DTD declarations. It may be necessary to special case all constraint
> module generation, I'm not sure yet.
>
> I'll add these issues as cards in the Trello board I set up for the
> remaining grammar generation issues:
> https://trello.com/b/y42J0sEq/rng-to-dtd-and-xsd-conversion
>
> Cheers,
>
> E.
>
> —————
> Eliot Kimber, Owner
> Contrext, LLC
> http://contrext.com
>
>
>
>
> On 10/20/14, 9:27 AM, "Robert D Anderson" <robander@us.ibm.com> wrote:
>
> >Hi,
> >
> >After Kris found the DTD issue with the (map mapgroup-d) token last week,
> >I realized that the copy of DITA 1.2 checked in to SVN did not match the
> >copy of DITA 1.2 shipped by OASIS. I did some comparison of the shipped
> >DITA 1.2 and SVN 1.2, and found many inconsistencies. Everything I've
> >listed below is also an issue for the DITA 1.3 DTDs. I found all of these
> >issues by comparing 1.2 level DTDs - not comparing the RNGs - so this
> >presumably mixes RNG issues with conversion tool issues, and also means
> >that this testing did not cover anything new to 1.3.
> >
> >1. All embedded files are missing public IDs. These have been present
> >since 1.0, and in 1.2 we explicitly made all internal public IDs use
> >version-specific values. This was done by TC request, and they should be
> >added back.
> >
> >2. Several entity declarations have been removed from the DTD. Some are
> >unlikely to cause problems but all could potentially be referenced in
> >specialized DTDs (I know of DTDs that reference the first one):
> >- %rel-atts in topic.mod: deprecated in 1.2 but left in for backwards
> >compatibility
> >- %hazard.cnt used to set content models in hazard domain
> >- In the tblDecl.mod, unlikely to cause problems but for completeness:
> >-- Entity %bodyatt; removed from table attribute list - was empty,
> >present as part of OASIS table model
> >-- Dropped entity %tbl.table.name; (value was "table") -- previously used
> >to define the element name
> >-- Dropped entity %titles; (value was "title?") -- it was declared but
> >unused in OASIS DITA 1.2
> >
> >3. The error Kris found in the mapgroup domain token is still present in
> >the glossref domain and learningMapDomain tokens (both claim to be based
> >on topic).
> >
> >4. The domain token for learningAssessment has changed - not sure if this
> >is intentional - from "(topic learningBase+learningInteractionBase-d
> >learningAssessment)" to "(topic learningAssessment)"
> >
> >5. In ditabase.dtd, the declaration of <dita> has dropped several
> >attributes. The following are present in DITA 1.2 but missing in DITA 1.3:
> >%arch-atts -- @ditaarch:DITAArchVersion, @xmlns:ditaarch,
> >%debug-atts -- @xtrc, @xtrc.
> >Related: in 1.3 we've added @translate and @dir to the existing @xml:lang
> >-- I think these should be added using the common localization-atts group
> >rather than declared independently?
> >
> >6. The task domain entity (topic task) has been dropped from
> >ditabase.dtd, task.dtd, and machineryTask.dtd
> >
> >7. glossary.dtd is broken: it is supposed to redirect to glossentry.dtd
> >but actually does nothing
> >
> >8. The topic containment entities are broken in learningContent.dtd. The
> >learningContent element previously allowed lots of children, currently
> >1.3 changed this to only allow <no-topic-nesting>:
> >1.2 as shipped: <!ENTITY % learningContent-info-types "((concept task
> >reference topic)*, (learningAssessment)?, (learningSummary)?)">
> >1.2 and 1.3 in SVN: <!ENTITY % learningContent-info-types
> >"no-topic-nesting"
> >>
> >
> >9. Related - the content model for <topic> inside the learningContent
> >doctype has also changed, pretty sure this was not intentional. DITA 1.2
> >shipped with this declaration in learningContent.dtd:
> ><!ENTITY % topic-info-types "(no-topic-nesting)?">
> >The SVN copies do not declare this entity, and as a result the default is
> >picked up, and <topic> now allows nesting of <topic>
> >
> >10. The strictTaskbodyConstraint domain entity name is invalid. The name
> >of the entity name has changed from "taskbody-constraints" to
> >"strictTaskbody-constraints". This will break any user of the constraint,
> >and also violates the spec (spec requires the entity use the literal
> >element name). I believe this error was introduced recently, it was not
> >in my copy of SVN until I updated on Friday.
> >
> >11. The map.mod module drops the default empty declaration of
> >included-domains (it is still in topic.mod).
> >
> >12. Every element has its attribute entity (such as alt.attributes)
> >declared twice. Strictly speaking this does not introduce errors but
> >really needs to be fixed (it's confusing and looks bad).
> >
> >13. Many of the DTD modules have bad headers (every ENT file I spot
> >checked and several of the MOD files). The ENT files appear to grab their
> >headers (including public ID info) from the equivalent MOD file.
> >
> >Nit-picks that we would fix if creating these by hand:
> >- In topic.mod, the entities "info-types" and "topic-info-types" are each
> >declared twice
> >- In machinery taskbody constraint, %prelreqs and %closereqs are each
> >declared twice
> >- DITA 1.2 /1.3 in SVN has added an entity idElements; it's empty and
> >unused, so not sure of the purpose
> >
> >Robert D Anderson
> >IBM Authoring Tools Development
> >Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ )
>
>

OK, we'll have this as an agenda item for tomorrow's TC call. Eliot, it sounds as if the TC needs to know more about your test processes, and that perhaps we need to have TC members do manual comparisons of the official 1.2 DTD files and the 1.2 DTD files in SVN that I assume you generated from the RNG. Robert, how far did you get with your manual comparison? We might also need to consider a date by which we freeze the generation of DTD from RNG, and handle everything manually after that point. We need to have packages publicly available so that we can gather statement of use ... Best, Kris Kristen James Eberlein Chair, OASIS DITA Technical Committee Principal consultant, Eberlein Consulting www.eberleinconsulting.com +1 919 682-2290; kriseberlein (skype) On 10/20/2014 10:27 AM, Robert D Anderson wrote: Hi, After Kris found the DTD issue with the (map mapgroup-d) token last week, I realized that the copy of DITA 1.2 checked in to SVN did not match the copy of DITA 1.2 shipped by OASIS. I did some comparison of the shipped DITA 1.2 and SVN 1.2, and found many inconsistencies. Everything I've listed below is also an issue for the DITA 1.3 DTDs. I found all of these issues by comparing 1.2 level DTDs - not comparing the RNGs - so this presumably mixes RNG issues with conversion tool issues, and also means that this testing did not cover anything new to 1.3. 1. All embedded files are missing public IDs. These have been present since 1.0, and in 1.2 we explicitly made all internal public IDs use version-specific values. This was done by TC request, and they should be added back. 2. Several entity declarations have been removed from the DTD. Some are unlikely to cause problems but all could potentially be referenced in specialized DTDs (I know of DTDs that reference the first one): - %rel-atts in topic.mod: deprecated in 1.2 but left in for backwards compatibility - %hazard.cnt used to set content models in hazard domain - In the tblDecl.mod, unlikely to cause problems but for completeness: -- Entity %bodyatt; removed from table attribute list - was empty, present as part of OASIS table model -- Dropped entity %tbl.table.name; (value was table ) -- previously used to define the element name -- Dropped entity %titles; (value was title? ) -- it was declared but unused in OASIS DITA 1.2 3. The error Kris found in the mapgroup domain token is still present in the glossref domain and learningMapDomain tokens (both claim to be based on topic). 4. The domain token for learningAssessment has changed - not sure if this is intentional - from (topic learningBase+learningInteractionBase-d learningAssessment) to (topic learningAssessment) 5. In ditabase.dtd, the declaration of <dita> has dropped several attributes. The following are present in DITA 1.2 but missing in DITA 1.3: %arch-atts -- @ditaarch:DITAArchVersion, @xmlns:ditaarch, %debug-atts -- @xtrc, @xtrc. Related: in 1.3 we've added @translate and @dir to the existing @xml:lang -- I think these should be added using the common localization-atts group rather than declared independently? 6. The task domain entity (topic task) has been dropped from ditabase.dtd, task.dtd, and machineryTask.dtd 7. glossary.dtd is broken: it is supposed to redirect to glossentry.dtd but actually does nothing 8. The topic containment entities are broken in learningContent.dtd. The learningContent element previously allowed lots of children, currently 1.3 changed this to only allow <no-topic-nesting>: 1.2 as shipped: < !ENTITY  % learningContent-info-types ((concept task reference topic)*, (learningAssessment)?, (learningSummary)?) > 1.2 and 1.3 in SVN: < !ENTITY  % learningContent-info-types no-topic-nesting > 9. Related - the content model for <topic> inside the learningContent doctype has also changed, pretty sure this was not intentional. DITA 1.2 shipped with this declaration in learningContent.dtd: < !ENTITY  % topic-info-types (no-topic-nesting)? > The SVN copies do not declare this entity, and as a result the default is picked up, and <topic> now allows nesting of <topic> 10. The strictTaskbodyConstraint domain entity name is invalid. The name of the entity name has changed from taskbody-constraints to strictTaskbody-constraints . This will break any user of the constraint, and also violates the spec (spec requires the entity use the literal element name). I believe this error was introduced recently, it was not in my copy of SVN until I updated on Friday. 11. The map.mod module drops the default empty declaration of included-domains (it is still in topic.mod). 12. Every element has its attribute entity (such as alt.attributes) declared twice. Strictly speaking this does not introduce errors but really needs to be fixed (it's confusing and looks bad). 13. Many of the DTD modules have bad headers (every ENT file I spot checked and several of the MOD files). The ENT files appear to grab their headers (including public ID info) from the equivalent MOD file. Nit-picks that we would fix if creating these by hand: - In topic.mod, the entities info-types and topic-info-types are each declared twice - In machinery taskbody constraint, %prelreqs and %closereqs are each declared twice - DITA 1.2 /1.3 in SVN has added an entity idElements; it's empty and unused, so not sure of the purpose Robert D Anderson IBM Authoring Tools Development Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ )

Hi Kris, I actually didn't get anywhere at all with a manual comparison. For some reason I was under the impression that the tool had generated exact copies of 1.2, but when I tried to do a file diff, it was too difficult due to all the changes (in spacing, in order of entity declarations, and with all the extra attribute entities). FWIW we had a lot of spacing changes from DITA 1.1 to 1.2, so the surprise was mostly that I couldn't diff actual 1.2 with (what I thought was) 1.2 in SVN. I ended up resurrecting my dreaded testing tools from the 1.2 process. It creates a relatively well formatted XML version of the DTD: alpha-sorted elements with content models and attribute lists/values. It also lists all declared entities. I ran that tool on the shipped copy of DITA 1.2, and then again on the copy of 1.2 in SVN, and compared the results. That comparison resulted in the full list below. FWIW this method also highlighted that the 1.2 doctypes in SVN declare <tasktroubleshooting> as a child of taskbody in the task constraint module; I didn't mention that earlier because it is not a problem for 1.3, but this would be a problem for anybody that thought they could get DITA 1.2 from SVN. Robert D Anderson IBM Authoring Tools Development Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ ) Kristen James Eberlein ---10/20/2014 10:32:04---OK, we'll have this as an agenda item for tomorrow's TC call.  Eliot, it sounds as if the TC needs t From: Kristen James Eberlein <kris@eberleinconsulting.com> To: dita@lists.oasis-open.org Date: 10/20/2014 10:32 Subject: Re: [dita] Issues in DTDs checked into SVN Sent by: <dita@lists.oasis-open.org> OK, we'll have this as an agenda item for tomorrow's TC call. Eliot, it sounds as if the TC needs to know more about your test processes, and that perhaps we need to have TC members do manual comparisons of the official 1.2 DTD files and the 1.2 DTD files in SVN that I assume you generated from the RNG. Robert, how far did you get with your manual comparison? We might also need to consider a date by which we freeze the generation of DTD from RNG, and handle everything manually after that point. We need to have packages publicly available so that we can gather statement of use ... Best, Kris Kristen James Eberlein Chair, OASIS DITA Technical Committee Principal consultant, Eberlein Consulting www.eberleinconsulting.com +1 919 682-2290; kriseberlein (skype) On 10/20/2014 10:27 AM, Robert D Anderson wrote: Hi, After Kris found the DTD issue with the (map mapgroup-d) token last week, I realized that the copy of DITA 1.2 checked in to SVN did not match the copy of DITA 1.2 shipped by OASIS. I did some comparison of the shipped DITA 1.2 and SVN 1.2, and found many inconsistencies. Everything I've listed below is also an issue for the DITA 1.3 DTDs. I found all of these issues by comparing 1.2 level DTDs - not comparing the RNGs - so this presumably mixes RNG issues with conversion tool issues, and also means that this testing did not cover anything new to 1.3. 1. All embedded files are missing public IDs. These have been present since 1.0, and in 1.2 we explicitly made all internal public IDs use version-specific values. This was done by TC request, and they should be added back. 2. Several entity declarations have been removed from the DTD. Some are unlikely to cause problems but all could potentially be referenced in specialized DTDs (I know of DTDs that reference the first one): - %rel-atts in topic.mod: deprecated in 1.2 but left in for backwards compatibility - %hazard.cnt used to set content models in hazard domain - In the tblDecl.mod, unlikely to cause problems but for completeness: -- Entity %bodyatt; removed from table attribute list - was empty, present as part of OASIS table model -- Dropped entity %tbl.table.name; (value was "table") -- previously used to define the element name -- Dropped entity %titles; (value was "title?") -- it was declared but unused in OASIS DITA 1.2 3. The error Kris found in the mapgroup domain token is still present in the glossref domain and learningMapDomain tokens (both claim to be based on topic). 4. The domain token for learningAssessment has changed - not sure if this is intentional - from " (topic learningBase+learningInteractionBase-d learningAssessment) " to " (topic learningAssessment) " 5. In ditabase.dtd, the declaration of <dita> has dropped several attributes. The following are present in DITA 1.2 but missing in DITA 1.3: %arch-atts -- @ditaarch:DITAArchVersion, @xmlns:ditaarch, %debug-atts -- @xtrc, @xtrc. Related: in 1.3 we've added @translate and @dir to the existing @xml:lang -- I think these should be added using the common localization-atts group rather than declared independently? 6. The task domain entity (topic task) has been dropped from ditabase.dtd, task.dtd, and machineryTask.dtd 7. glossary.dtd is broken: it is supposed to redirect to glossentry.dtd but actually does nothing 8. The topic containment entities are broken in learningContent.dtd. The learningContent element previously allowed lots of children, currently 1.3 changed this to only allow <no-topic-nesting>: 1.2 as shipped: < !ENTITY  % learningContent-info-types "((concept task reference topic)*, (learningAssessment)?, (learningSummary)?)" > 1.2 and 1.3 in SVN: < !ENTITY  % learningContent-info-types "no-topic-nesting" > 9. Related - the content model for <topic> inside the learningContent doctype has also changed, pretty sure this was not intentional. DITA 1.2 shipped with this declaration in learningContent.dtd: < !ENTITY  % topic-info-types "(no-topic-nesting)?" > The SVN copies do not declare this entity, and as a result the default is picked up, and <topic> now allows nesting of <topic> 10. The strictTaskbodyConstraint domain entity name is invalid. The name of the entity name has changed from "taskbody-constraints" to "strictTaskbody-constraints". This will break any user of the constraint, and also violates the spec (spec requires the entity use the literal element name). I believe this error was introduced recently, it was not in my copy of SVN until I updated on Friday. 11. The map.mod module drops the default empty declaration of included-domains (it is still in topic.mod). 12. Every element has its attribute entity (such as alt.attributes) declared twice. Strictly speaking this does not introduce errors but really needs to be fixed (it's confusing and looks bad). 13. Many of the DTD modules have bad headers (every ENT file I spot checked and several of the MOD files). The ENT files appear to grab their headers (including public ID info) from the equivalent MOD file. Nit-picks that we would fix if creating these by hand: - In topic.mod, the entities "info-types" and "topic-info-types" are each declared twice - In machinery taskbody constraint, %prelreqs and %closereqs are each declared twice - DITA 1.2 /1.3 in SVN has added an entity idElements; it's empty and unused, so not sure of the purpose Robert D Anderson IBM Authoring Tools Development Chief Architect, DITA Open Toolkit ( http://dita-ot.sourceforge.net/ )   --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php  

The test process is documented in the file doctypes/test/how-to.dita in SVN. The current test automation is only validating that the DTDs are syntactically valid, meaning that they are both valid in general and that they implement the intended rules, as verified by the test documents validated against them. It would be possible to automate deeper checks but I haven't had bandwidth for that--if anybody can contribute additional tests that would be good, otherwise we'll have to depend on inspection of the sort Robert did. Note that with Oxygen it's easy to do a diff of two directories, which is how I've been doing my compares. I also created a set of DITA 1.2 exemplars that I was using to check the generation--those reflected the unavoidable variances due to generation but otherwise reflected the original DITA 1.2 DTDs. I'm not sure where those are at the moment--I'll have to track them down. I have been doing manual comparisons against the 1.2 DTDs as I go but I haven't done so in a while and it looks like some issues have crept in to the generation process (like the duplicated .attributes parameter entity declarations). Doing any hand work of the DTDs except as a short-term expedience for review purposes is not acceptable: the generation process is such that any required syntax details can be handled as special case processing where necessary. If hand work is required then it undoes the value of the generation process. It is also a given that the generated DTDs cannot be an exact match to the 1.2 originals. But the variances should be minimal. I think everything Robert identified is legitimate and fixable without too much effort. Cheers, E. ————— Eliot Kimber, Owner Contrext, LLC http://contrext.com On 10/20/14, 10:31 AM, "Kristen James Eberlein" <kris@eberleinconsulting.com> wrote: > > > > > > OK, we'll have this as an agenda item > for tomorrow's TC call. > > Eliot, it sounds as if the TC needs to know more about your test > processes, and that perhaps we need to have TC members do manual > comparisons of the official 1.2 DTD files and the 1.2 DTD files in > SVN that I assume you generated from the RNG. Robert, how far did > you get with your manual comparison? > > We might also need to consider a date by which we freeze the > generation of DTD from RNG, and handle everything manually after > that point. > > We need to have packages publicly available so that we can gather > statement of use ... > > Best, > Kris > > Kristen James Eberlein > Chair, OASIS DITA Technical Committee > Principal consultant, Eberlein Consulting > www.eberleinconsulting.com < http://www.eberleinconsulting.com > > +1 919 682-2290; kriseberlein (skype) > > > On 10/20/2014 10:27 AM, Robert D Anderson wrote: > > > Hi, > > After Kris found the DTD issue > with the (map mapgroup-d) token last week, I realized that the > copy of DITA 1.2 checked in to SVN did not match the copy of > DITA 1.2 shipped by OASIS. I did some comparison of the > shipped DITA 1.2 and SVN 1.2, and found many inconsistencies. > Everything I've listed below is also an issue for the DITA 1.3 > DTDs. I found all of these issues by comparing 1.2 level DTDs > - not comparing the RNGs - so this presumably mixes RNG issues > with conversion tool issues, and also means that this testing > did not cover anything new to 1.3. > > 1. All embedded files are > missing public IDs. These have been present since 1.0, and in > 1.2 we explicitly made all internal public IDs use > version-specific values. This was done by TC request, and they > should be added back. > > 2. Several entity declarations > have been removed from the DTD. Some are unlikely to cause > problems but all could potentially be referenced in > specialized DTDs (I know of DTDs that reference the first > one): > - %rel-atts in topic.mod: > deprecated in 1.2 but left in for backwards compatibility > - %hazard.cnt used to set > content models in hazard domain > - In the tblDecl.mod, unlikely > to cause problems but for completeness: > -- Entity %bodyatt; removed > from table attribute list - was empty, present as part of > OASIS table model > -- Dropped entity > %tbl.table.name; (value was "table") -- previously used to > define the element name > -- Dropped entity %titles; > (value was "title?") -- it was declared but unused in OASIS > DITA 1.2 > > 3. The error Kris found in the > mapgroup domain token is still present in the glossref domain > and learningMapDomain tokens (both claim to be based on > topic). > > 4. The domain token for > learningAssessment has changed - not sure if this is > intentional - from "(topic > learningBase+learningInteractionBase-d learningAssessment)" to >"(topic learningAssessment)" > > 5. In ditabase.dtd, the > declaration of <dita> has dropped several attributes. > The following are present in DITA 1.2 but missing in DITA 1.3: > %arch-atts -- > @ditaarch:DITAArchVersion, @xmlns:ditaarch, > %debug-atts -- @xtrc, @xtrc. > Related: in 1.3 we've added > @translate and @dir to the existing @xml:lang -- I think these > should be added using the common localization-atts group > rather than declared independently? > > 6. The task domain entity > (topic task) has been dropped from ditabase.dtd, task.dtd, and > machineryTask.dtd > > 7. glossary.dtd is broken: it > is supposed to redirect to glossentry.dtd but actually does > nothing > > 8. The topic containment > entities are broken in learningContent.dtd. The > learningContent element previously allowed lots of children, > currently 1.3 changed this to only allow > <no-topic-nesting>: > 1.2 as shipped: <!ENTITY % learningContent-info-types "((concept > task > reference topic)*, (learningAssessment)?, > (learningSummary)?)"> > 1.2 and 1.3 in SVN: <!ENTITY % learningContent-info-types > "no-topic-nesting" > > > > 9. Related - the content model > for <topic> inside the learningContent doctype has also > changed, pretty sure this was not intentional. DITA 1.2 > shipped with this declaration in learningContent.dtd: > <!ENTITY % topic-info-types "(no-topic-nesting)?"> > The SVN copies do not declare > this entity, and as a result the default is picked up, and > <topic> now allows nesting of <topic> > > 10. The > strictTaskbodyConstraint domain entity name is invalid. The > name of the entity name has changed from > "taskbody-constraints" to "strictTaskbody-constraints". This > will break any user of the constraint, and also violates the > spec (spec requires the entity use the literal element name). > I believe this error was introduced recently, it was not in my > copy of SVN until I updated on Friday. > > 11. The map.mod module drops > the default empty declaration of included-domains (it is still > in topic.mod). > > 12. Every element has its > attribute entity (such as alt.attributes) declared twice. > Strictly speaking this does not introduce errors but really > needs to be fixed (it's confusing and looks bad). > > 13. Many of the DTD modules > have bad headers (every ENT file I spot checked and several of > the MOD files). The ENT files appear to grab their headers > (including public ID info) from the equivalent MOD file. > > Nit-picks that we would fix if > creating these by hand: > - In topic.mod, the entities > "info-types" and "topic-info-types" are each declared twice > - In machinery taskbody > constraint, %prelreqs and %closereqs are each declared twice > - DITA 1.2 /1.3 in SVN has added an > entity idElements; it's empty and unused, so not sure of the > purpose > > Robert D Anderson > IBM Authoring Tools Development > Chief Architect, DITA Open Toolkit >( http://dita-ot.sourceforge.net/ ) > > > > > > > >--------------------------------------------------------------------- >To unsubscribe from this mail list, you must leave the OASIS TC that >generates this mail. Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php > >