In starting to implement a working processor that does cross-publication linking in the way Michael and I have outlined I have come to realize that we've overlooked one aspect: references to elements within topics. In all the discussion to date, including my long exploration quoted below, we have only considered mapping *keys* to rendered targets. As Michael asserted and as I think I confirmed, this can be done by generating new key definitions to the rendered result of a given key-bound resource (e.g., what a topic becomes in a given rendered result). However... For references to elements within topics, it's not so easy. Consider the use case of two references from a topic in Map A to two figures in a topic in Map B., with the intent that the cross-publication references get you to the figures, not just it's containing topic: Map A: <map> ... <mapref scope="peer" processing-role="resource-only" href="../map-b/keydefs/map-b-keydefs.ditamap" /> ... <topicref href="a-01.dita"/> ... </map> Topic A-01.dita, used from map Map-A.ditamap: <topic id="a-01"><title>Crossrefs to Figs in Map B</title> <body> <p>See <xref keyref="topic-b-01/fig-01"/>.</p> <p>See <xref keyref="topic-b-01/fig-02"/>.</p> </body> </topic> Map B: Map B Keydefs (map-b/keydefs/map-b-keydefs.ditamap): <map> <keydef keys="topic-b-01" scope="peer" href="../topics/b-01.dita" /> </map> Topic b-01: <topic id="b-01"><title>Two Useful Figures</title> <body> <fig id="fig-01"> ... </fig> <fig id="fig-02"> ... </fig> </body> </topic> The references from topic a-01.dita to b-01.dita *as authored* are clear and can be resolved reliably in the source. However, because a key-based reference to a topic ID includes the element ID *at the point of reference*, not in the key definition, it means that simply rewriting the key definition to point to the topic as rendered is insufficient: you must also map the ID part of the original keyref into the appropriate output-specific address and there is no way to know what that is *except* to maintain an output-specific element-id=to-rendered-location map. There's no way to know what the output-specific address is because that's entirely a function of the output processor and it could do anything it wants. Even in the case of the current HTML transform, where there is a well-understood mapping from source IDs to output IDs, it's not 100% because of the use of @chunk. When you chunk you may create a situation where the processor has to synthesize new IDs and at that point you can't reliably predict what those might be. In the case of monolithic outputs like PDF, it is not possible to have a simple one-to-one mapping from source IDs to output IDs because result IDs need to be unique across the publication but source IDs only need to be unique within a given topic or map document. DITA, today, does not provide any interchange format for that mapping. That is, there is nothing provided by maps today that defines a maping of element IDs to anything--maps only address topics, maps, and non-DITA resources, and thus are incapable of saying anything about elements within topics. In my thinking about this in the past, I had presumed an all-knowing Process Manager that would maintain this mapping as a black box--in that case, there is no need for interchange because there is exactly one actor, the Process Manager. However, Michael makes a strong case for processor interoperation and I think that any solution we come up with has to support interchange of intermediate address resolution data among processors, which means we have to standardize the mapping. The mapping itself has to reflect the following information for each element mapped (e.g., as rows in a table): 1. The deliverable the mapping applies to, as represented by the Rendition Definition (as outlined in my information below). The Redition Definition includes the root map to be processed and all the processing options used to produce the rendition. E.g., "map-b.ditamap to PDF". If you have one mapping per deliverable then the deliverable can be a property of the mapping as a whole and need not be repeated in every row. 2. The containing topic or map, represented by the keys bound to it in the context of the deliverable's root map and the resource bound to those keys. A given topic or map may have any number of keys bound to it. 3. The source element ID (e.g., "fig-01" 4. The rendition-specific address, e.g., "map-b.pdf#anchor-d123" Entries are looked up by the keyname/element ID pair by which an element is referenced. In fact you could use the literal key-based fragment identifier as a lookup key for the data, since DITA's syntax for key-based element addresses is simple and consistent. Represented as a literal table the mapping table for the map-b-to-PDF rendition might look like this: +-----------------------------------------------+ keyname Element ID Rendition address -------------+------------+-------------------- topic-b-01 fig-01 anchor-d123 -------------+------------+-------------------- topic-b-01 fig-02 anchor-d546 +-----------------------------------------------+ If interchange is a requirement, which I think it is, then we need some XML markup for this. I think the right approach would be to define a specialized topic type that captures or points to the rendition definition metadata (which is on my plate to define) and that uses a specialized <simpletable> to capture the mapping pretty much as I've outlined it here. I would not worry about normalizing the data but would keep it as a simple as possible since the point is efficient run-time processing, not database purity. Such a table would be easy to query with XPath or could be read directly into SQL databases if necessary. I would expect a large-scale Production Manager to use a relational database to manage large volumes of mapping data but I could also see using an XQuery database in the context of an XML-primary CMS that already uses something like MarkLogic or eXist. I will add the definition of this mapping topic to my existing processing data representation design task. Cheers, Eliot On 9/13/11 1:45 PM, "Eliot Kimber" <
ekimber@reallysi.com> wrote: > In thinking about this more, I think that Michael's approach of thinking of > the rendition-specific key-to-target binding as being a literal DITA map > with literal key definitions is a useful one. It provides a clear syntax for > capturing the binding for interchange purposes, will always work for > distributed processing scenarios, and gives us a clear basis on which to > discuss data details. I will use this approach from now on in my > discussions. > > I explore the processing implications and possibilities in some detail > below, but I think my difference with Michael comes down to: > > Is it possible to keep the two key spaces for two publications distinct or > must you combine them? I say keep them distinct by enabling addressing of > keys in the context of specific root maps. Michael says combine them so that > existing processors "just work" once you swap in rendition-specific key > bindings, at the cost of requiring coordination of the key names across the > maps involved. > > If we stipulate that in both cases the actual rendition processing to create > working links is done by "swapping out" the target map as authored for an > equivalent map that contains rendition-specific key bindings, then there are > no actual processing differences in our two models--the only difference is > in the details of how those rendition-specific maps are coordinated or used, > which is all implementation detail. > > That is, the processing doesn't require or disallow my all-knowing > Processing Manager and fully allows Michael's completely informal and > distributed processing environment. All the differences in these two models > are implementation details. > > This "swapping out" must be done by specifying a mapping from the target map > as authored (e.g., "map-b.ditamap") to the rendition-specific map to use > instead (e.g., "map-b-PDF.ditamap") as an input to a rendition process. That > mapping has to be known regardless of how the processing is done. It could > be specified as a parameter or it could be specified as instructions to the > human setting up the production, who then reflects the knowledge by > modifying the input map. The functional result is the same. > > Given that mapping, a human or processor can thus reliably render key > references as authored to working links in the rendition, as long as the > rendition-specific key bindings are correct. > > Thus the mechanism by which rendition-specific keys are communicated to or > used by a processor is an implementation detail. The only question is what > does the processor have to do to resolve the keys? Do they always have > exactly one key space or do they need to handle one or more key spaces? > > My approach, which requires a new fragment identifier in order to point to > specific keys in the context of specific root maps, reliably keeps distinct > key spaces distinct and removes the need to coordinate names across key > spaces. I think this is essential. It requires processors to handle one or > more key spaces, but I don't think that should pose a problem in practice > because if you can construct one key space you can just as easily construct > 100 key spaces. Since the universe has more than one map I would hope that > engineers of DITA-aware systems instinctively provide for the possibility of > multiple key spaces. > > Michael's approach requires combining the key spaces of otherwise separate > publications into a single unified key space. This simplifies processing > where the rendition-specific maps are used literally to implement > cross-publication linking using DITA 1.2 processing, but at the cost of > requiring coordination across all the key spaces that might be combined. > > I think that this coordination is impossible in the general, distributed > case, because you may want to link to a publication over which you have no > control and that happens to duplicate some keys in your publication that you > do not want to resolve to that publication. > > The only solution in that case is to keep the key spaces separate. DITA 1.2 > clearly defines the notion of key space so there can't be any ambiguity > about what is intended when you address a key in the context of a given root > map and it shouldn't be a surprise to any processor that there might be more > than one key space in play at any given point in time (because the universe > contains more than one map). > > In the case where you have, for some reason, multiple maps that contribute > to a single rendered publication through some process, it would be up to > that process to generate the appropriate rendition-specific map but it could > do it. In that case there might be a many-to-one mapping from maps as > authored to intermediate maps, but the processing will still work just as it > would for the simpler case of one map exactly equal to one publication. > > So I think the question remains: do we allow referencing across key spaces > in a way that keeps key spaces distinct or do we require that all maps that > might want to participate in cross-publication links share a single unified > key space that requires coordination of all key names across those maps? > > I feel strongly that the latter is not acceptable or sustainable and that > the implementation cost of allowing cross-key-space referencing is low and > is, in fact, arguably inherent in the DITA 1.2 architecture because it > formally defines the concept of key space. In the case of the Toolkit in > particular, I will personally implement the processing required if that's a > barrier. > > It is certainly the case that key-aware editors and component management > systems already have to manage multiple key spaces if they allow management > of multiple maps, which they all do as far as I know (e.g., OxygenXML, > Arbortext Editor 6, XMetal 6). [I don't know of any CMS systems that today > actually manage keys or provide key-resolution services but there might be > some. I'm actively working on adding that functionality to our CMS products, > but it's a low product priority right now.] > > The purpose of the rest of this message is to try to define a general > abstract processing model or environment that fits both my > tightly-controlled approach and Michael's arbitrarily distributed model. My > intent is to define some common vocabulary and appropriate abstractions that > let us focus on the general requirements with out worrying too much about > implementation details. > > Michael is presuming (but not requiring) an environment where there is no > central all-knowing rendition system that maintains knowledge about all the > renditions and the key-to-rendition mappings. I was assuming an all-knowing > Production Manager. But I think for both of us those are implementation > details that don't really change the problem. We were both presuming that > *something* had required knowledge of the renditions involved and the > intents of the renderers--in my case it was a management system, in > Michael's it was the humans requesting the renditions. But I think the > knowledge required in both cases in the same, the only difference is how > that knowledge is captured or communicated, which is an implementation > detail. > > The following discussion reflects the real case of the DITA 1.2 spec, where > we have a single content set that needs to be published in at least two > ways: as a single publication combining the Architectural Spec and the > Language Reference and as two separate publications, the Architectural Spec > and the Language Reference, with cross references between the two > publications *as rendered*. I have tried to reflect this case with the > smallest illustrative data set. > > Note that in the case of the DITA spec all the content is authored by a > single, coordinated group, so it is possible to coordinate the key names > across all the publication packages that might be applied to the content. > This does not reflect the more general distributed case where you may want > to link to renditions of a publication you only have read-only access to and > for which there is no coordinate of its key names with your key names. > > Let us have three maps, Map A, Map B, and Map AB, and two topics, Topic 1 > and Topic 2. > > The author of Topic 1 creates a link to Topic 2 because Topic 1 depends > rhetorically on Topic 2. This is the DITA Spec case, where the arch spec > points to language reference topics (and visa versa). > > Topic 1 looks like this: > > <topic id="topic-01"> > <title>Topic One</title> > <body> > <p>See <xref keyref="topic-02"/>.</p> > </body. > </topic> > > Topic 2 looks like this: > > <topic id="topic-02"> > <title>Topic Two</title> > <body> > <p>Something important to Topic 1.</p> > </body. > </topic> > > Map AB includes both topics: > > <map> > <title>Map AB</title> > <keydef > keys="topic-01" > href="topics/topic-01.dita" > /> > <keydef > keys="topic-02" > href="topics/topic-02.dita" > / >> > <topicref keyref="topic-01"/> > <topicref keyref="topic-02"/> > </map> > > This is the full DITA spec case, where all the topics are used in the scope > of a single root map. No processing ambiguity. > > The other case is where we have two publications, Map A and Map B: > > Map A: > > <map> > <title>Map A</title> > <keydef > keys="topic-01" > href="topics/topic-01.dita" > /> > <keydef > keys="topic-02" > href="????" > format="????" > scope="????" > / >> > <topicref keyref="topic-01"/> > <!-- NOTE: No reference to topic-02 --> > </map> > > Map B: > > <map> > <title>Map B</title> > <keydef > keys="topic-02" > href="topics/topic-02.dita" > / >> > <!-- NOTE: No reference to topic-01 --> > <topicref keyref="topic-02"/> > </map> > > > Processing the publications: > > When Map B is rendered to a given output we can capture the key-to-address > mapping in some way, such as Michael's keydefs, e.g.: > > Map B-PDF: > > <map> > <title>Map B PDF-specific keys</title> > <keydef keys="topic-02" > href="/workspace/output/map-b/pdf/map-b.pdf#unique-01" > format="pdf" > scope="external" > /> > <map> > > That's as good as any other way to capture the information and I'm happy to > stipulate that this is how it is always captured for the purpose of > processing interchange. This leaves open the possibility of manual or > automatic inclusion of the map into the publication map as I think Michael > is describing in his processing model. How the map is used is an > implementation detail if the map is not literally included by a map author > separate from a specific rendition process action. > > When Map A is rendered the questions then are: > > Question 1. What should the keydef for key "topic-02" look like in Map A? > > My proposal is currently: > > <keydef keys="topic-02" > href="map-b.ditamap#keyname::topic-02" > format="ditamap" > scope="peer" > /> > > Where the fragment identifier is a strawman for a fragment ID that is > unambiguously a reference to a key in the scope of the key space defined by > root map map-b.ditamap. > > Michael's example is: > > <mapref processing-role="resource-only" href="map-b.ditamap"/> > > If I understand Michael's approach, he is simply including Map B as a > resource-only map so that the keys have a binding. However, his form of > inclusion doesn't make it clear that the intent is that those keys are > treated as a separate key space. I think that is essential. That is the > intent of my using scope="peer". It doesn't keep the two key spaces separate > and therefore requires that the key names not conflict between the two root > maps. > > His approach does allow swapping in of the rendition-specific bindings for > Map B given the map-as-authored-to-rendition-map mapping stipulated above as > a necessary parameter to the rendition process. But it still requires a > single unified key space across maps A and B. > > In the context of processing Map A as authored outside the context of a > specific rendition there would be nothing to indicate that map B's keys are > not defining resources directly required by Map A. For example, a process > that takes a map and produces a package of all of Map A's dependencies would > also gather up everything used by Map B even though they're not really > direct dependencies of Map A. (Such a processor is part of the open-source > DITA for Publishers project and is also in the Open Toolkit.) > > If the mapref specified scope="peer" that would avoid the dependency > confusion but wouldn't avoid the key space combination because there's no > separate direct binding of key in Map A to key in Map B as in my approach. > > In both cases we're pointing to the map defining the keys, the difference in > my approach is that I'm also pointing to the key within the map and using > @scope to make it clear that I'm not simply using Map B's key definitions to > include resources as part of Map A's content, which is otherwise the > implication per the DITA 1.2 rules. > > In my proposal, because there's an additional layer of indirection between > the key as referenced in the context of Map A and the key as referenced in > the key definition in Map A, the key names need not be coordinated between > the two maps. That is, if Map B defined the key for Topic 2 as > "second-topic", my form of keydef could be: > > <keydef keys="topic-02" > href="map-b.ditamap#keyname::second-topic" > format="ditamap" > scope="peer" > /> > > And the original reference from Topic 1 would continue to work in both Map A > and Map AB. > > I think that even if we don't address the keys via fragment ID that we have > to distinguish references to peer and external key sets. > > Question 2. How does the agent (person or processor) rendering Map A specify > which rendition of Map B some or all of the links to Map B should point to? > > That is, given that there is both a PDF rendition and an HTML rendition of > Map B, the choices are: > > - The PDF rendition > > - The HTML rendition > > - Both renditions (multiple links generated from a single source link, or > some intermediate fan-out link or whatever). > > Does this decision need to be made on a per-link basis or on a per-rendition > of Map A basis? > > My thinking to date had been that like would always link to like, but > Michael is correct to say that that can't be the only option, so it has to > be either a build-time decision or an authoring-time decision. > > I think it needs to be a build-time decision determined by how you define > the mapping of map-as-authored to rendition-specific map. Anything else > would require additional per-key-definition syntax or metadata conventions > that I think would be impractical in practice. I suppose if it came to it, > you could modify the rendition-specific map to reflect exactly what you > wanted and maintain that manually. > > Another fact, which I never stated but that Michael correctly pointed out, > is that a given rendition is not identified just by the rendition type (PDF, > HTML, etc.) but by all the runtime parameters that define it, including the > active DITAVAL conditions, any processor-specific runtime options, the > rendition-specific key-to-address mappings, etc. > > In my model, there is a Processing Manager that manages all rendition > processing applied to a set of known content, e.g., all the publications > managed within a given system. The Processing Manager abstracts the notion > of "rendition" through a Rendition Definition object, which captures all the > input parameters for a given rendition, e.g. "PDF, DITAVAL > platform="windows", PDF option set "foo". > > The implication of Rendition Definition is that the same input rendered > using the same Rendition Definition will produce the same (or functionally > equivalent) output. > > Michael's model assumes there is no Processing Manager but that processing > happens where it happens and people coordinate however they do it. However, > the abstract notion of Rendition Definition is the same: you have to now > what all the parameters were in order to reproduce the rendition. So in > Michael's distributed world the Rendition Definition might be implemented as > notes scribbled on your desk blotter or an email from the supplier of the > rendition you want to link to, or whatever, but the information content is > the same regardless. > > We can now define a "rendition instance" as being a Rendition > Definition/input map pair. Two different input maps that use the same > Rendition Definition will have "consistent" or "compatible" output (that is, > they'll reflect the same set of runtime options). > > This is all, I think, equivalent to Michael saying "the person who renders > the map has to specify the appropriate DITAVAL files, rendition-specific key > bindings, and on". > > So my notion of Rendition Definition is either literal, as in my Processing > Manager system, or virtual, reflected in the knowledge of the person doing > the rendering, but in both cases, the same information is represented. > > In my model all rendering is done by the same processors, so that > coordination of intermediate data (key-to-rendition-location mappings) is > obviously easy to do. > > But Michael say "no, you can't assume that--it has to be more disconnected > and distributed", which is true. > > But I think the degree of distribution becomes an implementation detail. > That is, if Rendition Definitions include the key-to-rendition bindings, > it's only a question of how those bindings get communicated among processing > systems, not how they are captured or represented. Michael presumes or > stipulates a map-based syntax because that is reliably interchanged and > processed by DITA processors, which is fine. > > So now to the processing: > > If we stipulate that rendition-target-type is a runtime parameter, then when > I process Map A to a particular rendition and want links to be to the PDF > renditions of the target publications, part of the Rendition Definition is > "render cross-publication links to PDF renditions". But in fact, it needs to > be "Render cross-publication links to the rendition created using Rendition > Definition X", that is, a specific Rendition Definition reflecting a > specific set of rendition options, not just the base output type. > > In the context of the Open Toolkit, this means all the Ant parameters plus > all the Toolkit Plugins and environment variables that contribute to the > configuration of the transformation type used. Any other processing system > will have the equivalent set of options and starting conditions. > > Given this background, we can now explore the different processing use > cases: > > Processing Use Case 1: Don't have rendition-specific key bindings for Map B. > > If I process Map A and I don't have the rendition-specific key binding for > Map B, the processor has three choices: > > 1. Process Map B using the Rendition Definition specified and then use the > result to complete processing Map A. Note that this could be a literal > process or it could be "get on the phone to the supplier of Map B and ask > for the rendition-specific key binding that reflects the Rendition > Defininition you want". > > 2. Process Map A with placeholder or otherwise unresolveable links. > > 3. Fail the rendition of A. > > Processing Use Case 2: Do have rendition-specific key bindings for Map B. > > If we are using my cross-publication key definition approach, then there > needs to be an association between the root map map-b.ditamap and the > corresponding set of rendition-specific keys. Abstractly this is part of the > Rendition Definition parameters: you simply say "for map file > "map-b.ditamap" use key definitions "map-b-PDF.ditamap" or whatever. It > could also be done by literally change "map-b.ditamap" to > "map-b-PDF.ditamap" in the map source before processing it normally. In any > case, given that association, the processor can resolve references to keys > nominally defined in map-b.ditamap to the keys as bound in the > rendition-specific binding. > > Using Michael's approach it's essentially the same: you define the mapping > as a rendition parameter or otherwise modify the map to be processed to > replace "map-b.ditamap" with "map-b-PDF.ditamap". > > In the context of the Open Toolkit this would be something done as part of > the general preparation of the intermediate files used to then create the > final rendition (e.g., as part of the map-pull process or whatever makes > sense). The data manipulation required is consistent with the sort > manipulation the Toolkit already does. > > In the context of Processing Manager the mapping might be hidden behind a > key resolution API that takes the rendition-specific key definitions into > account. > > In any case, the result is the same--the rendered links reflect the bindings > defined in the rendition-specific key map. > > The only difference is the interaction or potential interference of the two > key spaces. > > In my approach, as explained above, there's no possible interference of the > two key spaces because they are kept distinct, while in Michael's approach > they key spaces are combined. > > I'm sure there's more to say on this subject but I'm out of time for now. > But I think I've made my point about as clearly as I can. > > Cheers, > > E. > > -- > Eliot Kimber > Senior Solutions Architect > "Bringing Strategy, Content, and Technology Together" > Main: 512.554.9368 >
www.reallysi.com >
www.rsuitecms.com > > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail. Follow this link to all your TCs in OASIS at: >
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php > -- Eliot Kimber Senior Solutions Architect, RSI Content Solutions "Bringing Strategy, Content, and Technology Together" Main: 512.554.9368
www.rsicms.com www.rsuitecms.com Book: DITA For Practitioners, from XML Press,
http://xmlpress.net/publications/dita/practitioners-1/