Data Provenance (DPS) TC

 View Only
Expand all | Collapse all

Subject: Confirming initial work product details

  • 1.  Subject: Confirming initial work product details

    Posted 04-24-2025 16:33

    Hi everyone,

     

    To keep things moving, here's a quick summary of key decisions proposed so far for our initial work products:

     

    1. Metadata specification (Standards track – Committee Specification)

    ·         Editor: Stefan Hagen has volunteered

    ·         Format: Markdown

    ·         Abbreviation: dps-usecases

     

    1. Use cases document – Committee Note or Committee Specification?  Here we'd love feedback. The TC meeting surfaced two perspectives:

    ·         Option A: Treat use cases as a Committee Specification (as Duncan suggested), with potential for normative language and closer alignment to the metadata spec.

    ·         Option B: Treat it as a Committee Note (non-standards track), which is simpler procedurally and appropriate if the use cases are illustrative only.

     

    Please weigh in on which path makes the most sense. 

     

    We'll move forward if nobody objects to the Metadata specification format/editor/abbreviation suggestions by May 1.

     

    Thanks,

     

    Bryan, Fotis and Lisa

     

     

     

     

     

                                     



  • 2.  RE: Subject: Confirming initial work product details

    Posted 04-24-2025 17:38

    I think dps-usecases has issues. First, it is long. They used to limit us to 4 letters. Note this will be a filename precursor on multiple files. Second, it has a hyphen. Yes hyphens are allowed, but it turns out in practice they are also the separator used for what comes after the abbr so you end up with two kinds of hypens. Third - does DPS make sense? I assume it's short for Data Provenance Standard. We are a spec to start out with, hopefully will go all the way to standard. But aren't the use cases for data provenance (as opposed to for 'the' standard). I'd tend to leave off the "s" (ie either dp-usecases or better yet dpuc albeit that might be a horrible thing to say a lot). We might also consider "pv" instead of "dp". Most people think of "dp" as data processing. "v" is fairly unique so establishing us as the "provenance" experts (ie "pv" or "prv") might be better for general adoption. So maybe "prvu" and spec could be "prvs". 

    Nothing I'd fall on my sword over. Just things to consider. 



    ------------------------------
    Duncan Sparrell
    Chief Cyber Curmudgeion
    sFractal Consulting LLC
    Oakton VA
    703-828-8646
    ------------------------------



  • 3.  RE: Subject: Confirming initial work product details

    Posted 04-24-2025 17:46

    Wrt CN vs CS - I could go either way but favor CS. CS has more gravitas and can go to other standards bodies (eg ANSI/ITU/ISO/IEC) which CN cannot (because it hasn't been approved by all of OASIS, just our TC). I see the argument that "use cases aren't normative" but I would disagree. I expect in the future that the content of some of the information in the metadata could be spec'd separately for different use cases. For example use case 1 "shall" have field whatever filled with whatever but use case 2 only "should" have it filled (or maybe can't have it filled because you need to fill in field 2 instead, or it has a different set of 'whatever'). Just in the world of SBOMs for software supply chain, I can see that distinction mattering. If we were to want to do that, than the use cases would have to be normative.



    ------------------------------
    Duncan Sparrell
    Chief Cyber Curmudgeion
    sFractal Consulting LLC
    Oakton VA
    703-828-8646
    ------------------------------



  • 4.  RE: Subject: Confirming initial work product details

    Posted 04-25-2025 10:20

    I see normative specs as having conformance requirements and believe that the conformance section is required in a CS (Kelly can correct if I'm mistaken.)  And I don't see any use cases as being non-conforming – they tell a story and establish requirements, and levying conformance requirements on requirements creators does not make sense to me.  An example of a non-conforming use case might make the distinction clearer – something like "lawmakers make laws, but laws must be constitutional".

    So I favor CN.

     






  • 5.  RE: Subject: Confirming initial work product details

    Posted 04-30-2025 15:13

     

    As we transition from administrative setup to substantive work, we seek additional volunteers to help edit and lead the development of our key work products. Your involvement will be critical in shaping the final deliverables and ensuring we maintain momentum toward publication.

     

    We are currently looking for editors and contributors for the following deliverables:

    ·         Specification (Provenance model and metadata schema) – Stefan has kindly volunteered as editor, but we'd love to get contributors on this track

    ·         Executive briefing document (communicating the value and application of the standards) – we need at least one editor and several contributors

    ·         Use cases document (illustrating examples of standard application, pending final decision if normative or illustrative) - we need at least one editor and several contributors

     

    If you need help deciding whether to be an editor/contributor, here is the distinction: Editors collaborate with other TC members to collect, draft, and organize content. They coordinate contributions, suggest improvements, and help maintain consistency across documents. Editors are not responsible for creating all the content themselves (although they may contribute it), but for guiding, integrating, and shaping the overall deliverable.

     

    If you are interested in being an editor/contributor, please reply to this email and let us know:

    • Which deliverable(s) you would like to volunteer for
    • Whether you are volunteering as a primary editor, secondary editor, or contributor
    • Any areas of focus or expertise you bring that might be helpful

     

    We welcome and encourage multiple volunteers per deliverable to ensure continuity, a broader perspective, and a shared workload.

     

     

     

     

                                     






  • 6.  RE: Subject: Confirming initial work product details

    Posted 04-25-2025 10:30

    Oops – premature "Send".   I don't believe optionality affects the use cases themselves – UC A and UC B specify requirements, and the normative metadata CS must satisfy both A and B.  But how the CS satisfies the UCs is up to the designers; there could be separate structures A and B, or they could be a single structure with the fields not common to both UC A and UC B optional.

     






  • 7.  RE: Subject: Confirming initial work product details

    Posted 04-26-2025 09:42

    Although I argued for CS, I can live with CN. We also probably iterate it more often so that would favor CN as well. 



    ------------------------------
    Duncan Sparrell
    Chief Cyber Curmudgeion
    sFractal Consulting LLC
    Oakton VA
    703-828-8646
    ------------------------------



  • 8.  RE: Subject: Confirming initial work product details

    Posted 04-30-2025 15:20

    Use case deliverable – Standards track or illustrative

     

    My recommendation is to establish the use cases as illustrative examples, not as normative standards.

     

    Their purpose should be to demonstrate how the provenance metadata and model could be applied in different contexts, helping organizations understand practical uses and adapt the standard to their environments. They would also be used to check the metadata  standards to these real world examples knowing that they are examples but not all inclusive of the environments where the standard may be applied.

     

    The use cases must offer flexibility rather than imposing rigid or prescriptive requirements. Many organizations have their custom processes and systems, and we want the standards to be adaptable across diverse implementations, not limited by narrowly defined scenarios.

     

    In short:

    • Use cases = examples, not mandatory templates
    • They should inspire adoption, not constrain it
    • Our focus should stay on making the core standard robust, while keeping use cases as helpful, illustrative guidance

     

     

     

     

     

     

                                     






  • 9.  RE: Subject: Confirming initial work product details

    Posted 05-01-2025 19:40
    Dear members,

    On Wed, Apr 30, 2025, at 21:19, Lisa Bobbitt via OASIS wrote:
    > *Use case deliverable – Standards track or illustrative*
    >
    > My recommendation is to establish the use cases as illustrative examples, not as normative standards.
    >
    > Their purpose should be to demonstrate how the provenance metadata and model could be applied in different contexts, helping organizations understand practical uses and adapt the standard to their environments. They would also be used to check the metadata standards to these real world examples knowing that they are examples but not all inclusive of the environments where the standard may be applied.
    >
    > The use cases must offer flexibility rather than imposing rigid or prescriptive requirements. Many organizations have their custom processes and systems, and we want the standards to be adaptable across diverse implementations, not limited by narrowly defined scenarios.
    >
    > In short:
    >
    > • Use cases = examples, not mandatory templates
    > • They should inspire adoption, not constrain it
    > • Our focus should stay on making the core standard robust, while keeping use cases as helpful, illustrative guidance
    > [...]

    adding to that:

    1) Use cases as source and context for needed examples in core:

    I am assuming that we only have a JSON schema and a tool for SMB
    etc. support contributed as input for the core standard.
    So, I think we should consider using the contributed use cases
    mainly as examples in the core workproduct.
    We should not refer for such important albeit collateral
    functions from a normative to an informative-only document.
    Readers like a balanced and complete enough specification.

    Also, in reviewing IEEE standards during ballot I see a trend
    of adding extended informative annexes to specs.

    Main rationale: Examples are best informative parts adding to
    the "often terse and hard to understand and implement" normative
    parts in prose documents that were formerly known as a mix
    of schema and tools only.

    2) dedicated committee notes (informative-only) per use case:

    Maybe we should create committee notes per use case to really
    make them self-sufficient.

    Rationale: I expect specific use cases to not evolve really,
    but assume use cases will be added (so, the set will evolve).



    I kindly repeat my offer to support as editor for the core spec.

    > -------------------------------------------
    > Original Message:
    > Sent: 4/26/2025 9:42:00 AM
    > From: Duncan Sparrell
    > Subject: RE: Subject: Confirming initial work product details
    >
    > Although I argued for CS, I can live with CN. We also probably iterate it more often so that would favor CN as well.
    >
    >
    >
    > ------------------------------
    > Duncan Sparrell [...]

    All the best,
    Stefan.




  • 10.  RE: Subject: Confirming initial work product details

    Posted 05-03-2025 10:52

    This thread has several issues confounded together, all of which we need resolution on:

    The most substantive issue is should use cases be:

      • integral to committee spec (as in D&TA contribution)
        • my belief is consensus against this
      • it's own committee spec
        • my belief is consensus against this
      • it's own committee note
      • as one appendix to metadata CS for all use cases
      • as many appendices (one per use case) to metadata

    Note that we don't need to resolve this to 'start' work on the metadata spec (albeit it would be 'nicer' to since it makes it easier to start work on the use cases as well). Put also note that we DO need to resolve the other administrivia in this thread before we can start work on the metadata spec.

    One administrivia that we need at least an initial volunteer is editor for the metadata spec. Stefan has volunteered. I will also volunteer. Ideally someone else should also be an editor who has been involved in D&TA work prior to forming the TC. Note the editor is just that, an editor. Only the TC can make 'material' changes to the text. Earlier in this thread, the term editor and the term contributor were confounded. Any TC member can contribute and is encouraged to do so. If someone is an editor, they have to be explicit when proposing changes as to which role they are in when they make the change. Eg as editor, I can make a PR to conform to OASIS template language, or to correct a typo (as long as it's an immaterial change, material typos still require 'contribution'). Eg as contributor I can propose to adopt the D&TA paragraph whatever text (or some new text I wrote). Editor is a lot of administrivia and herding cats. However you did get you name on the final standard on the title page (every member who wants gets their name in the acknowledgments appendix) and you appear as the 'author' in any citations. I'll gladly give up being editor if too many other people volunteer. But in any case, we have enough that it is not a blocking item on starting.

    We do need to decide on our normative format - I believe we have consensus on markdown.

    We do need to decide on abbreviation. I have my opinions (as stated in early emails in the thread). I won't fall on my sword over but we do need to pick something before we can start. I think this should be a specific agenda topic if not resolved prior since it is a gating item. And any other gating mites should also get priority on agenda.



    ------------------------------
    Duncan Sparrell
    Chief Cyber Curmudgeion
    sFractal Consulting LLC
    Oakton VA
    703-828-8646
    ------------------------------



  • 11.  RE: Subject: Confirming initial work product details

    Posted 05-03-2025 11:49
    Dear members,

    On Sat, May 3, 2025, at 16:52, Duncan Sparrell via OASIS wrote:
    > This thread has several issues confounded together, all of which we need resolution on:
    >
    > The most substantive issue is should use cases be:
    >
    > * integral to committee spec (as in D&TA contribution)
    > * my belief is consensus against this
    > * it's own committee spec
    > * my belief is consensus against this
    > * it's own committee note
    > * as one appendix to metadata CS for all use cases
    > * as many appendices (one per use case) to metadata
    >

    Note, that I (Stefan) edited list item symbols as they were not
    rendered at all in my mail reader and thus the structure was lost]

    > Note that we don't need to resolve this to 'start' work on the metadata spec (albeit it would be 'nicer' to since it makes it easier to start work on the use cases as well). Put also note that we DO need to resolve the other administrivia in this thread before we can start work on the metadata spec.
    >
    > One administrivia that we need at least an initial volunteer is editor for the metadata spec. Stefan has volunteered. I will also volunteer. Ideally someone else should also be an editor who has been involved in D&TA work prior to forming the TC. Note the editor is just that, an editor. Only the TC can make 'material' changes to the text. Earlier in this thread, the term editor and the term contributor were confounded. Any TC member can contribute and is encouraged to do so. If someone is an editor, they have to be explicit when proposing changes as to which role they are in when they make the change. Eg as editor, I can make a PR to conform to OASIS template language, or to correct a typo (as long as it's an immaterial change, material typos still require 'contribution'). Eg as contributor I can propose to adopt the D&TA paragraph whatever text (or some new text I wrote). Editor is a lot of administrivia and herding cats. However you did get you name on the final standard on the title page (every member who wants gets their name in the acknowledgments appendix) and you appear as the 'author' in any citations. I'll gladly give up being editor if too many other people volunteer. But in any case, we have enough that it is not a blocking item on starting.

    I would love to collaborate with Duncan as editor. For some of you
    that may come as a surprise, given that Duncan and I are discussing
    in part controversial topics. My position on this is, that the
    prose we write will very muich profit from our differetn angles
    on the task. Especially, I would like us to construct a spec
    that at the same time has a convincing information model (not necessarily
    formulated in JADN) and is still accessible to everyone as it should.
    The latter to me comprises: Actionable information, human readability,
    and machine optimized accessibility where machines shine. The latter
    in verification, consistency, and transpiling schema information
    across language and format barriers.

    >
    > We do need to decide on our normative format - I believe we have consensus on markdown.

    Yes, please.

    > We do need to decide on abbreviation. I have my opinions (as stated in early emails in the thread). I won't fall on my sword over but we do need to pick something before we can start. I think this should be a specific agenda topic if not resolved prior since it is a gating item. And any other gating mites should also get priority on agenda.

    My experience since 2005 in OASIS and as editor is, that these
    abbreviations can wait until CSD01 and are best identified, when
    the content of these artifacts (where we need abbreviations
    as part of their file names for) is stabilized.


    Please, all have a great weekend!

    Cheers,
    Stefan.

    > Duncan Sparrell [...]
    > Original Message:
    > Sent: 05-01-2025 19:39
    > From: Stefan Hagen
    > Subject: Subject: Confirming initial work product details
    >
    > Dear members,
    >
    > On Wed, Apr 30, 2025, at 21:19, Lisa Bobbitt via OASIS wrote:
    > > *Use case deliverable – Standards track or illustrative*
    > >
    > > My recommendation is to establish the use cases as illustrative examples, not as normative standards.
    > >
    > > Their purpose should be to demonstrate how the provenance metadata and model could be applied in different contexts, helping organizations understand practical uses and adapt the standard to their environments. They would also be used to check the metadata standards to these real world examples knowing that they are examples but not all inclusive of the environments where the standard may be applied.
    > >
    > > The use cases must offer flexibility rather than imposing rigid or prescriptive requirements. Many organizations have their custom processes and systems, and we want the standards to be adaptable across diverse implementations, not limited by narrowly defined scenarios.
    > >
    > > In short:
    > >
    > > • Use cases = examples, not mandatory templates
    > > • They should inspire adoption, not constrain it
    > > • Our focus should stay on making the core standard robust, while keeping use cases as helpful, illustrative guidance
    > > [...]
    >
    > adding to that:
    >
    > 1) Use cases as source and context for needed examples in core:
    >
    > I am assuming that we only have a JSON schema and a tool for SMB
    > etc. support contributed as input for the core standard.
    > So, I think we should consider using the contributed use cases
    > mainly as examples in the core workproduct.
    > We should not refer for such important albeit collateral
    > functions from a normative to an informative-only document.
    > Readers like a balanced and complete enough specification.
    >
    > Also, in reviewing IEEE standards during ballot I see a trend
    > of adding extended informative annexes to specs.
    >
    > Main rationale: Examples are best informative parts adding to
    > the "often terse and hard to understand and implement" normative
    > parts in prose documents that were formerly known as a mix
    > of schema and tools only.
    >
    > 2) dedicated committee notes (informative-only) per use case:
    >
    > Maybe we should create committee notes per use case to really
    > make them self-sufficient.
    >
    > Rationale: I expect specific use cases to not evolve really,
    > but assume use cases will be added (so, the set will evolve).
    >
    >
    >
    > I kindly repeat my offer to support as editor for the core spec.
    >
    > > -------------------------------------------
    > > Original Message:
    > > Sent: 4/26/2025 9:42:00 AM
    > > From: Duncan Sparrell
    > > Subject: RE: Subject: Confirming initial work product details
    > >
    > > Although I argued for CS, I can live with CN. We also probably iterate it more often so that would favor CN as well.
    > >
    > >
    > >
    > > ------------------------------
    > > Duncan Sparrell [...]
    >
    > All the best,
    > Stefan.
    >
    >
    > Original Message:
    > Sent: 4/30/2025 3:20:00 PM
    > From: Lisa Bobbitt
    > Subject: RE: Subject: Confirming initial work product details
    >
    > *Use case deliverable – Standards track or illustrative*
    >
    > * *
    >
    > My recommendation is to establish the use cases as illustrative examples, not as normative standards.
    >
    >
    >
    > Their purpose should be to demonstrate how the provenance metadata and model could be applied in different contexts, helping organizations understand practical uses and adapt the standard to their environments. They would also be used to check the metadata standards to these real world examples knowing that they are examples but not all inclusive of the environments where the standard may be applied.
    >
    >
    >
    > The use cases must offer flexibility rather than imposing rigid or prescriptive requirements. Many organizations have their custom processes and systems, and we want the standards to be adaptable across diverse implementations, not limited by narrowly defined scenarios.
    >
    >
    >
    > In short:
    >
    > • Use cases = examples, not mandatory templates
    > • They should inspire adoption, not constrain it
    > • Our focus should stay on making the core standard robust, while keeping use cases as helpful, illustrative guidance
    >
    >
    >
    > Original Message:
    > Sent: 4/26/2025 9:42:00 AM
    > From: Duncan Sparrell
    > Subject: RE: Subject: Confirming initial work product details
    >
    > Although I argued for CS, I can live with CN. We also probably iterate it more often so that would favor CN as well.
    >
    >
    >
    > ------------------------------
    > Duncan Sparrell [...]




  • 12.  RE: Subject: Confirming initial work product details

    Posted 05-05-2025 10:35

    Re format,  I believe we have consensus on markdown too.

    I agree with Lisa on this item 8, I am leaning towards illustrative, but am flexible on this if other members feel strongly. Their purpose should be to demonstrate how the provenance metadata and model could be applied in different and broad contexts, promoting broad adoption and not an industry or use specific adoption. 

    I am interested in contributing on one or all of these three deliverables. While I am happy to work on any work product, I most prefer contributing to the Executive Briefing, and then Use Case Document. 



    ------------------------------
    Bryan Bortnick
    IBM
    ------------------------------