OASIS Static Analysis Results Interchange Format (SARIF) TC

Please read to the end for a not-quite-editorial change that you might want to comment on.   At TC #18, while reviewing the new values for file.roles , we discussed how to specify the run to which the files marked with these roles should be compared. It turns out that we had already encountered this issue with result.baselineState , where we wrote:   If the baselineId property (§3.11.4) of the containing run object (§3.11) is present, the baseline SHALL be the run specified by that baselineId .   If the baselineId property of the containing run object is absent, then a SARIF consumer will need out of band information available to determine the baseline.   But we can do better, now that we have the “Engineering system” conformance profile. So, following our usual practice of abstracting a concept we’ve encountered twice, we introduce the following definition in the Terminology section:   baseline run run that produces a baseline to which subsequent runs can be compared   With that definition in hand, we can improve our description of result.baselineState : 3.19.18 [result.]baselineState property A result object MAY contain a property named baselineState whose value is a string that specifies the state of this result with respect to some previous run, which we refer to as the “baseline run.” If baselineInstanceGuid (§3.11.4) is present on the containing run object (§3.11), its value SHALL specify the baseline run. If  baselineInstanceGuid is absent, the engineering system SHALL provide out of band information that determines the baseline run.   … and we can then use similar language for the new values of file.roles : 3.17.6 [file.]roles property …   The following role values denote files that have changed since the “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the containing run object (§3.11), its value SHALL specify the baseline run. If any of these role values are present but baselineInstanceGuid is absent, the engineering system SHALL provide out of band information that determines the baseline run.   NOTE: The statement that the new roles can be computed with respect to a run other than baselineInstanceGuid is not an editorial change. That is a substantive change that we agreed on in TC #18. The editorial change is to introduce that term “baseline run” and to use similar (and improved) language.   NOTE : Technically speaking, the new normative statement that “the engineering system SHALL provide out of band information…” – as opposed to the previous non-normative statement that “a SARIF consumer will need out of band information…” – is a substantive change, but I hope you’ll either (a) agree that it falls within editorial discretion, or (b) agree with the change! If not, let me know.   Larry  

I am uneasy about this change because it tries to specify things that are outside the scope of the standard. I think it would be better to have a non-normative note that a SARIF consumer will need out of band information baselineInstanceGuid is absent. David On 2018-05-17 11:42, Larry Golding (Comcast) wrote: /Please read to the end for a not-quite-editorial change that you might want to comment on./ At TC #18, while reviewing the new values for file.roles, we discussed how to specify the run to which the files marked with these roles should be compared. It turns out that we had already encountered this issue with result.baselineState, where we wrote: If the baselineId property (§3.11.4) of the containing run object (§3.11) is present, the baseline *SHALL* be the run specified by that baselineId. If the baselineId property of the containing run object is absent, then a SARIF consumer will need out of band information available to determine the baseline. But we can do better, now that we have the “Engineering system” conformance profile. So, following our usual practice of abstracting a concept we’ve encountered twice, we introduce the following definition in the Terminology section: baseline run run <#def_run> that produces a baseline <#def_baseline> to which subsequent runs can be compared With that definition in hand, we can improve our description of result.baselineState: *3.19.18 [result.]baselineState property*** A result object *MAY* contain a property named baselineState whose value is a string that specifies the state of this result with respect to some previous run, which we refer to as the “baseline run.” If baselineInstanceGuid (§3.11.4) is present on the containing run object (§3.11), its value *SHALL* specify the baseline run. If baselineInstanceGuid is absent, the engineering system *SHALL* provide out of band information that determines the baseline run. … and we can then use similar language for the new values of file.roles: *3.17.6 [file.]roles property*** … The following role values denote files that have changed since the “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the containing run object (§3.11), its value *SHALL* specify the baseline run. If any of these role values are present but baselineInstanceGuid is absent, the engineering system *SHALL* provide out of band information that determines the baseline run. *NOTE:* The statement that the new roles can be computed with respect to a run /other than/ baselineInstanceGuid is not an editorial change. That is a substantive change that we agreed on in TC #18. The editorial change is to introduce that term “baseline run” and to use similar (and improved) language. *NOTE*: Technically speaking, the new normative statement that “the engineering system *SHALL* provide out of band information…” – as opposed to the previous non-normative statement that “a SARIF consumer will need out of band information…” – is a substantive change, but I hope you’ll either (a) agree that it falls within editorial discretion, or (b) agree with the change! If not, let me know. Larry

Sorry, I meant "will need out of band information *if* baselineInstanceGuid is absent." David On 2018-05-17 15:42, David Keaton wrote:      I am uneasy about this change because it tries to specify things that are outside the scope of the standard.  I think it would be better to have a non-normative note that a SARIF consumer will need out of band information baselineInstanceGuid is absent.                     David On 2018-05-17 11:42, Larry Golding (Comcast) wrote: /Please read to the end for a not-quite-editorial change that you might want to comment on./ At TC #18, while reviewing the new values for file.roles, we discussed how to specify the run to which the files marked with these roles should be compared. It turns out that we had already encountered this issue with result.baselineState, where we wrote: If the baselineId property (§3.11.4) of the containing run object (§3.11) is present, the baseline *SHALL* be the run specified by that baselineId. If the baselineId property of the containing run object is absent, then a SARIF consumer will need out of band information available to determine the baseline. But we can do better, now that we have the “Engineering system” conformance profile. So, following our usual practice of abstracting a concept we’ve encountered twice, we introduce the following definition in the Terminology section: baseline run run <#def_run> that produces a baseline <#def_baseline> to which subsequent runs can be compared With that definition in hand, we can improve our description of result.baselineState:       *3.19.18 [result.]baselineState property*** A result object *MAY* contain a property named baselineState whose value is a string that specifies the state of this result with respect to some previous run, which we refer to as the “baseline run.” If baselineInstanceGuid (§3.11.4) is present on the containing run object (§3.11), its value *SHALL* specify the baseline run. If baselineInstanceGuid is absent, the engineering system *SHALL* provide out of band information that determines the baseline run. … and we can then use similar language for the new values of file.roles:         *3.17.6 [file.]roles property*** … The following role values denote files that have changed since the “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the containing run object (§3.11), its value *SHALL* specify the baseline run. If any of these role values are present but baselineInstanceGuid is absent, the engineering system *SHALL* provide out of band information that determines the baseline run. *NOTE:* The statement that the new roles can be computed with respect to a run /other than/ baselineInstanceGuid is not an editorial change. That is a substantive change that we agreed on in TC #18. The editorial change is to introduce that term “baseline run” and to use similar (and improved) language. *NOTE*: Technically speaking, the new normative statement that “the engineering system *SHALL* provide out of band information…” – as opposed to the previous non-normative statement that “a SARIF consumer will need out of band information…” – is a substantive change, but I hope you’ll either (a) agree that it falls within editorial discretion, or (b) agree with the change! If not, let me know. Larry --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

I don't agree that it's outside the scope of the standard. We just approved #166 which defined a Conformance Profile for the "engineering system". I proposed that change precisely so I we could make normative statements about how the engineering system behaves. We have a conformance profile for “result management system” for the same reason.   IMO if they have an engineering system that produces SARIF files that specify result.baselineState , but neglects to tell the consumer how to identify the baseline run, then they’re doing it wrong and we should say so.   Larry  
Original Message----- From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of David Keaton Sent: Thursday, May 17, 2018 3:45 PM To: Larry Golding (Comcast) <larrygolding@comcast.net>; sarif@lists.oasis-open.org Subject: Re: [sarif] Quasi-editorial change: the "baseline run"         Sorry, I meant "will need out of band information *if* baselineInstanceGuid is absent."                                                                                   David   On 2018-05-17 15:42, David Keaton wrote: >       I am uneasy about this change because it tries to specify things > that are outside the scope of the standard.  I think it would be > better to have a non-normative note that a SARIF consumer will need > out of band information baselineInstanceGuid is absent. > >                      David > > On 2018-05-17 11:42, Larry Golding (Comcast) wrote: >> /Please read to the end for a not-quite-editorial change that you >> might want to comment on./ >>   >> At TC #18, while reviewing the new values for file.roles, we >> discussed how to specify the run to which the files marked with these >> roles should be compared. It turns out that we had already >> encountered this issue with result.baselineState, where we wrote: >>   >> If the baselineId property (§3.11.4) of the containing run object >> (§3.11) is present, the baseline *SHALL* be the run specified by that >> baselineId. >>   >> If the baselineId property of the containing run object is absent, >> then a SARIF consumer will need out of band information available to >> determine the baseline. >>   >> But we can do better, now that we have the “Engineering system” >> conformance profile. So, following our usual practice of abstracting >> a concept we’ve encountered twice, we introduce the following >> definition in the Terminology section: >>   >> baseline run >>   >> run <#def_run> that produces a baseline <#def_baseline> to which >> subsequent runs can be compared >>   >> With that definition in hand, we can improve our description of >> result.baselineState: >>   >>   >>       *3.19.18 [result.]baselineState property*** >>   >> A result object *MAY* contain a property named baselineState whose >> value is a string that specifies the state of this result with >> respect to some previous run, which we refer to as the “baseline run.” >>   >> If baselineInstanceGuid (§3.11.4) is present on the containing run >> object (§3.11), its value *SHALL* specify the baseline run. If >> baselineInstanceGuid is absent, the engineering system *SHALL* >> provide out of band information that determines the baseline run. >>   >> … and we can then use similar language for the new values of file.roles: >>   >>   >>         *3.17.6 [file.]roles property*** >>   >> … >>   >> The following role values denote files that have changed since the >> “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the >> containing run object (§3.11), its value *SHALL* specify the baseline >> run. If any of these role values are present but baselineInstanceGuid >> is absent, the engineering system *SHALL* provide out of band >> information that determines the baseline run. >>   >> *NOTE:* The statement that the new roles can be computed with respect >> to a run /other than/ baselineInstanceGuid is not an editorial change. >> That is a substantive change that we agreed on in TC #18. The >> editorial change is to introduce that term “baseline run” and to use >> similar (and improved) language. >>   >> *NOTE*: Technically speaking, the new normative statement that “the >> engineering system *SHALL* provide out of band information…” – as >> opposed to the previous non-normative statement that “a SARIF >> consumer will need out of band information…” – is a substantive >> change, but I hope you’ll either (a) agree that it falls within >> editorial discretion, or (b) agree with the change! If not, let me know. >>   >> Larry >>   > > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail.  Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php     --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php  

I’m fine if the “engineering system” includes human beings who tell each other the answer. But it’s not ok to just forget what the answer was.   From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of Larry Golding (Comcast) Sent: Thursday, May 17, 2018 4:08 PM To: 'David Keaton' <dmk@dmk.com>; sarif@lists.oasis-open.org Subject: RE: [sarif] Quasi-editorial change: the "baseline run"   I don't agree that it's outside the scope of the standard. We just approved #166 which defined a Conformance Profile for the "engineering system". I proposed that change precisely so I we could make normative statements about how the engineering system behaves. We have a conformance profile for “result management system” for the same reason.   IMO if they have an engineering system that produces SARIF files that specify result.baselineState , but neglects to tell the consumer how to identify the baseline run, then they’re doing it wrong and we should say so.   Larry  
Original Message----- From: sarif@lists.oasis-open.org < sarif@lists.oasis-open.org > On Behalf Of David Keaton Sent: Thursday, May 17, 2018 3:45 PM To: Larry Golding (Comcast) < larrygolding@comcast.net >; sarif@lists.oasis-open.org Subject: Re: [sarif] Quasi-editorial change: the "baseline run"         Sorry, I meant "will need out of band information *if* baselineInstanceGuid is absent."                                                                                   David   On 2018-05-17 15:42, David Keaton wrote: >       I am uneasy about this change because it tries to specify things > that are outside the scope of the standard.  I think it would be > better to have a non-normative note that a SARIF consumer will need > out of band information baselineInstanceGuid is absent. > >                      David > > On 2018-05-17 11:42, Larry Golding (Comcast) wrote: >> /Please read to the end for a not-quite-editorial change that you >> might want to comment on./ >>   >> At TC #18, while reviewing the new values for file.roles, we >> discussed how to specify the run to which the files marked with these >> roles should be compared. It turns out that we had already >> encountered this issue with result.baselineState, where we wrote: >>   >> If the baselineId property (§3.11.4) of the containing run object >> (§3.11) is present, the baseline *SHALL* be the run specified by that >> baselineId. >>   >> If the baselineId property of the containing run object is absent, >> then a SARIF consumer will need out of band information available to >> determine the baseline. >>   >> But we can do better, now that we have the “Engineering system” >> conformance profile. So, following our usual practice of abstracting >> a concept we’ve encountered twice, we introduce the following >> definition in the Terminology section: >>   >> baseline run >>   >> run <#def_run> that produces a baseline <#def_baseline> to which >> subsequent runs can be compared >>   >> With that definition in hand, we can improve our description of >> result.baselineState: >>   >>   >>       *3.19.18 [result.]baselineState property*** >>   >> A result object *MAY* contain a property named baselineState whose >> value is a string that specifies the state of this result with >> respect to some previous run, which we refer to as the “baseline run.” >>   >> If baselineInstanceGuid (§3.11.4) is present on the containing run >> object (§3.11), its value *SHALL* specify the baseline run. If >> baselineInstanceGuid is absent, the engineering system *SHALL* >> provide out of band information that determines the baseline run. >>   >> … and we can then use similar language for the new values of file.roles: >>   >>   >>         *3.17.6 [file.]roles property*** >>   >> … >>   >> The following role values denote files that have changed since the >> “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the >> containing run object (§3.11), its value *SHALL* specify the baseline >> run. If any of these role values are present but baselineInstanceGuid >> is absent, the engineering system *SHALL* provide out of band >> information that determines the baseline run. >>   >> *NOTE:* The statement that the new roles can be computed with respect >> to a run /other than/ baselineInstanceGuid is not an editorial change. >> That is a substantive change that we agreed on in TC #18. The >> editorial change is to introduce that term “baseline run” and to use >> similar (and improved) language. >>   >> *NOTE*: Technically speaking, the new normative statement that “the >> engineering system *SHALL* provide out of band information…” – as >> opposed to the previous non-normative statement that “a SARIF >> consumer will need out of band information…” – is a substantive >> change, but I hope you’ll either (a) agree that it falls within >> editorial discretion, or (b) agree with the change! If not, let me know. >>   >> Larry >>   > > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail.  Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php     --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php  

I'm just letting you know that this is a frequent problem that standards run into. Once you start talking normatively about something that is out of band, it is like pulling a thread on a sweater. Some day, maybe a month from now, or maybe a year from now, someone will say that since you specified this one out-of-band item normatively, you have to specify something else, and the scope expands from there. It would be better to structure out-of-band items as non-normative if possible. I won't object; I'm just delivering the information. David On 05/17/2018 04:33 PM, Larry Golding (Comcast) wrote: I’m fine if the “engineering system” includes human beings who tell each other the answer. But it’s not ok to just forget what the answer was. *From:* sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> *On Behalf Of *Larry Golding (Comcast) *Sent:* Thursday, May 17, 2018 4:08 PM *To:* 'David Keaton' <dmk@dmk.com>; sarif@lists.oasis-open.org *Subject:* RE: [sarif] Quasi-editorial change: the "baseline run" I don't agree that it's outside the scope of the standard. We just approved #166 which defined a Conformance Profile for the "engineering system". I proposed that change precisely so I we could make normative statements about how the engineering system behaves. We have a conformance profile for “result management system” for the same reason. IMO if they have an engineering system that 1. produces SARIF files that specify result.baselineState, but 2. neglects to tell the consumer how to identify the baseline run, then they’re doing it wrong and we should say so. Larry
Original Message----- From: sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> On Behalf Of David Keaton Sent: Thursday, May 17, 2018 3:45 PM To: Larry Golding (Comcast) <larrygolding@comcast.net < mailto:larrygolding@comcast.net >>; sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > Subject: Re: [sarif] Quasi-editorial change: the "baseline run"       Sorry, I meant "will need out of band information *if* baselineInstanceGuid is absent."                                                                                 David On 2018-05-17 15:42, David Keaton wrote: >       I am uneasy about this change because it tries to specify things > that are outside the scope of the standard.  I think it would be > better to have a non-normative note that a SARIF consumer will need > out of band information baselineInstanceGuid is absent. > >                      David > > On 2018-05-17 11:42, Larry Golding (Comcast) wrote: >> /Please read to the end for a not-quite-editorial change that you >> might want to comment on./ >> >> At TC #18, while reviewing the new values for file.roles, we >> discussed how to specify the run to which the files marked with these >> roles should be compared. It turns out that we had already >> encountered this issue with result.baselineState, where we wrote: >> >> If the baselineId property (§3.11.4) of the containing run object >> (§3.11) is present, the baseline *SHALL* be the run specified by that >> baselineId. >> >> If the baselineId property of the containing run object is absent, >> then a SARIF consumer will need out of band information available to >> determine the baseline. >> >> But we can do better, now that we have the “Engineering system” >> conformance profile. So, following our usual practice of abstracting >> a concept we’ve encountered twice, we introduce the following >> definition in the Terminology section: >> >> baseline run >> >> run <#def_run> that produces a baseline <#def_baseline> to which >> subsequent runs can be compared >> >> With that definition in hand, we can improve our description of >> result.baselineState: >> >> >>       *3.19.18 [result.]baselineState property*** >> >> A result object *MAY* contain a property named baselineState whose >> value is a string that specifies the state of this result with >> respect to some previous run, which we refer to as the “baseline run.” >> >> If baselineInstanceGuid (§3.11.4) is present on the containing run >> object (§3.11), its value *SHALL* specify the baseline run. If >> baselineInstanceGuid is absent, the engineering system *SHALL* >> provide out of band information that determines the baseline run. >> >> … and we can then use similar language for the new values of file.roles: >> >> >>         *3.17.6 [file.]roles property*** >> >> … >> >> The following role values denote files that have changed since the >> “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the >> containing run object (§3.11), its value *SHALL* specify the baseline >> run. If any of these role values are present but baselineInstanceGuid >> is absent, the engineering system *SHALL* provide out of band >> information that determines the baseline run. >> >> *NOTE:* The statement that the new roles can be computed with respect >> to a run /other than/ baselineInstanceGuid is not an editorial change. >> That is a substantive change that we agreed on in TC #18. The >> editorial change is to introduce that term “baseline run” and to use >> similar (and improved) language. >> >> *NOTE*: Technically speaking, the new normative statement that “the >> engineering system *SHALL* provide out of band information…” – as >> opposed to the previous non-normative statement that “a SARIF >> consumer will need out of band information…” – is a substantive >> change, but I hope you’ll either (a) agree that it falls within >> editorial discretion, or (b) agree with the change! If not, let me know. >> >> Larry >> > > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail.  Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

That makes a lot of sense. I'm just surprised because we've been moving in the direction of defining conformance profiles for "out of band" participants in the SARIF ecosystem: #154: Define a "result management system" conformance profile #166: Define an "engineering system" conformance profile ... and then taking what were formerly "NOTEs" and turning them into normative statements about what these things should do. So could you please clarify: - Are you saying that we should not define these conformance profiles? - Or that we should define them, but not make -any- normative statements around them? - Or that it's ok to make normative statements around them, but only SHOULD or MAY statements (not SHALL statements)? - Or something else? I know you say you won't object, but I'm asking what your recommendation is, from among those options, or some other option I haven't thought of. Larry
Original Message----- From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of David Keaton Sent: Thursday, May 17, 2018 6:55 PM To: Larry Golding (Comcast) <larrygolding@comcast.net>; sarif@lists.oasis-open.org Subject: Re: [sarif] Quasi-editorial change: the "baseline run" I'm just letting you know that this is a frequent problem that standards run into. Once you start talking normatively about something that is out of band, it is like pulling a thread on a sweater. Some day, maybe a month from now, or maybe a year from now, someone will say that since you specified this one out-of-band item normatively, you have to specify something else, and the scope expands from there. It would be better to structure out-of-band items as non-normative if possible. I won't object; I'm just delivering the information. David On 05/17/2018 04:33 PM, Larry Golding (Comcast) wrote: > I’m fine if the “engineering system” includes human beings who tell > each other the answer. But it’s not ok to just forget what the answer was. > > *From:* sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> *On > Behalf Of *Larry Golding (Comcast) > *Sent:* Thursday, May 17, 2018 4:08 PM > *To:* 'David Keaton' <dmk@dmk.com>; sarif@lists.oasis-open.org > *Subject:* RE: [sarif] Quasi-editorial change: the "baseline run" > > I don't agree that it's outside the scope of the standard. We just > approved #166 which defined a Conformance Profile for the "engineering > system". I proposed that change precisely so I we could make normative > statements about how the engineering system behaves. We have a > conformance profile for “result management system” for the same reason. > > IMO if they have an engineering system that > > 1. produces SARIF files that specify result.baselineState, but 2. > neglects to tell the consumer how to identify the baseline run, > > then they’re doing it wrong and we should say so. > > Larry > >
Original Message----- > From: sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> On > Behalf Of David Keaton > Sent: Thursday, May 17, 2018 3:45 PM > To: Larry Golding (Comcast) <larrygolding@comcast.net > < mailto:larrygolding@comcast.net >>; sarif@lists.oasis-open.org > < mailto:sarif@lists.oasis-open.org > > Subject: Re: [sarif] Quasi-editorial change: the "baseline run" > > Sorry, I meant "will need out of band information *if* > baselineInstanceGuid is absent." > > > David > > On 2018-05-17 15:42, David Keaton wrote: > > > I am uneasy about this change because it tries to specify > things > > > that are outside the scope of the standard. I think it would be > > > better to have a non-normative note that a SARIF consumer will need > > > out of band information baselineInstanceGuid is absent. > > > > > > David > > > > > > On 2018-05-17 11:42, Larry Golding (Comcast) wrote: > > >> /Please read to the end for a not-quite-editorial change that you > > >> might want to comment on./ > > >> > > >> At TC #18, while reviewing the new values for file.roles, we > > >> discussed how to specify the run to which the files marked with > these > > >> roles should be compared. It turns out that we had already > > >> encountered this issue with result.baselineState, where we wrote: > > >> > > >> If the baselineId property (§3.11.4) of the containing run object > > >> (§3.11) is present, the baseline *SHALL* be the run specified by > that > > >> baselineId. > > >> > > >> If the baselineId property of the containing run object is absent, > > >> then a SARIF consumer will need out of band information available > to > > >> determine the baseline. > > >> > > >> But we can do better, now that we have the “Engineering system” > > >> conformance profile. So, following our usual practice of > abstracting > > >> a concept we’ve encountered twice, we introduce the following > > >> definition in the Terminology section: > > >> > > >> baseline run > > >> > > >> run <#def_run> that produces a baseline <#def_baseline> to which > > >> subsequent runs can be compared > > >> > > >> With that definition in hand, we can improve our description of > > >> result.baselineState: > > >> > > >> > > >> *3.19.18 [result.]baselineState property*** > > >> > > >> A result object *MAY* contain a property named baselineState whose > > >> value is a string that specifies the state of this result with > > >> respect to some previous run, which we refer to as the “baseline run.” > > >> > > >> If baselineInstanceGuid (§3.11.4) is present on the containing run > > >> object (§3.11), its value *SHALL* specify the baseline run. If > > >> baselineInstanceGuid is absent, the engineering system *SHALL* > > >> provide out of band information that determines the baseline run. > > >> > > >> … and we can then use similar language for the new values of file.roles: > > >> > > >> > > >> *3.17.6 [file.]roles property*** > > >> > > >> … > > >> > > >> The following role values denote files that have changed since the > > >> “baseline run”. If baselineInstanceGuid (§3.11.4) is present on > the > > >> containing run object (§3.11), its value *SHALL* specify the > baseline > > >> run. If any of these role values are present but > baselineInstanceGuid > > >> is absent, the engineering system *SHALL* provide out of band > > >> information that determines the baseline run. > > >> > > >> *NOTE:* The statement that the new roles can be computed with > respect > > >> to a run /other than/ baselineInstanceGuid is not an editorial change. > > >> That is a substantive change that we agreed on in TC #18. The > > >> editorial change is to introduce that term “baseline run” and to > use > > >> similar (and improved) language. > > >> > > >> *NOTE*: Technically speaking, the new normative statement that > “the > > >> engineering system *SHALL* provide out of band information…” – as > > >> opposed to the previous non-normative statement that “a SARIF > > >> consumer will need out of band information…” – is a substantive > > >> change, but I hope you’ll either (a) agree that it falls within > > >> editorial discretion, or (b) agree with the change! If not, let me know. > > >> > > >> Larry > > >> > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe from this mail list, you must leave the OASIS TC > that > > > generates this mail. Follow this link to all your TCs in OASIS at: > > > > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php > > --------------------------------------------------------------------- > > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail. Follow this link to all your TCs in OASIS at: > > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php > --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

Larry, Your intuition is good that there is a continuum. This one item struck me because it is at the far end, explicitly saying something is out of band but is specified normatively. Let me think about this over the weekend and maybe discuss with you further offline. Whatever we do, I do not want to cause a rewrite of a lot of text; I just wanted to point out a problem I saw. David On 05/18/2018 09:11 AM, Larry Golding (Comcast) wrote: That makes a lot of sense. I'm just surprised because we've been moving in the direction of defining conformance profiles for "out of band" participants in the SARIF ecosystem: #154: Define a "result management system" conformance profile #166: Define an "engineering system" conformance profile ... and then taking what were formerly "NOTEs" and turning them into normative statements about what these things should do. So could you please clarify: - Are you saying that we should not define these conformance profiles? - Or that we should define them, but not make -any- normative statements around them? - Or that it's ok to make normative statements around them, but only SHOULD or MAY statements (not SHALL statements)? - Or something else? I know you say you won't object, but I'm asking what your recommendation is, from among those options, or some other option I haven't thought of. Larry
Original Message----- From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of David Keaton Sent: Thursday, May 17, 2018 6:55 PM To: Larry Golding (Comcast) <larrygolding@comcast.net>; sarif@lists.oasis-open.org Subject: Re: [sarif] Quasi-editorial change: the "baseline run" I'm just letting you know that this is a frequent problem that standards run into. Once you start talking normatively about something that is out of band, it is like pulling a thread on a sweater. Some day, maybe a month from now, or maybe a year from now, someone will say that since you specified this one out-of-band item normatively, you have to specify something else, and the scope expands from there. It would be better to structure out-of-band items as non-normative if possible. I won't object; I'm just delivering the information. David On 05/17/2018 04:33 PM, Larry Golding (Comcast) wrote: I’m fine if the “engineering system” includes human beings who tell each other the answer. But it’s not ok to just forget what the answer was. *From:* sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> *On Behalf Of *Larry Golding (Comcast) *Sent:* Thursday, May 17, 2018 4:08 PM *To:* 'David Keaton' <dmk@dmk.com>; sarif@lists.oasis-open.org *Subject:* RE: [sarif] Quasi-editorial change: the "baseline run" I don't agree that it's outside the scope of the standard. We just approved #166 which defined a Conformance Profile for the "engineering system". I proposed that change precisely so I we could make normative statements about how the engineering system behaves. We have a conformance profile for “result management system” for the same reason. IMO if they have an engineering system that 1. produces SARIF files that specify result.baselineState, but 2. neglects to tell the consumer how to identify the baseline run, then they’re doing it wrong and we should say so. Larry
Original Message----- From: sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> On Behalf Of David Keaton Sent: Thursday, May 17, 2018 3:45 PM To: Larry Golding (Comcast) <larrygolding@comcast.net < mailto:larrygolding@comcast.net >>; sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > Subject: Re: [sarif] Quasi-editorial change: the "baseline run" Sorry, I meant "will need out of band information *if* baselineInstanceGuid is absent." David On 2018-05-17 15:42, David Keaton wrote: > I am uneasy about this change because it tries to specify things > that are outside the scope of the standard. I think it would be > better to have a non-normative note that a SARIF consumer will need > out of band information baselineInstanceGuid is absent. > > David > > On 2018-05-17 11:42, Larry Golding (Comcast) wrote: >> /Please read to the end for a not-quite-editorial change that you >> might want to comment on./ >> >> At TC #18, while reviewing the new values for file.roles, we >> discussed how to specify the run to which the files marked with these >> roles should be compared. It turns out that we had already >> encountered this issue with result.baselineState, where we wrote: >> >> If the baselineId property (§3.11.4) of the containing run object >> (§3.11) is present, the baseline *SHALL* be the run specified by that >> baselineId. >> >> If the baselineId property of the containing run object is absent, >> then a SARIF consumer will need out of band information available to >> determine the baseline. >> >> But we can do better, now that we have the “Engineering system” >> conformance profile. So, following our usual practice of abstracting >> a concept we’ve encountered twice, we introduce the following >> definition in the Terminology section: >> >> baseline run >> >> run <#def_run> that produces a baseline <#def_baseline> to which >> subsequent runs can be compared >> >> With that definition in hand, we can improve our description of >> result.baselineState: >> >> >> *3.19.18 [result.]baselineState property*** >> >> A result object *MAY* contain a property named baselineState whose >> value is a string that specifies the state of this result with >> respect to some previous run, which we refer to as the “baseline run.” >> >> If baselineInstanceGuid (§3.11.4) is present on the containing run >> object (§3.11), its value *SHALL* specify the baseline run. If >> baselineInstanceGuid is absent, the engineering system *SHALL* >> provide out of band information that determines the baseline run. >> >> … and we can then use similar language for the new values of file.roles: >> >> >> *3.17.6 [file.]roles property*** >> >> … >> >> The following role values denote files that have changed since the >> “baseline run”. If baselineInstanceGuid (§3.11.4) is present on the >> containing run object (§3.11), its value *SHALL* specify the baseline >> run. If any of these role values are present but baselineInstanceGuid >> is absent, the engineering system *SHALL* provide out of band >> information that determines the baseline run. >> >> *NOTE:* The statement that the new roles can be computed with respect >> to a run /other than/ baselineInstanceGuid is not an editorial change. >> That is a substantive change that we agreed on in TC #18. The >> editorial change is to introduce that term “baseline run” and to use >> similar (and improved) language. >> >> *NOTE*: Technically speaking, the new normative statement that “the >> engineering system *SHALL* provide out of band information…” – as >> opposed to the previous non-normative statement that “a SARIF >> consumer will need out of band information…” – is a substantive >> change, but I hope you’ll either (a) agree that it falls within >> editorial discretion, or (b) agree with the change! If not, let me know. >> >> Larry >> > > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail. Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php