OASIS Static Analysis Results Interchange Format (SARIF) TC

I still prefer my design. I do not think SARIF should couple the mechanisms for resolving relative paths and version control provenance. Currently, nothing in SARIF requires the use of UriBaseIds, and nothing imposes any restrictions on UriBaseIds. If we use Michael's design, it comes with the requirement to use UriBaseIds. If you want to make general restrictions on UrIBaseIds so they work without surprises then at least the following restrictions must be enforced: 1) no two UriBaseIds can refer to the same path, 2) if the physical location of UriBaseId A is under the physical location of UriBaseId B, but no others, then the UriBaseId must use B as its uriBaseId, and 3) all fileLocations need to generated using the closest ancestor UriBaseId. Most tools I've seen output either an absolute path or a paths relative to a single location. Even without submodules, there may still be multiple directories checked out independent version control systems. Now there needs to at one UriBaseId for each, and tools (and engineering systems) will need to use these consistently and can't just us a BaseUriId to refer to the SOURCE or PACKAGE directory. With Michael's design a tool or engineering system adding version control details, needs to make add these UriBaseIds and possibly modify every location to conform to the restrictions above changing every baseUriIds and file location in the SARIF file. With my proposal, the versionControlDetails can be easily added as a mostly static piece of text without affecting any other fileLocation or BaseUriIds. Jim On 12/10/18 4:24 PM, Larry Golding (Myriad Consulting Inc) wrote: > I d like to hear from Jim before reverting to Michael s design. Jim, > please take a look at the change draft, and consider the arguments below. > > Thanks, > > Larry > > *From:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com> > *Sent:* Monday, December 10, 2018 9:53 AM > *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>; > Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC > Discussion List <sarif@lists.oasis-open.org> > *Subject:* RE: Change draft for #248: Associate file location with repo > > Oh, and I m sorry I keep saying my design , but actually the idea of > linking a versionControlDetails to a fileLocation /via/ a uriBaseId was > Michael s. > > *From:* sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> *On > Behalf Of *Larry Golding (Myriad Consulting Inc) > *Sent:* Monday, December 10, 2018 9:51 AM > *To:* Michael Fanning <Michael.Fanning@microsoft.com > < mailto:Michael.Fanning@microsoft.com >>; OASIS SARIF TC Discussion List > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> > *Subject:* [sarif] RE: Change draft for #248: Associate file location > with repo > > Both my design and Jim s accomplish that. The difference is that in my > design, the link from a versionControlsDetails object to a fileLocation > is purely a uriBaseId, whereas in Jim s design, the link is a > full-fledged fileLocation object (versionControlDetails.mappedTo). Jim s > point (as I understood it; see my inline commentary on his design in the > issue) was that his design simplified life for the SARIF producer > because the producer didn t need a separate uriBaseId for each repo. > > Having said that, and having taken the time to write the change draft, > I m now coming around to your point of view: that we ve traded producer > convenience for consumer complexity. Each consumer has to implement that > algorithm to get the right mapping. > > The counter-argument is: There are lots more producers (tools) than > consumers (viewers and result management systems). So we should favor > convenience for consumers. This is actually a design principle we ve > followed in the past. > > The counter-counter-argument is: This really only helps producers in > the case where there are submodules. In other cases, you would generally > have one uriBaseId per repo. Why are we complicating life for consumers, > just for the sake of consumer convenience in one scenario, and not the > most common scenario at that? > > One more thing: even though only consumers need to /implement/ Jim s > algorithm, both producers and consumers need to /read and understand > it/, because producers need to understand how to populate > originalUriBaseIds and versionControlDetails.mappedTo in Jim s design. > > TL;DR: I m coming back around to preferring the simplicity of my design. > > Larry > > *From:* Michael Fanning <Michael.Fanning@microsoft.com > < mailto:Michael.Fanning@microsoft.com >> > *Sent:* Monday, December 10, 2018 9:36 AM > *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com > < mailto:v-lgold@microsoft.com >>; Larry Golding (Myriad Consulting Inc) > <v-lgold@microsoft.com < mailto:v-lgold@microsoft.com >>; OASIS SARIF TC > Discussion List <sarif@lists.oasis-open.org > < mailto:sarif@lists.oasis-open.org >> > *Subject:* RE: Change draft for #248: Associate file location with repo > > I am concerned at the difficult in understanding this proposal. I also > am wondering whether we have clearly identified the goals of this design. > > Here s the scenario that originally drove this request > > 1. I have a SARIF log file and would like to recreate the enlistment > locally in order to get source files that match the analysis > 2. I see an originalUriBase that points to the non-deterministic root > of the enlistment > 3. I d like to find the version control details for this root and to > sync to it locally. > > Michael > > *From:* sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> *On > Behalf Of *Larry Golding (Myriad Consulting Inc) > *Sent:* Sunday, December 9, 2018 5:04 PM > *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com > < mailto:v-lgold@microsoft.com >>; OASIS SARIF TC Discussion List > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> > *Subject:* [sarif] RE: Change draft for #248: Associate file location > with repo > > That probably read snarkier than I intended ð Jim s description is > accurate and much more compact than it would have been if he d expressed > it without the symbols. It s just that readers of the spec will need > some help to understand it. > > *From:* sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org > > <sarif@lists.oasis-open.org < mailto:sarif@lists.oasis-open.org >> *On > Behalf Of *Larry Golding (Myriad Consulting Inc) > *Sent:* Sunday, December 9, 2018 5:01 PM > *To:* OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org > < mailto:sarif@lists.oasis-open.org >> > *Subject:* [sarif] Change draft for #248: Associate file location with repo > > I pushed a change draft for Issue #248 > < https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foasis-tcs%2Fsarif-spec%2Fissues%2F248&data=02%7C01%7Cv-lgold%40microsoft.com%7C538bb4ec21274f013e0008d65ec84d79%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636800611716554327&sdata=18EsQo85bKMbOCBmBPqOvuj7PQRnroPl%2B1CfNhS2DvY%3D&reserved=0 >: > Version control details not strongly associated with results : > > Documents/ChangeDrafts/Active/sarif-v2.0-issue-248-versionControlProvenance-file-mapping.docx > < https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foasis-tcs%2Fsarif-spec%2Fblob%2Fmaster%2FDocuments%2FChangeDrafts%2FActive%2Fsarif-v2.0-issue-248-versionControlProvenance-file-mapping.docx&data=02%7C01%7Cv-lgold%40microsoft.com%7C538bb4ec21274f013e0008d65ec84d79%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636800611716564335&sdata=5URBYh3J654dRERJ1Qs3FHzozkj4AmgzF%2Bf6BKXFBtk%3D&reserved=0 > > > We will move its adoption at TC 29 on Wednesday December 12^th . > > We had previously proposed a design for this, but Jim objected and > proposed what, after a careful reading, I think is a better one. The new > change draft follows Jim s design. > > Jim, academic that he is ð, expressed his mapping algorithm rather > abstractly. Fortunately, I was also an academic in a former life, so I > interspersed some comments into Jim s description of his design > < https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foasis-tcs%2Fsarif-spec%2Fissues%2F248%23issuecomment-437942306&data=02%7C01%7Cv-lgold%40microsoft.com%7C538bb4ec21274f013e0008d65ec84d79%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636800611716574344&sdata=N5dgtx1BV8nJUaBDh%2Fihg1lFD2XBcGZ8q9e9dMxPa7M%3D&reserved=0 >, > and extensively annotated his example in the text of the change draft. > *Please follow the example closely* to make sure you know what Jim is > proposing. > > Thanks, > > Larry >

I find your argument compelling, with one minor
quibble . Inline:
 

Original Message-----
From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of James Kupsch
Sent: Tuesday, December 11, 2018 12:55 PM
To: sarif@lists.oasis-open.org
Subject: Re: [sarif] RE: Change draft for #248: Associate file location with repo
 
I still prefer my design.  I do not think SARIF should couple the mechanisms for resolving relative paths and version control provenance.
 
[Larry] Yes, the mechanism for resolving relative paths is the uriBaseId. I agree it's not logically necessary for the relative path resolution mechanism to be the same as the version control provenance resolution mechanism.
OTOH , there's nothing inherently wrong with using the same mechanism for more than one purpose, if it's fit for both purposes; I think characterizing that reuse as "coupling" is a bit loaded. Someone
arguing the other side could say "Isn't it nice that we can _reuse_ the same mechanism for relative paths and version control provenance!" But that's moot, because...
 
[Larry] The feature of your design that I like most is that version control provenance resolution is mediated by a fileLocation -- a fundamental concept that appears throughout the specification. It is a higher-level concept than uriBaseId
(which it encapsulates), and you're right -- resolution via fileLocation means you don't have to use uriBaseId if you don't want to.
 
Currently, nothing in SARIF requires the use of UriBaseIds, and nothing imposes any restrictions on UriBaseIds.

 
[Larry] True.
 
If we use Michael's design, it comes with the requirement to use UriBaseIds.
 
[Larry] True.
 
If you want to make general restrictions on UrIBaseIds so they work without surprises then at least the following restrictions must be enforced:

1) no two UriBaseIds can refer to the same path,
2) if the physical location of UriBaseId A is under the physical location of UriBaseId B, but no others, then the UriBaseId must use B as its uriBaseId, and
3) all fileLocations need to generated using the closest ancestor UriBaseId.
 
[Larry] I confess I don t understand why Michael s design implies these three restrictions, but I don t think I need to, because I find the rest of your argument compelling even if these restrictions weren t necessary:
 
Most tools I've seen output either an absolute path or a paths relative to a single location.  Even without submodules, there may still be multiple directories checked out independent version control systems.

Now there needs to at one UriBaseId for each, and tools (and engineering systems) will need to use these consistently and can't just us a BaseUriId to refer to the SOURCE or PACKAGE directory.
 
[Larry] True. If your source tree is laid out from a single root, with various repos underneath them, your design allows
all locations to be expressed relative to the tree s root, whereas Michael s design requires one uriBaseId per repo.
 
With Michael's design a tool or engineering system adding version control details, needs to
make add these UriBaseIds and possibly modify every location to conform to the restrictions above changing every baseUriIds and file location in the SARIF file.
 
[Larry] True.
 
With my proposal, the versionControlDetails can be easily added as a mostly static piece of text without affecting any other fileLocation or BaseUriIds.
 
[Larry] True. In your design, any fileLocation will work, because the first step in your algorithm is to fully expand the fileLocation , so it doesn t matter which uriBaseId, if any, the fileLocation started with. If you want to add
version control details to an existing file, all you have to do is add a mappedTo property to the versionControlDetails object and the mappedTo doesn t have to use a uriBaseId, either.
 
[Larry] Bottom line, I think these advantages outweigh the complexity of the resolution algorithm (which really
isn t that complex, when you see an example it just needs to be expressed formally, as I ve previously observed).
 
Jim
 
 
 
On 12/10/18 4:24 PM, Larry Golding (Myriad Consulting Inc) wrote:
> I d like to hear from Jim before reverting to Michael s design. Jim,

> please take a look at the change draft, and consider the arguments below.
>
> Thanks,
>
> Larry
>
> *From:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>
> *Sent:* Monday, December 10, 2018 9:53 AM
> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>;

> Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC

> Discussion List <sarif@lists.oasis-open.org>
> *Subject:* RE: Change draft for #248: Associate file location with

> repo
>
> Oh, and I m sorry I keep saying my design , but actually the idea

> of linking a versionControlDetails to a fileLocation /via/ a uriBaseId

> was Michael s.
>
> *From:* sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On

> Behalf Of *Larry Golding (Myriad Consulting Inc)
> *Sent:* Monday, December 10, 2018 9:51 AM
> *To:* Michael Fanning <Michael.Fanning@microsoft.com

> <mailto:Michael.Fanning@microsoft.com>>; OASIS SARIF TC Discussion

> List <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>>
> *Subject:* [sarif] RE: Change draft for #248: Associate file location

> with repo
>
> Both my design and Jim s accomplish that. The difference is that in my

> design, the link from a versionControlsDetails object to a

> fileLocation is purely a uriBaseId, whereas in Jim s design, the link

> is a full-fledged fileLocation object
> (versionControlDetails.mappedTo). Jim s point (as I understood it; see

> my inline commentary on his design in the
> issue) was that his design simplified life for the SARIF producer

> because the producer didn t need a separate uriBaseId for each repo.
>
> Having said that, and having taken the time to write the change draft,

> I m now coming around to your point of view: that we ve traded

> producer convenience for consumer complexity. Each consumer has to

> implement that algorithm to get the right mapping.
>
> The counter-argument is: There are lots more producers (tools) than

> consumers (viewers and result management systems). So we should favor

> convenience for consumers. This is actually a design principle we ve

> followed in the past.
>
> The counter-counter-argument is: This really only helps producers in

> the case where there are submodules. In other cases, you would

> generally have one uriBaseId per repo. Why are we complicating life

> for consumers, just for the sake of consumer convenience in one

> scenario, and not the most common scenario at that?
>
> One more thing: even though only consumers need to /implement/ Jim s

> algorithm, both producers and consumers need to /read and understand

> it/, because producers need to understand how to populate

> originalUriBaseIds and versionControlDetails.mappedTo in Jim s design.
>
> TL;DR: I m coming back around to preferring the simplicity of my design.
>
> Larry
>
> *From:* Michael Fanning <Michael.Fanning@microsoft.com

> <mailto:Michael.Fanning@microsoft.com>>
> *Sent:* Monday, December 10, 2018 9:36 AM
> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com

> <mailto:v-lgold@microsoft.com>>; Larry Golding (Myriad Consulting Inc)

> <v-lgold@microsoft.com <mailto:v-lgold@microsoft.com>>; OASIS SARIF TC

> Discussion List <sarif@lists.oasis-open.org
> <mailto:sarif@lists.oasis-open.org>>
> *Subject:* RE: Change draft for #248: Associate file location with

> repo
>
> I am concerned at the difficult in understanding this proposal. I also

> am wondering whether we have clearly identified the goals of this design.
>
> Here s the scenario that originally drove this request
>
>  1. I have a SARIF log file and would like to recreate the enlistment
>     locally in order to get source files that match the analysis  2. I

> see an originalUriBase that points to the non-deterministic root
>     of the enlistment
>  3. I d like to find the version control details for this root and to
>     sync to it locally.
>
> Michael
>
> *From:* sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On

> Behalf Of *Larry Golding (Myriad Consulting Inc)
> *Sent:* Sunday, December 9, 2018 5:04 PM
> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com

> <mailto:v-lgold@microsoft.com>>; OASIS SARIF TC Discussion List

> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>>
> *Subject:* [sarif] RE: Change draft for #248: Associate file location

> with repo
>
> That probably read snarkier than I intended