energyinterop-comment

As requested.



Cheers,

Marty



From: Marty Burns [mailto:burnsmarty@aol.com]
Sent: Saturday, April 02, 2011 11:26 AM
To: 'William Cox'
Cc: 'Aaron Snyder'; bbartell@xtensible.net; 'Gerald Gray'; ed@akuacom.com;
Toby.Considine@gmail.com; 'Holmberg, David'; 'Stuart McCafferty'; 'Erich W.
Gunther'; jamie.clark@oasis-open.org; eluzcando@midwestiso.org; 'Don
Sturek'; jbooe@naesb.org; 'Marty Burns'
Subject: Discussions on PAP030409



Bill,



I am following up on our discussion at Nashville. Aaron and I are pushing
for path modifications in EMIX and EI that will result, we believe, in a
full throated support of OASIS standards emerging this year from the major
utilities participating in the SGIP. We have discussed with numerous parties
their reasons for the reluctance to get behind your efforts and we think
that some process alterations will make an enormous difference.



These concepts are our own and not developed or promoted by any
organization. However, they are based on our knowledgeable perceptions of
status and opinions on the standards to date.



We believe that OpenADR, IRC, EI/EMIX/WSC, SEP2.0 represent an ecosystem of
interactions. These interactions represent the repertoire of wholesale and
retail demand response messaging proposed from the ISO down to
customer-owned devices. As such, there needs to be a continuum of models and
messaging over this scope in order to minimize mismatch, translation
complexity, and information loss. Each standards community has provided its
own perspective on the requirements for this messaging. EI recognizes this
concept explicitly through its notion of VirtualTopNode (VTN) and
VirtualEndNote(VEN) as a recursive actor/role concept throughout the
network.



There has been a large amount of sound thinking and envisioning behind the
EMIX and EI work. This has been sometimes obscured by the separation between
the thinking and the detail work methodology pursued to date. We applaud the
recent movements towards better UML modeling and inclusion of examples for
some of the schemas. Many participants in the information model standards of
the Smart Grid have converged on some best practices that maximize the
ability of information models to meet original requirements, maintain
consistency between models, documents, and messaging schemas, and provide
for formal documentation as well as implementer's tools.



We suggest that it is valuable, in order to achieve the stated goals, that
the EI/EMIX/WSC standards pursue the following plan or its equivalent in
trying to converge its work to completion expeditiously and effectively:



1) Take the OpenADR and IRC classes and combine them into a unified
(union) information model - this model should clearly identify inheritance
and extensions to CIM (this had been substantially done previously - it
should be resurrected and refined).

2) Capture the EMIX model as currently envisioned by the current
schemas (rev 22?) into the same EA project.

3) The resulting EA file will have: CIM, OpenADR, IRC, EI, and EMIX
packages only.

4) Massage the result into the EI UML model without significant
renaming or refactoring. This can be assisted by using the CIM profiling
tools that allow this kind of massage without losing traceability (CIMEA).
This approach provides built-in traceability to the key contributed works
provided to the committee including the NAESB work.

5) Generate equivalent schemas (OASIS style) from the UML model.
Generate all schemas and table documentation directly from the UML model.
This is the only way to ensure that they really have the equivalent content.
This may require some work with the CIM tools that allow tailoring of the
schemas generated to get the OASIS desired style correct but should not be
too difficult. Worst case is there may be a small list of repeatable "post
processing" manual fixes to the schemas after export. Bruce and his company
have extensive experience with this method and he can get quick help if he
has any problems helping us meet these goals. We are definitely not
proposing maintaining a "round-trip" capability. Only UML->Schema (with
appropriate post-processing if needed) and UML->Document (to produce table
extracts).

6) Alignment with SEP 2 means that you should be able to produce a one
to one correspondence between the elements and containment hierarchy of the
two. Side by side equivalent examples will assist in demonstrating this.

7) We need examples of full messages that prove the theories of how the
models work. These sample messages should be validated against the schemas
that are the normative part of the standard. Best practices include
verification with at least JAX and XMSpy. Experience shows that batch
validations can be produced by a simple script file invoking the various
parsers reducing the tedium of repeating these validations as the products
evolve.



Bill, we identify that there are several UML and XMLSchema capable experts
on the EMIX/EI teams with existing substantial time commitment to the effort
- including Bruce Bartell, Gerald Gray, Ed Koch, Edgardo Luzcando, of course
added to Bill Cox and Toby Considine to name a few. Divide and conquer is
the metaphor for getting work done quickly and effectively as you know. I
think that such an approach to implementing this plan would reliably
increase the quality and adoption of the results while not slowing down the
process. I believe that focus on this kind of plan followed by several weeks
of internal and external review would result in a draft that would progress
through the OASIS public review cycles with minimum friction and achieve the
goals of this email. One or two concentrated workshops might help get the
steps 1-4 completed quickly.



Bill, I hope you find this email an encouragement toward an excellent
conclusion of the EMIX/EI process and that we can collaborate on bringing it
to fruition.



Cheers,

Marty

(To all on the comment list, most of this response was sent to the original distribution list earlier this week)

Marty -

We sincerely appreciate your input and are happy to have feedback on our work. It is very worthwhile looking carefully at how to get increased support for Energy Interop and how to move the standardization process forward most efficiently. We agree in spirit with much or most of what you suggested, and I'll address your points in detail below (see the inline notes below).

As you have done, we urge others to contribute specific comments on the specifications and technical work via the respective comment lists to Energy Interoperation<http://www.oasis-open.org/committtees/energyinteroperation> [http://www.oasis-open.org/committtees/energyinteroperation] and Energy Market Information Exchange<http://www.oasis-open.org/committees/emix> [http://www.oasis-open.org/committees/emix]. All comments are given careful consideration by the technical committee and all comments and resolutions are posted publicly.

(For those not familiar with that process, go to the TC home page (links above) and click on the oval "Send A Comment" button and follow the instructions. Current documents including the latest specification and schemas and UML models are at the "Documents" link on the right sidebar of the TC page.)

To the issue of getting buy-in, we believe that getting good examples and good alignment between UML and schema and text in the standard, and getting that out for public review asap is the main goal. The input we got most clearly from the recent UCA meeting was the request for the full schema and examples. To that end, the current work plan is focused on schema refinement and example development by multiple individuals. As the schema is tested and modified, the model will get updated to match. This seems the most efficient path forward. Schema validation is also an ongoing activity with different tools. I believe we have already adopted the "divide and conquer" approach and are making good progress. This past week members from the OpenADR Alliance have proposed some schema changes for the EiEvent service, and we expect some other tweaks to the schema in the coming week leading up to a vote for a committee draft spec and public review this month.

We are getting close to releasing a clean version and hope to get even more engagement at that point.

Always your friend,
David Holmberg


From: Marty Burns [mailto:burnsmarty@aol.com<mailto:burnsmarty@aol.com?>]
Sent: Saturday, April 02, 2011 11:26 AM
To: 'William Cox'
Cc: 'Aaron Snyder'; bbartell@xtensible.net<mailto:bbartell@xtensible.net>; 'Gerald Gray'; ed@akuacom.com<mailto:ed@akuacom.com>; Toby.Considine@gmail.com<mailto:Toby.Considine@gmail.com>; Holmberg, David; 'Stuart McCafferty'; 'Erich W. Gunther'; jamie.clark@oasis-open.org<mailto:jamie.clark@oasis-open.org>; eluzcando@midwestiso.org<mailto:eluzcando@midwestiso.org>; 'Don Sturek'; jbooe@naesb.org<mailto:jbooe@naesb.org>; 'Marty Burns'
Subject: Discussions on PAP030409

Bill,

I am following up on our discussion at Nashville. Aaron and I are pushing for path modifications in EMIX and EI that will result, we believe, in a full throated support of OASIS standards emerging this year from the major utilities participating in the SGIP. We have discussed with numerous parties their reasons for the reluctance to get behind your efforts and we think that some process alterations will make an enormous difference.

These concepts are our own and not developed or promoted by any organization. However, they are based on our knowledgeable perceptions of status and opinions on the standards to date.

We believe that OpenADR, IRC, EI/EMIX/WSC, SEP2.0 represent an ecosystem of interactions. These interactions represent the repertoire of wholesale and retail demand response messaging proposed from the ISO down to customer-owned devices. As such, there needs to be a continuum of models and messaging over this scope in order to minimize mismatch, translation complexity, and information loss. Each standards community has provided its own perspective on the requirements for this messaging. EI recognizes this concept explicitly through its notion of VirtualTopNode (VTN) and VirtualEndNote(VEN) as a recursive actor/role concept throughout the network.
[[dgh]] Yes, we would agree with this perspective. There are more participants in the ecosystem, such as SPC201P, LonMark, ISA, and others. We would add that the VTN and VEN act as service interfaces and that the ideal goal is that we have lossless mapping of information at these points. Currently the IRC requirements and OpenADR base (and requirements) have been brought together into a single DR information model in EI.

There has been a large amount of sound thinking and envisioning behind the EMIX and EI work. This has been sometimes obscured by the separation between the thinking and the detail work methodology pursued to date. We applaud the recent movements towards better UML modeling and inclusion of examples for some of the schemas. Many participants in the information model standards of the Smart Grid have converged on some best practices that maximize the ability of information models to meet original requirements, maintain consistency between models, documents, and messaging schemas, and provide for formal documentation as well as implementer's tools.
[[dgh]] I believe we understand the best practices in use and benefits. More discussion on that below.

We suggest that it is valuable, in order to achieve the stated goals, that the EI/EMIX/WSC standards pursue the following plan or its equivalent in trying to converge its work to completion expeditiously and effectively:

1) Take the OpenADR and IRC classes and combine them into a unified (union) information model - this model should clearly identify inheritance and extensions to CIM (this had been substantially done previously - it should be resurrected and refined).
[[dgh]] We believe that this is what has in fact been done. The requirements of the IRC (they provided UML, but it is not implementation UML, rather business requirements), has been melded with the OpenADR requirements, built on the OpenADR 1.0 specification. We have tracking information for each of the classes/attributes, but many classes and attributes that exist in EI cannot be tied directly to a class/attribute in OpenADR because the EI class/attribute was developed based on requirements not met in OpenADR. The IRC itself based its requirements on an analysis of the requirements and proprietary protocols of multiple different ISOs. We are open to discussion on how best to handle traceability for public visibility.

2) Capture the EMIX model as currently envisioned by the current schemas (rev 22?) into the same EA project.
[[dgh]] it is.

3) The resulting EA file will have: CIM, OpenADR, IRC, EI, and EMIX packages only.
[[dgh]] The relevant aspects of the CIM (via the IRC and the OpenADR Task Force) are currently captured along with EMIX in the EI EAP. The OpenADR and IRC models are not included in EI, as follows from the discussion in (1) above.

4) Massage the result into the EI UML model without significant renaming or refactoring. This can be assisted by using the CIM profiling tools that allow this kind of massage without losing traceability (CIMEA). This approach provides built-in traceability to the key contributed works provided to the committee including the NAESB work.
[[dgh]] As noted in (1), there was a significant amount of refactoring and renaming required to arrive at a robust DR standard that met OpenADR and IRC requirements and was built on the SG inter-domain standards for calendar (WS-Cal) and price and product communication (EMIX). Part of the resulting model has roots in the CIM, and whenever possible (most obviously with electrical products in EMIX and dispatch related elements of the IRC requirements) CIM attributes are used directly. Because the DR model is not a simple collection of pre-existing classes it is therefore not possible to assemble it in the manner you are recommending.

5) Generate equivalent schemas (OASIS style) from the UML model. Generate all schemas and table documentation directly from the UML model. This is the only way to ensure that they really have the equivalent content. This may require some work with the CIM tools that allow tailoring of the schemas generated to get the OASIS desired style correct but should not be too difficult. Worst case is there may be a small list of repeatable "post processing" manual fixes to the schemas after export. Bruce and his company have extensive experience with this method and he can get quick help if he has any problems helping us meet these goals. We are definitely not proposing maintaining a "round-trip" capability. Only UML->Schema (with appropriate post-processing if needed) and UML->Document (to produce table extracts).
[[dgh]] This sounds good, but there are real concerns. First, there is a higher level issue here. In OASIS, the schema is the normative standard, not UML, and the schema must follow OASIS guidelines and be NIEM compliant. We agree that a UML model is very important for examples and easier discussion, but EA does not produce schema to meet OASIS requirements and output from EA must be massaged to get schema. Every time changes are made to the EA UML, the schema would need to be remade and revalidated. Since the schema is the normative standard, it is not a simple matter to rely on EA UML and a conversion of that to schema. Either direction, UML/schema conversion must be massaged. The UML remains a tool for visual presentation and model coordination. The short answer is that this recommended approach of starting from existing models in EA is not possible (4 above), and getting the required quality schema out of EA is not possible without a significant time delay and little benefit (in our judgment).

6) Alignment with SEP 2 means that you should be able to produce a one to one correspondence between the elements and containment hierarchy of the two. Side by side equivalent examples will assist in demonstrating this.
[[dgh]] We agree that SEP and EI should map cleanly at the interface. I think one real issue is the extension of CIM for SEP versus what we have in EI. EI and SEP do not have the same MRD and TRD, as it were. EI is an inter-domain DR standard. SEP has a smaller application domain as a residential DR management protocol. The two have taken different tacks to address DR. I would like to see SEP align with 201P to address the potential mismatches, and we have the opportunity to make tweaks to both EI and SEP in the coming weeks to address any issues. We also certainly need examples for DR signals moving from EI to SEP. Personally, I hope that you and I can sit down and work through that, as your input there would be a valuable contribution. Perhaps we can schedule a block of time next week to sit and work through the model mapping.

7) We need examples of full messages that prove the theories of how the models work. These sample messages should be validated against the schemas that are the normative part of the standard. Best practices include verification with at least JAX and XMSpy. Experience shows that batch validations can be produced by a simple script file invoking the various parsers reducing the tedium of repeating these validations as the products evolve.
[[dgh]] agreed.

Bill, we identify that there are several UML and XMLSchema capable experts on the EMIX/EI teams with existing substantial time commitment to the effort - including Bruce Bartell, Gerald Gray, Ed Koch, Edgardo Luzcando, of course added to Bill Cox and Toby Considine to name a few. Divide and conquer is the metaphor for getting work done quickly and effectively as you know. I think that such an approach to implementing this plan would reliably increase the quality and adoption of the results while not slowing down the process. I believe that focus on this kind of plan followed by several weeks of internal and external review would result in a draft that would progress through the OASIS public review cycles with minimum friction and achieve the goals of this email. One or two concentrated workshops might help get the steps 1-4 completed quickly.
[[dgh]] We believe these guys are fully engaged in schema evaluation and revision. We do not believe any extra workshops are needed at this point, but rather getting the schema and examples out for public review. Previous working drafts have been discussed in detail at several UCAIug meetings, at Grid-Interop, and elsewhere. Many of the questions and issues raised are those that implementors would ask.

In summary, we believe the best way to address your intent to converge the work and complete it expeditiously and effectively is to ensure that we have complete and coherent schema and UML, then build the examples off those and then get out for public review. We don't believe that using EA and CIM tools to take UML to schema is efficient, nor can we tie all the attributes and classes in EI back to existing attributes and classes in CIM/OpenADR/EMIX. It seems that traceability is the real issue there, and we are happy to discuss how best to show traceability.

Thanks-David

Bill, I hope you find this email an encouragement toward an excellent conclusion of the EMIX/EI process and that we can collaborate on bringing it to fruition.

Cheers,
Marty