OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC

  • 1.  Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 04-27-2017 02:55
    Hi all, Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.   On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity.  We can encourage new projects to use it for verification and use their feedback to improve it gradually, but  it's not reasonable to force the early adapters of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them.  We have discussed this issue back and forth for a long time, how could we make a decision about it? Regards, Huabing Original Mail Sender:  <lishitao@huawei.com>; To:  <michael@gigaspaces.com>; <lauwers@ubicity.com>; CC:  <mrutkows@us.ibm.com>; zhaohuabing10201488; <tosca@lists.oasis-open.org>; Date:  2017/04/27 10:20 Subject:  [tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded Hi Michael   I think the intention here is to support what has already been supported in TOSCA v1.0 that external workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it. As I said, this proposal is just a start, if we close the door at this moment, it will make the gap between implementation and standard even bigger.   Regards shitao   ??? : tosca@lists.oasis-open.org [mailto:tosca@lists.oasis-open.org] ?? Michael Brenner ???? : 2017 ? 4 ? 27 ?  4:22 ??? : Chris Lauwers ?? : Matt Rutkowski; Huabing Zhao; tosca@lists.oasis-open.org ?? : Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Chris,   I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.   Let me try to parse what I am saying:-)   When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is workflow implementation neutral (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to date a standard, and it won't last.   But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations or making them more complex than  necessary).   Do we want to take the red pill or the blue pill?   Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago  - so I am sharing it here.   http://cloudify.co/brochures/tosca-workflows-Apr-2017.pdf   Best regards, Michael   On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers < lauwers@ubicity.com > wrote: I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it J )   It appears we have differing opinions about the best way to get there. We could try to  discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.   Chris     From: tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ] On Behalf Of Michael Brenner Sent: Wednesday, April 26, 2017 7:07 AM To: Matt Rutkowski < mrutkows@us.ibm.com > Cc: Huabing Zhao < zhao.huabing@zte.com.cn >; tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Matt,   Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not always coordinate:-). Like for many of us... I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.   Best regards, Michael   On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: Hi Michael, Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.    In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as black boxes that can return us to a known state (consuming inputs  in the model and providing access to outputs) that is the general view of processing.  Of course, we have other proposals for operational hints on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue. All agreed, that an instance model/API would be needed to allow full introspection after the black boxes complete and that these workflows should not cause changes to the other  parts of the topology.  These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc. In addition, we agreed not to call this delegate as our existing delegate does not match this behavior proposed. Can you attend the WG call in subsequent weeks? Kind regards, Matt From:         Michael Brenner < michael@gigaspaces.com > To:         Huabing Zhao < zhao.huabing@zte.com.cn > Cc:         tosca@lists.oasis-open.org Date:         04/26/2017 07:57 AM Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded Sent by:         < tosca@lists.oasis-open.org > Hi Huabing, all: I reviewed the proposed changes to support external workflow engines via delegate mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring  interactions between external entities. The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all  the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution. Here are some of the issues I observe are unsolved: 1. TOSCA philosophy is to specify what (intent) and not how (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy  is now infringed by the proposal, because of several issues - see below. 1.a: If we standardize the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from what to how . And we also create a precedent that would require constant changes - because other such keynames  will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec. 1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation. 2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts  for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22. 2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification  of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from what to how . 2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate  artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact. 3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to  the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements  and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external  workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how. 4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results. On the how/when, I can see several options that would have to be defined: 4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected  from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about. 4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template  to describe how to wait/block for those results later on. 4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results. 4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue  execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later. There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing  it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better. Best regards, Michael On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao < zhao.huabing@zte.com.cn > wrote: Document Name : Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx No description provided. Download  Latest Revision Public Download Link Submitter :  Huabing Zhao Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC Folder : Working Documents Date submitted : 2017-04-25 19:47:19 -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource            -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource              -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           


  • 2.  Re: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 04-28-2017 09:21
    Mat and Chris summary i think that your summary represent a pragmatic view on the general direction that TOSCA should strive to. It also provide clear direction toward a model that makes the best use of TOSCA. I think that this discussion bring is a clear definition and process on what should be considered part of "backward computability" and we should also adopt a clear deprecation model to address future changes to the spec. Huabing while i agree with the pragmatic approach to allow both imperative and declarative models i think that the backward computability argument is fairly weak. For a start Its hard to argue backward computability for something that was never defined as part of the standard. So while i would agree that backward computability is important, non of the references that you pointed out (JAVA, IETF) committed for backward computability of things that were implemented outside of the standard and there are many examples of cases where existing implementation had to change to adopt the new standard. Using Open-O and OpenEcomp as a references for this argument is also a fairly weak argument as both are fairly young projects that didn't even reached their first production phase and are now being pivoted to yet a new project - ONAP in which theres still lots of room for change. From many of the discussion that i've been involved with theres a much greater opening now within ONAP community to consider more native approach to TOSCA rather limiting it to be yet another configuration input. We have to consider the tradeoff as well and their impact on the use of TOSCA in the future. The tradeoff is that if we will not have clear direction toward native approach we will create a legacy issue and backward compatibility issue for all upcoming releases of TOSCA . Is that what we want? What about forward compatibility? To be clear i'm totally fine with Chris description that we should treat workflow execution as "blackbox" with more clear interface on on the interaction between the model on those "blackbox" implementation. If this is the general consensus that i think that we made a good progress in this discussion.  Huabing, Shitao i would ask you to look also whether the direction that your pushing for serve best the future direction that we all want to strive to especially now that we still have a chance to change past decision and implementations. Nati S.   On Thu, Apr 27, 2017 at 5:54 AM < zhao.huabing@zte.com.cn > wrote: Hi all, Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.   On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity.  We can encourage new projects to use it for verification and use their feedback to improve it gradually, but  it's not reasonable to force the early adapters of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them.  We have discussed this issue back and forth for a long time, how could we make a decision about it? Regards, Huabing Original Mail Sender:  < lishitao@huawei.com >; To:  < michael@gigaspaces.com >; < lauwers@ubicity.com >; CC:  < mrutkows@us.ibm.com >; zhaohuabing10201488; < tosca@lists.oasis-open.org >; Date:  2017/04/27 10:20 Subject:  [tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded Hi Michael   I think the intention here is to support what has already been supported in TOSCA v1.0 that external workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it. As I said, this proposal is just a start, if we close the door at this moment, it will make the gap between implementation and standard even bigger.   Regards shitao   ??? : tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ] ?? Michael Brenner ???? : 2017 ? 4 ? 27 ?  4:22 ??? : Chris Lauwers ?? : Matt Rutkowski; Huabing Zhao; tosca@lists.oasis-open.org ?? : Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Chris,   I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.   Let me try to parse what I am saying:-)   When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to "date" a standard, and it won't last.   But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations or making them more complex than  necessary).   Do we want to take the red pill or the blue pill?   Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago  - so I am sharing it here.   http://cloudify.co/brochures/tosca-workflows-Apr-2017.pdf   Best regards, Michael On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers < lauwers@ubicity.com > wrote: I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it J )   It appears we have differing opinions about the best way to get there. We could try to  discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.   Chris     From: tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ] On Behalf Of Michael Brenner Sent: Wednesday, April 26, 2017 7:07 AM To: Matt Rutkowski < mrutkows@us.ibm.com > Cc: Huabing Zhao < zhao.huabing@zte.com.cn >; tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Matt,   Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not always coordinate:-). Like for many of us... I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.   Best regards, Michael On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: Hi Michael, Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.    In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming inputs  in the model and providing access to outputs) that is the general view of processing.  Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue. All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes to the other  parts of the topology.  These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc. In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed. Can you attend the WG call in subsequent weeks? Kind regards, Matt From:         Michael Brenner < michael@gigaspaces.com > To:         Huabing Zhao < zhao.huabing@zte.com.cn > Cc:         tosca@lists.oasis-open.org Date:         04/26/2017 07:57 AM Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded Sent by:         < tosca@lists.oasis-open.org > Hi Huabing, all: I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring  interactions between external entities. The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all  the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution. Here are some of the issues I observe are unsolved: 1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy  is now infringed by the proposal, because of several issues - see below. 1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"  will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec. 1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation. 2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts  for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22. 2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification  of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how". 2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate  artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact. 3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to  the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements  and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external  workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how. 4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results. On the how/when, I can see several options that would have to be defined: 4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected  from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about. 4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template  to describe how to wait/block for those results later on. 4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results. 4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue  execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later. There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing  it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better. Best regards, Michael On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao < zhao.huabing@zte.com.cn > wrote: Document Name : Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx No description provided. Download  Latest Revision Public Download Link Submitter :  Huabing Zhao Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC Folder : Working Documents Date submitted : 2017-04-25 19:47:19 -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource            -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource              -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource            --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php


  • 3.  RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 05-05-2017 17:37




    I completely agree with Nati. We should (pragmatically) support what Huabing wants to accomplish because it will at least result in broader adoption of TOSCA
    as the way in which one models/describes services.
     
    However, our ultimate goal should not be to simply act as an “input format” for imperative workflows. Our goal should be to drive and support fully declarative
    orchestration architectures based not just on TOSCA’s modeling features but also based on its orchestration features. To that end, we should leverage ONAP use cases to help identify the necessary improvements in TOSCA that will allow fully-declarative orchestration
    in most cases, with the occasional “assist” from TOSCA’s imperative workflows.
     
    Chris
     
     
     
    From: Nati Shalom [mailto:natis@gigaspaces.com]

    Sent: Friday, April 28, 2017 2:21 AM
    To: zhao.huabing@zte.com.cn; lishitao@huawei.com
    Cc: michael@gigaspaces.com; Chris Lauwers <lauwers@ubicity.com>; mrutkows@us.ibm.com; tosca@lists.oasis-open.org
    Subject: Re: [tosca] Re:[tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded
     

    Mat and Chris summary i think that your summary represent a pragmatic view on the general direction that TOSCA should strive to.

    It also provide clear direction toward a model that makes the best use of TOSCA.


     


    I think that this discussion bring is a clear definition and process on what should be considered part of "backward computability" and we should also adopt a clear deprecation model to address future changes to the spec.


     




    Huabing while i agree with the pragmatic approach to allow both imperative and declarative models i think that the backward computability argument is fairly weak.


     


    For a start Its hard to argue backward computability for something that was never defined as part of the standard. So while i would agree that backward computability is important, non of the references that you pointed out (JAVA, IETF)
    committed for backward computability of things that were implemented outside of the standard and there are many examples of cases where existing implementation had to change to adopt the new standard.


     


    Using Open-O and OpenEcomp as a references for this argument is also a fairly weak argument as both are fairly young projects that didn't even reached their first production phase and are now being pivoted to yet a new project - ONAP in
    which theres still lots of room for change.


     


    From many of the discussion that i've been involved with theres a much greater opening now within ONAP community to consider more native approach to TOSCA rather limiting it to be yet another configuration input.


     


    We have to consider the tradeoff as well and their impact on the use of TOSCA in the future.


     


    The tradeoff is that if we will not have clear direction toward native approach
    we will create a legacy issue and backward compatibility issue for all upcoming releases of TOSCA .


     


    Is that what we want?


    What about forward compatibility?


     


    To be clear i'm totally fine with Chris description that we should treat workflow execution as "blackbox" with more clear interface on on the interaction between the model on those "blackbox" implementation.


    If this is the general consensus that i think that we made a good progress in this discussion. 


     


    Huabing, Shitao i would ask you to look also whether the direction that your pushing for serve best the future direction that we all want to strive to especially now that we still have a chance to change past decision and implementations.


     


    Nati S.


     


     


     


     


     


     


     


     


     


     


    On Thu, Apr 27, 2017 at 5:54 AM < zhao.huabing@zte.com.cn > wrote:



    Hi all,
     
    Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.  
     
    On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity. We can encourage new projects to use it for verification and use their feedback to improve it gradually, but it's not reasonable to force the early adapters
    of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them. 
     
    We have discussed this issue back and forth for a long time, how could we make a decision about it?
     
    Regards,
    Huabing
     
     
     
     



    Original Mail




    Sender: 
    < lishitao@huawei.com > ;


    To: 
    < michael@gigaspaces.com > ;
    < lauwers@ubicity.com > ;


    CC: 
    < mrutkows@us.ibm.com > ;zhaohuabing10201488;
    < tosca@lists.oasis-open.org > ;


    Date:  2017/04/27 10:20


    Subject: [tosca]
    ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded


     












    Hi Michael
     
    I think the intention here is to support what has already been supported in TOSCA v1.0 that external
    workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned
    are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people
    still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it.
    As I said, this proposal is just a start, if we close the door at this moment, it will make the gap
    between implementation and standard even bigger.
     
    Regards
    shitao
     
    ??? :
    tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ]
    ??
    Michael Brenner
    ???? :
    2017 ? 4 ? 27 ?
     4:22
    ??? : Chris Lauwers
    ?? : Matt Rutkowski;
    Huabing Zhao; tosca@lists.oasis-open.org
    ? ? :
    Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded
     

    Hi Chris,
     

    I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with
    Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.

     

    Let me try to parse what I am saying:-)

     

    When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming
    orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use
    of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever
    else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage
    one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to "date" a standard,
    and it won't last.

     

    But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that
    once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations
    or making them more complex than  necessary).

     

    Do we want to take the red pill or the blue pill?

     

    Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered
    you showed interest a few weeks ago  - so I am sharing it here.

     

    http://cloudify.co/brochures/tosca-workflows-Apr-2017.pdf

     

    Best regards,


    Michael


















    On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers
    < lauwers@ubicity.com > wrote:





















    I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with
    Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA
    mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases)
    to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the
    point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it
    J )
     
    It appears we have differing opinions about the best way to get there. We could try to  discourage
    the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative
    logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.

     
    Chris
     
     
    From:
    tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ]
    On Behalf Of Michael Brenner
    Sent: Wednesday, April 26, 2017 7:07 AM
    To: Matt Rutkowski
    < mrutkows@us.ibm.com >
    Cc: Huabing Zhao
    < zhao.huabing@zte.com.cn > ;
    tosca@lists.oasis-open.org



    Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded




























     

    Hi Matt,
     

    Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not
    always coordinate:-). Like for many of us...


    I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions,
    works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.

     

    Best regards,


    Michael
































    On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski
    < mrutkows@us.ibm.com > wrote:

































    Hi Michael,

    Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.   

    In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers
    and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming inputs  in the model and providing
    access to outputs) that is the general view of processing.  Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue.

    All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes to the other  parts of the topology. 
    These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc.

    In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed.

    Can you attend the WG call in subsequent weeks?

    Kind regards,
    Matt




    From:         Michael Brenner
    < michael@gigaspaces.com >
    To:         Huabing Zhao
    < zhao.huabing@zte.com.cn >
    Cc:         tosca@lists.oasis-open.org
    Date:         04/26/2017 07:57 AM
    Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded
    Sent by:         < tosca@lists.oasis-open.org >










































    Hi Huabing, all:

    I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring
     interactions between external entities.

    The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all
     the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution.

    Here are some of the issues I observe are unsolved:

    1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy
     is now infringed by the proposal, because of several issues - see below.
    1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"
     will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec.
    1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation.
    2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts
     for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22.
    2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification
     of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how".
    2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate
     artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator
    implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this.
    The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact.
    3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to
     the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements
     and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external
     workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how.






































    4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described
    above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager
    of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based
    on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results.







































    On the how/when, I can see several options that would have to be defined:
    4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected
     from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about.
    4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template
     to describe how to wait/block for those results later on.
    4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results.
    4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue
     execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later.

    There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing
     it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better.

    Best regards,
    Michael



    On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao
    < zhao.huabing@zte.com.cn > wrote:










































    Document Name :
    Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx



    No description provided.

    Download
     Latest Revision
    Public Download Link



    Submitter :  Huabing Zhao
    Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC
    Folder : Working Documents
    Date submitted : 2017-04-25 19:47:19








    --










































    Michael Brenner,  Chief Architect NFV
















    M: +1-732-895-5772



    http://getcloudify.org



    @cloudifysource




          












     



































     
    --






    Michael Brenner,  Chief Architect NFV


















    M: +1-732-895-5772



    http://getcloudify.org



    @cloudifysource





            












     



























     
    --






    Michael Brenner,  Chief Architect NFV


















    M: +1-732-895-5772



    http://getcloudify.org



    @cloudifysource





            












     














     




     


    ---------------------------------------------------------------------
    To unsubscribe from this mail list, you must leave the OASIS TC that
    generates this mail.  Follow this link to all your TCs in OASIS at:
    https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php













  • 4.  RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 05-05-2017 21:01
    I thought we had resolved going forward to acknowledge BPMN and BPEL (by name) as popular examples of opaque ("Black box" artifact) work flow languages and list all the issues/problems/caveats around using them in conjunction within a declarative topology (for the Orchestrator and portability).  To be clear, we would not endorse them as normative, but neither would we reject them and follow our philosophy of "meeting customer where they are at in their (declarative) journey; that is, not leave them behind, ignore them or force them to choose to rewrite to TOSCA or leave. We would then seek, as a community (and by example over time) the advantages of adopting the declarative approach; which has been our philosophy since we first decided to create the Simple Profile (and even before when we created normative types and the first interop. samples and demos). Kind regards, Matt Rutkowski From:         Chris Lauwers <lauwers@ubicity.com> To:         Nati Shalom <natis@gigaspaces.com>, "zhao.huabing@zte.com.cn" <zhao.huabing@zte.com.cn>, "lishitao@huawei.com" <lishitao@huawei.com> Cc:         "michael@gigaspaces.com" <michael@gigaspaces.com>, "mrutkows@us.ibm.com" <mrutkows@us.ibm.com>, "tosca@lists.oasis-open.org" <tosca@lists.oasis-open.org> Date:         05/05/2017 12:36 PM Subject:         RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded Sent by:         <tosca@lists.oasis-open.org> I completely agree with Nati. We should (pragmatically) support what Huabing wants to accomplish because it will at least result in broader adoption of TOSCA as the way in which one models/describes services.   However, our ultimate goal should not be to simply act as an “input format” for imperative workflows. Our goal should be to drive and support fully declarative orchestration architectures based not just on TOSCA’s modeling features but also based on its orchestration features. To that end, we should leverage ONAP use cases to help identify the necessary improvements in TOSCA that will allow fully-declarative orchestration in most cases, with the occasional “assist” from TOSCA’s imperative workflows.   Chris       From: Nati Shalom [ mailto:natis@gigaspaces.com ] Sent: Friday, April 28, 2017 2:21 AM To: zhao.huabing@zte.com.cn; lishitao@huawei.com Cc: michael@gigaspaces.com; Chris Lauwers <lauwers@ubicity.com>; mrutkows@us.ibm.com; tosca@lists.oasis-open.org Subject: Re: [tosca] Re:[tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Mat and Chris summary i think that your summary represent a pragmatic view on the general direction that TOSCA should strive to. It also provide clear direction toward a model that makes the best use of TOSCA.   I think that this discussion bring is a clear definition and process on what should be considered part of "backward computability" and we should also adopt a clear deprecation model to address future changes to the spec.   Huabing while i agree with the pragmatic approach to allow both imperative and declarative models i think that the backward computability argument is fairly weak.   For a start Its hard to argue backward computability for something that was never defined as part of the standard. So while i would agree that backward computability is important, non of the references that you pointed out (JAVA, IETF) committed for backward computability of things that were implemented outside of the standard and there are many examples of cases where existing implementation had to change to adopt the new standard.   Using Open-O and OpenEcomp as a references for this argument is also a fairly weak argument as both are fairly young projects that didn't even reached their first production phase and are now being pivoted to yet a new project - ONAP in which theres still lots of room for change.   From many of the discussion that i've been involved with theres a much greater opening now within ONAP community to consider more native approach to TOSCA rather limiting it to be yet another configuration input.   We have to consider the tradeoff as well and their impact on the use of TOSCA in the future.   The tradeoff is that if we will not have clear direction toward native approach we will create a legacy issue and backward compatibility issue for all upcoming releases of TOSCA .   Is that what we want? What about forward compatibility?   To be clear i'm totally fine with Chris description that we should treat workflow execution as "blackbox" with more clear interface on on the interaction between the model on those "blackbox" implementation. If this is the general consensus that i think that we made a good progress in this discussion.   Huabing, Shitao i would ask you to look also whether the direction that your pushing for serve best the future direction that we all want to strive to especially now that we still have a chance to change past decision and implementations.   Nati S.                     On Thu, Apr 27, 2017 at 5:54 AM < zhao.huabing@zte.com.cn > wrote: Hi all,   Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.     On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity. We can encourage new projects to use it for verification and use their feedback to improve it gradually, but it's not reasonable to force the early adapters of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them.   We have discussed this issue back and forth for a long time, how could we make a decision about it?   Regards, Huabing         Original Mail Sender:   < lishitao@huawei.com > ; To:   < michael@gigaspaces.com > ; < lauwers@ubicity.com > ; CC:   < mrutkows@us.ibm.com > ;zhaohuabing10201488; < tosca@lists.oasis-open.org > ; Date: 2017/04/27 10:20 Subject: [tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Michael   I think the intention here is to support what has already been supported in TOSCA v1.0 that external workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it. As I said, this proposal is just a start, if we close the door at this moment, it will make the gap between implementation and standard even bigger.   Regards shitao   ??? : tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ] ?? Michael Brenner ???? : 2017 ? 4 ? 27 ?  4:22 ??? : Chris Lauwers ?? : Matt Rutkowski; Huabing Zhao; tosca@lists.oasis-open.org ? ? : Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Chris,   I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.   Let me try to parse what I am saying:-)   When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to "date" a standard, and it won't last.   But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations or making them more complex than  necessary).   Do we want to take the red pill or the blue pill?   Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago  - so I am sharing it here.   http://cloudify.co/brochures/tosca-workflows-Apr-2017.pdf   Best regards, Michael On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers < lauwers@ubicity.com > wrote: I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it J )   It appears we have differing opinions about the best way to get there. We could try to  discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.   Chris     From: tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open.org ] On Behalf Of Michael Brenner Sent: Wednesday, April 26, 2017 7:07 AM To: Matt Rutkowski < mrutkows@us.ibm.com > Cc: Huabing Zhao < zhao.huabing@zte.com.cn >; tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Matt,   Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not always coordinate:-). Like for many of us... I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.   Best regards, Michael On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: Hi Michael, Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.   In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming inputs  in the model and providing access to outputs) that is the general view of processing.  Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue. All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes to the other  parts of the topology.  These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc. In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed. Can you attend the WG call in subsequent weeks? Kind regards, Matt From:         Michael Brenner < michael@gigaspaces.com > To:         Huabing Zhao < zhao.huabing@zte.com.cn > Cc:         tosca@lists.oasis-open.org Date:         04/26/2017 07:57 AM Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded Sent by:         < tosca@lists.oasis-open.org > Hi Huabing, all: I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring  interactions between external entities. The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all  the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution. Here are some of the issues I observe are unsolved: 1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy  is now infringed by the proposal, because of several issues - see below. 1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"  will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec. 1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation. 2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts  for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22. 2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification  of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how". 2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate  artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact. 3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to  the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements  and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external  workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how. 4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results. On the how/when, I can see several options that would have to be defined: 4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected  from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about. 4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template  to describe how to wait/block for those results later on. 4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results. 4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue  execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later. There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing  it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better. Best regards, Michael On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao < zhao.huabing@zte.com.cn > wrote: Document Name : Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx No description provided. Download  Latest Revision Public Download Link Submitter :  Huabing Zhao Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC Folder : Working Documents Date submitted : 2017-04-25 19:47:19 -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource         -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource             --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php


  • 5.  RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 05-05-2017 21:19




    Yes, I think we’re all saying the same thing



  • 6.  Re: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 05-06-2017 13:01
    Matt, all: I would support a pragmatic approach, once we specify a complete proposal - where one not only allows delegation out to external entities, but also indicates at the time how the control returns to TOSCA once the external entity completes. THAT IS what ONAP will really need. The current proposal is actually what Open-O needs/needed - because in that community TOSCA was only used to generate the model, and then other entities leveraged it. In that latter case, one does not care to return to the TOSCA topology, because they do not count on a TOSCA orchestrator to traverse it.  So if we do this, we have to do it right for everyone - so those that do take full advantage of TOSCA via a TOSCA-based orchestrator can delegate to an external workflow engine when they handle some lifecycle management event for a particular node, but return the control to the TOSCA orchestrator once the external handling ends. Short of that - we only serve a community that uses TOSCA in the most superficial and handicapped manner - i.e. its ability to describe the model, that then gets translated to some other format that other orchestrators work with. Not what TOSCA TC envisioned. Let's do this right folks. Michael On Fri, May 5, 2017 at 5:00 PM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: I thought we had resolved going forward to acknowledge BPMN and BPEL (by name) as popular examples of opaque ("Black box" artifact) work flow languages and list all the issues/problems/caveats around using them in conjunction within a declarative topology (for the Orchestrator and portability).  To be clear, we would not endorse them as normative, but neither would we reject them and follow our philosophy of "meeting customer where they are at in their (declarative) journey; that is, not leave them behind, ignore them or force them to choose to rewrite to TOSCA or leave. We would then seek, as a community (and by example over time) the advantages of adopting the declarative approach; which has been our philosophy since we first decided to create the Simple Profile (and even before when we created normative types and the first interop. samples and demos). Kind regards, Matt Rutkowski From:         Chris Lauwers < lauwers@ubicity.com > To:         Nati Shalom < natis@gigaspaces.com >, " zhao.huabing@zte.com.cn " < zhao.huabing@zte.com.cn >, " lishitao@huawei.com " < lishitao@huawei.com > Cc:         " michael@gigaspaces.com " < michael@gigaspaces.com >, " mrutkows@us.ibm.com " < mrutkows@us.ibm.com >, " tosca@lists.oasis-open.org " < tosca@lists.oasis-open.org > Date:         05/05/2017 12:36 PM Subject:         RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded Sent by:         < tosca@lists.oasis-open.org > I completely agree with Nati. We should (pragmatically) support what Huabing wants to accomplish because it will at least result in broader adoption of TOSCA as the way in which one models/describes services.   However, our ultimate goal should not be to simply act as an “input format” for imperative workflows. Our goal should be to drive and support fully declarative orchestration architectures based not just on TOSCA’s modeling features but also based on its orchestration features. To that end, we should leverage ONAP use cases to help identify the necessary improvements in TOSCA that will allow fully-declarative orchestration in most cases, with the occasional “assist” from TOSCA’s imperative workflows.   Chris       From: Nati Shalom [ mailto:natis@gigaspaces.com ] Sent: Friday, April 28, 2017 2:21 AM To: zhao.huabing@zte.com.cn ; lishitao@huawei.com Cc: michael@gigaspaces.com ; Chris Lauwers < lauwers@ubicity.com >; mrutkows@us.ibm.com ; tosca@lists.oasis-open.org Subject: Re: [tosca] Re:[tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Mat and Chris summary i think that your summary represent a pragmatic view on the general direction that TOSCA should strive to. It also provide clear direction toward a model that makes the best use of TOSCA.   I think that this discussion bring is a clear definition and process on what should be considered part of "backward computability" and we should also adopt a clear deprecation model to address future changes to the spec.   Huabing while i agree with the pragmatic approach to allow both imperative and declarative models i think that the backward computability argument is fairly weak.   For a start Its hard to argue backward computability for something that was never defined as part of the standard. So while i would agree that backward computability is important, non of the references that you pointed out (JAVA, IETF) committed for backward computability of things that were implemented outside of the standard and there are many examples of cases where existing implementation had to change to adopt the new standard.   Using Open-O and OpenEcomp as a references for this argument is also a fairly weak argument as both are fairly young projects that didn't even reached their first production phase and are now being pivoted to yet a new project - ONAP in which theres still lots of room for change.   From many of the discussion that i've been involved with theres a much greater opening now within ONAP community to consider more native approach to TOSCA rather limiting it to be yet another configuration input.   We have to consider the tradeoff as well and their impact on the use of TOSCA in the future.   The tradeoff is that if we will not have clear direction toward native approach we will create a legacy issue and backward compatibility issue for all upcoming releases of TOSCA .   Is that what we want? What about forward compatibility?   To be clear i'm totally fine with Chris description that we should treat workflow execution as "blackbox" with more clear interface on on the interaction between the model on those "blackbox" implementation. If this is the general consensus that i think that we made a good progress in this discussion.   Huabing, Shitao i would ask you to look also whether the direction that your pushing for serve best the future direction that we all want to strive to especially now that we still have a chance to change past decision and implementations.   Nati S.                     On Thu, Apr 27, 2017 at 5:54 AM < zhao.huabing@zte.com.cn > wrote: Hi all,   Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.     On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity. We can encourage new projects to use it for verification and use their feedback to improve it gradually, but it's not reasonable to force the early adapters of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them.   We have discussed this issue back and forth for a long time, how could we make a decision about it?   Regards, Huabing         Original Mail Sender:   < lishitao@huawei.com > ; To:   < michael@gigaspaces.com > ; < lauwers@ubicity.com > ; CC:   < mrutkows@us.ibm.com > ; zhaohuabing10201488; < tosca@lists.oasis-open.org > ; Date: 2017/04/27 10:20 Subject: [tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Michael   I think the intention here is to support what has already been supported in TOSCA v1.0 that external workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it. As I said, this proposal is just a start, if we close the door at this moment, it will make the gap between implementation and standard even bigger.   Regards shitao   ??? : tosca@lists.oasis-open.org [mailto: tosca@lists.oasis- open.org ] ?? Michael Brenner ???? : 2017 ? 4 ? 27 ?  4:22 ??? : Chris Lauwers ?? : Matt Rutkowski; Huabing Zhao; tosca@lists.oasis-open.org ? ? : Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Chris,   I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.   Let me try to parse what I am saying:-)   When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to "date" a standard, and it won't last.   But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations or making them more complex than  necessary).   Do we want to take the red pill or the blue pill?   Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago  - so I am sharing it here.   http://cloudify.co/brochures/ tosca-workflows-Apr-2017.pdf   Best regards, Michael On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers < lauwers@ubicity.com > wrote: I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it J )   It appears we have differing opinions about the best way to get there. We could try to  discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.   Chris     From: tosca@lists.oasis-open.org [ mailto: tosca@lists.oasis-open. org ] On Behalf Of Michael Brenner Sent: Wednesday, April 26, 2017 7:07 AM To: Matt Rutkowski < mrutkows@us.ibm.com > Cc: Huabing Zhao < zhao.huabing@zte.com.cn >; tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Matt,   Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not always coordinate:-). Like for many of us... I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.   Best regards, Michael On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: Hi Michael, Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.   In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming inputs  in the model and providing access to outputs) that is the general view of processing.  Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue. All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes to the other  parts of the topology.  These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc. In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed. Can you attend the WG call in subsequent weeks? Kind regards, Matt From:         Michael Brenner < michael@gigaspaces.com > To:         Huabing Zhao < zhao.huabing@zte.com.cn > Cc:         tosca@lists.oasis-open.org Date:         04/26/2017 07:57 AM Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded Sent by:         < tosca@lists.oasis-open.org > Hi Huabing, all: I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring  interactions between external entities. The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all  the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution. Here are some of the issues I observe are unsolved: 1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy  is now infringed by the proposal, because of several issues - see below. 1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"  will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec. 1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation. 2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts  for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22. 2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification  of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how". 2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate  artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact. 3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to  the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements  and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external  workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how. 4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results. On the how/when, I can see several options that would have to be defined: 4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected  from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about. 4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template  to describe how to wait/block for those results later on. 4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results. 4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue  execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later. There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing  it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better. Best regards, Michael On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao < zhao.huabing@zte.com.cn > wrote : Document Name : Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx No description provided. Download  Latest Revision Public Download Link Submitter :  Huabing Zhao Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC Folder : Working Documents Date submitted : 2017-04-25 19:47:19 -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource         -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource             ------------------------------ ------------------------------ --------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/ apps/org/workgroup/portal/my_ workgroups.php -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           


  • 7.  ??: Re: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 05-06-2017 16:19
    Hi Michael and all I suggest we follow what we have discussed in ONAP yesterday that focusing on the use case first. Then figure out what is the requirement from ONAP for workflow. At this moment is really hard to judge whether the requirement comes from ONAP or Open-o (since ONAP is the merged project between OpenECOMP and Open-o , from my perspective, open-o requirement is the potential requirement for ONAP). I think what Matt suggested is the best way we can do at this moment, that we can work on a in-normative method first to fit the  requirement we have on the table now.  Regards  Shitao  ??? rick (M)+86-13404161483   (E) lishitao@huawei.com ???????-????????? ???: Michael Brenner ???: Matt Rutkowski< mrutkows@us.ibm.com > ??: Chris Lauwers< lauwers@ubicity.com >;Lishitao< lishitao@huawei.com >;Nati Shalom< natis@gigaspaces.com >; tosca@lists.oasis-open.org ; zhao.huabing@zte.com.cn ??: Re: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded ??: 2017-05-06 09:01:16 Matt, all: I would support a pragmatic approach, once we specify a complete proposal - where one not only allows delegation out to external entities, but also indicates at the time how the control returns to TOSCA once the external entity completes. THAT IS what ONAP will really need. The current proposal is actually what Open-O needs/needed - because in that community TOSCA was only used to generate the model, and then other entities leveraged it. In that latter case, one does not care to return to the TOSCA topology, because they do not count on a TOSCA orchestrator to traverse it.  So if we do this, we have to do it right for everyone - so those that do take full advantage of TOSCA via a TOSCA-based orchestrator can delegate to an external workflow engine when they handle some lifecycle management event for a particular node, but return the control to the TOSCA orchestrator once the external handling ends. Short of that - we only serve a community that uses TOSCA in the most superficial and handicapped manner - i.e. its ability to describe the model, that then gets translated to some other format that other orchestrators work with. Not what TOSCA TC envisioned. Let's do this right folks. Michael On Fri, May 5, 2017 at 5:00 PM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: I thought we had resolved going forward to acknowledge BPMN and BPEL (by name) as popular examples of opaque ("Black box" artifact) work flow languages and list all the issues/problems/caveats around using them in conjunction within a declarative topology (for the Orchestrator and portability).  To be clear, we would not endorse them as normative, but neither would we reject them and follow our philosophy of "meeting customer where they are at in their (declarative) journey; that is, not leave them behind, ignore them or force them to choose to rewrite to TOSCA or leave. We would then seek, as a community (and by example over time) the advantages of adopting the declarative approach; which has been our philosophy since we first decided to create the Simple Profile (and even before when we created normative types and the first interop. samples and demos). Kind regards, Matt Rutkowski From:         Chris Lauwers < lauwers@ubicity.com > To:         Nati Shalom < natis@gigaspaces.com >, " zhao.huabing@zte.com.cn " < zhao.huabing@zte.com.cn >, " lishitao@huawei.com " < lishitao@huawei.com > Cc:         " michael@gigaspaces.com " < michael@gigaspaces.com >, " mrutkows@us.ibm.com " < mrutkows@us.ibm.com >, " tosca@lists.oasis-open.org " < tosca@lists.oasis-open.org > Date:         05/05/2017 12:36 PM Subject:         RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded Sent by:         < tosca@lists.oasis-open.org > I completely agree with Nati. We should (pragmatically) support what Huabing wants to accomplish because it will at least result in broader adoption of TOSCA as the way in which one models/describes services.   However, our ultimate goal should not be to simply act as an “input format” for imperative workflows. Our goal should be to drive and support fully declarative orchestration architectures based not just on TOSCA’s modeling features but also based on its orchestration features. To that end, we should leverage ONAP use cases to help identify the necessary improvements in TOSCA that will allow fully-declarative orchestration in most cases, with the occasional “assist” from TOSCA’s imperative workflows.   Chris       From: Nati Shalom [ mailto:natis@gigaspaces.com ] Sent: Friday, April 28, 2017 2:21 AM To: zhao.huabing@zte.com.cn ; lishitao@huawei.com Cc: michael@gigaspaces.com ; Chris Lauwers < lauwers@ubicity.com >; mrutkows@us.ibm.com ; tosca@lists.oasis-open.org Subject: Re: [tosca] Re:[tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Mat and Chris summary i think that your summary represent a pragmatic view on the general direction that TOSCA should strive to. It also provide clear direction toward a model that makes the best use of TOSCA.   I think that this discussion bring is a clear definition and process on what should be considered part of "backward computability" and we should also adopt a clear deprecation model to address future changes to the spec.   Huabing while i agree with the pragmatic approach to allow both imperative and declarative models i think that the backward computability argument is fairly weak.   For a start Its hard to argue backward computability for something that was never defined as part of the standard. So while i would agree that backward computability is important, non of the references that you pointed out (JAVA, IETF) committed for backward computability of things that were implemented outside of the standard and there are many examples of cases where existing implementation had to change to adopt the new standard.   Using Open-O and OpenEcomp as a references for this argument is also a fairly weak argument as both are fairly young projects that didn't even reached their first production phase and are now being pivoted to yet a new project - ONAP in which theres still lots of room for change.   From many of the discussion that i've been involved with theres a much greater opening now within ONAP community to consider more native approach to TOSCA rather limiting it to be yet another configuration input.   We have to consider the tradeoff as well and their impact on the use of TOSCA in the future.   The tradeoff is that if we will not have clear direction toward native approach we will create a legacy issue and backward compatibility issue for all upcoming releases of TOSCA .   Is that what we want? What about forward compatibility?   To be clear i'm totally fine with Chris description that we should treat workflow execution as "blackbox" with more clear interface on on the interaction between the model on those "blackbox" implementation. If this is the general consensus that i think that we made a good progress in this discussion.   Huabing, Shitao i would ask you to look also whether the direction that your pushing for serve best the future direction that we all want to strive to especially now that we still have a chance to change past decision and implementations.   Nati S.                     On Thu, Apr 27, 2017 at 5:54 AM < zhao.huabing@zte.com.cn > wrote: Hi all,   Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.     On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity. We can encourage new projects to use it for verification and use their feedback to improve it gradually, but it's not reasonable to force the early adapters of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them.   We have discussed this issue back and forth for a long time, how could we make a decision about it?   Regards, Huabing         Original Mail Sender:   < lishitao@huawei.com > ; To:   < michael@gigaspaces.com > ; < lauwers@ubicity.com > ; CC:   < mrutkows@us.ibm.com > ; zhaohuabing10201488; < tosca@lists.oasis-open.org > ; Date: 2017/04/27 10:20 Subject: [tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Michael   I think the intention here is to support what has already been supported in TOSCA v1.0 that external workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it. As I said, this proposal is just a start, if we close the door at this moment, it will make the gap between implementation and standard even bigger.   Regards shitao   ??? : tosca@lists.oasis-open.org [mailto: tosca@lists.oasis- open.org ] ?? Michael Brenner ???? : 2017 ? 4 ? 27 ?  4:22 ??? : Chris Lauwers ?? : Matt Rutkowski; Huabing Zhao; tosca@lists.oasis-open.org ? ? : Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Chris,   I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.   Let me try to parse what I am saying:-)   When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to "date" a standard, and it won't last.   But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations or making them more complex than  necessary).   Do we want to take the red pill or the blue pill?   Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago  - so I am sharing it here.   http://cloudify.co/brochures/ tosca-workflows-Apr-2017.pdf   Best regards, Michael On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers < lauwers@ubicity.com > wrote: I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it J )   It appears we have differing opinions about the best way to get there. We could try to  discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.   Chris     From: tosca@lists.oasis-open.org [ mailto: tosca@lists.oasis-open. org ] On Behalf Of Michael Brenner Sent: Wednesday, April 26, 2017 7:07 AM To: Matt Rutkowski < mrutkows@us.ibm.com > Cc: Huabing Zhao < zhao.huabing@zte.com.cn >; tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Matt,   Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not always coordinate:-). Like for many of us... I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.   Best regards, Michael On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: Hi Michael, Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.   In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming inputs  in the model and providing access to outputs) that is the general view of processing.  Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue. All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes to the other  parts of the topology.  These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc. In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed. Can you attend the WG call in subsequent weeks? Kind regards, Matt From:         Michael Brenner < michael@gigaspaces.com > To:         Huabing Zhao < zhao.huabing@zte.com.cn > Cc:         tosca@lists.oasis-open.org Date:         04/26/2017 07:57 AM Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded Sent by:         < tosca@lists.oasis-open.org > Hi Huabing, all: I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring  interactions between external entities. The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all  the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution. Here are some of the issues I observe are unsolved: 1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy  is now infringed by the proposal, because of several issues - see below. 1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"  will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec. 1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation. 2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts  for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22. 2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification  of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how". 2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate  artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact. 3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to  the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements  and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external  workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how. 4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results. On the how/when, I can see several options that would have to be defined: 4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected  from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about. 4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template  to describe how to wait/block for those results later on. 4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results. 4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue  execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later. There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing  it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better. Best regards, Michael On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao < zhao.huabing@zte.com.cn > wrote : Document Name : Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx No description provided. Download  Latest Revision Public Download Link Submitter :  Huabing Zhao Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC Folder : Working Documents Date submitted : 2017-04-25 19:47:19 -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource         -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource             ------------------------------ ------------------------------ --------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/ apps/org/workgroup/portal/my_ workgroups.php -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           


  • 8.  Re: Re: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded

    Posted 05-06-2017 17:19
    Shitao, I am 100% for supporting all ONAP use cases. And both following use cases where discussed (both in the room and afterwards): 1. from OpenECOMP perspective they want to invoke a TOSCA orchestrator for a sub-domain for which it has complete control. In that case, they may still need support during the execution of the TOSCA-based declarative model, to leverage external workflows. This is the use case for which it is insufficient to delegate externally, but need a declarative mechanism by which one defines how control is returned to the TOSCA-based orchestrator. This is the use case for which the current ZTE/Huawei proposal to TOSCA falls short 2. from Open-O perspective, TOSCA is not used as an orchestrator. It is only used as a language to describe the topology and relationship, and the desire is to indicate that workflows are executed by an external engine. In this use case, some other orchestrator (not TOSCA based) is traversing the TOSCA defined/parsed model, and deciding how the execution of workflows is done, and how to get away from or back to the model - hence it does not need a TOSCA description for that. This is the use case for which the current ZTE/Huawei proposal may be sufficient. My point all along, and in yesterday's discussion is that in ONAP we should do everything that is needed for ONAP, and if the 2nd use case is more urgent than the 1st, so be it. However, as part of the TOSCA TC we promote the entire set of TOSCA features, and as such want to promote the use of a TOSCA-based orchestrator, not just a limited use of TOSCA as a description language and parser. That is why from TOSCA TC perspective we need to focus on solving use case 1, and by solving use case 1 swe implicitly solve use case 2 as well, since it is a subset of use case 1. My opposition to solve only (or even only initially) use case 2 first is that we may have to completely revisit the solution for use case 1 later (perhaps rendering the spec not backward compatible), as well as creating a situation for some may be less interested to solve use case 1 after use case 2 has been addressed, or even worse getting to an extreme situation where use case 1 will never be addressed, because some may oppose it. The issue is that these 2 use cases have never clearly been presented (as described below) in TOSCA TC. If they would have, I am pretty sure that TOSCA TC principals would all be favorable to addressing the broader use case first and consistently, before focusing on a subset that may then be difficult to grow into supporting the full use case. Bottom-line - TOSCA TC is supposed to be a consistent standard. Let's do the work consistently, trying to address things in the right order, and solve the problems from the TOSCA-entire community-serving perspective, rather than unilaterally satisfying a de-facto implementation of a slice of the community. This should not be too difficult, and should not delay things too much. From my vantage point, I would like to understand why the proposers of the half-way solution show resistance to what I propose. Best regards, Michael On Sat, May 6, 2017 at 12:17 PM, Lishitao < lishitao@huawei.com > wrote: Hi Michael and all I suggest we follow what we  have discussed in ONAP  yesterday that focusing on  the use case first. Then  figure out what is the  requirement from ONAP for  workflow. At this moment is  really hard to judge whether  the requirement comes from  ONAP or Open-o (since ONAP is  the merged project between  OpenECOMP and Open-o , from  my perspective, open-o  requirement is the potential  requirement for ONAP). I think what Matt suggested  is the best way we can do at  this moment, that we can work  on a in-normative method  first to fit the  requirement  we have on the table now.  Regards  Shitao  ??? rick (M) +86-13404161483    (E) lishitao@huawei.com ???????-????????? ???: Michael Brenner ???: Matt Rutkowski< mrutkows@us.ibm.com > ??: Chris Lauwers< lauwers@ubicity.com >; Lishitao< lishitao@huawei.com >; Nati Shalom< natis@gigaspaces.com >; t osca@lists.oasis-open.org ; zhao .huabing@zte.com.cn ??: Re: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded ??: 2017-05-06 09:01:16 Matt, all: I would support a pragmatic approach, once we specify a complete proposal - where one not only allows delegation out to external entities, but also indicates at the time how the control returns to TOSCA once the external entity completes. THAT IS what ONAP will really need. The current proposal is actually what Open-O needs/needed - because in that community TOSCA was only used to generate the model, and then other entities leveraged it. In that latter case, one does not care to return to the TOSCA topology, because they do not count on a TOSCA orchestrator to traverse it.  So if we do this, we have to do it right for everyone - so those that do take full advantage of TOSCA via a TOSCA-based orchestrator can delegate to an external workflow engine when they handle some lifecycle management event for a particular node, but return the control to the TOSCA orchestrator once the external handling ends. Short of that - we only serve a community that uses TOSCA in the most superficial and handicapped manner - i.e. its ability to describe the model, that then gets translated to some other format that other orchestrators work with. Not what TOSCA TC envisioned. Let's do this right folks. Michael On Fri, May 5, 2017 at 5:00 PM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: I thought we had resolved going forward to acknowledge BPMN and BPEL (by name) as popular examples of opaque ("Black box" artifact) work flow languages and list all the issues/problems/caveats around using them in conjunction within a declarative topology (for the Orchestrator and portability).  To be clear, we would not endorse them as normative, but neither would we reject them and follow our philosophy of "meeting customer where they are at in their (declarative) journey; that is, not leave them behind, ignore them or force them to choose to rewrite to TOSCA or leave. We would then seek, as a community (and by example over time) the advantages of adopting the declarative approach; which has been our philosophy since we first decided to create the Simple Profile (and even before when we created normative types and the first interop. samples and demos). Kind regards, Matt Rutkowski From:         Chris Lauwers < lauwers@ubicity.com > To:         Nati Shalom < natis@gigaspaces.com >, " zhao.huabing@zte.com.cn " < zhao.huabing@zte.com.cn >, " lishitao@huawei.com " < lishitao@huawei.com > Cc:         " michael@gigaspaces.com " < michael@gigaspaces.com >, " mrutkows@us.ibm.com " < mrutkows@us.ibm.com >, " tosca@lists.oasis-open.org " < tosca@lists.oasis-open.org > Date:         05/05/2017 12:36 PM Subject:         RE: [tosca] Re:[tosca] ??: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded Sent by:         < tosca@lists.oasis-open.org > I completely agree with Nati. We should (pragmatically) support what Huabing wants to accomplish because it will at least result in broader adoption of TOSCA as the way in which one models/describes services.   However, our ultimate goal should not be to simply act as an “input format” for imperative workflows. Our goal should be to drive and support fully declarative orchestration architectures based not just on TOSCA’s modeling features but also based on its orchestration features. To that end, we should leverage ONAP use cases to help identify the necessary improvements in TOSCA that will allow fully-declarative orchestration in most cases, with the occasional “assist” from TOSCA’s imperative workflows.   Chris       From: Nati Shalom [ mailto:natis@gigaspaces.com ] Sent: Friday, April 28, 2017 2:21 AM To: zhao.huabing@zte.com.cn ; lishitao@huawei.com Cc: michael@gigaspaces.com ; Chris Lauwers < lauwers@ubicity.com >; mrutkows@us.ibm.com ; tosca@lists.oasis-open.org Subject: Re: [tosca] Re:[tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Mat and Chris summary i think that your summary represent a pragmatic view on the general direction that TOSCA should strive to. It also provide clear direction toward a model that makes the best use of TOSCA.   I think that this discussion bring is a clear definition and process on what should be considered part of "backward computability" and we should also adopt a clear deprecation model to address future changes to the spec.   Huabing while i agree with the pragmatic approach to allow both imperative and declarative models i think that the backward computability argument is fairly weak.   For a start Its hard to argue backward computability for something that was never defined as part of the standard. So while i would agree that backward computability is important, non of the references that you pointed out (JAVA, IETF) committed for backward computability of things that were implemented outside of the standard and there are many examples of cases where existing implementation had to change to adopt the new standard.   Using Open-O and OpenEcomp as a references for this argument is also a fairly weak argument as both are fairly young projects that didn't even reached their first production phase and are now being pivoted to yet a new project - ONAP in which theres still lots of room for change.   From many of the discussion that i've been involved with theres a much greater opening now within ONAP community to consider more native approach to TOSCA rather limiting it to be yet another configuration input.   We have to consider the tradeoff as well and their impact on the use of TOSCA in the future.   The tradeoff is that if we will not have clear direction toward native approach we will create a legacy issue and backward compatibility issue for all upcoming releases of TOSCA .   Is that what we want? What about forward compatibility?   To be clear i'm totally fine with Chris description that we should treat workflow execution as "blackbox" with more clear interface on on the interaction between the model on those "blackbox" implementation. If this is the general consensus that i think that we made a good progress in this discussion.   Huabing, Shitao i would ask you to look also whether the direction that your pushing for serve best the future direction that we all want to strive to especially now that we still have a chance to change past decision and implementations.   Nati S.                     On Thu, Apr 27, 2017 at 5:54 AM < zhao.huabing@zte.com.cn > wrote: Hi all,   Totally agree with Shitao,If we take a look at the standards/Specifications of both the IT and CT industries(JAVA and IETF are two good example), most of them take the backward compatible issue very seriously when publishing a new version.     On the other hand, the workflow in the TOSCA v1.1 is still in its first stage, far away from maturity. We can encourage new projects to use it for verification and use their feedback to improve it gradually, but it's not reasonable to force the early adapters of TOSCA to change their tremendous existing codes, it's a lot of work, also very risky for them.   We have discussed this issue back and forth for a long time, how could we make a decision about it?   Regards, Huabing         Original Mail Sender:   < lishitao@huawei.com > ; To:   < michael@gigaspaces.com > ; < lauwers@ubicity.com > ; CC:   < mrutkows@us.ibm.com > ;zhaohua bing10201488; < tosca@lists.oasis-open.org > ; Date: 2017/04/27 10:20 Subject: [tosca] ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Michael   I think the intention here is to support what has already been supported in TOSCA v1.0 that external workflow engine should be allowed in TOSCA.  And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some of the issues you mentioned are still there, I agree with it. And now  this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1 support this or not,  people still use external workflow in there TOSCA  applications today. The requirement is there, we cannot just ignore it. As I said, this proposal is just a start, if we close the door at this moment, it will make the gap between implementation and standard even bigger.   Regards shitao   ??? : tosca@lists.oasis-open.org [mailto: tosca@lists.oasis-open .org ] ?? Michael Brenner ???? : 2017 ? 4 ? 27 ?  4:22 ??? : Chris Lauwers ?? : Matt Rutkowski; Huabing Zhao; tosca@lists.oasis-open.org ? ? : Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Chris,   I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of  is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.   Let me try to parse what I am saying:-)   When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow, and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same)  that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through what mechanism it does so (and that could include the use of BPEL/BPMN by some particular  TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine, BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line  with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent the best way possible, and does not favor or disadvantage one implementation or another.  Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing is one the fireproof mechanism to "date" a standard, and it won't last.   But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues to be portable/inter-operable - which means that once we go down the slippery slope of  explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability (let aside of delaying implementations or making them more complex than  necessary).   Do we want to take the red pill or the blue pill?   Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago  - so I am sharing it here.   http://cloudify.co/brochures/t osca-workflows-Apr-2017.pdf   Best regards, Michael On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers < lauwers@ubicity.com > wrote: I agree with Michael that our goal should be to strengthen the appeal of TOSCA. But I  also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations I’m aware  of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very specific  use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we need to  get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it J )   It appears we have differing opinions about the best way to get there. We could try to  discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing service-specific  imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.   Chris     From: tosca@lists.oasis-open.org [mai lto: tosca@lists.oasis-open.org ] On Behalf Of Michael Brenner Sent: Wednesday, April 26, 2017 7:07 AM To: Matt Rutkowski < mrutkows@us.ibm.com > Cc: Huabing Zhao < zhao.huabing@zte.com.cn >; tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded   Hi Matt,   Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and my committed tasks have the tendency  to be able to not always coordinate:-). Like for many of us... I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement on the issues/requirements before  you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.   Best regards, Michael On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski < mrutkows@us.ibm.com > wrote: Hi Michael, Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)...  We touched on most of these issues with Luc echoing many of your thoughts.   In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that we wanted to  embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative.  In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming inputs  in the model and providing access to outputs) that is the general view of processing.  Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should continue. All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes to the other  parts of the topology.  These are all things we will discuss in text.  Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc. In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed. Can you attend the WG call in subsequent weeks? Kind regards, Matt From:         Michael Brenner < michael@gigaspaces.com > To:         Huabing Zhao < zhao.huabing@zte.com.cn > Cc:         tosca@lists.oasis-open.org Date:         04/26/2017 07:57 AM Subject:         Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx  uploaded Sent by:         < tosca@lists.oasis-open.org > Hi Huabing, all: I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring  interactions between external entities. The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all  the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution. Here are some of the issues I observe are unsolved: 1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy  is now infringed by the proposal, because of several issues - see below. 1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"  will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec. 1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation. 2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts  for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22. 2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification  of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how". 2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate  artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation  to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution  would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact. 3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to  the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements  and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external  workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how. 4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does  not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.  It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model.. Therefore, any mechanism delegating  work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results. On the how/when, I can see several options that would have to be defined: 4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected  from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about. 4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template  to describe how to wait/block for those results later on. 4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results. 4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue  execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later. There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing  it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better. Best regards, Michael On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao < zhao.huabing@zte.com.cn > wrote : Document Name : Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx No description provided. Download  Latest Revision Public Download Link Submitter :  Huabing Zhao Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC Folder : Working Documents Date submitted : 2017-04-25 19:47:19 -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource         -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource           -- Michael Brenner, Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource             ------------------------------ ------------------------------ --------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/app s/org/workgroup/portal/my_work groups.php -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource            -- Michael Brenner,  Chief Architect NFV M: +1-732-895-5772 http://getcloudify.org @cloudifysource