Hi Michael,
I fully agree with you and that is exactly what I stressed out in the Tuesday call and I was the one feeling lonely. I also have the sentiment that during the
past year we tried to close the gaps and missing elements that existed in the 1.0 yaml spec and 1.0 xml spec that both had not enough elements for interoperability. This goes from detailed (maybe still not enough) information written in the spec on how workflows
should be generated and how relationships impact them to how to define workflow in a way that taken as is has all the details on how to execute TOSCA defined in the nodes operations, change node states, get output from the operation (through environment variable
and get_operation_output function which we all know is not the most perfect way but that is at least a specified and interoperable way).
Some people claim backward compatibility but the truth is that 1.0 XML spec and 1.1 yaml spec are not, and not just about workflows. Parsing is the most obvious
but there is actually more than that. Workflow I insist are not part of 1.0 specification, there is just a vague reference that BPML or BPEL COULD be used. But there is no clear specification that this is we way to go. Moreover, there is no specification on
how an operation from TOSCA get generated in BPMN, I know some people have made work and extension of BPMN for TOSCA but here again this is not standard stuff neither done in BPMN working groups or TOSCA working groups.
I also fully agree that there is interesting work to be done to try to make all initiatives reach a common way but I also think that we should not rush to please
people and lose any runtime value of the standard as the result will be that all implementers will have different ways to define things, that we will acknowledge actually not only BPMN but anything ant that if we allow that there will be no way to keep a backward
compatibility with everyone AND keep interoperability.
As Chris stated we are improving the Operation support so any black-box can be called with a clear contract (as the contract we have with shell script and python
that is clear even if not perfect) and in a way that will improve the standard while not reducing its portability (actually this will be the opposite).
We can do the same on workflows but as Chris explained this would have to be a next step as it is from a logical point of view and from a TC bandwidth point of
view (and actually it should leverage the work started and not completed on operations). So in my opinion we should not keep losing time on having the same workflow discussion that consumed around 3/4 yaml calls while we put on hold the discussions around
Artifact support that would make us go forward (both on the immediate operation thing and the potential future extended workflow support).
My opinion is that for now vendors that wanted to support non-normative things have custom DSL “close to TOSCA” but not TOSCA, we have one, GigaSpaces has one
and I guess that while we don’t support Workflow extensions in a portable way this should be the way to go for vendors clearly marking that this template is not a portable TOSCA template but a Vendor specific “close to TOSCA” template that may have specific
requirements on their solution. (for example, tosca_definition_version: my-vendor-dsl-1.0.0).
The other option that is the one currently supported by most of people (except Michael and myself if we follow this thread) is that we should acknowledge the
fact that TOSCA support calling any kind of workflow that will leave the Orchestrator unaware of anything deployed, whether it is compliant or not with elements defined in the template and that for such use-case the orchestrator is actually not able to orchestrate
anything, will not have any knowledge of the elements deployed will not be able to provide any instance model because he won’t know states or attributes of any nodes as there is right now, no ways to feed them. For such use-case the orchestrator will deploy
things but won’t be able to make them interoperate with any other TOSCA nodes or deployments as too many information will be missing. In such use-case the TOSCA orchestrator will be no more than executing a command line and providing a schema that may or may
not be related to the actual deployment. Note that actually such workflows may have dependencies and potentially on another TOSCA orchestrator. For example, it could refer to Cloudify specific workflow generation language that leverages the Cloudify API so
the template will be TOSCA compliant and portable on any TOSCA orchestrator with the requirement to first deploy the required orchestrator. Or I could even from a TOSCA standard template refer to a vendor specific template so it makes it a “portable” TOSCA
template.
Best regards,
Luc
From:
<
tosca@lists.oasis-open.org> on behalf of Michael Brenner <
michael@gigaspaces.com>
Date: Thursday, 27 April 2017 at 05:27
To: shitao li <
lishitao@huawei.com>
Cc: Chris Lauwers <
lauwers@ubicity.com>, Matthew Rutkowski <
mrutkows@us.ibm.com>, Huabing Zhao <
zhao.huabing@zte.com.cn>, "tosca@lists.oasis-open.org" <
tosca@lists.oasis-open.org>
Subject: [tosca] Re: ?? : [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded
I understand there is a desire to re-use work done via BPEL/BPMN, and have no issue with that. What I do not understand is why we rush to make a half-standard, when we could support initially an implementation that would do exactly that,
without the backing of a standard. Seems like rubber-stamping something that is half-cooked, instead of working through the problems. Not the way good standards are created.
Btw - the backward compatibility however is a red herring issue, since we are talking of quite different 2 specs, and the companies that use BPEL/BPMN workflow engines (for a good reason, solving B2B issues) in fact do not use is with the
XML version of TOSCA. We are bound to create a spec that cannot ensure portability and inter-operability, I cannot repeat this enough. This will NOT be a good standard though - since it creates more problems than solutions.
I would have preferred that the community works to provide a good/complete solution to the problem, rather than introducing partial solutions in the standard. We can live with partial solution in the context of a community (e.g. ONAP) until
we implement (there or elsewhere) a complete solution, then go and standardize it.
Bottom-line, I cannot support this approach, and at the same time I realize I may be a lonely voice - so the community will decide what it decides. No hard feelings either way:-)
Best regards,
Michael
On Wed, Apr 26, 2017 at 10:19 PM, Lishitao <
lishitao@huawei.com > wrote:
Hi Michael
I think the intention here is to support what has already been supported
in TOSCA v1.0 that external workflow engine should be allowed in TOSCA. And after several years of publishing the first version of TOSCA standard (since 2013), many TOSCA applications have still chosen to use this mechanism, such as OpenECOMP and Open-O. Some
of the issues you mentioned are still there, I agree with it. And now this proposal is just a start, it contains just what TOSCA 1.0 has mentioned, nothing more, and the purpose is for backward compatible. No meter tosca-simple-profile-yaml version 1.0, 1.1
support this or not, people still use external workflow in there TOSCA applications today. The requirement is there, we cannot just ignore it.
As I said, this proposal is just a start, if we close the door at this moment,
it will make the gap between implementation and standard even bigger.
Regards
shitao
? ?? :
tosca@lists.oasis-open.org [mailto:
tosca@lists.oasis-open.org ]
??
Michael Brenner
? ? ?? :
2017 ? 4 ? 27 ?
4:22
??? :
Chris Lauwers
?? :
Matt Rutkowski; Huabing Zhao;
tosca@lists.oasis-open.org ? ? :
Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded
Hi Chris,
I appreciate the challenges. Let me clarify that I do not discourage the native imperative workflows defined in TOSCA, and
I don't think that what I suggest is in conflict with Matt's view. What I am not a big fan of is half-measures that lead to lack of portability/inter-operability ... and that was the main point of my comments.
Let me try to parse what I am saying:-)
When TOSCA promotes its native (declarative OR imperative workflows), all one describes is the intent at the end of the workflow,
and nothing more. I interpret this (and any TOSCA-consuming orchestrator would do the same) that using the service template and the provided artifacts, the TOSCA-consuming orchestrator is able to accomplish the task - and it absolutely does not matter through
what mechanism it does so (and that could include the use of BPEL/BPMN by some particular TOSCA-based orchestrator, if that is how it implements workflow). But TOSCA service template in this case is "workflow implementation neutral" (i.e. unaware of the engine,
BPEL/BPMN/embedded in the orchestrator, or whatever else you may have) which is in-line with the TOSCA philosophy of under-specifying things (a good principle when it comes to standards that are build to last). It allows ANY orchestrator to fulfill the intent
the best way possible, and does not favor or disadvantage one implementation or another. Which makes the spec support portability/inter-operability. And it gets you away from specifying all the complexities I mentioned to make the spec right. Over-standardizing
is one the fireproof mechanism to "date" a standard, and it won't last.
But if we choose to over-specify (I don't have a Quijotian syndrome:-) then we MUST specify in such a way that the spec continues
to be portable/inter-operable - which means that once we go down the slippery slope of explicitly supporting/endorsing external workflow engines, we better specify the mechanism in full, because otherwise we deliberately introduce lack of portability and inter-operability
(let aside of delaying implementations or making them more complex than necessary).
Do we want to take the red pill or the blue pill?
Btw Chris, here is the attached write-up on workflows. By now we have advanced to further analyzing the topic in-depth on this
thread and in the TOSCA TC meetings, but I remembered you showed interest a few weeks ago - so I am sharing it here.
http://cloudify.co/brochures/tosca-workflows-Apr-2017.pdf Best regards,
Michael
On Wed, Apr 26, 2017 at 2:52 PM, Chris Lauwers <
lauwers@ubicity.com >
wrote:
I agree with Michael that our goal should be to strengthen the appeal of
TOSCA. But I also agree with Matt that we should be pragmatic about this. The reality we’re faced with is that (as far as I know) there aren’t any interoperable implementations of TOSCA out there today that adopt a truly declarative paradigm. The implementations
I’m aware of use TOSCA mostly for modeling but not necessarily for orchestration. Instead, they rely largely on imperative logic (such as the BPNM workflows in Open-O) provided by the service template designer (or worse, hardcoded in the Orchestrator for very
specific use cases) to get a TOSCA service model deployed. What we need to get to instead is a “true” TOSCA orchestrator that provides a “TOSCA runtime” that is able to process any standards-based declarative TOSCA service template. Said a different way, we
need to get to the point where people use TOSCA for its orchestration features rather than (or in addition to) its modeling features. We’re not quite there yet (although some of us are working on it
J )
It appears we have differing opinions about the best way to get there. We
could try to discourage the “imperative” use of TOSCA (as Michael suggests) with the goal of promoting the declarative paradigm, or we could support it with the goal of getting broader adoption of TOSCA models (as Matt suggests) with the goal of replacing
service-specific imperative logic (such as BPMN workflows) with generic declarative support in the Orchestrator over time.
Chris
From:
tosca@lists.oasis-open.org [mailto:
tosca@lists.oasis-open.org ]
On Behalf Of Michael Brenner
Sent: Wednesday, April 26, 2017 7:07 AM
To: Matt Rutkowski <
mrutkows@us.ibm.com >
Cc: Huabing Zhao <
zhao.huabing@zte.com.cn >;
tosca@lists.oasis-open.org Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx uploaded
Hi Matt,
Thanks for the feedback, and certainly, I wish too to have been part of the discussion ... it just my wishes/interests and
my committed tasks have the tendency to be able to not always coordinate:-). Like for many of us...
I'll do my best to be in other calls dedicated to this topic - but discussing over email, and reaching some tentative agreement
on the issues/requirements before you propose solutions, works best for the kind of schedule I have, at least. I think if the requirements are agreed, the solution may be more straightforward to be agreed.
Best regards,
Michael
On Wed, Apr 26, 2017 at 9:42 AM, Matt Rutkowski <
mrutkows@us.ibm.com >
wrote:
Hi Michael,
Wish you had been on the call yesterday (perhaps you can read my meeting notes posted to TOSCA?)... We touched on most of these issues with Luc echoing many of
your thoughts.
In the end, TOSCA has taken the philosophy of recognizing that exiting providers use existing tooling for installs/configs (including BPEL/BPMN workflows) and that
we wanted to embrace these providers and recognize/point out the pitfalls and encourage them (when able) to move to declarative. In addition, if we can treat these tools (including ansible, or other) as "black boxes" that can return us to a known state (consuming
inputs in the model and providing access to outputs) that is the general view of processing. Of course, we have other proposals for operational "hints" on expected processing time (will not call it timeout) for the orchestrator to gauge how long it should
continue.
All agreed, that an instance model/API would be needed to allow full introspection after the "black boxes" complete and that these workflows should not cause changes
to the other parts of the topology. These are all things we will discuss in text. Please also account for the Artifact Processor entity which will also have information for the orchestrator (for invocation) and can also have properties/configurations, etc.
In addition, we agreed not to call this "delegate" as our existing delegate does not match this behavior proposed.
Can you attend the WG call in subsequent weeks?
Kind regards,
Matt
From: Michael Brenner <
michael@gigaspaces.com >
To: Huabing Zhao <
zhao.huabing@zte.com.cn >
Cc:
tosca@lists.oasis-open.org Date: 04/26/2017 07:57 AM
Subject: Re: [tosca] Groups - Issue_TOSCA318_Lack
of BPMN BPEL support-v-2017-04-25.pptx uploaded
Sent by: <
tosca@lists.oasis-open.org >
Hi Huabing, all:
I reviewed the proposed changes to support external workflow engines via "delegate" mechanism, and I think this is something that can complement TOSCA in special cases - e.g. re-using workflows previously implemented outside of TOSCA, in particular those requiring
interactions between external entities.
The problem that I have is that the current proposal leaves us with many unsolved issue, and renders a TOSCA-consuming orchestrator with more questions than the answers that an external workflow engine can provide. I would prefer us to work on resolving all
the issues to provide a workable and portable solution, rather than inserting a half-baked mechanism that we do not know yet how it will converge to the full solution.
Here are some of the issues I observe are unsolved:
1. TOSCA philosophy is to specify "what" (intent) and not "how" (implementation). How a TOSCA consuming orchestrator resolves the TOSCA-driven requirements is typically irrelevant, but the expectation is that the result fulfills the requirements. This philosophy
is now infringed by the proposal, because of several issues - see below.
1.a: If we "standardize" the artifact, i.e. have some standard artifact name or suffix that a TOSCA parser needs to recognize, we are going from "what" to "how". And we also create a precedent that would require constant changes - because other such "keynames"
will emerge. Perhaps this could be solved with a separate registry, but I don't think it should be standardized as part of the TOSCA spec.
1.b: on the other hand, if the parser does not have a way to identify whom to delegate, it is at a loss about what to do with the artifact. So it is sort of a catch 22 situation.
2. Assuming we find the right solution for 1. above, we still have a major portability issue. Again, typically a TOSCA-consuming orchestrator should not understand/parse the content of the artifacts, but just execute them or pass them along to other artifacts
for consumption. The proposal does not specify what the TOSCA-consuming orchestrator is supposed to do with the artifact that is passed to the delegate workflow. And that gets us into a different catch22.
2.a: If the delegate artifact is opaque to TOSCA-consuming orchestrators, 2 different orchestrators may process the artifact quite differently in the best case, or would not know what to do it (beyond parsing) in the worst case. So this requires a clear specification
of how the TOSCA-consuming orchestrator is supposed to process the artifact, in order to consistently get the same result. Obviously, that moves us from "what" to "how".
2.b: Assuming there is a good solution to 2.a above, we run into a different issue, and that becomes an implementation and compliance issue. Each TOSCA-consuming orchestrator would have to be able to support the mechanism described for processing the delegate
artifact, but for every different artifact (e.g. BPEL vs. BPMN ... and this is just the beginning precedent), the mechanism would have to have specific parts, in addition to the generic parts, most likely. That basically forces a TOSCA-based orchestrator implementation
to support in a standard way ANY potential mechanism to delegate to any potential workflow engine. This is not something one should ask a vendor that will be forced into compliance, and it is completely atypical for a standard to ask for this. The solution
would be to define a one-time generic mechanism of passing the artifact, regardless of which what external workflow engine is required to process the artifact.
3. TOSCA service template defines a topology, with nodes that have certain states. When control is being passed to the delegated external workflow engine, it is not clear in the proposal what the expectations (and constraints) are with respect to changes to
the topology and/or nodes and/or states of the nodes by the external entity. A stable solution would be advantaged by a single source of truth for topology, nodes and states - in this case that source of truth is the TOSCA-consuming orchestrator that implements
and maintains the TOSCA service template. That would argue that the delegated workflow engine cannot change topology, nodes or node states? If this is the intent, it should be stated in the changes to the TOSCA spec. If the intent is different (i.g. external
workflow engine can change topology, nodes, and their states) then how do we ensure stability of the solution? No mechanism is included in the proposal for communicating back to the TOSCA-consuming orchestrator what has changed and how.
4. That brings us to another general issue. While the proposal attempts to define a mechanism that gives an external entity the delegation to execute a workflow (and I described above why that portion needs more work to ensure portability), the proposal does
not address how the control is returned to the TOSCA-consuming orchestrator, neither is it defined when the control is returned. TOSCA-consuming orchestrator is the master/manager of the topology/nodes/states that it deployed or is in course of deploying.
It uses the topology and the relationship between nodes to traverse the topology in a certain order, and execute the lifecycle management of the different nodes in the topology based on the declaration of intent in the model. Therefore, any mechanism delegating
work outside the TOSCA-consuming orchestrator is incomplete unless it describes how the control is returned to the TOSCA-consuming orchestrator, and when, and with what results.
On the how/when, I can see several options that would have to be defined:
4. a: delegate and forget. In this case, the TOSCA-consuming orchestrator does not block to wait the result of the external workflow execution, and instead continues to work on implementing the intent described in the service template. No results are expected
from the external workflow processing that are needed by the TOSCA-consuming orchestrator to know about.
4.b: delegate and sync up later. This is a similar case as 4.a, but in this case there may be results from the external workflow execution available later on, that are of interest. In this case, a mechanism needs to be defined to allow a TOSCA service template
to describe how to wait/block for those results later on.
4.c: delegate and wait/block indefinitely. In this case, as soon as TOSCA-consuming orchestrator delegates a workflow externally, it would stop executing any further, and wait for the workflow to complete and return control - possibly with results.
4.d: delegate and wait/block for a period of time. This is similar to 4.c above, but a timer is associated, and if the external workflow engine does not return control to the TOSCA-based orchestrator within the defined time interval, the latter will continue
execution. A further option would be to add also the mechanism described in 4.b, i.e. to allow for a timed-out case to sync up again later.
There may be other situation we need to consider. In any case, the main point I want to make is that we do need to think about these issues and provide a consistent and portable solution. This is a complex mechanism if we want to do it right. And not doing
it right ... well, it is just not right, and weakens, rather than strengthening the appeal of TOSCA specification. I am sure that we can all agree that our interest is to make the TOSCA spec better.
Best regards,
Michael
On Tue, Apr 25, 2017 at 10:47 PM, Huabing Zhao <
zhao.huabing@zte.com.cn > wrote:
Document Name :
Issue_TOSCA318_Lack of BPMN BPEL support-v-2017-04-25.pptx
No description provided.
Download
Latest Revision
Public Download Link
Submitter : Huabing Zhao
Group : OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC
Folder : Working Documents
Date submitted : 2017-04-25 19:47:19
--
Michael Brenner, Chief Architect NFV
M:
+1-732-895-5772
http://getcloudify.org @cloudifysource
--
Michael Brenner, Chief Architect NFV
M:
+1-732-895-5772
http://getcloudify.org @cloudifysource
--
Michael Brenner, Chief Architect NFV
M:
+1-732-895-5772
http://getcloudify.org @cloudifysource
--
Michael Brenner, Chief Architect NFV
M: +1-732-895-5772
http://getcloudify.org @cloudifysource