OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC

 View Only
  • 1.  Re: [tosca] Operation implementations

    Posted 09-21-2021 09:49




    Hi,
     

    I imagine that an artifact processor should be a generic prerequisite that should be available in the environment where the TOSCA orchestrator executes. If an artifact
    is defined as an implementation of an operation etc. and the orchestrator cannot find it, then the orchestrator should flag an error (either as a runtime or during the validation phase), but not when importing a profile.
    Yes, agree. Maybe the fact that an artifact is an implementation should be described in the description part of the artifact definition, then anybody that could
    use it would know, so isImplementation may not be that useful. Yes, regarding the get_artifact it would be more useful to return something like the artifact definition map, thus sending all all the properties. This would avoid
    also the confustion that if two artifact definition point to the same path, you cannot distinguish them in the get_artifact function.
     
    BR/Calin
     
     

    From:
    Chris Lauwers <lauwers@ubicity.com>
    Date: Tuesday, 21 September 2021 at 04:56
    To: Calin Curescu <calin.curescu@ericsson.com>, adam souzis <adam@souzis.com>, Tal Liron <tliron@redhat.com>
    Cc: "tosca@lists.oasis-open.org" <tosca@lists.oasis-open.org>
    Subject: RE: [tosca] Operation implementations


     

    Hi Calin,
     
    Thanks, this is a nice summary. A couple of comments:
     


    Yes, we could explicitly specify that a specify type of artifact is intended to be used as an implementation. Does that mean that the profile that contains the artifact definition must also include code that can be used as the artifact processor?
    Will the orchestrator flag an error if no artifact processor is specified?
    Alternatively, the mere fact that an artifact is used as an operation implementation would implicitly specify that this is an implementation artifact . An orchestrator would then flag an error at orchestration time if there is no artifact
    processor/configurator defined for that artifact type
    We need to revisit the get_artifact intrinsic function. As defined right now, the get_artifact function returns a string that contains the path name of the artifact, but it doesn t say anything about the context in which that path is
    valid. More importantly, there is no mechanism for passing artifact properties as part of the get_artifact function, which makes artifact properties useless for artifacts that are not used as operation implementations.
     
    Thanks,
     
    Chris
     
     


    From: Calin Curescu <calin.curescu@ericsson.com>

    Sent: Monday, September 20, 2021 10:30 AM
    To: adam souzis <adam@souzis.com>; Tal Liron <tliron@redhat.com>; Chris Lauwers <lauwers@ubicity.com>
    Cc: tosca@lists.oasis-open.org
    Subject: Re: [tosca] Operation implementations


     
    Hi Adam, Tal, Chris,
     
    How I read the TOSCA specification, is that artifacts are used to designate information that is used/is executed on the outer side of the membrane that is the boundary between the TOSCA scope and the outside world (including
    the underlying implementations). Thus from the point of view of TOSCA they are artifacts as the TOSCA parser/orchestrator cannot understand or control what and how they do things. I agree that there are 2 classes of artifacts:

    artifacts that are used by an artifact processor to execute a specific action, triggered by the TOSCA orchestrator. These I consider implementation artifacts and
    will represent a script, a code, a specific DSL snippet, that the artifact processor is interpreting, or just-in-time compiling to execute that specific action.
    the foremost example here are the artifacts that are operations implementations we have not discussed how policies are implemented in the context of TOSCA yet, but I could envision that such an implementation artifact could be associated to
    a policy
    artifacts that represent a specific data/information, that is not executed by an artifact processor, but is used as additional external input to another artifact
    that is an implementation artifact executed by an artifact processor
    here for me are VM images, or JPEG pictures, or location data, etc I do not think that these artifacts should have an intrinsic meaning to TOSCA based on the node type as it was the case in TOSCA 1.2 where the image artifact was
    associate to Compute nodes as providing the VM image
    I think such VM images should be given as input to the e.g. create operation, and its implementation explicitly via the get_artifact function.


    So for both these cases I think that name artifact is fitting as for TOSCA they are external objects.
     
    I also think it could be useful for the designer specifying artifacts in a profile to be able to signal if an artifact is intended to be used as an implementation artifact that is implemented by an artifact processor,
    or not, so we could introduce a new keyword such as isImplementation to signal that this is an artifact that is assumed to be interpreted by an artifact processor to implement a specific action triggered by the TOSCA orchestrator.
     
    BR,
    /Calin
     
     

    From:
    < tosca@lists.oasis-open.org > on behalf of adam souzis < adam@souzis.com >
    Date: Wednesday, 8 September 2021 at 21:48
    To: Tal Liron < tliron@redhat.com >
    Cc: Chris Lauwers < lauwers@ubicity.com >, " tosca@lists.oasis-open.org " < tosca@lists.oasis-open.org >
    Subject: Re: [tosca] Operation implementations


     


    Yes, an artifact represents a file-like bag of data, and for example, a DevOps deployment process can interact with artifacts in a myriad ways, whether its container images, build process artifacts, artifact registries
    like JFrog, files destined for a cloud object store like AWS S3, deployment artifacts like Terraform state files, etc.  I see its use as representing an implementation as secondary and probably confusing for most users if we overload its meaning to mean any
    interface to an executable process.


     




    á

     


    On Wed, Sep 8, 2021 at 11:46 AM Tal Liron < tliron@redhat.com > wrote:





    On Wed, Sep 8, 2021 at 1:32 PM Chris Lauwers < lauwers@ubicity.com > wrote:




    Hi Tal, would you mind clarifying the two roles that artifact types take on? Based on my understanding, the artifact type is only there to inform the orchestrator
    which code needs to be executed to call the artifact (or, using Adam s terminology, which Configurator to use). Do you see another role for artifact types?




     


    I see many roles for artifacts that have nothing to with operations:


     


    1. Virtual machine images attached to nodes


    2. Policy definitions for specialized agents


    3. Schemas for special data type validation (e.g. YANG)


    ...











  • 2.  Re: [tosca] Operation implementations

    Posted 09-30-2021 17:44
    Sorry it took me a while to write this up, but finally here is a draft of what I mean by "typed implementations". The proposed grammar is actually very simple. Instead of "implementation" having that weird structure of a "primary" string and a string-list of "dependencies", which semantically were never very clear, I suggest that it be given a data type using a "type" keyword for both the operation definition and assignment (a refinement of the definition). This data type provides syntactical validation for "implementation", but more importantly it represents the processor's mechanism for reacting to the operation event. In the example below I'm showing a node with two operations. One of them is a gRPC call and the other is remote execution using ssh. Both of them are "remote", and the default for the Remote data type from which they both derive assumes that the node has an "address" attribute. However, this default address can be overridden and given a different value if necessary, for example if the node is not a server itself, or if another component (an orchestrator) is handling execution. The gRPC call has nothing to do with artifacts per se. However, it is expected that there is a gRPC client somewhere that can handle these calls, perhaps statically compiled from a ".proto" file, which perhaps can be included in the CSAR. But that ".proto" file defines the entire membrane, not individual calls. It's also possible to imagine a ".proto"-to-TOSCA translator that would create derived data types and for each supported call and one big interface type. The ssh implementation is actually quite similar, but it does have optional support for artifacts. The idea is that the artifacts would be copied over (scp) before executing the ssh command. Here we can extract TOSCA artifacts from the CSAR using the "get_artifact" function or just use any file from a URL. One issue is what to do with inputs: setting environment variables may be impossible (PermitUserEnvironment is set to false for most sshd deployments). So, what I am suggesting here is that the processor would know how to pack the inputs into a command-line-compatible string. The result is something which I think is most similar to the spirit of the existing "primary + dependencies" structure of "implementation", but I did want to present some of the complexities regarding where the command runs and how to construct it. Also note a simple use of metadata in order to configure specific aspects of execution for specific orchestrators. This approach raises a more general issue regarding interfaces in TOSCA. In much of our thinking and examples until now we've treated interfaces as a design element coming from TOSCA. In the Simple Profile specifically they represent the lifecycle as defined by TOSCA. So, our job in that case is to then map that paradigm onto whatever exists in the real world. This mapping needs to be done not just in TOSCA but likely also with some kind of adapter layer. In Ubicity this layer is a collection of executable scripts. But I want to suggest that there's another approach in which we actually start from the real world and find ways to expose it to TOSCA. For example above I mention that an existing ".proto" file ( or a Swagger schema) can be translated into TOSCA. In this approach the designer actually has no freedom to create arbitrary interfaces, because they are defined by the environment, and there's no adapter layer: it is a direct membrane to the environment. TOSCA's job, then, is to map data from the topology into that given interface. And that's where better topology traversal functions (get_property/get_attribute) would be really useful. There are a lot of issues I'm not addressing here, but I want to start the discussion simply! The example: interface_types: Maintenance: operations: maintenance-on: inputs: mode: type: string constraints: - valid_values: [ staging, production ] maintenance-off: type: SSH # data type for the implementation. if not declared, must be set in assignment inputs: immediate: type: bool data_types: Remote: properties: address: type: string default: { get_attribute: [ SELF, address ] } credentials: type: Credentials default: { get_input: credentials } GRPC: derived_from: Remote properties: rpc: type: string timeout: type: scalar-unit.time required: false user-agent: type: string default: TOSCA SSH: derived_from: Remote metadata: myorchestrator.executor: fabric properties: command: type: list entry_schema: string work-dir: type: string default: /tmp files: description: Copy over these files (optional) type: list entry_schema: string node_types: Server: interfaces: Maintenance: type: Maintenance topology_template: node_templates: server: type: Server interfaces: Maintenance: operations: maintenance-on: type: GRPC implementation: rpc: StartMaintenanceWithMode timeout: 10 s inputs: mode: production maintenance-off: implementation: command: [ /usr/bin/bash, -c, '/home/tosca/off.sh $INPUTS > off.log' ] files: - { get_artifact: [ SELF, main ] } - { get_artifact: [ SELF, utils ] } - http://acme.org/utils/lib.tar.gz inputs: # This will be packed as "--immediate=true" in $INPUTS immediate: true artifacts: main: file: off.sh deploy_path: /home/tosca utils: file: utils.sh deploy_path: /home/tosca