OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC

 View Only
  • 1.  RE: TOSCA inputs and outputs for shell scripts

    Posted 10-07-2020 02:56










     
      Clearly using environment variable for outputs doesn t work. As alternatives, we discussed passing a file descriptor, or an environment variable that specifies a location to which to write outputs. Have you given any more
    thought to this?




    Yes I considered implementing something like that where the shell scripts could echo outputs to a file descriptor that was set in an environment variable and the orchestrator would read from that file after the
    script terminated. But I didn't find implementing this was necessary since my orchestrator has TOSCA extensions that lets the user configure the environment variables passed to the shell script or process and also a template which has access to the script's
    stdout, stderr and returncode. So essentially the user is free to implement any private protocol between the orchestrator and shell script as needed. This is particularly useful if you need to work with pre-existing scripts.
    I m not sure how having access to the script s stdout and stderr allows the orchestrator to retrieve the appropriate operation output values. If a script has to return output values, doesn t it need to be designed
    to do so?
    When my orchestrator invokes the artifact processor for bash scripts, it will clone the stdout file descriptor to a new file descriptor (typically nr. 3), and then redirect stdout and stderr to various log files.
    The script will then echo any required output values to the cloned file descriptor.



     


    Regarding file descriptors, their main benefits compared to a file path are that file descriptors are more secure: easier to avoid potential race conditions and information leaks, and they can point to a private
    temp file not visible on the file system. They are also more flexible (e.g. they don't require access to a file system, they can point to pipe so they can more easily support streaming, etc.)


     





            
    Even for input values, environment variable are not always convenient. What is the standard way for encoding maps, lists, or complex data types so they can be carried in an environment variable? I m curious how your implementation
    handles this.




     



    My implementation handles this by trying to parse the variable's value as JSON, and if parsing fails, treat it as a string. This approach is easy to implement and handles numbers, lists, and maps pretty much
    as the user would expect.
    That s a reasonable and elegant approach, but why wouldn t we always encode all input values as JSON?

     
    Thanks,
     
    Chris
     










  • 2.  Re: TOSCA inputs and outputs for shell scripts

    Posted 10-07-2020 23:30
    On Tue, Oct 6, 2020 at 7:56 PM Chris Lauwers <lauwers@ubicity.com> wrote:

    > · Clearly using environment variable for outputs doesn’t work. As
    > alternatives, we discussed passing a file descriptor, or an environment
    > variable that specifies a location to which to write outputs. Have you
    > given any more thought to this?
    >
    > Yes I considered implementing something like that where the shell scripts
    > could echo outputs to a file descriptor that was set in an environment
    > variable and the orchestrator would read from that file after the script
    > terminated. But I didn't find implementing this was necessary since my
    > orchestrator has TOSCA extensions that lets the user configure the
    > environment variables passed to the shell script or process and also a
    > template which has access to the script's stdout, stderr and returncode. So
    > essentially the user is free to implement any private protocol between the
    > orchestrator and shell script as needed. This is particularly useful if you
    > need to work with pre-existing scripts.
    >
    > I’m not sure how having access to the script’s stdout and stderr allows
    > the orchestrator to retrieve the appropriate operation output values. If a
    > script has to return output values, doesn’t it need to be designed to do so?
    >
    > When my orchestrator invokes the artifact processor for bash scripts, it
    > will “clone” the stdout file descriptor to a new file descriptor (typically
    > nr. 3), and then redirect stdout and stderr to various log files. The
    > script will then “echo” any required output values to the “cloned” file
    > descriptor.
    >
    >
    >

    Yes, I just mean that the user defined templates can implement a protocol
    like yours or something different as needed. For example, to support
    Terraform it can read outputs from its Terraform state file (which is
    encoded as json).



    > Regarding file descriptors, their main benefits compared to a file path
    > are that file descriptors are more secure: easier to avoid potential race
    > conditions and information leaks, and they can point to a private temp file
    > not visible on the file system. They are also more flexible (e.g. they
    > don't require access to a file system, they can point to pipe so they can
    > more easily support streaming, etc.)
    >
    >
    >
    > · Even for input values, environment variable are not always
    > convenient. What is the standard way for encoding maps, lists, or complex
    > data types so they can be carried in an environment variable? I’m curious
    > how your implementation handles this.
    >
    >
    >
    > My implementation handles this by trying to parse the variable's value as
    > JSON, and if parsing fails, treat it as a string. This approach is easy to
    > implement and handles numbers, lists, and maps pretty much as the user
    > would expect.
    >
    > That’s a reasonable and elegant approach, but why wouldn’t we always
    > encode all input values as JSON?
    >
    >
    >

    Yes that makes sense... in my case also needs to be able to handle external
    environment variables that weren't set by the orchestrator.


    > Thanks,
    >
    >
    >
    > Chris
    >
    >
    >