Hi Tal,
These are very good questions:
First, we should debate the usefulness of this feature. This feels like yet one more mechanism to derive/refine entities that are described elsewhere. Do we really need it? If we re going to keep it, then we should discuss it in the context of the current discussion about entity refinements, to make sure the same rules apply. For example, your question
about lists: if a derived type defines a requirement called host that is already defined in the parent type, does this mean that the derived type wants to refine the host requirement, or add a second host requirement. My interpretation is the former,
but your rules would suggest the latter. Having different rules will be confusing.
Thanks,
Chris
From:
tosca@lists.oasis-open.org <
tosca@lists.oasis-open.org>
On Behalf Of Tal Liron
Sent: Sunday, February 16, 2020 9:55 AM
To:
tosca@lists.oasis-open.org Subject: [tosca] Implementing the node/relationship template "copy" keyword
This is what we have in the spec:
"The optional (symbolic) name of another node template to copy into (all keynames and values) and use as a basis for this node template."
Actually, this deceptively simple description raises many questions:
1. What is included in "all keynames and values"? "Keynames" is fairly clear, but I understand "values" to also mean nested values. For example, if we have a complex nested data type, do we completely overwrite the basis or perform a nested
merge of the map values? E.g.:
server1:
type: MyType
properties:
mydata:
field1: hello!
field2: world!
server2:
copy: server1
properties:
mydata:
field2: other world!
The question is: should server2 have "mydata.field1 = hello!"? Neither "mydata" nor "field1" are keywords, but they are structural components of the value. So I think, yes, it is expected that this nested value be copied over. Moreover,
this should happen at all nested levels:
server1:
type: MyType
properties:
mydata:
myfield:
subfield1: hello!
server2:
copy: server1
properties:
mydata:
myfield:
subfield2: world!
I think it's understood here that server2 should have both "subfield1" and "subfield2" values.
2. What to do with lists? Specifically let's look at the "requirements" keyword:
server1:
type: MyType
requirements:
- host: mymachine
server2:
copy: server1
requirements:
- db: mydb
The question is: should server2 have the "host" requirements from server1? In other words: when overriding a list, are we replacing it or appending to it? I think server2 should
not have that requirement. The reason is that an element in a list is not a structural component of that value. When we specify the "requirements" keyword we do not mean "and also these requirements" but instead mean "these are the requirements". So,
I think lists as a rule should be replaced, not merged together, when overriding.
Note that lists can also occur in other places, such as complex data types.
3. What about recursion? E.g.:
server1:
type: MyType
properties:
mydata1: hello
server2:
copy: server1
properties:
mydata2: world
server3:
copy: server2
It seems fairly clear to me that server3 should have the "mydata1" property from server1 and the "mydata2" keyword from server2. That means that we're not just copying from another node template, we are also making sure that node template
is "resolved" in terms of what it needs to copy before we copy from it. Recursion is necessary. And that also means that a good parser should be able to detect loops and fail nicely rather than getting stuck in that endless loop (and possibly running
out of stack memory depending on your implementation).
So, what I am suggesting here that "copy" is a fairly low-level mechanism. What is meant is:
"copy" is a recursive YAML copy-and-merge, whereby: 1) map values are recursively merged into existing ones, and 2) new lists replace existing lists.
That, or a more detailed variation of it, should be the description. To me it is the most comprehensible, straightforward, and least confusing way to understand what the "copy" keyword intends. This talk about "keywords and values" raises
more questions than it answers.
Finally, not sure why "(symbolic)" is there in the original description. We just mean a node template name, right? Best to remove that, it's confusing.
One more note about implementation -- depending on how you architected your parser, this could be a very challenging feature to implement! For example, if you use some kind of YAML mapping parser to move directly from YAML to your programming
language data structures (classes, structs, etc.) then it might be non-trivial to copy that data from another instance and then merge it recursively. You might need to do reflection.
Probably the "cleanest" way to handle this feature is in the pre-reading phase, in raw YAML. That way you can set up all the YAML in the right way and have it read as-is into your data structures. Again, depending on how you architected
your parser it might be challenging to implement such a phase. (This "pre-read" phase how I've handled it in Puccini.)