All,
While I agree with the concepts I would urge you to be very
careful here. Gary Ham taught me many years ago the “A Standard Ai’nt
a Standard if it isn’t used”. In his white paper he focuses
on ease of implementation as a key issue. What you are describing below
is a series of best practices that should be recommendations from the
Adoption Committee. To consider this as part of the guidance of using the
standard will make many developers just turn away from it and the result will
be less implementation – thus less interoperability.
Profiles are nothing more than further restraints and element definition
enhancements/restrictions to the approved standard that need to be understood
by two or more exchange partners. By ensuring that the “Profile”
validates against the original schema than any other entity that uses complete/original
Standard Schema can consume it….sharing your “further restrained”
schema may be desirable from an implementation standpoint, but that depends on
your intended use – and we cannot assume that everyone intends to use profiles
exactly as we do….in many cases a profile will be shared between only 2
exchange partners and no one else needs to know the restrictions enforced by
the profile….
Thanks,
Lee
The aim of education should be to teach us rather how to think,
than what to think - rather to improve our minds, so as to enable us to think
for ourselves, than to load the memory with thoughts of other men. ~Bill
Beattie
From: Ron Lake
[mailto:rlake@galdosinc.com]
Sent: Sunday, March 14, 2010 4:39 AM
To: Ram Kumar
Cc: Gary Ham; David RR Webber (XML); Lee Tincher;
emergency@lists.oasis-open.org; Dwarkanath,Sukumar - INTL; McGarry,Donald P.
Subject: RE: [emergency] HAVE Conformance vs. Documentation vs. Released
Schemas
I
would go farther and state that we need registries in support of this
interoperability governance - registries that manage schemas, code lists,
schema documentation etc. Simply posting schemas on a web site (and
claiming this is a registry) is NOT sufficient and it will not work.
From: Ram Kumar [mailto:kumar.sydney@gmail.com]
Sent: Sun 3/14/2010 12:23 AM
To: Ron Lake
Cc: Gary Ham; David RR Webber (XML); Lee Tincher;
emergency@lists.oasis-open.org; Dwarkanath,Sukumar - INTL; McGarry,Donald P.
Subject: Re: [emergency] HAVE Conformance vs. Documentation vs. Released
Schemas
If we want to achieve interoperability, two things are
required:
1. Interoperability of data - schemas are required
2. Guidelines on how the schemas should be used (what is optional,
what is not, what code lists to use, etc) to enable interoperability. This will
help the interoperating parties to use these guidelines to ensure consistent
implementation of the schemas. - This is part of interoperability governance
Therefore, using a set of schemas and expecting systems
implementing the schemas without any guidelines to ensure consistent
implementation, to interoperate is virtually impossible.
xPIL and other CIQ artifacts have been designed to be
application independent and vertical industry independent, and importantly
global (ability to handle 240+ country addresses and many name formats), it is
up to the users using these schemas to ensure that they define proper
guidelines to customise these schemas for implementation to enable
interoperability.
On 14 March 2010 19:06, Ron Lake <rlake@galdosinc.com> wrote:
Would
one of you be interested in coming to GeoWeb and giving a workshop on Emergency
Response Standards and Technologies? You could cover NIEM, EDXL, CAP etc.
Let
me know what you think.
From: Gary Ham [mailto:gham@grandpaham.com]
Sent: Sat 3/13/2010 10:40 PM
To: David RR Webber (XML)
Cc: Lee Tincher; emergency@lists.oasis-open.org;
Dwarkanath,Sukumar - INTL; McGarry,Donald P.
Subject: Re: [emergency] HAVE Conformance vs. Documentation vs. Released
Schemas
On Mar 14, 2010, at 12:04 AM, "David RR Webber \(XML\)" <david@drrw.info> wrote:
This is precisely what having a CAM template is doing for
you. Every item in your bullet points.
But instead of someone having to guess at which optional items you
are omitting - or restrictions you have added to extensible items - or code
values - these are all documented in the template in a formal manner that is
machine parsible - and allows you to generate the human readable documentation
as well.
BTW - this is all intensely deja vu - I went back and looked at
how EDI defined interoperability - and it is precisely in terms of
Implementation Conventions - "IC's" - aka profiles and
templates. The more things change, the more they stay the same.
-------- Original Message
--------
Subject: RE: [emergency] HAVE Conformance vs. Documentation vs.
Released Schemas
From: "Lee Tincher" <ltincher@evotecinc.com>
Date: Sat, March 13, 2010 11:25 pm
To: "'McGarry, Donald P.'" <dmcgarry@mitre.org>, "'David RR Webber
(XML)'" <david@drrw.info>
Cc: <emergency@lists.oasis-open.org>,
"'Dwarkanath,Sukumar - INTL'"
<Sukumar_Dwarkanath@sra.com>
These are the guidelines we have been using (this one is for
HAVE – but applies to all of our profile work)
The
following are the attributes of a HAVE Haiti Profile message that are required:
· A HAVE
Haiti Profile message must NOT become a new or additional
“standard” (e.g. another Hospital Availability standard or another
HAVE 1.0 “version”).
· A HAVE
Haiti Profile message must NOT be a Proprietary Format.
· A HAVE
Haiti Profile message must comply with the HAVE 1.0 standard.
o A HAVE
Haiti Profile message must validate against the HAVE 1.0 standard
schema.
o A HAVE
Haiti Profile message must validate within the HAVE 1.0 standard
namespace with no changes to root elements.
o A HAVE
Haiti Profile message must use all required elements (i.e. no
deletion of required elements are allowed).
o A HAVE
Haiti Profile message must not change attributes for required
fields.
The
following are recommendations for clarity:
· A HAVE
Haiti Profile message may further constrain the HAVE 1.0
standard.*
(* may be thought of as a “constraint schema” against the
standard)
· A HAVE
Haiti Profile message may add to required element definitions.*
(* only to extend or interpret the definition)
· A HAVE
Haiti Profile message may limit size of required elements.
· A HAVE
Haiti Profile message may exclude optional elements.
· A HAVE
Haiti Profile message may use optional elements in a specific way
– as defined for the profile.
The aim of education should be to teach us rather how to think,
than what to think - rather to improve our minds, so as to enable us to think
for ourselves, than to load the memory with thoughts of other men. ~Bill
Beattie
I’m sorry...Standards are to guarantee
interoperability. That’s why they are called standards.
HTML, HTTP, XML, TCP, UDP, IP, 802.11, XHTML, Unicode,
CSS, SOAP, WSDL, XSLT, XML Schema, Ethernet, DNS, Arp, RIP, ICMP, Telnet, FTP,
SMTP to name a few.
What if cisco made their own profile for RIP?
What if Sun made their own profile for TCP/IP in unix?
EDXL-HAVE and RM need to work without a developer pow-wow
beforehand. It’s not CIQ’s fault, we just copy-pasted their
schema. If we’re all gonna go off and make our own profiles…why
have the standard? I think when you combine the context above standards
list into what the “internet” is today you see why…The
TC’s official answer to documentation issues and referenced schemas
shouldn’t be to tell developers to go off and make their own
profiles…I think we are just shooting ourselves in the foot.
NIEM is not a standard…it’s a standard process model
for developing data interchanges based on standard terminology; similar to what
goes on in a TC, or in Engineering shops across the world every day, it’s
a great process and model for developing defined data interchanges based on a
common dataset and allowing for cross organization reuse.
I hear you but I don't believe that a standard can guarantee
interoperability - and especially not through the use of XSD schema
alone. May be if there is only one XML instance that everyone
has to adher to - but that is not what people expect.
Notice OASIS standards in general - provide the schema framework
for the exchange content - implementers expect to have to test conformance (see
Drummond Group work on OASIS conformance testing) and declare
interoperability - and someone can still send you something that passes
the schema but breaks your backend application.
And to Gary's point - yes - optional is not the schema default -
but most standards use optional since the context is unknown and rather than
have a situation where a required element is being fudged - its made
optional. CIQ is a point in case - which part of an address is
required? That is impossible to determine for all 207 postal authorities
and then in country mail handling. E.g. USA has 5 possible address
formats that the USPS will accept.
Mentioning context - that is another weakness in XSD Schema design
- no explicit context mechanism - that allows you to control when something is
mandatory or optional. You will be shocked to know that OASIS CAM has
explicit context mechanisms - so you can dynamically control that.
Don - at this point in the process here - the schema is what it
is. My suggest is to augment that with additional profile tools that can provide
the types of interoperability measures you are looking for.
BTW - OASIS CIQ now have the v3 format which is a significant
improvement on matching addressing needs and removing the ugly from CIQ
v2.
-------- Original Message --------
Subject: RE: [emergency] HAVE Conformance vs. Documentation vs.
Released Schemas
From: "McGarry, Donald P." <dmcgarry@mitre.org>
Date: Tue, March 09, 2010 12:34 pm
To: "David RR Webber (XML)" <david@drrw.info>,
"Dwarkanath,Sukumar - INTL" <Sukumar_Dwarkanath@sra.com>
Cc: "emergency@lists.oasis-open.org"
<emergency@lists.oasis-open.org>
By this assessment what distinguishes a standard from a common
data dictionary? I envision a standard as defining interoperability in
that if two systems have never “met” before being able to expect
exactly what the other system is generating so that they can generate messages
for data sharing and process the other messages system. By this
assessment, if I go by the schemas I have to implement the entire xPIL
standard, if I go by the HAVE document, I’m not exactly sure what out of
xPIL I have to implement, which means that I could conceivably represent my
Hospital’s location information by its stock ticker symbol. I
don’t think this is what we intended to do as a TC. If we all go
off making our own tailored profiles, then when our two systems meet we will
discover they can’t interoperate because the “MITRE” profile
only works with stock symbols, while the “Other” profile only works
with Membership information. This doesn’t seem like what a standard
is supposed to do.
You are encountering the limitation of schema itself.
Everything has to be defined as optional.
If you are following the NIEM IEPD approach - you would publish
your IEPD and subset schemas as your profile.
The CAM toolkit provides full support for this.
Ingest the HAVE XSD into CAM template - tailor that as you desire
- use excludeTree() rules to prune out pieces you don't need (to match
EDXL conformance) - and then add other rules as desired to show
dependencies on other parts that you do, and or your content
restrictions. Then run File / Export / Compress process - to complete
your template. You can then generate the subset schema, via File / Export
/ Template to XSD - to build either a flattened schema, or a NIEM compliant
subset schema (depending on what type of application development tooling you
are using).
You can also build the business documentation, XML examples,
cross-reference to NIEM spreadsheet and NIEM wantlist - all as required for
NIEM IEPD publishing.
This gives you a true complete profile of your use of EDXL HAVE,
derived from the original OASIS schema.
Interoperability is then dependent on the conformance to that
profile.
There is also the CAMV engine - which you can use in lieu of
schema checks for production runtime. This has added benefit of providing
graduated failure levels - error, warning, info - rather than the XSD which only
has error. This allows you to tailor the runtime actions of your backend
systems to respond to differences in XML instances. An upcoming
Developerworks article will be covering this with an example use case.
-------- Original Message --------
Subject: RE: [emergency] HAVE Conformance vs. Documentation vs.
Released Schemas
From: "Dwarkanath, Sukumar - INTL" <Sukumar_Dwarkanath@sra.com>
Date: Tue, March 09, 2010 10:08 am
To: "McGarry, Donald P." <dmcgarry@mitre.org>,
<emergency@lists.oasis-open.org>
The restrictions on using CIQ were considered to be business rules
and the intention was not create a profile as far as I remember. I am not
against creating a CIQ Profile but if we go down that path, we should consider
requirements across the other standards such as EDXL RM, DE etc. We have dealt
with this particular issue quite a few times and it is a balance –
offering flexibility vs ensuring interoperability.
After spending some time doing some coding this weekend I noticed
something that we may want to address:
- HAVE uses xPil which in turn uses xAL and xNL
- We included the full schemas for all of these
referenced schemas on the OASIS page to download the standards.
I think the problem here is that when I went to implement this the
documentation states that we are using a “profile” recommendation
to limit the choices for xPil to “maximize interoperability”.
It then goes on to state that <have:Organization> should have the
sub-elements OrganizationInformation and OrginizationGeoLocation.
OrganizationInformation should have the sub-elements as defined in
the CIQ standard:
- OrganisationName
- OrganisationInfo
- Addresses
- ContactNumbers
- CommentText
It also states that we won’t use georss but will use the gml
in the OrganizationGeoLocation Section.
It also refers me to Appendix C which suggests that I refer to the
CIQ TC website, and also states that for the purpose of HAVE the naming &
location elements are used. The use of other elements is left to implementation
choices.
Conformance is defined in the document as:
- Validating to the schema
- Meets the mandatory requirements of section 3
My concern is that the referenced xPil schemas (and in turn the
xAL and xNL) are the FULL SCHEMAS. There is no restriction in the
HAVE schema enforcing our smaller profile of CIQ. Additionally the
reference to the georss namespace or elements was not removed.
Furthermore, the document is somewhat confusing in that it states what elements
to use, but then tells the develop that it’s an implementation choice whether
to use the other elements or not. Right now as it stands I can generate
an XML document that has a bunch of xPIL fields that we didn’t include in
our documentation, but will validate against our schemas. With the
vagueness in the document I could argue that this was an implementation choice
and my document is valid according to the conformance section, but I suspect my
document may break some systems.
So which is it? If I am building an XML processor to ingest
HAVE documents I need to know what to expect. If I need to be prepared to
handle Accounts, Documents, Revenues, Stocks, etc. as defined in xPIL because
some system out there decided that they wanted to do it, that makes HAVE more
heavyweight that I think the designers intended. If indeed we are using a
CIQ “profile” we should develop the schema for that profile and
post it with the standard and add some more info to our documentation so it
isn’t as vague. I’ll upload my generated sample file as
HAVE_FullToSchemaButNotDocument.xml to the TC page so you can check it
out. This example validated against the schemas from our page. I
added in Geo-RSS as well (which will validate if you reference the georss
schema)…
---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that generates
this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php