OASIS ebXML Messaging Services TC

 View Only

Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId

  • 1.  Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId

    Posted 12-05-2001 13:21
    That's fine then.
    
    Martin W Sachs wrote:
    
    > Chris,
    > 
    > I agree with everything you say. You can't possibly use persistDuration to
    > control how long the conversationId has to be persisted.  No argument at
    > all.
    > 
    > I am simply less trusting than you are of "reasonable implementers" so I
    > suggested underlining that persistDuration is not applicable to message
    > ordering and conversationId retention.  That's all. If I really had my way,
    > I would capture a number of the points from your posting in the
    > non-normative statement that persistDuration is not applicable  to meesage
    > ordering and conversationId retention.
    > 
    > Regards,
    > Marty
    > 
    > *************************************************************************************
    > 
    > Martin W. Sachs
    > IBM T. J. Watson Research Center
    > P. O. B. 704
    > Yorktown Hts, NY 10598
    > 914-784-7287;  IBM tie line 863-7287
    > Notes address:  Martin W Sachs/Watson/IBM
    > Internet address:  mwsachs @ us.ibm.com
    > *************************************************************************************
    > 
    > 
    > 
    > Christopher Ferris <chris.ferris@sun.com> on 12/05/2001 10:17:12 AM
    > 
    > To:    ebxml-msg@lists.oasis-open.org
    > cc:
    > Subject:    Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId
    > 
    > 
    > 
    > Marty,
    > 
    > I respectfully disagree. I don't think that we need to say
    > anything.
    > 
    > persistDuration is a parameter that the parties have agreed
    > to use as the minimum amount of time that the party shall
    > commit to preserving message artifacts necessary for
    > support of the RM function (in many cases, messageId is
    > sufficient so that possible duplicate transmissions
    > of A MESSAGE can be detected).
    > 
    > A conversation will potentially have a SIGNIFICANTLY
    > longer duration than the lifetime of a single message
    > for purposes of reliable delivery. It could be weeks or
    > months. IMO it would be foolish to reuse the persistDuration
    > for purposes of determining how long to preserve the
    > conversational state of an exchange between parties
    > which would include SequenceNumber when used in conjunction
    > with message ordering.
    > 
    > Conversational state MUST be preserved for the life of the
    > conversation, period, full stop. One could include the list
    > of messageIds of all messages for a conversation in the
    > conversational state, but that would be potentially inefficient
    > w/r/t persistent store, especially given that after persistDuration
    > has expired the sending MSH should not (must not?) attempt
    > to redeliver a message to the receiving party that has been
    > unacknowledged.
    > 
    > The point is that the purpose of the persistDuration parameter
    > is to provide a reasonable limit on the amount of time that
    > a receiving party needs to preserve messageId in persistent
    > store for purposes of filtering out potential duplicate
    > transmissions of a single message. It has NOTHING to do with
    > the conversational state of an exchaneg between parties.
    > 
    > Cheers,
    > 
    > Chris
    > 
    > Martin W Sachs wrote:
    > 
    > 
    >>You need to say enough in the MSG spec to inform an implementer that
    >>persistDuration follows different rules for conversationId and
    >>last-ordered-message than for reliable messaging.  Considering the amount
    >>of discussion we have had on this point, we cannot assume that a
    >>"reasonable implementer" will know what to do. There are plenty of
    >>
    > examples
    > 
    >>in the newspapers about deadly mistakes made by reasonable implementers.
    >>The Risks Forum mut be full of them.
    >>
    >>Regards,
    >>Marty
    >>
    >>
    >>
    > *************************************************************************************
    > 
    > 
    >>Martin W. Sachs
    >>IBM T. J. Watson Research Center
    >>P. O. B. 704
    >>Yorktown Hts, NY 10598
    >>914-784-7287;  IBM tie line 863-7287
    >>Notes address:  Martin W Sachs/Watson/IBM
    >>Internet address:  mwsachs @ us.ibm.com
    >>
    >>
    > *************************************************************************************
    > 
    > 
    >>
    >>
    >>Christopher Ferris <chris.ferris@sun.com> on 12/04/2001 09:39:42 PM
    >>
    >>To:    David Fischer <david@drummondgroup.com>
    >>cc:    Martin W Sachs/Watson/IBM@IBMUS, Dan Weinreb
    >>
    > <dlw@exceloncorp.com>,
    > 
    >>       ebxml-msg@lists.oasis-open.org
    >>Subject:    Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId
    >>
    >>
    >>
    >>The thing I have trouble with here is why we have to say anything
    >>in the spec. This is too suggestive of implementation detail
    >>IMO (although probably accurate). Why do we need to say anything
    >>about this at all?
    >>
    >>Cheers,
    >>
    >>Chris
    >>
    >>David Fischer wrote:
    >>
    >>
    >>
    >>>OK, I think that will work, but it is not the whole message which needs
    >>>
    >>>
    >>to be
    >>
    >>
    >>>saved.  Once the message is delivered to the application, just the
    >>>
    >>>
    >>MessageId,
    >>
    >>
    >>>CPAId, persistDuration, ConversationId, SequenceNumber (did I get
    >>>
    >>>
    >>everything?)
    >>
    >>
    >>>need to be saved.  The spec says now only the MessageId needs to be saved
    >>>
    >>>
    >>but
    >>
    >>
    >>>that's not enough for MessageOrder.
    >>>
    >>>David.
    >>>
    >>>-----Original Message-----
    >>>From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
    >>>Sent: Tuesday, December 04, 2001 5:01 PM
    >>>To: Christopher Ferris
    >>>Cc: David Fischer; Dan Weinreb; ebxml-msg@lists.oasis-open.org;
    >>>ebxml-cppa@lists.oasis-open.org
    >>>Subject: Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId
    >>>
    >>>
    >>>
    >>>Then it should only be necessary to add to the MSG spec words that say:
    >>>
    >>>  The MSH shall preserve in persistent storage the last message in the
    >>>  conversation that was sent in order to the application until either a
    >>>  subseqent in-order message arrives or until the conversation ends
    >>>  (regardless of how long that takes). These words probably should be
    >>>  modified if preserving some header information is all that is needed.
    >>>
    >>>  The MSH shall preserve in persistent storage the conversationId of an
    >>>  in-progress conversation until that conversation ends (no matter how
    >>>  long that takes).
    >>>
    >>>
    >>>No change is needed to the CPPA spec unless it now does not say that
    >>>persistDuration is the mininum length of time... (see Chris' words
    >>>
    >>>
    >>below).
    >>
    >>
    >>>Regards,
    >>>Marty
    >>>
    >>>
    >>>
    >>>
    > ********************************************************************************
    > 
    > 
    >>
    >>>*****
    >>>
    >>>Martin W. Sachs
    >>>IBM T. J. Watson Research Center
    >>>P. O. B. 704
    >>>Yorktown Hts, NY 10598
    >>>914-784-7287;  IBM tie line 863-7287
    >>>Notes address:  Martin W Sachs/Watson/IBM
    >>>Internet address:  mwsachs @ us.ibm.com
    >>>
    >>>
    >>>
    > ********************************************************************************
    > 
    > 
    >>
    >>>*****
    >>>
    >>>
    >>>
    >>>Christopher Ferris <chris.ferris@sun.com> on 12/04/2001 05:46:11 PM
    >>>
    >>>To:    Martin W Sachs/Watson/IBM@IBMUS
    >>>cc:    David Fischer <david@drummondgroup.com>, Dan Weinreb
    >>>      <dlw@exceloncorp.com>, ebxml-msg@lists.oasis-open.org
    >>>Subject:    Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId
    >>>
    >>>
    >>>
    >>>right now, the spec suggests #1 which is that pd is the
    >>>minimum length of time that a party will commit to
    >>>preserving a message (or its relevant artifacts) in
    >>>persistent store for purposes of filtering duplicate
    >>>messages.
    >>>
    >>>Cheers,
    >>>
    >>>Chris
    >>>
    >>>Martin W Sachs wrote:
    >>>
    >>>
    >>>
    >>>
    >>>>One of these possibilities:
    >>>>
    >>>>1. If the MSG spec states that persistDuration is a minimum length of
    >>>>
    >>>>
    >>>>
    >>>time,
    >>>
    >>>
    >>>
    >>>>then all you need is to add words to the MSG spec conveying additional
    >>>>rules when message ordering is in effect.
    >>>>
    >>>>2. If the MSG spec states that messages that live until persistDuration
    >>>>MUST be discarded right away, then we probably need additional text in
    >>>>
    >>>>
    >>>>
    >>>both
    >>>
    >>>
    >>>
    >>>>the MSG and the CPA spec.
    >>>>
    >>>>Regards,
    >>>>Marty
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    > ********************************************************************************
    > 
    > 
    >>
    >>>*****
    >>>
    >>>
    >>>
    >>>
    >>>>Martin W. Sachs
    >>>>IBM T. J. Watson Research Center
    >>>>P. O. B. 704
    >>>>Yorktown Hts, NY 10598
    >>>>914-784-7287;  IBM tie line 863-7287
    >>>>Notes address:  Martin W Sachs/Watson/IBM
    >>>>Internet address:  mwsachs @ us.ibm.com
    >>>>
    >>>>
    >>>>
    >>>>
    > ********************************************************************************
    > 
    > 
    >>
    >>>*****
    >>>
    >>>
    >>>
    >>>
    >>>>David Fischer <david@drummondgroup.com> on 12/04/2001 02:39:43 PM
    >>>>
    >>>>To:    Martin W Sachs/Watson/IBM@IBMUS
    >>>>cc:    Dan Weinreb <dlw@exceloncorp.com>, ebxml-msg@lists.oasis-open.org
    >>>>Subject:    RE: [ebxml-msg] Re: Comments on the 1.09 about
    >>>>
    > ConversationId
    > 
    >>>>
    >>>>
    >>>>Yes Marty, absolutely.  I think there are potential problems (discussed
    >>>>below)
    >>>>which make MessageOrder quite fragile.  I am shuddering over the thought
    >>>>
    >>>>
    >>>>
    >>>of
    >>>
    >>>
    >>>
    >>>>changing the persistDuration rules depending upon whether or not
    >>>>MessageOrder is
    >>>>present (wouldn't this constitute a CPA override!).
    >>>>
    >>>>I still like your other solution.
    >>>>
    >>>>Regards,
    >>>>
    >>>>David Fischer
    >>>>Drummond Group.
    >>>>
    >>>>-----Original Message-----
    >>>>From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
    >>>>Sent: Tuesday, December 04, 2001 10:50 AM
    >>>>To: David Fischer
    >>>>Cc: Dan Weinreb; ebxml-msg@lists.oasis-open.org
    >>>>Subject: RE: [ebxml-msg] Re: Comments on the 1.09 about ConversationId
    >>>>
    >>>>
    >>>>
    >>>>Some comments below.
    >>>>
    >>>>Regards,
    >>>>Marty
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    > ********************************************************************************
    > 
    > 
    >>
    >>>>*****
    >>>>
    >>>>Martin W. Sachs
    >>>>IBM T. J. Watson Research Center
    >>>>P. O. B. 704
    >>>>Yorktown Hts, NY 10598
    >>>>914-784-7287;  IBM tie line 863-7287
    >>>>Notes address:  Martin W Sachs/Watson/IBM
    >>>>Internet address:  mwsachs @ us.ibm.com
    >>>>
    >>>>
    >>>>
    >>>>
    > ********************************************************************************
    > 
    > 
    >>
    >>>>*****
    >>>>
    >>>>
    >>>>
    >>>>David Fischer <david@drummondgroup.com> on 12/04/2001 11:16:21 AM
    >>>>
    >>>>To:    Dan Weinreb <dlw@exceloncorp.com>
    >>>>cc:    ebxml-msg@lists.oasis-open.org
    >>>>Subject:    RE: [ebxml-msg] Re: Comments on the 1.09 about
    >>>>
    > ConversationId
    > 
    >>>>
    >>>>
    >>>>Dan, did you mean to imply that messages are no longer stored once they
    >>>>
    >>>>
    >>>>
    >>>are
    >>>
    >>>
    >>>
    >>>>acknowledged?  This is not the case.  Messages must be stored longer
    >>>>
    > than
    > 
    >>>>persistDuration.  In your scenario, when 12 got to the Receiving MSH,
    >>>>
    >>>>
    >>>>
    >>>1-10
    >>>
    >>>
    >>>
    >>>>are
    >>>>still in the store.  If you wait so long that 1-10 have passed
    >>>>persistDuration,
    >>>>then I think you have waited so long that it does not matter any more.
    >>>>
    >>>>I see your point about what happens if there is nothing in the store and
    >>>>the
    >>>>Receiving MSH gets a message with a number larger than 1.  However, this
    >>>>implies
    >>>>the last ordered message/MessageId (which has been delivered in order)
    >>>>
    >>>>
    >>>>
    >>>must
    >>>
    >>>
    >>>
    >>>>be
    >>>>stored indefinitely (forever?) which is probably not acceptable.
    >>>>
    >>>>MWS:  For most conversations, there will be an indication of an end, at
    >>>>which time
    >>>>the last ordered message can be discarded.  I guess if storage is tight,
    >>>>you
    >>>>only need to keep the sequence number and conversationId of the last
    >>>>ordered message
    >>>>
    >>>>
    >>>>This also
    >>>>points out the conflict between Message Order and persistDuration.  What
    >>>>
    >>>>
    >>>>
    >>>if
    >>>
    >>>
    >>>
    >>>>messages 1,2,3,4 have persistDuration of 2 days.  Messages 1,2,4 are
    >>>>delivered.
    >>>>The MSH hands 1,2 to the application but holds 4.  For some reason, 3 is
    >>>>not
    >>>>delivered for 3 days.  In this case 3,4 have already passed
    >>>>
    >>>>
    >>>>
    >>>persistDuration
    >>>
    >>>
    >>>
    >>>>so
    >>>>they are not delivered.  The next day, 5 is delivered.  Since 3,4 were
    >>>>
    >>>>
    >>>>
    >>>not
    >>>
    >>>
    >>>
    >>>>delivered, then 5 cannot be delivered (nor any subsequent messages) even
    >>>>though
    >>>>it is still within TTL, and the ordering is forever broken for this
    >>>>ConversationId.  We may have to say Message Ordering does not work if
    >>>>
    > any
    > 
    >>>>message in the order fails to be delivered within its persistDuration
    >>>>
    > (or
    > 
    >>>>maybe
    >>>>TTL)?  Of course, all this can be adjusted at run time by increasing the
    >>>>value
    >>>>of persistDuration in the CPA.  Once again, MessageOrdering is based on
    >>>>persistDuration.
    >>>>
    >>>>MWS:  All of which says that some additional rules are needed on
    >>>>
    >>>>
    >>>>
    >>>discarding
    >>>
    >>>
    >>>
    >>>>messages
    >>>>in the presence of ordering.  The last ordered message cannot be
    >>>>
    >>>>
    >>>>
    >>>discarded
    >>>
    >>>
    >>>
    >>>>until
    >>>>either the next message in the sequence arrives or the application ends
    >>>>
    >>>>
    >>>>
    >>>the
    >>>
    >>>
    >>>
    >>>>conversation,
    >>>>regardless of its persist duration.  Similary, if there is a gap in the
    >>>>sequence numbers,
    >>>>all the out of sequence messages must be preserved, regardless of
    >>>>persistDuration,
    >>>>until they can be delivered.  The MSG or CPPA specification must provide
    >>>>advice on
    >>>>how long to set persistDuration depending on what combination of
    >>>>
    > reliable
    > 
    >>>>messaging
    >>>>and ordering is being used.
    >>>>
    >>>>I am wondering why are we messing with all of this?  Did I see Marty
    >>>>suggest, if
    >>>>messages are order dependant, just wait for acknowledgment from the end
    >>>>before
    >>>>sending the next message?  Sounds good.
    >>>>
    >>>>MWS:  Ahhh... the light begins to dawn.
    >>>>
    >>>>Regards,
    >>>>
    >>>>David.
    >>>>
    >>>>-----Original Message-----
    >>>>From: Dan Weinreb [mailto:dlw@exceloncorp.com]
    >>>>Sent: Monday, December 03, 2001 9:40 PM
    >>>>To: david@drummondgroup.com
    >>>>Cc: rberwanger@btrade.com; ebxml-msg@lists.oasis-open.org
    >>>>Subject: Re: [ebxml-msg] Re: Comments on the 1.09 about ConversationId
    >>>>
    >>>>
    >>>> Date: Mon, 03 Dec 2001 20:30:08 -0600
    >>>> From: David Fischer <david@drummondgroup.com>
    >>>>
    >>>>        If there
    >>>> are no messages in the persistent store pertaining to a conversation,
    >>>> then
    >>>>there
    >>>> is no need to store it any longer for the purposes of maintaining
    >>>> message
    >>>>order.
    >>>>
    >>>>I'm not sure that would work.  Suppose two parties establish a
    >>>>conversation with ID 5551212.  Party A sends messages numbered 1
    >>>>through 10 to Party B, and party B's application reads all the
    >>>>messages and acknowledges them, so that there aren't any messages
    >>>>still waiting to be delivered to B.  Now a message arrives at B with
    >>>>conversation ID 5551212 and a sequence number of 12.  The MSH at B's
    >>>>end has to realize that is should not deliver this message to the
    >>>>application because message 11 hasn't arrived yet.  So the MSH at B's
    >>>>end has to be storing somewhere the fact that the most recent message
    >>>>delivered to the application from conversation 5551212 is 10.
    >>>>
    >>>>-- Dan
    >>>>
    >>>>
    >>>>
    >>>>----------------------------------------------------------------
    >>>>To subscribe or unsubscribe from this elist use the subscription
    >>>>manager: <http://lists.oasis-open.org/ob/adm.pl>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>----------------------------------------------------------------
    >>>>To subscribe or unsubscribe from this elist use the subscription
    >>>>manager: <http://lists.oasis-open.org/ob/adm.pl>
    >>>>
    >>>
    >>>----------------------------------------------------------------
    >>>To subscribe or unsubscribe from this elist use the subscription
    >>>manager: <http://lists.oasis-open.org/ob/adm.pl>
    >>>
    >>>
    >>>
    >>>
    >>>----------------------------------------------------------------
    >>>To subscribe or unsubscribe from this elist use the subscription
    >>>manager: <http://lists.oasis-open.org/ob/adm.pl>
    >>>
    >>>
    >>>----------------------------------------------------------------
    >>>To subscribe or unsubscribe from this elist use the subscription
    >>>manager: <http://lists.oasis-open.org/ob/adm.pl>
    >>>
    >>
    >>
    >>
    >>
    >>
    >>----------------------------------------------------------------
    >>To subscribe or unsubscribe from this elist use the subscription
    >>manager: <http://lists.oasis-open.org/ob/adm.pl>
    >>
    > 
    > 
    > 
    > ----------------------------------------------------------------
    > To subscribe or unsubscribe from this elist use the subscription
    > manager: <http://lists.oasis-open.org/ob/adm.pl>
    > 
    > 
    >