[Gambas-user] gb3: using string as stream for sequential write operations of arbitrary datatypes

[Kevin Fishburne]

On 08/30/2011 04:35 AM, Bruce Bruen wrote:
> On Tue, 2011-08-30 at 01:52 -0400, Kevin Fishburne wrote:
>> When a transaction isn't received and I need to resend it, I'd like
>> to
>> be able to refer to the transaction history and resend the data
>> (QueueOut[id].Data) without having to recalculate it, especially
>> since
>> the data may have changed during the timeout period. I don't see any
>> way
>> to store a completely arbitrary set of datatypes as a single
>> structure
>> property however. I could write them to a file, then read the file
>> into
>> a string and load it into a property of the array of structures, but
>> that's just crazy.
> Kevin,
>
> I can't give you an answer to the question you ask, mainly because I
> kept seeing bigger problems as I read your post.  From what I glean,
> this is a classic TX->ACK situation and I think you are basing the code
> on some type of timeout rule, such that TX->?->TX.  However, there are
> some problems with that model.  Firstly, where did the failure occur?
> a) firstly, the transaction was never "sent" from end A (why? is
> immaterial. What is, is that end B will not have any possibility of
> processing the transaction or sending the ACK)
> b) second, the transaction was fully received by end B, but for some
> reason it never ACK'ed it
>      b1) it did process the full transaction, but never sent the ACK
>      b2) it did not process the transaction and never sent the ACK
> c) the transaction was fully received, processed and the ACK sent, but
> it was never received by end A.
> Since at the end of the timeout, end A has no knowledge of the state of
> end B simply retransmitting the same transaction is considered "bad
> design" in network theory. (I could lookup the references for you, but
> they are on rotting pieces of paper somewhere in a mouldy box in the
> garage, and...)
> > From memory, there are at least two well-formed models regarding these
> problems:
> i) any retransmission is marked as such.  This puts a need on the server
> to audit any retransmission and either process it or ignore it.
> ii) any failed transmission is assumed to have been processed by end B
> and a reversal transaction is sent by end A before retrying the
> transaction.  This means that end B must implement a "reversal"
> procedure, but puts the onus on end A to ensure that the transaction
> ONLY HAPPENS ONCE (not yelling, just emphasising).
>
> Disregarding your need to store the sent transaction for a moment, it
> might be wise of you to consider how these.

All those are valid observations about sending and receiving with acks. 
My implementation works even with repeated and unwanted resends of data 
from multiple sources (DoS). It keeps the place of each transaction for 
each player, authenticating each through either an IP or 
username/password lookup, and not incrementing the associated 
transaction queue pointer unless processed properly. Incoming 
transactions are placed in the queue position that their ID dictates, 
their ID provided by the incoming transaction data. This way 
out-of-order packets are stored in the incoming transaction queue 
without affecting the execution order of the queue. The trick is to 
place ack sends, ack receives, and transaction retransmissions outside 
the scope (don't store them) of the incoming and outgoing queues. Just 
make them happen, and as long as the .Acknowledged property is 
maintained everything will flow smoothly.

Network programming like this is a hell of a thing, though. God help us 
all...

-- 
Kevin Fishburne
Eight Virtues
www: http://sales.eightvirtues.com
e-mail: sales at ...1887...
phone: (770) 853-6271