[Gambas-user] HttpClient question, now massively parallel!

[Caveat]

Hi Ron_2nd

You're welcome!  I made a couple of improvements while I thought of
them.  It's probably better not to add the leading zero to the name of
the HttpClient, so just do something simple like NEW HttpClient AS
"tube" & freeSlot, then your event handling routines can be simply
tube0_Finished(), tube1_Finished()... tube200_Finished() etc.  You'll
probably need to take account of other events too like Connect, Error,
and Read (I presume in Read you'll actually be getting your data, so
you'll need to handle a collection of buffers or something...).  You'll
most likely also need to allow for timeouts.

I'd also suggest to make the numbered event handlers simple one-liners
that call back to a real handler... so for the Finished event:

' This is the real handler for Finished, it takes a param of the index
' of the httpClient that raised the Finished event
PUBLIC SUB tube_Finished(clientIndex as Integer)
  ' indicate that the slot has come free
  slots[clientIndex] = FALSE
  PRINT "Slot " & clientIndex & " finished"
  PRINT "Processed GET: " & getters[clientIndex].URL
  ... do some real work!
END 

' Each of the Finished event handlers is now super-simple and
' should never need to change, as the real event handler (above) does
' the work
PUBLIC SUB tube0_Finished()
  tube_Finished(0)
END

PUBLIC SUB tube1_Finished()
  tube_Finished(1)
END

...
PUBLIC SUB tube200_Finished()
  tube_Finished(200)
END

I'm wondering if there isn't a way to programmatically generate the
event handlers, and have them know which event fired... freeing you from
the pain of copy-pasting a bunch of one-line SUBs.

You may want to get real fancy and start queuing requests once all your
slots are used up, or maybe just deny requests (note I return -1 if
there's no free slot) until something comes free again... 

Have fun, let me know if anything is not clear.

Regards,
Caveat

P.S.  Should I have added a disclaimer that my code is not certified for
use in real-time radiation monitoring systems upon which the fate of the
world may depend?  ;-)


On Wed, 2011-03-30 at 13:54 +0200, Ron wrote:
> Hi Caveat,
> 
> I wanted to create a module for my software so I can fetches sensor data 
> from public pachubes sensors in japen, to get nuclear radiation values.
> 
> I will adapt your code, thanks!
> 
> Regards,
> Ron_2nd.
> > Hi Ron_2nd,
> >
> > Benoit is (need I say it!) 100% correct.
> >
> > But I figured out a kind of simplistic way to allow you to do your gets
> > in a MASSIVELY PARALLEL fashion...  :-D
> >
> > http://pastebin.com/LiY3g4Lt
> >
> >
> > Oh and Rolf, I notice PasteBin has a specific Gambas syntax highlighter
> > already :-D
> >
> > Regards,
> > Caveat
> >
> >
> > On Wed, 2011-03-30 at 01:54 +0200, Benoît Minisini wrote:
> >>> It seems that if you call HttpClient twice after each other with
> >>> different urls in async mode, like in example below the _finished event
> >>> is only called once (with the last call/url)
> >>>
> >>> So they are not really async/background, or is this a bug?
> >>> Or are they overwritten, due to false usage/code?
> >>>
> >>> This happens in Gambas2 and 3...
> >>>
> >>> ' Gambas module
> >>>
> >>> Public hPachubeFetch As HttpClient
> >>>
> >>> Public Sub Main()
> >>>
> >>>     Dim rFeeds As String[] = ["21017", "3711"]
> >>>     Dim sUrl, sFeed As String
> >>>
> >>>     For Each sFeed In rFeeds
> >>>       Debug sfeed
> >>>       hPachubeFetch = New HttpClient As "hPachubeFetch"
> >>>       hPachubeFetch.URL = sUrl
> >>>       hPachubeFetch.TimeOut = 8
> >>>       hPachubeFetch.Tag = sFeed&  "|0|12" ' feed | datastream | device id
> >>>       hPachubeFetch.Auth = 1
> >>>       hPachubeFetch.User = "user"
> >>>       hPachubeFetch.Password = "password"
> >>>       hPachubeFetch.Async = True
> >>>       hPachubeFetch.Get()
> >>>    Next
> >>>
> >>> End
> >>>
> >>> Public Sub hPachubeFetch_Finished()
> >>>
> >>>      'do stuff
> >>>
> >>> End
> >>>
> >>> Regards,
> >>> Ron_2nd.
> >>>
> >> I think you have to keep reference on the HttpClient object until the Get() is
> >> finished. Otherwise the Finished event will be lost.
> >>
> >> Regards,
> >>
> >>
> >
> >
> > ------------------------------------------------------------------------------
> > Enable your software for Intel(R) Active Management Technology to meet the
> > growing manageability and security demands of your customers. Businesses
> > are taking advantage of Intel(R) vPro (TM) technology - will your software
> > be a part of the solution? Download the Intel(R) Manageability Checker
> > today! http://p.sf.net/sfu/intel-dev2devmar
> > _______________________________________________
> > Gambas-user mailing list
> > Gambas-user at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
>