[Gambas-user] Does gambas3 have the guts to make it?

Bruce Bruen bbruen at ...2308...
Mon Oct 17 09:23:23 CEST 2011


Well you may have noticed a little drop off in my bug reports lately.  I
have just finished a little ETL exercise of a reasonable sized database
(7.5million rows over 28 tables, heavily "foreign-keyed", postgresql 8.3
backends on both sides).  It took me a few days longer than I thought it
would because I tried some "major" linux ETL tools first.

Of the three I tried, al were java based, two were free and open source
and one was commercial licence linux (30day non-crippled trial).  One
could not cope with any datatype stronger than an integer and insisted
that all floating point numeric types have a scale of 1245678?????
(Guess which one).  Of the two free ones, one just gave up and pointed
it toes toward the sky on the first occurrence of more than 1000 rows in
the transform set, the other is still going on a test machine (6 days
now?).

 So. I gave up.

And did it myself.

I would have done it in gb2 but I thought I'd just see how gb3 went.
The job took 6 projects, 4 to build nice clean base tables (including
one of 1.1M rows) and two to clean, transform, recode and load the
dependent tables.  If we "excuse" the mistakes that I made in the DDL
for the new database, which meant two re-runs, the whole development
time was 14 hours and the elapsed time for the final run was 3hours and
12minutes.

Nett result, 7.43million rows in new database in 16 tables.  (How can 5
users create 70,000 junk records in a year? )

I have now run the 60 odd test queries to prove the data is correct (26
hours) with zero errors.  

So Benoît, I reckon gb3 definitely has got what it takes!  :-) :-) :-)

Thanks a million, or in fact thanks 7.4million!
regards
Bruce

p.s. No I don't use Ubuntu! :-)



More information about the User mailing list