[Gambas-user] Runaway memory consumption not released until program exit

[Benoît Minisini]

> On Friday 07 May 2010 20:31:25 Fabien Bodard wrote:
> > frame.Load(SourceImage).Stretch(NewWidth,
> > NewHeight).Save(DestinationImage)
> > 
> > 2010/5/7 Richard <richard.j.walker at ...247...>:
> > > On Friday 07 May 2010 19:08:49 Fabien Bodard wrote:
> > >> frame = frame.Stretch(NewWidth, NewHeight)
> > >> 
> > >> if you do not re assign frame stretching returned image ... the image
> > >> will not be changed
> > >> 
> > >> http://gambasdoc.org/help/comp/gb.image.imlib/image/stretch?v3
> > >> 
> > >> 2010/5/7 Richard <richard.j.walker at ...247...>:
> > >> > Hi everybody,
> > >> > 
> > >> > I am using Gambas 2.20.0 on Mandriva 2010.0. The desktop is LXDE and
> > >> > the gui toolkit might possibly be GTK.
> > >> > 
> > >> > I have written a button event handler to process hundreds or
> > >> > thousands of .png images in a directory. The main memory available
> > >> > when the program starts is around 1.8 GBytes. After processing just
> > >> > 744 images (in testing) this falls to about 74 MBytes. The memory
> > >> > is not freed on exit from the event handler, only when I exit from
> > >> > the program.
> > >> > 
> > >> > I use only 1 Image object; a variable called "frame". I have tried
> > >> > declaring this as a local in the event handler and as a
> > >> > class-visible variable with no apparent change in the way memory is
> > >> > eaten and never disgorged.
> > >> > 
> > >> > I do three things with my frame variable in a loop which iterates
> > >> > for all frames in a directory;
> > >> > 
> > >> > 1. I load it from an image on disc
> > >> > 2. I "stretch" it to make its width smaller(!)
> > >> > 3. I save the altered frame to a different directory
> > >> > 
> > >> > Specifically the significant lines of code look like;
> > >> > 
> > >> > frame = Image.Load(SourceImage)
> > >> > frame.Stretch(NewWidth, NewHeight)
> > >> > frame.Save(DestinationImage)
> > >> > 
> > >> > Outside the loop I now do frame = NULL, but this has no effect.
> > >> > 
> > >> > Is it possible that my code is allocating multiple "anonymous" frame
> > >> > objects and not releasing them? I have read Benoit's explanation in
> > >> > this list of Gambas memory management (Re: [Gambas-user] Memory
> > >> > Management, on 08/09/2007 09:19).
> > >> > 
> > >> > I turned to Gambas to program this function because my first choice,
> > >> > imagemagick, would always run out of memory without processing a
> > >> > single frame. The Gambas solution works, but no more work can be
> > >> > done without killing the program and restarting it - not very
> > >> > convenient. I anticipate that typical uses of the program will have
> > >> > to handle 10-30 thousand png images at a time.
> > >> > 
> > >> > Can anyone suggest how I find out what is so hungry for my memory,
> > >> > and hopefully how to suppress its appetite?
> > >> > 
> > >> > Richard
> > >> > 
> > >> > --------------------------------------------------------------------
> > >> > --
> > >> >
> > >> >--- -----
> > >> >
> > >> > _______________________________________________
> > >> > Gambas-user mailing list
> > >> > Gambas-user at lists.sourceforge.net
> > >> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > >> 
> > >> ----------------------------------------------------------------------
> > >> --
> > >>
> > >>--- ---
> > >>
> > >> _______________________________________________
> > >> Gambas-user mailing list
> > >> Gambas-user at lists.sourceforge.net
> > >> https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > 
> > > Thanks for reminding me, Fabien, about a fascinating side issue.
> > > Originally I had tried frame.Resize(NewWidth, NewHeight) but I was
> > > disappointed to discover it doesn't resize the image at all; it just
> > > re-frames it by cropping the right hand side.
> > > 
> > > I changed to Stretch by just typing over Resize and was quite a bit
> > > surprised when that didn't seem to work either! That was when I
> > > realised that while Resize(W,H) is a procedure, Stretch(W,H) is, as
> > > you said, a function! I was a bit hasty in providing my example code
> > > lines so I repeated the mistake which I had corrected in my event
> > > handler and which you have corrected above.
> > > 
> > > The code which I copied for my example does not assign the result of
> > > the Stretch function, but immediately writes it back to disc. It
> > > reads:
> > > 
> > > frame.Stretch(NewWidth,NewHeight).Save(DestinationImage)
> > > 
> > > It works. This is what I do now, although I had originally used the
> > > version you suggested and assigned the "stretched" image to my frame
> > > variable before saving. I have been trying to nail down the greedy
> > > memory culprit and the frame.Stretch().Save() version made no
> > > difference, but I was too lazy to change it back to
> > > frame=frame.Stretch() followed by frame.Save().
> > > 
> > > I have since discovered that it is the frame=Image.Load() step which is
> > > to blame. Each png is at most 1MiB, though most are much less. 740 of
> > > these when loaded in turn by  frame=Image.Load() uses 1.8GiB of main
> > > memory and nearly 2GiB of swap. This seems excessive to me, especially
> > > since I don't seem able to get it back without quitting from the
> > > program. Any ideas about that?
> > 
> > frame=null ?
> > 
> > > Richard
> > > 
> > > -----------------------------------------------------------------------
> > > --
> > >
> > >-----
> > >
> > > _______________________________________________
> > > Gambas-user mailing list
> > > Gambas-user at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > 
> > -------------------------------------------------------------------------
> > --
> >
> >---
> >
> > _______________________________________________
> > Gambas-user mailing list
> > Gambas-user at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> 
> Yes. Right after the FOR loop which iterates through all the frames for
> loading and processing.
> 
> That is one of the reasons I thought I would ask for help on this. As far
> as I can tell I have complied with all of Benoit's hints and tips on
> managing memory. The frame variable is declared local to the event
> handler: DIM frame AS NEW Image
> The event handler is the only code which uses this variable. There is a
> WAIT statement in the FOR loop so that the ProgressBar works, but the
> memory is eaten up at the same ferocious rate either with or without the
> WAIT.
> 
> What I can tell for certain is that the memory is only allocated while my
> code is executing the FOR loop. The frame variable is created and
> destroyed outside this loop which strongly suggests that I have not
> programmed the code which is using the memory. By deleting all code within
> the loop EXCEPT the frame = Image.Load() line I can run the code and watch
> as it runs through my memory at an estimated rate of something like 4MiB
> per sub-1MiB frame. This just doesn't look right. If it is then I cannot
> use Gambas to apply simple changes to very large numbers of PNG images.
> That can't be right either.
> 
> For completeness I should also mention that I have run the FOR loop with
> both Image handling lines removed (load/stretch and save) and there is no
> sign of memory loss; just blindness 'cos there are no pictures:-)
> 
> Richard
> 

Can you send your project, or better isolate the image processing that burns 
your memory in a little project? 

Regards,

-- 
Benoît Minisini