[Gambas-user] Runaway memory consumption not released until program exit

Sat May 8 01:38:18 CEST 2010

On Friday 07 May 2010 23:11:02 Benoît Minisini wrote:
> > On Friday 07 May 2010 20:31:25 Fabien Bodard wrote:
> > > frame.Load(SourceImage).Stretch(NewWidth,
> > > NewHeight).Save(DestinationImage)
> > >
> > > 2010/5/7 Richard <richard.j.walker at ...247...>:
> > > > On Friday 07 May 2010 19:08:49 Fabien Bodard wrote:
> > > >> frame = frame.Stretch(NewWidth, NewHeight)
> > > >>
> > > >> if you do not re assign frame stretching returned image ... the
> > > >> image will not be changed
> > > >>
> > > >> http://gambasdoc.org/help/comp/gb.image.imlib/image/stretch?v3
> > > >>
> > > >> 2010/5/7 Richard <richard.j.walker at ...247...>:
> > > >> > Hi everybody,
> > > >> >
> > > >> > I am using Gambas 2.20.0 on Mandriva 2010.0. The desktop is LXDE
> > > >> > and the gui toolkit might possibly be GTK.
> > > >> >
> > > >> > I have written a button event handler to process hundreds or
> > > >> > thousands of .png images in a directory. The main memory available
> > > >> > when the program starts is around 1.8 GBytes. After processing
> > > >> > just 744 images (in testing) this falls to about 74 MBytes. The
> > > >> > memory is not freed on exit from the event handler, only when I
> > > >> > exit from the program.
> > > >> >
> > > >> > I use only 1 Image object; a variable called "frame". I have tried
> > > >> > declaring this as a local in the event handler and as a
> > > >> > class-visible variable with no apparent change in the way memory
> > > >> > is eaten and never disgorged.
> > > >> >
> > > >> > I do three things with my frame variable in a loop which iterates
> > > >> > for all frames in a directory;
> > > >> >
> > > >> > 1. I load it from an image on disc
> > > >> > 2. I "stretch" it to make its width smaller(!)
> > > >> > 3. I save the altered frame to a different directory
> > > >> >
> > > >> > Specifically the significant lines of code look like;
> > > >> >
> > > >> > frame = Image.Load(SourceImage)
> > > >> > frame.Stretch(NewWidth, NewHeight)
> > > >> > frame.Save(DestinationImage)
> > > >> >
> > > >> > Outside the loop I now do frame = NULL, but this has no effect.
> > > >> >
> > > >> > Is it possible that my code is allocating multiple "anonymous"
> > > >> > frame objects and not releasing them? I have read Benoit's
> > > >> > explanation in this list of Gambas memory management (Re:
> > > >> > [Gambas-user] Memory Management, on 08/09/2007 09:19).
> > > >> >
> > > >> > I turned to Gambas to program this function because my first
> > > >> > choice, imagemagick, would always run out of memory without
> > > >> > processing a single frame. The Gambas solution works, but no more
> > > >> > work can be done without killing the program and restarting it -
> > > >> > not very convenient. I anticipate that typical uses of the program
> > > >> > will have to handle 10-30 thousand png images at a time.
> > > >> >
> > > >> > Can anyone suggest how I find out what is so hungry for my memory,
> > > >> > and hopefully how to suppress its appetite?
> > > >> >
> > > >> > Richard
> > > >> >
> > > >> > ------------------------------------------------------------------
> > > >> >-- --
> > > >> >
> > > >> >--- -----
> > > >> >
> > > >> > _______________________________________________
> > > >> > Gambas-user mailing list
> > > >> > Gambas-user at lists.sourceforge.net
> > > >> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > >>
> > > >> --------------------------------------------------------------------
> > > >>-- --
> > > >>
> > > >>--- ---
> > > >>
> > > >> _______________________________________________
> > > >> Gambas-user mailing list
> > > >> Gambas-user at lists.sourceforge.net
> > > >> https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > >
> > > > Thanks for reminding me, Fabien, about a fascinating side issue.
> > > > Originally I had tried frame.Resize(NewWidth, NewHeight) but I was
> > > > disappointed to discover it doesn't resize the image at all; it just
> > > > re-frames it by cropping the right hand side.
> > > >
> > > > I changed to Stretch by just typing over Resize and was quite a bit
> > > > surprised when that didn't seem to work either! That was when I
> > > > realised that while Resize(W,H) is a procedure, Stretch(W,H) is, as
> > > > you said, a function! I was a bit hasty in providing my example code
> > > > lines so I repeated the mistake which I had corrected in my event
> > > > handler and which you have corrected above.
> > > >
> > > > The code which I copied for my example does not assign the result of
> > > > the Stretch function, but immediately writes it back to disc. It
> > > > reads:
> > > >
> > > > frame.Stretch(NewWidth,NewHeight).Save(DestinationImage)
> > > >
> > > > It works. This is what I do now, although I had originally used the
> > > > version you suggested and assigned the "stretched" image to my frame
> > > > variable before saving. I have been trying to nail down the greedy
> > > > memory culprit and the frame.Stretch().Save() version made no
> > > > difference, but I was too lazy to change it back to
> > > > frame=frame.Stretch() followed by frame.Save().
> > > >
> > > > I have since discovered that it is the frame=Image.Load() step which
> > > > is to blame. Each png is at most 1MiB, though most are much less. 740
> > > > of these when loaded in turn by  frame=Image.Load() uses 1.8GiB of
> > > > main memory and nearly 2GiB of swap. This seems excessive to me,
> > > > especially since I don't seem able to get it back without quitting
> > > > from the program. Any ideas about that?
> > >
> > > frame=null ?
> > >
> > > > Richard
> > > >
> > > > ---------------------------------------------------------------------
> > > >-- --
> > > >
> > > >-----
> > > >
> > > > _______________________________________________
> > > > Gambas-user mailing list
> > > > Gambas-user at lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > >
> > > -----------------------------------------------------------------------
> > >-- --
> > >
> > >---
> > >
> > > _______________________________________________
> > > Gambas-user mailing list
> > > Gambas-user at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >
> > Yes. Right after the FOR loop which iterates through all the frames for
> > loading and processing.
> >
> > That is one of the reasons I thought I would ask for help on this. As far
> > as I can tell I have complied with all of Benoit's hints and tips on
> > managing memory. The frame variable is declared local to the event
> > handler: DIM frame AS NEW Image
> > The event handler is the only code which uses this variable. There is a
> > WAIT statement in the FOR loop so that the ProgressBar works, but the
> > memory is eaten up at the same ferocious rate either with or without the
> > WAIT.
> >
> > What I can tell for certain is that the memory is only allocated while my
> > code is executing the FOR loop. The frame variable is created and
> > destroyed outside this loop which strongly suggests that I have not
> > programmed the code which is using the memory. By deleting all code
> > within the loop EXCEPT the frame = Image.Load() line I can run the code
> > and watch as it runs through my memory at an estimated rate of something
> > like 4MiB per sub-1MiB frame. This just doesn't look right. If it is then
> > I cannot use Gambas to apply simple changes to very large numbers of PNG
> > images. That can't be right either.
> >
> > For completeness I should also mention that I have run the FOR loop with
> > both Image handling lines removed (load/stretch and save) and there is no
> > sign of memory loss; just blindness 'cos there are no pictures:-)
> >
> > Richard
>
> Can you send your project, or better isolate the image processing that
> burns your memory in a little project?
>
> Regards,

Glad to; I have been running a simple test project which reveals what I now 
believe to be a bug of some long standing. I have only been able to test 2.4, 
2,16 and 2.20 so far. 2.4 works without consuming memory, 2.16.and 2.20 have 
the bug.

I put together a simple form with a progress bar and two buttons. I observe 
memory and swap consumption while the progress bar moves towards 100% using 
gkrellm - 'cos it's always there.

The test program merely reloads a single image file many times so it is 
quicker to run than the "real thing" where there may be hundreds  of 
different files. Nevertheless its behaviour is just the same, suggesting to 
me that there may be a problem in the Image.Load() method. The same memory 
consumption problem occurs if the object is of type Picture and 
Picture.Load() is used.

Hope it's easy to find because I would really like to get this working for my 
brother.

Richard

' Gambas class file

PUBLIC SUB _new()

END

PUBLIC SUB Form_Open()

END

PUBLIC SUB Quit_Click()

  ME.Close

END

PUBLIC SUB Test_Click()
DIM n AS Integer
DIM Frame AS NEW Image

    ProgressBar1.Visible = TRUE
    FOR n = 0 TO 799
        Frame = Image.Load("/home/richard/1024x576.png")
        ProgressBar1.Value = n / 499
        WAIT 
    NEXT 
    ProgressBar1.Visible = FALSE

END