[Gambas-user] Runaway memory consumption not released until program exit

Benoît Minisini gambas at ...1...
Sat May 8 01:50:04 CEST 2010


> On Friday 07 May 2010 23:11:02 Benoît Minisini wrote:
> > > On Friday 07 May 2010 20:31:25 Fabien Bodard wrote:
> > > > frame.Load(SourceImage).Stretch(NewWidth,
> > > > NewHeight).Save(DestinationImage)
> > > > 
> > > > 2010/5/7 Richard <richard.j.walker at ...247...>:
> > > > > On Friday 07 May 2010 19:08:49 Fabien Bodard wrote:
> > > > >> frame = frame.Stretch(NewWidth, NewHeight)
> > > > >> 
> > > > >> if you do not re assign frame stretching returned image ... the
> > > > >> image will not be changed
> > > > >> 
> > > > >> http://gambasdoc.org/help/comp/gb.image.imlib/image/stretch?v3
> > > > >> 
> > > > >> 2010/5/7 Richard <richard.j.walker at ...247...>:
> > > > >> > Hi everybody,
> > > > >> > 
> > > > >> > I am using Gambas 2.20.0 on Mandriva 2010.0. The desktop is LXDE
> > > > >> > and the gui toolkit might possibly be GTK.
> > > > >> > 
> > > > >> > I have written a button event handler to process hundreds or
> > > > >> > thousands of .png images in a directory. The main memory
> > > > >> > available when the program starts is around 1.8 GBytes. After
> > > > >> > processing just 744 images (in testing) this falls to about 74
> > > > >> > MBytes. The memory is not freed on exit from the event handler,
> > > > >> > only when I exit from the program.
> > > > >> > 
> > > > >> > I use only 1 Image object; a variable called "frame". I have
> > > > >> > tried declaring this as a local in the event handler and as a
> > > > >> > class-visible variable with no apparent change in the way
> > > > >> > memory is eaten and never disgorged.
> > > > >> > 
> > > > >> > I do three things with my frame variable in a loop which
> > > > >> > iterates for all frames in a directory;
> > > > >> > 
> > > > >> > 1. I load it from an image on disc
> > > > >> > 2. I "stretch" it to make its width smaller(!)
> > > > >> > 3. I save the altered frame to a different directory
> > > > >> > 
> > > > >> > Specifically the significant lines of code look like;
> > > > >> > 
> > > > >> > frame = Image.Load(SourceImage)
> > > > >> > frame.Stretch(NewWidth, NewHeight)
> > > > >> > frame.Save(DestinationImage)
> > > > >> > 
> > > > >> > Outside the loop I now do frame = NULL, but this has no effect.
> > > > >> > 
> > > > >> > Is it possible that my code is allocating multiple "anonymous"
> > > > >> > frame objects and not releasing them? I have read Benoit's
> > > > >> > explanation in this list of Gambas memory management (Re:
> > > > >> > [Gambas-user] Memory Management, on 08/09/2007 09:19).
> > > > >> > 
> > > > >> > I turned to Gambas to program this function because my first
> > > > >> > choice, imagemagick, would always run out of memory without
> > > > >> > processing a single frame. The Gambas solution works, but no
> > > > >> > more work can be done without killing the program and
> > > > >> > restarting it - not very convenient. I anticipate that typical
> > > > >> > uses of the program will have to handle 10-30 thousand png
> > > > >> > images at a time.
> > > > >> > 
> > > > >> > Can anyone suggest how I find out what is so hungry for my
> > > > >> > memory, and hopefully how to suppress its appetite?
> > > > >> > 
> > > > >> > Richard
> > > > >> > 
> > > > >> > ----------------------------------------------------------------
> > > > >> > --
> > > > >> >
> > > > >> >-- --
> > > > >> >
> > > > >> >--- -----
> > > > >> >
> > > > >> > _______________________________________________
> > > > >> > Gambas-user mailing list
> > > > >> > Gambas-user at lists.sourceforge.net
> > > > >> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > > >> 
> > > > >> ------------------------------------------------------------------
> > > > >> --
> > > > >>
> > > > >>-- --
> > > > >>
> > > > >>--- ---
> > > > >>
> > > > >> _______________________________________________
> > > > >> Gambas-user mailing list
> > > > >> Gambas-user at lists.sourceforge.net
> > > > >> https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > > > 
> > > > > Thanks for reminding me, Fabien, about a fascinating side issue.
> > > > > Originally I had tried frame.Resize(NewWidth, NewHeight) but I was
> > > > > disappointed to discover it doesn't resize the image at all; it
> > > > > just re-frames it by cropping the right hand side.
> > > > > 
> > > > > I changed to Stretch by just typing over Resize and was quite a bit
> > > > > surprised when that didn't seem to work either! That was when I
> > > > > realised that while Resize(W,H) is a procedure, Stretch(W,H) is, as
> > > > > you said, a function! I was a bit hasty in providing my example
> > > > > code lines so I repeated the mistake which I had corrected in my
> > > > > event handler and which you have corrected above.
> > > > > 
> > > > > The code which I copied for my example does not assign the result
> > > > > of the Stretch function, but immediately writes it back to disc.
> > > > > It reads:
> > > > > 
> > > > > frame.Stretch(NewWidth,NewHeight).Save(DestinationImage)
> > > > > 
> > > > > It works. This is what I do now, although I had originally used the
> > > > > version you suggested and assigned the "stretched" image to my
> > > > > frame variable before saving. I have been trying to nail down the
> > > > > greedy memory culprit and the frame.Stretch().Save() version made
> > > > > no difference, but I was too lazy to change it back to
> > > > > frame=frame.Stretch() followed by frame.Save().
> > > > > 
> > > > > I have since discovered that it is the frame=Image.Load() step
> > > > > which is to blame. Each png is at most 1MiB, though most are much
> > > > > less. 740 of these when loaded in turn by  frame=Image.Load() uses
> > > > > 1.8GiB of main memory and nearly 2GiB of swap. This seems
> > > > > excessive to me, especially since I don't seem able to get it back
> > > > > without quitting from the program. Any ideas about that?
> > > > 
> > > > frame=null ?
> > > > 
> > > > > Richard
> > > > > 
> > > > > -------------------------------------------------------------------
> > > > > --
> > > > >
> > > > >-- --
> > > > >
> > > > >-----
> > > > >
> > > > > _______________________________________________
> > > > > Gambas-user mailing list
> > > > > Gambas-user at lists.sourceforge.net
> > > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > > 
> > > > ---------------------------------------------------------------------
> > > > --
> > > >
> > > >-- --
> > > >
> > > >---
> > > >
> > > > _______________________________________________
> > > > Gambas-user mailing list
> > > > Gambas-user at lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > 
> > > Yes. Right after the FOR loop which iterates through all the frames for
> > > loading and processing.
> > > 
> > > That is one of the reasons I thought I would ask for help on this. As
> > > far as I can tell I have complied with all of Benoit's hints and tips
> > > on managing memory. The frame variable is declared local to the event
> > > handler: DIM frame AS NEW Image
> > > The event handler is the only code which uses this variable. There is a
> > > WAIT statement in the FOR loop so that the ProgressBar works, but the
> > > memory is eaten up at the same ferocious rate either with or without
> > > the WAIT.
> > > 
> > > What I can tell for certain is that the memory is only allocated while
> > > my code is executing the FOR loop. The frame variable is created and
> > > destroyed outside this loop which strongly suggests that I have not
> > > programmed the code which is using the memory. By deleting all code
> > > within the loop EXCEPT the frame = Image.Load() line I can run the
> > > code and watch as it runs through my memory at an estimated rate of
> > > something like 4MiB per sub-1MiB frame. This just doesn't look right.
> > > If it is then I cannot use Gambas to apply simple changes to very
> > > large numbers of PNG images. That can't be right either.
> > > 
> > > For completeness I should also mention that I have run the FOR loop
> > > with both Image handling lines removed (load/stretch and save) and
> > > there is no sign of memory loss; just blindness 'cos there are no
> > > pictures:-)
> > > 
> > > Richard
> > 
> > Can you send your project, or better isolate the image processing that
> > burns your memory in a little project?
> > 
> > Regards,
> 
> Glad to; I have been running a simple test project which reveals what I now
> believe to be a bug of some long standing. I have only been able to test
> 2.4, 2,16 and 2.20 so far. 2.4 works without consuming memory, 2.16.and
> 2.20 have the bug.
> 
> I put together a simple form with a progress bar and two buttons. I observe
> memory and swap consumption while the progress bar moves towards 100% using
> gkrellm - 'cos it's always there.
> 
> The test program merely reloads a single image file many times so it is
> quicker to run than the "real thing" where there may be hundreds  of
> different files. Nevertheless its behaviour is just the same, suggesting to
> me that there may be a problem in the Image.Load() method. The same memory
> consumption problem occurs if the object is of type Picture and
> Picture.Load() is used.
> 
> Hope it's easy to find because I would really like to get this working for
> my brother.
> 
> Richard
> 
> ' Gambas class file
> 
> PUBLIC SUB _new()
> 
> END
> 
> PUBLIC SUB Form_Open()
> 
> END
> 
> PUBLIC SUB Quit_Click()
> 
>   ME.Close
> 
> END
> 
> PUBLIC SUB Test_Click()
> DIM n AS Integer
> DIM Frame AS NEW Image
> 
>     ProgressBar1.Visible = TRUE
>     FOR n = 0 TO 799
>         Frame = Image.Load("/home/richard/1024x576.png")
>         ProgressBar1.Value = n / 499
>         WAIT
>     NEXT
>     ProgressBar1.Visible = FALSE
> 
> END
> 

Do you use gb.qt or gb.gtk?

-- 
Benoît Minisini




More information about the User mailing list