[Gambas-user] (To Benoit) - About SHELL slowlyness

[Doriano Blengino]

I tried to investigate whether is true that SHELL is slow.

I searched for a directory having a few short file, and tried with 
/usr/share/services (188 files and some directory).

Then I typed:
time for a in *; do sh -c "cp $a /tmp"; done
cp: directory `kaddressbook' omessa   <-- "omessa" = "omitted"
cp: directory `kded' omessa
cp: directory `kontact' omessa
cp: directory `kresources' omessa
cp: directory `searchproviders' omessa
cp: directory `useragentstrings' omessa

real    0m1.784s
user    0m0.768s
sys     0m0.912s

-------------------------
Then I did:
time for a in /tmp/*; do sh -c "rm $a"; done
rm: impossibile rimuovere `/tmp/gambas.0': Is a directory
rm: impossibile rimuovere `/tmp/kde-root': Is a directory
rm: impossibile rimuovere `/tmp/ksocket-root': Is a directory
rm: impossibile rimuovere `/tmp/ssh-DtpWml2479': Is a directory

real    0m1.425s
user    0m0.632s
sys     0m0.756s
----------------------------

Then I tried to do the same thing inside the gambas app. The code for 
the copy operation is as follows:

  initime = Timer()
  FOR EACH fname IN fvMain.Selection
    mess("Copying '" & fname & "'...")         ' <<<--- sets a label and 
issues a WAIT to update the screen

    IF Exist(fvAux.dir &/ fname) THEN
      reply = Message.Question("\"" & fname & "\" already exist. 
Overwrite?", "&No", "&Yes", "&Stop!")
      IF reply = 1 THEN CONTINUE
      IF reply = 3 THEN
        status("Interrupted: only copied " & cnt & " files.")
        RETURN
      ENDIF
    ENDIF

    SHELL "cp -r \"" & fvMain.Dir &/ fname & "\" \"" & fvAux.Dir & "\" 
2>&1" TO res
    IF res <> "" THEN
      reply = Message.error("Copying, shell replied: " & res, "&Stop", 
"Go on")
      IF reply = 1 THEN
        mess("Interrupted: only copied " & cnt & " files.")
        BREAK
      ENDIF
    ELSE
      INC cnt
    ENDIF
  NEXT
 
  status("Copied " & cnt & " files. " & (Timer() - initime) & "secs.")
  fvAux.Reload

This routine took 3.133 seconds (after selecting the 188 files in 
/usr/share/services and issued a copy to /tmp).
-------------------------------------------

Then I tried the delete operation. The code is the following one:

  initime = Timer()
  FOR EACH name IN lastfv.Selection
    fname = lastfv.Dir & "/" & name
    IF IsDir(fname) THEN
      IF askdirs THEN
        reply = Message.Question("About to delete a directory. Sure to 
delete " & fname & "?", "&Yes", "Yes to &All", "&No")
        IF reply = 3 THEN RETURN
        IF reply = 2 THEN askdirs = FALSE
      ENDIF
      SHELL "rm -r \"" & fname & "\" 2>&1" TO name
      IF name <> "" THEN
        Message.Info(name)
      ELSE
        dvMain.deldir(fname)
      ENDIF
    ELSE
      SHELL "rm \"" & fname & "\" 2>&1" TO name
      IF name <> "" THEN Message.Info(name)
    ENDIF
  NEXT
  mess("Took " & (Timer() - initime) & "seconds.")

This routine took 1.86 seconds for 188 files (no directories selected) 
in /tmp.

To summarize:
  copy operation on 188 short files (mostly 1K):
    bash: 1.78 seconds; gambas: 3.13 seconds
    ...the gambas code has the burden of updating a user message (a 
label); my graphic card is a slow one

  remove operation on 188 files:
    bash: 1.42 seconds; gambas: 1.86
    ...in this case, there is no burden - only the minimal necessary 
things are done

File system type is ext2 for both / (where /tmp resides) and /usr 
(mounted on /).
My machine is a Duron 900 with 256K ram (yes, I know, a monster 
machine!), running Debian stable and an ATI Radeon as video card.

Now some consideration.
When opening my /home/mp3 directory, which contains 702 items (songs), 
gambas is far faster than KDE. The routine is basically a slightly 
modified version of FileView - but it does nearly the same things: open 
dir, stat() (two times!) every file, raise event to get icon, add the 
filename to a ListView. My conclusion is that the interpreted bytecode 
is fast - at least faster than bash interpreting a "for" cycle. So the 
slowlyness should not belong to bytecode.

When copying 188 files, the gambas code does three things:
  1. Update a label to let the user know what file is under copy
  2. Test for the destination existance
  3. SHELL to "cp file ..."

Apart from the label stuff, all the things are also done by the bash 
command line. But the time for gambas is double. One could assume this 
is because the label stuff (assuming that bytecode interpretation is 
faster than bash interpretation - but I think this is easily and 
certainly true).

When deleting 188 files, gambas is marginally slower than bash. But this 
time there is no burden about labels; the gambas code does almost the 
same things bash does, but is still slower. I point my attention to the 
SHELL instruction, but I could be wrong.

Benoit, you are right when you say "there is no reason why SHELL should 
be slow", but I tested my impressions with some more scientific tests. 
This is not a precise test, and a few assumptions are made. Assumptions 
can often be wrong, of course; but... impressions rarely are. The fact 
my machine is slow amplifies problems.

Just to do some more test, I did:
time cp * /tmp
cp: directory `kaddressbook' omessa
cp: directory `kded' omessa
cp: directory `kontact' omessa
cp: directory `kresources' omessa
cp: directory `searchproviders' omessa
cp: directory `useragentstrings' omessa

real    0m0.044s
user    0m0.012s
sys     0m0.016s

As you see, without using the "sh -c ...." time is really really faster: 
(1.42 secs vs 0.044).

And removing files gives:
time rm /tmp/*
rm: impossibile rimuovere `/tmp/gambas.0': Is a directory
rm: impossibile rimuovere `/tmp/kde-root': Is a directory
rm: impossibile rimuovere `/tmp/ksocket-root': Is a directory
rm: impossibile rimuovere `/tmp/ssh-DtpWml2479': Is a directory

real    0m0.014s
user    0m0.008s
sys     0m0.008s

Now we see where the burden is, for both bash and gambas: the EXEC (or 
SHELL) instruction. But gambas is slower than bash, and I think that 
bash should interpret its code far slowly than gambas. This is why I say 
that SHELL is slow in gambas.
There is a lot of uncertainity though; for example, what about kernel 
caches? I suppose they have a timeout - who knows when they are flushed 
to disk? Perhaps this plays a role, perhaps not. But my impression 
remains...


Anyway, who cares about milliseconds! I can wait 10 seconds for 
Konqueror to load a /home/mp3 directory. And I can wait 3 seconds for 
gambas to copy 188 files. This is why I decided to ignore the issue, 
until you asked me to explore better.

Salut, (and prosit - long life to gambas!)... :-)

-- 
Doriano Blengino

"Listen twice before you speak.
This is why we have two ears, but only one mouth."