[Gambas-user] Need Gambas3 Made Apps.

[Rolf-Werner Eilert]

Am 07.02.2013 18:16, schrieb Tobias Boege:
> On Thu, 07 Feb 2013, Rolf-Werner Eilert wrote:
>> Hi Tobias,
>>
>> following your thread, I found
>>
>>   > About soft(ware?) for media (streams?), that's a point I would
>> support: I'm
>>   > not aware of easy methods for bit-fiddling or using SIMD, etc. like
>> one can
>>   > do it with gcc intrinsics. If there a library for it (I don't know) this
>>   > could be a target for a component.
>>
>> But one could write the functions in C or C++ and include them as a
>> library. At the end of the day, there is an end user who wants to use
>> the functionality, not the functions.
>>
>>
>> I simply wondered about what you said here, so I'm curious:
>>
>>> That clearly depends on _how_ you use the files. This is the same with all
>>> programming languages: Never read a file into memory for processing it but
>>> use a Stream instead. The number of files opened and their size should not
>>> matter then.
>>>
>>
>> I must confess, this is the first time I've heared about this. Could you
>> explain, where is the difference and how do I manage the data in the
>> file then? A pointer to the bytes? And how do I read/write while working
>> on the data? (I'm thinking of picture manipulation or a video cutting
>> program, real big data, not a text file of course, or did I get you
>> wrong here?)
>
> You can apply this concept to text files, too. A common example is to parse
> a configuration file. Normally, these contain line-oriented, human readable
> data on Unices. You can equally easily use Split(File.Load("config"), "\n")
> and LINE INPUT to process lines individually but the stream thing is much
> superior from a memory point of view:
>
> The first methods loads the entire file at once and splits it into an array
> of lines, which temporarily leaves the entire file in memory twice. Once in
> the array, once in the return value of File.Load() which is subsequently
> freed. Using LINE INPUT you keep a line at a time in memory. Less memory
> usage means less swap usage, etc..
>
> It is a different thing when the underlying mechanisms use mmap() to map a
> file into memory which does not appear to be the case for File.Load().
>
> How you use streams to process your data depends on the data. gbx3 uses
> mmap() to load executable Gambas archives[0]. These are uncompressed
> archives and so the interpreter can examine the header of that archive to
> find the files it needs. The file is perhaps not completely loaded in one
> particular execution of the program but only the needed parts. (I only
> glanced at the sources. Benoit knows this for sure.)
>
> Regards,
> Tobi
>
> [0] http://gambasdoc.org/help/doc/arch?v3
>

That makes sense to me, though in today's OSs it is a question of how 
many (small) files I want to browse or how big they are. When I don't 
have to look into many hundred or thousand files but only a few but for 
many times, why not make use of the caching mechanism of the file system?

But it sounds interesting, I'll have a look at it tomorrow. So thanks a 
lot for your explanation!

Rolf