[Gambas-user] Immutable strings and c functions returning allocated blocks, mmaped strings/objects

[Brian G]

I have attached a project which uses GNU readline(Prompt as string) as pointer

This c function allocates a block of memory from the heap to return as the read buffer. The function provided as editable input line, and history etc. The returned string must be released back to the heap after use.

When that happens it leaves a few hanging pointers, from this example and the program stops working... sometimes.

This is just one example... I happened to want to use in a project. I can avoid this, write my own. but is it possible to have Gambas just drop the reference if its outside its heap, not free it. With some compile time flag. To Allow Gambas to work with memory references. It adds a tremendous amount of flexibility to the external library interface.


The Shared Memory Component should Copy the buffer from application to application, not a real issue, or between tasks in Gambas. Allowing somewhat multi threaded like application development. With less OS overhead, that other transfer methods allow.

I guess what I am really asking is for some way that a component could prevent or control the release of memory for objects and string etc. 

Thanks for your detailed and informative reply. I appreciate you taking the time. 

If you could have a look at the project provided as an example of what I am talking about as
 far as pointer go. 

I have to say that as a programming and development platform Gambas is excellent and I have used it for a number of projects and number of years. I really appreciate the work that has been put into this language and environment. The web interface builder is just amazing. Thank you for all the hard work.

Thank You
Brian G

----- Original Message -----
From: "Tobias Boege" <taboege at gmail.com>
To: "Gambas mailing list" <user at lists.gambas-basic.org>
Sent: Saturday, February 15, 2020 6:41:02 AM
Subject: Re: [Gambas-user] Immutable strings and c functions returning allocated blocks, mmaped strings/objects

On Sat, 15 Feb 2020, Benoît Minisini wrote:
> Le 15/02/2020 à 08:51, Brian G a écrit :
> > I am having an interesting time with functions that return allocated
> > blocks that represent zero terminated strings
> > 
> > such as gnu readline(prompt as string) as string. This function provided
> > and editable line on the terminal, it returns the input as a string
> > buffer allocated from the heap. I do this from some c functions as well,
> > but into the shared memory buffers used by task to transfer information.
> > The Returned pointer is freed after use, but bad pointers are left.
> > 
> > It appears that You have implemented string as an immutable value, even
> > mid simply return a pointer in the original string and manages a length
> > internally without changing the string or making any copy, which is
> > fast.
> > 
> > no matter what you do, the string is immutable, so a = b, just creates a
> > copy of a pointer to the original string, its immutable right.
> > so adding it to a collection, or string array, or anything else leaves a
> > bad pointer when its freed later. Strange and unpredictable results. How
> > does the garbage collector handle the pointers if the value is not freed
> > and the variable goes out of scope. It appears to sometimes cause a
> > crash, could just be my bad code.
> > 
> > The same issue arises with shared memory segment, that contains string
> > information. Instead this time the pointers are all into the area not
> > managed by the Gambas gc. When the variables assigned for example with s
> > = string@(pSharedMem) <mailto:string@(pSharedMem)> goes out of scope
> > what happens? Does that change if a global points to the same string in
> > shared memory as well?... umm Boom segfault, if the unmap is done before
> > the variable goes out of scope.
> > 
> > How is it handled, should I be making copies of strings, is there a way
> > to tell the gc not to collect pointers for certain variables when they
> > go out of scope.
> > This is a little complex, but I am running into some of these issues,
> > there seems to be only one way of getting a copy right now, that's x =
> > string(1,string@(pointer)) maybe there is a faster way. Is the gc smart
> > enough to know that a pointer is used in a higher level scope.

To give you a more concrete idea of what's happening: the "garbage collector"
in Gambas is just plain old reference counting. So no, the garbage collector
is not smart enough for just about any lifetime analysis. All of it happens
at runtime. If you use a value, you borrow a ref, if you don't need it any-
more, you release the ref. This is done automatically if you write in Gambas
and semi-automatically if you use the interpreter C API. Note that the very
act of keeping the reference count requires you to import any data into the
Gambas interpreter's structs, most robustly by copying it, so that Gambas
can manage its copy normally.

As long as a value is referenced somewhere, it survives. Do not think about
it in terms of scope. A variable (in scope) referencing a value keeps its
ref count above zero, but the value itself exists independently of the
variable that holds a reference to it. If you assign the same value into
a Collection or an Array or a property of an object, that's another reference
to the value. If the original variable goes out of scope, the value's ref
count is reduced but the value is not destroyed, because the Collection or
whatever now keeps the ref count positive. It is not possible (barring
incorrectness of Gambas' memory management) that a dangling pointer to
a Gambas-owned value exists in some Gambas container because it sitting
in there keeps the value from being destroyed.

If you can make the interpreter crash because of double frees, then either
you do something forbidden or the interpreter's ref counting mechanism has
a bug. Please show a minimal example of such a crash. Another way to make
Gambas crash is to smuggle a string pointer into its system which does not
sit inside a proper Gambas STRING struct (which seems to be what you're
after, but I doubt that this is possible from a Gambas program).

If you want to keep a value around indefinitely, simply increase its ref
count without the intention of decreasing it, but this would be called a
memory leak. I do not think that manipulating ref counts is exposed to
Gambas programs anyway. I don't know what you're trying to do, but if you
want to add shared memory utilities to Gambas, I bet you would be happier
with the C API, i.e. writing a native component.

Regards,
Tobias

-- 
"There's an old saying: Don't change anything... ever!" -- Mr. Monk

----[ http://gambaswiki.org/wiki/doc/netiquette ]----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testpointers-0.0.1.tar.gz
Type: application/x-compressed-tar
Size: 11826 bytes
Desc: not available
URL: <https://lists.gambas-basic.org/pipermail/user/attachments/20200215/e48291be/attachment-0001.bin>