[Gambas-user] bizarre invisible chars

[Bruce Steers]

On Sat, 7 Jan 2023 at 00:30, Christof Thalhofer <chrisml at deganius.de> wrote:

> Hi Bruce,
>
> Am 06.01.23 um 23:35 schrieb Bruce Steers:
>
> > Attached is a script file that has me very puzzled.
> >
> > It's a thing i've seen a bit of recently using my gambas text editor, no
> > idea the cause.
> >
> > the attached script has some lines that appear to be just some spaces but
> > they are not.
> > I have tried all sorts to clear them, Trim does not trim them.
>
> Look at it in a hex viewer (picture attached). These are UTF8
> non-breaking-spaces:
>
> https://en.wikipedia.org/wiki/Non-breaking_space#Encodings
>
> Alles Gute
>
> Christof Thalhofer
>

aaah  cheers Chris

I've now managed to make a function to clean the text, either selected text
or whole doc.
(posted below if anyone is interested)
Had to iterate through each char and check using String.Mid and String.Code
If i removed all codes outside of 0 to 128 i lost all the leading spaces
so had to change all non std chars to a space then rtrim each line after.

Anyone know of a better way to clean a docs text of non std chars?

Me tired , me need bed.
Cheers
BruceS

Public Sub FixText()

      Dim sTxt As String
      Dim sNew As String
      If tEditor.Selected Then sTxt = tEditor.SelectedText Else sTxt =
tEditor.Text
      For c As Integer = 1 To Len(sTxt)
        Select String.Code(String.Mid(sTxt, c, 1))
          Case 0 To 128
            sNew &= String.Mid(sTxt, c, 1)
          Case Else
            sNew &= " "
        End Select
      Next

      If sNew <> sTxt Then
        If Message.Warning("Non valid chars found", "Fix", "Ignore") = 2
Then Return
      Else
        Return
      Endif
      Dim aLines As String[] = Split(sNew, "\n")
      For c = 0 To alines.Max
        aLines[c] = RTrim(aLines[c])
      Next
      sNew = aLines.Join("\n")
      If tEditor.Selected Then tEditor().Insert(sNew) Else tEditor.Text =
sNew

End
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gambas-basic.org/pipermail/user/attachments/20230107/4690c447/attachment.htm>