[Gambas-user] help with data import into postgres - funny characters> import fails

[Robert Moss]

Is the error a Gambas error or a Postgres error? It might be they are using
a charset with special keys spefic to their industry, which might indicate
more data, or, more likely, the data is errorenous and got bit-shifted, and
now, the high-order bit might be too large to be part of the UTF-8 charset.
I would see if you can get the binary/hex of those keys, and shift the bits
each way by one, (try two also), and then if one of them makes sense, you
know its a communication error (that got uncorrected??? doesn't sound like
TCP).

Since you've never encountered these characters, the most likely scenario is
data corruption. Try to correct it with a bitshift, and if that works, write
code to search for high-order bits that are incorrect, and shift the
sequence appropriately.

Let me know what you think ^_^

-Robert

On Thu, Feb 26, 2009 at 5:57 PM, richard terry <rterry at ...1946...>wrote:

> Hi ,
>
> I'm trying to import the hl7 from a local radiology provider:
> It's got funny characters in the file and postgres baulks when I go to save
> the line with the message attatched, but it basically says - invalid byte
> sequence for encoding UTF8.
>
> My postgres database is created with UTF8 encoding
>
> CREATE DATABASE "27Feb09"
>  WITH OWNER = richard
>       ENCODING = 'UTF8';
>
> snipped is the offending bit:
>
> with a maximum AP diameter\.br\of 1.0cm and a less than 50% stenosis in its
> proximal segment.  \.br\\.br\Continued�����/2\.br\In the mid superficial
> femoral artery there is staccato flow and an
>
> I've imported thousands and thousands of lines from a pathology provider
>  and
> never encountered these characters.
>
> Thanks in anticipation.
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco,
> CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the
> Enterprise
> -Strategies to boost innovation and cut costs with open source
> participation
> -Receive a $600 discount off the registration fee with the source code:
> SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Gambas-user mailing list
> Gambas-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>