[HeavyConnect] ODBC BigQuery - Can not read certain string values

Follow
Avatar
Jose Eduardo Reinoso Andrade

Hello,

I have just installed and configured the ODBC driver for BigQuery. While it works fine generally, I am encountering issues when attempting to read records that contain accented characters.

The error message I receive is:

[Odbc error SQLSTATE = [01004]. Native Error Code = [10160]. Details: "[Simba][ODBC] (10160) Output string data right truncation: string data is too big for the output data buffer and has been truncated."]

It occurs when I'm trying to read a FOREIGN TABLE. Please note that the error does not occur if the dataset I want to read from BigQuery does not contain accented characters.

This table can be read correctly:

While this one generates the error described above:

According to the documentation, the Simba Google BigQuery ODBC Connector supports many common data formats, converting between BigQuery data types and SQL data types. In this case, STRING is converted into: SQL_VARCHAR or SQL_WVARCHAR, and those types are permitted and handled by the HeavyConnector.

I have tried to read the columns in these HeavyDB datatypes: TEXT ENCODING DICT and TEXT ENCODING NONE, but the same issue is shown.

Following the Simba Google BigQuery ODBC documentation, I have also tried to specify the conversion from STRING to SQL_WVARCHAR always, but the issue persists.

I believe the issue might not be with the Simba Google BigQuery ODBC Connector itself, as I have tried reading the same datasets from the console using the isql command, and it works fine, displaying the accents in both tables correctly

 

SPECS:

HeavyAi v8.0.2
unixODBC 2.3.1
Simba Google BigQuery ODBC Data Connector 3.0.5
CentOS 7, 24cpu, 32GB RAM
Tesla T4 16GB



Thank you very much for the help! If you need further information, please let me know!

Jose E.

1

Comments

3 comments

Sort by Date Votes
  • Avatar
    Candido Dessanti

    Hi,

    Probably the source data contains a multibyte char, but the connector is allocating the buffers as it was composed by single bytes.

    We support multibyte characters in the database, but there is probably something wrong with the ODBC connector, which is still in beta state.

    Candido

    0
    Comment actions Permalink
  • Avatar
    Jose Eduardo Reinoso Andrade

    Hello Candido after some test. You are right!!! 

    Here you have the workaround that i have found.

    - Apply this "replace" in all string columns that you want to connect with the ODBC. 

    REGEXP_REPLACE(col_name, r'[^\x00-\x7F]', '_') as col_name
    0
    Comment actions Permalink
  • Avatar
    Candido Dessanti

    Hi,

    I'm glad you found the workaround. I will open an internal ticket to get that fixed. I just need to reproduce the issue.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post