Datatype from a parquet column is not matched with HeavDB table

Comments

1 comment

  • Avatar
    Candido Dessanti

    Hi  Abdul Rahim Kamaludeen,

    Could you share the parquet file you are trying to ingest or some sample rows?

    In any case, could you run the \detect_parquet command from the heavysql command?

    It'd return field names, datatypes, some sample data, and a DDL of the table.

    \detect_parquet /opt/opendata/flights/flights_none.parquet
    [cut output]
    CREATE TABLE your_table_name(flight_year SMALLINT, flight_mmonth SMALLINT, flight_dayofmonth SMALLINT, flight_dayofweek SMALLINT, deptime SMALLINT, crsdeptime SMALLINT, arrtime SMALLINT, crsarrtime SMALLINT, uniquecarrier TEXT ENCODING DICT, flightnum SMALLINT, tailnum TEXT ENCODING DICT, actualelapsedtime SMALLINT, crselapsedtime SMALLINT, airtime SMALLINT, arrdelay SMALLINT, depdelay SMALLINT, origin TEXT ENCODING DICT, dest TEXT ENCODING DICT, distance SMALLINT, taxiin SMALLINT, taxiout SMALLINT, cancelled SMALLINT, cancellationcode TEXT ENCODING DICT, diverted SMALLINT, carrierdelay SMALLINT, weatherdelay SMALLINT, nasdelay SMALLINT, securitydelay SMALLINT, lateaircraftdelay SMALLINT, dep_timestamp TIMESTAMP(3), arr_timestamp TIMESTAMP(3), carrier_name TEXT ENCODING DICT, plane_type TEXT ENCODING DICT, plane_manufacturer TEXT ENCODING DICT, plane_issue_date TEXT ENCODING DICT, plane_model TEXT ENCODING DICT, plane_status TEXT ENCODING DICT, plane_aircraft_type TEXT ENCODING DICT, plane_engine_type TEXT ENCODING DICT, plane_year SMALLINT, origin_name TEXT ENCODING DICT, origin_city TEXT ENCODING DICT, origin_state TEXT ENCODING DICT, origin_country TEXT ENCODING DICT, origin_lat FLOAT, origin_lon FLOAT, dest_name TEXT ENCODING DICT, dest_city TEXT ENCODING DICT, dest_state TEXT ENCODING DICT, dest_country TEXT ENCODING DICT, dest_lat FLOAT, dest_lon FLOAT);

    And post the reulteing DDLs?

    Bests,
    Candido

     

    0
    Comment actions Permalink

Please sign in to leave a comment.