Long query string throws a `thrift.transport.TTransport.TTransportException: unexpected exception`
Hello --
I'm using `pymapd` to run Postgres queries. I've noticed that when the query string is very long, the client raises an error (`TTransportException: unexpected exception`) when trying to run the query:
```
Exception ignored in: <function Connection.__del__ at 0x7f8d8c392790> Traceback (most recent call last): File "/usr/local/webapp/flask/lib/python3.8/site-packages/pymapd/connection.py", line 348, in __del__ self.close() File "/usr/local/webapp/flask/lib/python3.8/site-packages/pymapd/connection.py", line 364, in close self._client.disconnect(self._session) File "/usr/local/webapp/flask/lib/python3.8/site-packages/omnisci/thrift/OmniSci.py", line 867, in disconnect self.send_disconnect(session) File "/usr/local/webapp/flask/lib/python3.8/site-packages/omnisci/thrift/OmniSci.py", line 876, in send_disconnect self._oprot.trans.flush() File "/usr/local/webapp/flask/lib/python3.8/site-packages/thrift/transport/TTransport.py", line 179, in flush self.__trans.write(out) File "/usr/local/webapp/flask/lib/python3.8/site-packages/thrift/transport/TSocket.py", line 161, in write raise TTransportException(message="unexpected exception", inner=e) thrift.transport.TTransport.TTransportException: unexpected exception
```
Immediately after, Omnisci seems to crash:
```
Traceback (most recent call last): File "/usr/local/webapp/flask/lib/python3.8/site-packages/thrift/transport/TSocket.py", line 99, in open addrs = self._resolveAddr() File "/usr/local/webapp/flask/lib/python3.8/site-packages/thrift/transport/TSocket.py", line 37, in _resolveAddr return socket.getaddrinfo(self.host, File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/webapp/flask/lib/python3.8/site-packages/pymapd/connection.py", line 281, in __init__ self._transport.open() File "/usr/local/webapp/flask/lib/python3.8/site-packages/thrift/transport/TTransport.py", line 155, in open return self.__trans.open() File "/usr/local/webapp/flask/lib/python3.8/site-packages/thrift/transport/TSocket.py", line 103, in open raise TTransportException(type=TTransportException.NOT_OPEN, message=msg, inner=gai) thrift.transport.TTransport.TTransportException: failed to resolve sockaddr for omniscidb:6274 The above exception was the direct cause of the following exception: Traceback (most recent call last): [omitted...] File "/usr/local/webapp/flask/lib/python3.8/site-packages/retrying.py", line 49, in wrapped_f return Retrying(*dargs, **dkw).call(f, *args, **kw) File "/usr/local/webapp/flask/lib/python3.8/site-packages/retrying.py", line 212, in call raise attempt.get() File "/usr/local/webapp/flask/lib/python3.8/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/usr/local/webapp/flask/lib/python3.8/site-packages/six.py", line 719, in reraise raise value File "/usr/local/webapp/flask/lib/python3.8/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(*args, **kwargs), attempt_number, False) [omitted...] File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) [omitted...] File "/usr/local/webapp/flask/lib/python3.8/site-packages/pymapd/connection.py", line 116, in connect return Connection( File "/usr/local/webapp/flask/lib/python3.8/site-packages/pymapd/connection.py", line 285, in __init__ raise err from e pymapd.exceptions.OperationalError: Could not connect to database
```
I was able to consistently make this error go away by artificially shortening the query string, as well as make it reappear by lengthening the query string. A query with 10,447 chars fails, and a query with 8,749 chars passes. Furthermore, when the query is artificially split up into two smaller queries (by storing partial results in a temp table), it also passes.
I couldn't find any mention of a query string length limit in the documentation. Is there a char limit? Is this known behavior?
Thanks!
-
Hi Victoria,
Sorry for the late reply, but the system wasn't showing some messages.We haven't such a lower limit for the length of the query; I just tried with a query using over 88000 characters using the heavysql tool without any issue
mapd@zion-tr:/opt/heavyai$ wc -c test_sql
80049 test_sql
mapd@zion-tr:/opt/heavyai$ cat test_sql | bin/heavysql -p HyperInteractive tpch40
User admin connected to database tpch40
No rows returned.
User admin disconnected from database tpch40So I'm unsure if the issue is the length of the query itself; could you post the query that's failing?
Could you please tell me which database version you are using? pymapd is quite old and has been superseded by pyomnisci before and now by heavyai
You can find the installation instructions here
https://pypi.org/project/heavyai/
Regards,
Candido
Please sign in to leave a comment.
Comments
1 comment