Integrated jupyterlab

Follow
Avatar
Terry Lines

I'm running omnisci on AWS, and pulling information from the database into jupyterlab is very slow. This is the precompiled AMI, with both jupyterlab and omnsci running on the same instance, but there is some sort of bottleneck that is messing things up. A select function of a 2 column table is linear with length, and 100,000 rows takes over 10 seconds. Running via sqleditor takes a few ms as you might expect. Any ideas as to what might be the cause?

0

Comments

3 comments

Sort by Date Votes
  • Avatar
    Candido Dessanti

    Hi,

    Yes, you are right. However, using the internal connection turns out on a prolonged response time.

    As a fast workaround, you can connect to the database directly in binary mode.

    immagine|690x224

    This should fix the performance problem. I will ask internally if there is a more straight full workaround and come back to you after the weekend.

    Regards, Candido

    1
    Comment actions Permalink
  • Avatar
    Candido Dessanti

    Hi,

    can you supply a code example?

    something like that?

    con = connect(user="admin", password= "HyperInteractive", host="my.host.com", dbname="omnisci")
c = con.cursor()
c.execute("SELECT depdelay, arrdelay FROM flights LIMIT 100000")

    10000ms is really high, but the SQLEditor is goin gto return just the first 1000 records. The first time it can by very high expeciallly if you are using dictionary encoding datatypes.

    Regards, Candido

    0
    Comment actions Permalink
  • Avatar
    Terry Lines

    This is on AWS in JupyterLab:

    con = omnisci_connect()
c=con.con.cursor()

%%time
c.execute('SELECT "ADDRID", "PLACEKEY" FROM placekey LIMIT 100000')

    CPU times: user 9.04 s, sys: 6.23 ms, total: 9.05 s Wall time: 9.28 s

    Running the equivalent on the Mac version on my MacBook takes 11ms....

    My guess is that it's due to the connection between the containers being http?

    con.con

    Connection([omnisci://None:***@http://omniscidb:6278/None?protocol=http]

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post