Direct Storage to GPU

Follow
Avatar
Laurent

Hi Team,

Is direct storage to GPU tech (RTX IO, NetGPU) on the roadmap for future versions?

Do you see this as a game changer for 1st time memory loads?

Thanks, Laurent

0

Comments

8 comments

Sort by Date Votes
  • Avatar
    Todd Mostak

    Hi @Laurent,

    Thanks for joining our forum! Your question is well timed, as we are actually testing Nvidia's new GPU Direct Storage capabilities, both locally and via network storage and exploring how we can leverage it in our platform. Early results are quite promising, particularly on a system like a DGX A100, where you have a ton of bandwidth over the 8 infiniband ports directly to the GPUs.

    As you might understand, we don't have a firm timeline yet on how and when this capability might make it into our product, but would love to understand your particular use case better so as to better inform our path forward. Would you be using network flash storage or are you more interested in just optimizing loads from local nvme?

    Best,

    Todd

    0
    Comment actions Permalink
  • Avatar
    Laurent

    Hi Todd,

    Thanks for the reply. I heard of Direct Storage with PG-Strom a while ago (SSD2GPU) but found its constraints too restrictive at the time despite promising performance uplifts. My current understanding is that Nvidia & Facebook efforts (on the NetGPU side) are still ongoing so it's good to hear this is already on the radar. With the new Ampere architecture, Mellanox inter-connect and added PCIE4 benefits, I can only imagine what the performance will look like on a DGX A100, looking forward to see how Omnisci scales once all the pieces come together!

    For now, my own use case is more modest, using Omnisci Open Source on a Threadripper workstation, occasionally on a laptop with both RTX/GTX GPUs and local NVME drives. Proportionally to my hardware, performance is rock solid and the fastest I have experienced, with only that first load taking more time and ETL adjustments for string operations as current constraints. That said, between legacy offerings and workable but complex solutions such as Spark, I have to say it really hits the sweet spot of usability Vs performance. Jensen Huang said you were onto a good thing in one of his keynotes, I couldn't agree more, hope you guys keep the momentum going.

    Best, Laurent

    0
    Comment actions Permalink
  • Avatar
    Todd Mostak

    Thanks @Laurent for the kinds words, and very glad to hear that you're having a good experience with OmniSci! As for the string functions, we're actively looking into this, but if you're brave you can use an undocumented sqlite interoperability flag (enable-interoperability) to get some capabilities for basic projections. For example:

    create table pa_house_district_map as select leg_distri as district_int, substring(url,96,3) || 'TH LEGISLATIVE DISTRICT' as district_name from pa_state_house_district_boundaries where mod(leg_distri, 10) >= 4 OR mod(leg_distri, 10) = 0;
insert into pa_house_district_map select leg_distri as district_int, substring(url,96,3) || 'RD LEGISLATIVE DISTRICT' as district_name from pa_state_house_district_boundaries where mod(leg_distri, 10) = 3;
insert into pa_house_district_map select leg_distri as district_int, substring(url,96,3) || 'ND LEGISLATIVE DISTRICT' as district_name from pa_state_house_district_boundaries where mod(leg_distri, 10) = 2;
insert into pa_house_district_map select leg_distri as district_int, substring(url,96,3) || 'ST LEGISLATIVE DISTRICT' as district_name from pa_state_house_district_boundaries where mod(leg_distri, 10) = 1;
alter table pa_house_district_map add column district_name2 text; /*because the original column is not dictionary encoded*/
update pa_house_district_map set district_name2 = district_name;

    Then I was able to use an update via subquery to add the transformed column to the original table.

    alter table pa_state_house_district_boundaries add column district_name text;
update
  pa_state_house_district_boundaries
set
  district_name = 
    '13TH LEGISLATIVE DISTRICT' WHERE district_name = '13RD LEGISLATIVE DISTRICT';

    Several things don't work fully, like direct update via these functions, and additional steps on top of the projected results, but we're looking into these. However I know it can be painful to have to go outside the system for things which could be done in simple ELT, so wanted to note the above as a potential early workaround.

    Thanks again for diving into our product, and let us know if you have any other thoughts or questions we can help with!

    Todd

    0
    Comment actions Permalink
  • Avatar
    Laurent

    Thanks for taking the time to share Todd, good to hear there are workarounds while string functions are being looked at. My own use case isn't too impacted as I have opted for ETL early on by using Python for light Transforms, but I can see how this will greatly simplify complex Enterprise ELT scenarios and reduce data preparation latencies.

    Coming back to Direct Storage Access tech, it seems there is potential for even better optimisation. On top of what we discussed with shortcut access between storage and GPUs, i/o between CPU and GPU is also heading for an uplift with Nvidia's implementation of AMD's Smart Access Memory (https://videocardz.com/newz/nvidia-developing-similar-technology-to-amd-smart-access-memory-for-ampere-gpus).

    For now, I have upgraded my GPU to Ampere (RTX 3090) and running through some expected compatibility issues (running 5.4.1 on Nvidia 455.23.04, Ubuntu 18.04 , Kernel 5.4.0 for reference). GPU mode is disabled but running fine on CPU, I will make sure to reach out if any issues arise once all the updated drivers are released, the joys of early adoption :slight_smile:

    Best, Laurent

    0
    Comment actions Permalink
  • Avatar
    Gopinath Jaganmohan

    Hi, Does this workaround still works? Do we have any other undocumented string manipulation supported?

    0
    Comment actions Permalink
  • Avatar
    Candido Dessanti

    Hi,

    Yes it's still working, assuming the parameter enable-interoperability is turned on.

    Essentially you can use almost all sqlite string functions, but just in projection queries; it's a useful feature if you need to do some transformations on you data without using any externa tool.

    Regards, Candido.

    0
    Comment actions Permalink
  • Avatar
    Gopinath Jaganmohan

    Thanks, Candido. I tried enabling the enable-interoperability while starting omnisc-server like this

    /bin/omnisci_server /var/data --enable-runtime-udf --enable-table-functions --enable-interoperability

    and tried the below query. "select substring(product,2,3) from epf"

    Got error "TOmniSciException(error_msg='Function SUBSTRING(TEXT, INTEGER, INTEGER) not supported.')"

    Am'i doing anything wrong

    0
    Comment actions Permalink
  • Avatar
    Candido Dessanti

    Hi,

    Sorry, but looks that from 5.5 the substring function isn't here anymore, and the only thing that works is the concat.

    had to check internally if something has gone wrong

    regards, Candido

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post