Immerse black screen after updating to Omnisci 5.8
Hello, after updating to 5.8 Immerse shows a black screen without login or dashboard list: omnisci_server and web_server are correctly loaded, and all is fine, without error into the logfiles. The browser pointing to: localhost:6273/omnisci/dashboards return a back screen without error: it seems to me that omnisci is working, but not displaying anything. I Already removed from browser all from file cache and offline data files, I'm running out of ideas. Details:
Fri Oct 15 11:43:08 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |
| 37% 62C P0 47W / 180W | 1064MiB / 8085MiB | 2% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1357 G /usr/lib/xorg/Xorg 584MiB |
| 0 N/A N/A 1771 G /usr/bin/kwin_x11 168MiB |
| 0 N/A N/A 1858 G /usr/bin/plasmashell 119MiB |
| 0 N/A N/A 57584 G /usr/lib/firefox/firefox 173MiB |
| 0 N/A N/A 65466 G /usr/lib/firefox/firefox 1MiB |
+-----------------------------------------------------------------------------+
ps -ef|grep omnisci
omnisci 10084 1 0 10:04 ? 00:00:06 /opt/omnisci/bin/omnisci_web_server --config /var/lib/omnisci/omnisci.conf
omnisci 94093 1 16 11:44 ? 00:00:00 /opt/omnisci/bin/omnisci_server --config /var/lib/omnisci/omnisci.conf
omnisci 94110 94093 83 11:44 ? 00:00:01 -Xmx1024m -DMAPD_LOG_DIR=/var/lib/omnisci/data -jar /opt/omnisci/bin/calcite-1.0-SNAPSHOT-jar-with-dependencies.jar -e /opt/omnisci/QueryEngine/ -d /var/lib/omnisci/data -p 6279 -m 6274 -T -P -Y -Z -c /var/lib/omnisci/omnisci.conf
This is the back screen as shown:
Ubuntu GTX1080 8GB Omnisci free GPU edition Firefox browser
here a snippet of the current log:
2021-10-15T12:51:32.894641 I 148763 0 0 MapDServer.cpp:394 OmniSciDB starting up
2021-10-15T12:51:32.897539 I 148763 0 0 DBHandler.cpp:310 OmniSci Server 5.8.0-20211011-b40c4f681d
2021-10-15T12:51:33.124482 I 148763 0 0 CudaMgr.cpp:368 Using 1 Gpus.
2021-10-15T12:51:33.124569 I 148763 0 0 CudaMgr.cpp:67 Warming up the GPU JIT Compiler... (this may take several seconds)
2021-10-15T12:51:33.237265 I 148763 0 0 CudaMgr.cpp:70 GPU JIT Compiler initialized.
2021-10-15T12:51:33.237351 I 148763 0 0 ArrowForeignStorage.cpp:850 CSV backed temporary tables has been activated. Create table `with (storage_type='CSV:path/to/file.csv');`
2021-10-15T12:51:33.237365 I 148763 0 0 ArrowForeignStorage.cpp:1028 CSV backed temporary tables has been activated. Create table `with (storage_type='CSV:path/to/file.csv');`
2021-10-15T12:51:33.237393 I 148763 0 0 DataMgr.cpp:230 Min CPU Slab Size is 256MB
2021-10-15T12:51:33.237410 I 148763 0 0 DataMgr.cpp:231 Max CPU Slab Size is 4096MB
2021-10-15T12:51:33.237417 I 148763 0 0 DataMgr.cpp:232 Max memory pool size for CPU is 98304MB
2021-10-15T12:51:33.237425 I 148763 0 0 DataMgr.cpp:250 Reserved GPU memory is 1337.67MB includes render buffer allocation
2021-10-15T12:51:33.237437 I 148763 0 0 DataMgr.cpp:269 Min GPU Slab size for GPU 0 is 256MB
2021-10-15T12:51:33.237445 I 148763 0 0 DataMgr.cpp:271 Max GPU Slab size for GPU 0 is 4096MB
2021-10-15T12:51:33.237451 I 148763 0 0 DataMgr.cpp:273 Max memory pool size for GPU 0 is 32768MB
2021-10-15T12:51:33.237605 I 148763 0 0 FileMgr.cpp:230 Completed Reading table's file metadata, Elapsed time : 0ms Epoch: 0 files read: 0 table location: '/var/lib/omnisci/data/mapd_data/table_0_0'
2021-10-15T12:51:33.237642 I 148763 0 0 Calcite.cpp:371 Creating Calcite Handler, Calcite Port is 6279 base data dir is /var/lib/omnisci/data
2021-10-15T12:51:33.237655 I 148763 0 0 Calcite.cpp:279 Running Calcite server as a daemon
2021-10-15T12:51:33.752701 I 148763 0 0 Calcite.cpp:319 Calcite server start took 500 ms
2021-10-15T12:51:33.752813 I 148763 0 0 Calcite.cpp:320 ping took 9 ms
2021-10-15T12:51:33.776755 I 148763 0 0 DBHandler.cpp:493 Started in GPU mode
2021-10-15T12:51:33.779220 I 148763 0 0 VulkanPlatform.cpp:156 Vulkan Instance API version: 1.2.162
2021-10-15T12:51:33.897649 I 148763 0 1 **MapDServer.cpp:368 Interrupt signal (11) received.**
where there is a MapDServer.cpp:368 Interrupt signal (11) received.
I can observe that the omnisci_server process disappears from memory (using ps command) , then reappears again.
-
Hi Gianfranco,
I suggest you to turn off the vulkan rendered adding this parameter to your omnisci.conf file
renderer-use-vulkan-driver=falseprobably you are having some trouble with our new renderer. if confirmed, that you are having troubles with that, you can try to fix installing vulkan libraries needed; please refer to the vulkan renderer documentation.
I f you are continuing to get crashes, please report back, so I will file the problem to the renderer team.
Candido
-
Thank you, I already checked vulkan lib, it was part of installation procedure, and it seems correctly installed:
$ sudo apt show libvulkan1 Package: libvulkan1 Version: 1.2.162.0-1 Priority: optional Section: libs Source: vulkan-loader Origin: Ubuntu Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Original-Maintainer: Debian X Strike Force <debian-x@lists.debian.org> Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 422 kB Depends: libc6 (>= 2.29) Recommends: mesa-vulkan-drivers | vulkan-icd Breaks: libvulkan-dev (<< 1.1.70+dfsg1-2), vulkan-loader Replaces: libvulkan-dev (<< 1.1.70+dfsg1-2), vulkan-loader Homepage: https://github.com/KhronosGroup/Vulkan-Loader Task: ubuntu-desktop-minimal, ubuntu-desktop, ubuntu-desktop-raspi, kubuntu-desktop, xubuntu-core, xubuntu-desktop, lubuntu-desktop, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-budgie-desktop, ubuntu-budgie-desktop-raspi Download-Size: 97,2 kB APT-Manual-Installed: yes APT-Sources: http://it.archive.ubuntu.com/ubuntu hirsute/main amd64 Packages Description: Vulkan loader library The Loader implements the main VK library. It handles layer management and driver management. The loader fully supports multi-gpu operation. As part of this, it dispatches API calls to the correct driver, and to the correct layers, based on the GPU object selected by the application. . This package includes the loader library.Anyway, after disabling vulkan and restarted omnisci_server and omnisci_web_server, and cleared the browser cache, I got exactly the same behavior with the black screen.
-
So the server is keep crashing and restarting? I guess that's the reason why immerse is returning a blank screen.
have you tried what's happens if you start the omnisci_server from command line with this command?
/opt/omnisci/bin/omnisci_server --config /var/lib/omnisci/omnisci.confCandido.
-
That command returns:
/opt/omnisci/bin/omnisci_server --config /var/lib/omnisci/omnisci.conf terminate called after throwing an instance of 'boost::wrapexcept<boost::filesystem::filesystem_error>' what(): Failed to open file for writing: Input/output error: "/var/lib/omnisci/data/mapd_log/omnisci_server.INFO.20211015-152846.log" Aborted (core dumped)omnisci_server keep crashing and restarting automatically with a different pid every few seconds, while no message is reported in /MAPD_LOG folder.
Omnisci folder and subfolder are owned by omnisci user with : -rw-r--r--
-
Have you enough space and/or the right permissions to write into /var/lib/omnisci/data/ directory?
it seems that errors you are getting when you run the DB as a service or from cli are different.
try to run from cli with the same user that's running the service and add
--log-severity-clog=DEBUG4if you are running the omnsci DB as root runsudo /opt/omnisci/bin/omnisci_server --data /var/lib/omnisci/data --log-severity-clog=DEBUG4this way the server will be run with default parameters and should output a super detailed log on standard output.
Candido
-
The detailed log activated with DEBUG4 doesn't write any info in ERROR and FATAL log files.
In addition, I have to point out that the system is plenty of space, and that many several omnisci_server.INFO logfiles are continuously created there.
Therefore, this should not be a problem of permission, considering that the owner of /var/lib/omnisci folder (omnisci) has the write permission and the logfiles are regularly created.
Lastly, I cannot see any error in initdb.info.
-
And now I run omnisci with the same user of the service, as you suggested:
I got the super detailed log in the standard output, here the last few lines:
2021-10-15T16:38:05.898118 I 330635 0 0 DBHandler.cpp:493 Started in GPU mode 2021-10-15T16:38:05.901606 I 330635 0 0 VulkanPlatform.cpp:156 Vulkan Instance API version: 1.2.162 2021-10-15T16:38:06.023318 1 330635 0 1 MapDServer.cpp:363 heartbeat thread exiting 2021-10-15T16:38:06.023387 I 330635 0 1 MapDServer.cpp:368 Interrupt signal (11) received. Segmentation fault (core dumped) -
Hi @Gianfranco_Campana,
I have the same crash on Ubuntu 20.04 with the 460.91.03 drivers and Vulkan render enabled. switching off the vulkan render and then using the old opengl one is fixing the crash, so everything works.
Anyway, to fix this crash being able to use the Vulkan render, it's necessary to remove the mesa-vulkan-drivers that in ubuntu are installed by default, with this command
sudo apt remove mesa-vulkan-driversor you can et this ENV while running the server if you need the mesa drivers.
export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia_icd.jsonnow runnig the server
bin/omnisci_server --data /mapd_storage/data_test/ --log-severity-clog=DEBUG4 --renderer-use-vulkan-driver=trueeverything works. Let me know if this is working on your install too
Regards, Candido
Please sign in to leave a comment.
Comments
13 comments