New beta 9.3.0 on Linux crashes

[gavindi]

Hi,

I've installed the new beta and upon launch the following error occurs:

Code: Select all

terminate called without an active exception
Killed

Waveform doesn't start.

[gavindi]

Update

I deleted the Waveform9Log.txt file so I could capture a clean log file for you guys (the devs) and now that I've deleted the log file, Waveform9 9.3.0 is starting up.

Problem solved!

[dRowAudio]

Odd, I would have thought you'd need to install curl. No idea why the log file was causing problems. Probably a red herring...

[gavindi]

Odd, I would have thought you'd need to install curl. No idea why the log file was causing problems. Probably a red herring...

Yeah, it may be coincidence. I reinstalled the deb file for 9.2.5 then installed the deb file for 9.3.0 and still no go. Then I reinstalled 9.2.5 again and at that point I thought of capturing the log file for you guys so I reinstalled 9.3.0 again. That's when I deleted the log. So, there was a few reinstalls along the way too.

[gavindi]

I've got home from a long day and booted up Ubuntu 18.04.
Waveform9 is now crashing to the desktop again. Hmmmmmm.
Something is not right here.

Here's the last part of the log file:

Code: Select all

17 Aug 2018 17:18:30  Creating Default Controllers...
17 Aug 2018 17:18:30  Loading Custom Controllers...
17 Aug 2018 17:18:30  MIDI External controller assigned: Axiom A.I.R. Mini32 MIDI 1
17 Aug 2018 17:18:31  Reloading Windows...
17 Aug 2018 17:18:39  !! killing thread by force !!
17 Aug 2018 17:18:39  Crash Location: UnknownLocation
17 Aug 2018 17:18:39  Crashed
17 Aug 2018 17:18:39  

17 Aug 2018 17:18:39  Thread 0:
17 Aug 2018 17:18:39    0: MainWindow.cpp:MainWindow:2935
17 Aug 2018 17:18:39    1: WindowManager.cpp:createNewWindow:280
17 Aug 2018 17:18:39    2: ApplicationStartup.cpp:initialise:449

[gavindi]

Hi,

Although I have the right libcurl installed (and symlinked libcurl.so.4.5.0 -> libcurl.so) I've taken your idea and ran with it.
I can reliably get Waveform9 (9.3.0) to run if I do the following exec:

Code: Select all

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0 Waveform9

SInce I've got the symlink in place, I'm a bit befuddled why it wont work without the LD_PRELOAD.

Cheers,
Gavin

[gavindi]

Some more information:

I ran the following trace:

Code: Select all

strace Waveform9 2>&1| grep '^open.*[0-9]$' | grep -i curl

and got:

Code: Select all

openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libcurl.so", O_RDONLY|O_CLOEXEC) = 6

My symlinks for libcurl are as follows:

Code: Select all

ls -lah /usr/lib/x86_64-linux-gnu/libcurl*
lrwxrwxrwx 1 root root   19 Jul  5 00:18 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.3 -> libcurl-gnutls.so.4
lrwxrwxrwx 1 root root   23 Jul  5 00:18 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 -> libcurl-gnutls.so.4.5.0
-rw-r--r-- 1 root root 499K Jul  5 00:18 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.5.0
lrwxrwxrwx 1 root root   12 Jul  4 20:00 /usr/lib/x86_64-linux-gnu/libcurl.so -> libcurl.so.4
lrwxrwxrwx 1 root root   16 Jul  5 00:18 /usr/lib/x86_64-linux-gnu/libcurl.so.4 -> libcurl.so.4.5.0
-rw-r--r-- 1 root root 507K Jul  5 00:18 /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0

[rhydermike]

I'm getting the same error on Ubuntu 18.04 here too. Back to 9.2.1 for me, for now.

This is the final log output:

1 Sep 2018 8:01:07 Initialising ReWire...
1 Sep 2018 8:01:07 Initialising controllers...
1 Sep 2018 8:01:07 Creating Default Controllers...
1 Sep 2018 8:01:07 Loading Custom Controllers...
1 Sep 2018 8:01:08 Reloading Windows...
1 Sep 2018 8:01:17 !! killing thread by force !!
1 Sep 2018 8:01:17 Crash Location: UnknownLocation
1 Sep 2018 8:01:17 Crashed
1 Sep 2018 8:01:17

1 Sep 2018 8:01:17 Thread 0:
1 Sep 2018 8:01:17 0: MainWindow.cpp:MainWindow:2935
1 Sep 2018 8:01:17 1: WindowManager.cpp:createNewWindow:280
1 Sep 2018 8:01:17 2: ApplicationStartup.cpp:initialise:449
1 Sep 2018 8:01:17 Thread 1:
1 Sep 2018 8:01:17 0: PresetLibrary.cpp:runJob:549

[gavindi]

I'm getting the same error on Ubuntu 18.04 here too. Back to 9.2.1 for me, for now.

This is the final log output:

1 Sep 2018 8:01:07 Initialising ReWire...
1 Sep 2018 8:01:07 Initialising controllers...
1 Sep 2018 8:01:07 Creating Default Controllers...
1 Sep 2018 8:01:07 Loading Custom Controllers...
1 Sep 2018 8:01:08 Reloading Windows...
1 Sep 2018 8:01:17 !! killing thread by force !!
1 Sep 2018 8:01:17 Crash Location: UnknownLocation
1 Sep 2018 8:01:17 Crashed
1 Sep 2018 8:01:17

1 Sep 2018 8:01:17 Thread 0:
1 Sep 2018 8:01:17 0: MainWindow.cpp:MainWindow:2935
1 Sep 2018 8:01:17 1: WindowManager.cpp:createNewWindow:280
1 Sep 2018 8:01:17 2: ApplicationStartup.cpp:initialise:449
1 Sep 2018 8:01:17 Thread 1:
1 Sep 2018 8:01:17 0: PresetLibrary.cpp:runJob:549

Have to tried my little LD_PRELOAD trick above?

[rhydermike]

I cut and pasted your load string without success, thanks. I checked and

/usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0

is present.

I'd been using a variation on that technique to load 9.2.1 on Ubuntu 18.04. See: viewtopic.php?f=22&t=504172&p=7070367#p7070367

[dRowAudio]

This could be a race condition on the new libcurl initialisation. I'll try to get a new beta out this week to see if this addresses it.
(It's tricky though as I spent about half an hour simply opening W9 on Linux on Friday and closing it again to try and get it to crash. Must have opened it at least 50 times and not a single crash.)

I'd be really interested to know what is different about our systems. Mine is a fairly new Ubuntu 18.04 download with pretty much just the libs I need to develop installed and curl.

[gavindi]

This could be a race condition on the new libcurl initialisation. I'll try to get a new beta out this week to see if this addresses it.
(It's tricky though as I spent about half an hour simply opening W9 on Linux on Friday and closing it again to try and get it to crash. Must have opened it at least 50 times and not a single crash.)

I'd be really interested to know what is different about our systems. Mine is a fairly new Ubuntu 18.04 download with pretty much just the libs I need to develop installed and curl.

Granted my Ubuntu has been upgraded from previous versions time and time again however it's still rather 'clean'. Maybe it's a race condition and maybe I'm susceptible to race conditions since I'm using an SSD drive - I don't know.

What I don't understand is that libcurl 4.5 is the only version I have installed anyway and I've symlinked libcurl.so to libcurl.so.4.5.0 so it can see the correct shared object:

Code: Select all

lrwxrwxrwx 1 root root     19 Jul  5 00:18 libcurl-gnutls.so.3 -> libcurl-gnutls.so.4
lrwxrwxrwx 1 root root     23 Jul  5 00:18 libcurl-gnutls.so.4 -> libcurl-gnutls.so.4.5.0
-rw-r--r-- 1 root root 510408 Jul  5 00:18 libcurl-gnutls.so.4.5.0
lrwxrwxrwx 1 root root     16 Aug 18 14:59 libcurl.so -> libcurl.so.4.5.0
lrwxrwxrwx 1 root root     16 Jul  5 00:18 libcurl.so.4 -> libcurl.so.4.5.0
-rw-r--r-- 1 root root 518600 Jul  5 00:18 libcurl.so.4.5.0

So in this case, I don't understand why the LD_PRELOAD makes a difference between it loading and not loading since there is no other libcurl installed.

[dRowAudio]

You shouldn't need to preload the library, I imagine that could cause more harm than good...
This is how we load the curl symbols: https://github.com/WeAreROLI/JUCE/blob/ ... etwork.cpp

You'll see we iterate through the libraries like so: "libcurl.so", "libcurl.so.4", "libcurl.so.3". So in your case, it should be resolving that to "libcurl.so.4.5.0" which should work.

The other thing you'll see in that juce link is some lock additions which I added on Friday to try and reduce the number of race conditions. As I said, I'll try and get a build out this week to see if this has resolved the issue as I simply can't replicate it on any of my machines.

[mikoatkvr]

I too have been experiencing the frequent crash concerning Libcurl on Linux Mint-18 here's output of Valgrind :_

==13022== Command: Waveform9
==13022==
==13022== Thread 21 VideoManager:
==13022== Jump to the invalid address stated on the next line
==13022== at 0x12DAC024: ???
==13022== by 0x1FFFFFFFE: ???
==13022== by 0x1DC0E2E7: ???
==13022== by 0x801: ???
==13022== by 0x1202D3B: ??? (in /usr/bin/Waveform9)
==13022== Address 0x12dac024 is not stack'd, malloc'd or (recently) free'd
==13022==
==13022== Jump to the invalid address stated on the next line
==13022== at 0x243C6: ???
==13022== by 0x12DAC030: ??? (in /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2)
==13022== by 0x12DAC147: ??? (in /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2)
==13022== by 0x12DE4D25: gnutls_x509_trust_list_add_trust_file (in /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2)
==13022== by 0x12DE510E: ??? (in /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2)
==13022== by 0x12DE516A: gnutls_x509_trust_list_add_trust_dir (in /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2)
==13022== by 0x12D918D9: gnutls_certificate_set_x509_trust_dir (in /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2)
==13022== by 0x11CA00C9: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0)
==13022== by 0x11CA10BF: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0)
==13022== by 0x11C5E91C: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0)
==13022== by 0x11C6FA80: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0)
==13022== by 0x11C83F3D: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0)
==13022== Address 0x243c6 is not stack'd, malloc'd or (recently) free'd
==13022==
Killed


The trick of preloading the Lib works for me too.

[dRowAudio]

Ok, think I've finally found the cause of this but will take a day or two to get the fix approved and new build up.
The problem stems from libcurl being unloaded if multiple threads are using it at once which will cause a crash.

Will post when a new version is available to verify this fix works.
Thanks for everyone's patience in this.