Greetings!
I am looking to do some real-time remote collaboration. I can't seem to find the right software to meet my needs. What I am looking for is a method to transmit high quality audio over a VPN. Latency is not really a show stopper, for example, 500-1000ms will still functionally meet the requirements.
I have explored a solution using Dante VIA that worked on a local-network but support at Dante stated that it will not work over the VPN which has 40ms latency. I know that VIA requires systems to be on the same network but I can work around that using a datacenter technology called VXLAN.
Can anyone recommend a suitable software? VST is ideal, but not an absolute requirement. The remote end doesn't even have to be using a DAW as long as the audio quality is high.
Thanks,
Daniel
Real-Time Remote Collaboration
-
- KVRer
- Topic Starter
- 6 posts since 17 Jan, 2014
-
- KVRAF
- 7134 posts since 8 Feb, 2003 from London, UK
You've not stated the data transfer rate of the connection - i.e. how many bits per second of data would saturate the connection?
Generally, for bi-directional transfer over a connection, you'd take the lowest speed link in the chain, divide its rate by two to get an "ideal maximum" data rate, then start deciding what you want to use that bandwidth for. Some of it will need to go on other network traffic and the protocol layers you're using to support your communication.
If you want "reliable" data transfer, you'll also have to allow for retransmission overheads (and the resultant hold ups - this is possibly where Dante falls down). Otherwise you'll be accepting an "unreliable" connection where you may well experience drop outs.
When you say "collaboration", also, are you intending anything other than audio? In which case could you be streaming at the highest data rate you can with a low data-rate audio chat?
Generally, for bi-directional transfer over a connection, you'd take the lowest speed link in the chain, divide its rate by two to get an "ideal maximum" data rate, then start deciding what you want to use that bandwidth for. Some of it will need to go on other network traffic and the protocol layers you're using to support your communication.
If you want "reliable" data transfer, you'll also have to allow for retransmission overheads (and the resultant hold ups - this is possibly where Dante falls down). Otherwise you'll be accepting an "unreliable" connection where you may well experience drop outs.
When you say "collaboration", also, are you intending anything other than audio? In which case could you be streaming at the highest data rate you can with a low data-rate audio chat?
-
- KVRer
- Topic Starter
- 6 posts since 17 Jan, 2014
The VPN typically sees 20 Mbps bi-directional throughput. Stereo audio at high quality is the only requirement for this application. I have other applications which meet the chat audio and screen sharing requirements.
-
- KVRAF
- 7134 posts since 8 Feb, 2003 from London, UK
If you could live with even a few seconds delay in preference to not having drop outs, then using NINJAM with a client like Jamtaba2 (or ReaNINJAM if you're using Reaper) is one method. Because it's not trying very hard to be real time, you get better audio. You can set up a private server and have both parties connect to that. The "latency" is then down to the "interval length" (in beats per interval and beats per second). Jamtaba2 is available as a VST and standalone.
For streaming, I've not used a plug-in that does this but I'd hope someone's written one.
All the "real" "real time" protocols I know want to squash the audio as small as possible to get it to arrive as fast as possible and drop buffers to stop falling behind. You sacrifice quality for latency, as you've seen.
For streaming, I've not used a plug-in that does this but I'd hope someone's written one.
All the "real" "real time" protocols I know want to squash the audio as small as possible to get it to arrive as fast as possible and drop buffers to stop falling behind. You sacrifice quality for latency, as you've seen.
-
- KVRer
- Topic Starter
- 6 posts since 17 Jan, 2014
I've used Ninjam in the past, maybe 2 years ago but I found the quality of the compression was too far off of the original. I am looking to collaborate on mixing and sound design.
The latency of Ninjam was acceptable for this use case.
The latency of Ninjam was acceptable for this use case.
-
- Banned
- 1780 posts since 26 Aug, 2012
I heard Protools 12 incorporated that type of technology. I dont know anythin' about it though. I'm still on 11.
-
- KVRer
- Topic Starter
- 6 posts since 17 Jan, 2014
I've seen similar with cubase, we are using multiple DAWs so we would prefer a DAW agnostic solution, but we could potentially switch.Kinh wrote:I heard Protools 12 incorporated that type of technology. I dont know anythin' about it though. I'm still on 11.
-
- KVRAF
- 8823 posts since 6 Jan, 2017 from Outer Space
For jamming, you need real time, and the limitation is the compression you have to put on the streamed audio. Probably won't be much better than with Ninjam...ensigndaniel wrote:I've used Ninjam in the past, maybe 2 years ago but I found the quality of the compression was too far off of the original. I am looking to collaborate on mixing and sound design.
The latency of Ninjam was acceptable for this use case.
Mixing and sound design would not need to stream anything. Just exchange the material non-real time. Sound design in terms of programming synths, just requires to exchange patches...
The latency could always be fixed to a bar, like with Ninjam, but there the collaborators hear and play not the same music. You combine your playing with the collaborators last bar, and vice versa...
-
- KVRAF
- 7134 posts since 8 Feb, 2003 from London, UK
If you need to use a better codec than the one used by default in NINJAM, both the client code (all clients) and the server code are open source, so you could get that done.
Ohm Studio was an early internet collaboration tool that supported shared projects - tied to the same DAW, of course.
@ensigndaniel - you've not really explained the roles of the different parties involved in the collaboration and what facilities they'd need, now I look back. You've made it sound like there's one person with the DAW and one or more people chatting with them -- just you really want to have high quality audio from the DAW. It's not clear what that high quality audio is needed for by those remote participants.
Ohm Studio was an early internet collaboration tool that supported shared projects - tied to the same DAW, of course.
@ensigndaniel - you've not really explained the roles of the different parties involved in the collaboration and what facilities they'd need, now I look back. You've made it sound like there's one person with the DAW and one or more people chatting with them -- just you really want to have high quality audio from the DAW. It's not clear what that high quality audio is needed for by those remote participants.
-
- KVRer
- Topic Starter
- 6 posts since 17 Jan, 2014
It isn't for jamming, but the real time is needed as the work flow of recording, mixing, transferring to shared storage, and waiting for the other party is time consuming in a way that is not effective for me and my collaborator. We are both working professionals and music is a hobby, so when we are going to dedicate time for working together we have to be able to spend that time fully engaged.For jamming, you need real time, and the limitation is the compression you have to put on the streamed audio. Probably won't be much better than with Ninjam...
Mixing and sound design would not need to stream anything. Just exchange the material non-real time. Sound design in terms of programming synths, just requires to exchange patches...
The latency could always be fixed to a bar, like with Ninjam, but there the collaborators hear and play not the same music. You combine your playing with the collaborators last bar, and vice versa...
-
- KVRer
- Topic Starter
- 6 posts since 17 Jan, 2014
You've described the workflow correctly, I thought I had described it, but I can try to be more thorough. I appreciate the efforts to assist me in.If you need to use a better codec than the one used by default in NINJAM, both the client code (all clients) and the server code are open source, so you could get that done.
Ohm Studio was an early internet collaboration tool that supported shared projects - tied to the same DAW, of course.
@ensigndaniel - you've not really explained the roles of the different parties involved in the collaboration and what facilities they'd need, now I look back. You've made it sound like there's one person with the DAW and one or more people chatting with them -- just you really want to have high quality audio from the DAW. It's not clear what that high quality audio is needed for by those remote participants.
Say for example a song or soundtrack is being worked on by party A and and party B. Party A hosts the DAW and audio equipment. Party B is participating in the process via remote presence software such as WebEx, AnyDesk, Nomachine etc. They have a real-time view of the DAW and can participate, take control, etc. The audio with any of these applications is not of acceptable quality for this purpose. I am able to route sound around through Dante VIA or ASIO from hardware to other playback devices, virtual or otherwise, and I can even move this audio to remote systems within the local network of Party A. Party B needs to be able to monitor the DAW audio at the same quality.
-
- KVRAF
- 7134 posts since 8 Feb, 2003 from London, UK
If you want someone moving a control in remote control software to affect what they're hearing in real time, you're not going to get high quality audio, as it's going to be compressed. I was envisaging one person at the controls, the others simply providing feedback, with the first person doing the changes, then hitting play again for further comment. You're not going to be getting a filter slider moved by a remote party and that party hearing the effect at 96000 x 32 bit samples per second (i.e. 3Mb/s) between multiple parties over a 20Mb/s link without a central server - you'll saturate the link. Putting a central server in place will immediately make remote control have a high latency (probably two or more seconds), which means you move the fader and ... ... ... then hear what you did and thing "Ouch!" and react and .. .. .. etc.
To lower latency, you need to go peer to multi-peer - i.e. cut out the middle man - which can cut the latency down do 20plus milliseconds which, whilst better, still "feels odd". But you'll be transmitting compressed audio to achieve that. Compression takes CPU time - higher quality compression takes more CPU time and increases the bandwidth (eating your latency and saturating your connection more quickly). Hence why you end up going for Ogg Vorbis, which Jamulus and NINJAM use.
(Gigabit Ethernet makes this more feasible locally, of course.)
Even when typing over remote control software, you can feel the latency. Imagine that after pressing a key, before you got a response, you had to render the letter as a sound, compress it and despatch it to multiple peers. It's not going to be pleasant...
What software like Ohm Studio do is let you work locally, sending your updates to a central server in the background. This synchronises with updates from other clients updating the same project. Where you add audio to a project, it takes a low quality "thumbnail" and gets that up to the server quickly and out to the other clients, streaming the high quality audio in the background. It lets you keep working together and you can apply effects and hear the result immediately - and others can also, as the change to an effect setting can be exchanged like pressing a key. The rendered audio isn't exchanged.
To lower latency, you need to go peer to multi-peer - i.e. cut out the middle man - which can cut the latency down do 20plus milliseconds which, whilst better, still "feels odd". But you'll be transmitting compressed audio to achieve that. Compression takes CPU time - higher quality compression takes more CPU time and increases the bandwidth (eating your latency and saturating your connection more quickly). Hence why you end up going for Ogg Vorbis, which Jamulus and NINJAM use.
(Gigabit Ethernet makes this more feasible locally, of course.)
Even when typing over remote control software, you can feel the latency. Imagine that after pressing a key, before you got a response, you had to render the letter as a sound, compress it and despatch it to multiple peers. It's not going to be pleasant...
What software like Ohm Studio do is let you work locally, sending your updates to a central server in the background. This synchronises with updates from other clients updating the same project. Where you add audio to a project, it takes a low quality "thumbnail" and gets that up to the server quickly and out to the other clients, streaming the high quality audio in the background. It lets you keep working together and you can apply effects and hear the result immediately - and others can also, as the change to an effect setting can be exchanged like pressing a key. The rendered audio isn't exchanged.
-
- KVRer
- 1 posts since 7 May, 2004 from Houston
AUTO-ADMIN: Non-MP3, WAV, OGG, SoundCloud, YouTube, Vimeo, Twitter and Facebook links in this post have been protected automatically. Once the member reaches 5 posts the links will function as normal.
I came across this thread while researching an article and thought I'd contribute - there's an online course on this subject put on by Stanford that appears to be free:https://online.stanford.edu/courses/soh ... technology (https://online.stanford.edu/courses/sohs-music0001-online-jamming-and-concert-technology)
Full disclosure: it's presented by my former adviser when I was there.
But I have no stake, it's just something I noticed matched your query here Cheers!
-- Aaron
-
- KVRer
- 6 posts since 4 Dec, 2020
You can use koord.live for Real-Time Remote Collaboration. If you are within 500Km from one of the currently available real-time regions you typically get below 30ms of latency. It's super easy to create a private session. No tech skills are required. The VPN will always add a lot of latency to the experience and eventually make it unusable for a proper real-time rehearsal experience.
-
- KVRian
- 670 posts since 7 May, 2002 from Phoenix, Arizona, USA
have you already considered https://sonobus.net/ ?
oops - necro thread!
Didn't know about https://koord.live/ , thanks for posting!
oops - necro thread!
Didn't know about https://koord.live/ , thanks for posting!
