Login / Register  0 items | $0.00 New#KVRDeals
Mayae
KVRian
 
520 posts since 1 Jan, 2013, from Denmark

Postby Mayae; Tue Jan 28, 2014 11:19 am Audio Programming Environment 0.3 - x32/x64 on win + mac (vst/au)!

This is something I've been working on for a long time, and to say the least, im pretty thrilled to hear what you guys think about it.

Intro
Image
To put it simple, it's a audio effect that wraps a compiler and a texteditor into your audiostream. To elaborate a bit, it includes a dynamic compiler bridge which theoretically allows you to write in any language so long as it can provide a C bridge. The editor is based on the scintilla scilexer (e: can also use juce codeeditor now), which most of you probably know. Together with these tools and a console, it provides a small API that allows to create controls, metering tools and diagnostics.

APE's intended use is the development process - testing out and fine-tuning algorithms in an efficient manner before integrating it into your primary project. It allows for on-the-fly compilation and testing. Also, it's intended to provide a simple introduction to writing DSP code in a manner where you don't have to worry about anything but the actual relevant code. It is therefore an ideal tool for teaching/educational projects and demonstrating small algorithms and / or effects.

APE currently ships with a C compiler that, together with the API, provides a suite of tools and a system that makes it extremely easy to work with DSP-code. Here's an example 'script':

Code: Select all
 #include <CInterface.h>

 /*******************************************************************
 *                  Limiter
 *            Written by Janus Thorborg   
 * Description:                     
 *       Limiting is basically a special case of
 *      compression, where your ratio is infinite and
 *      your attack is zero.            
 *******************************************************************/

struct PluginData
{
   float threshold; // 0 .. 1
   float env; // envelope follower
   float release; // 0 .. 1000
};

GlobalData("limiter.c");

enum Status onLoad()
{
   // set threshold to 1, this will just make sure nothing clips
   self.threshold = 1.0f;
   // release to 100 ms
   self.release = 0.1;
   // reset env
   self.env = 1;
   // knobs etc
   api.createKnob("Treshold", &self.threshold, knobType.decibel);
   api.createKnob("Release", &self.release, knobType.ms);
   api.createLabel("Envelope", "%f", &self.env);

   // print out starting settings
   api.printLine(color.black, "Limiter loaded. Threshold: %f, release: %f",
         self.threshold,
         self.release);

   return status.ready;
}

enum Status onUnload()
{
   return status.ok;
}


enum Status processReplacing(float ** in, float ** out, int sampleFrames)
{
   // amount of channels were working with
   int numChannels = api.getNumInputs();
   // this is our temporary sample
   float sample;
   // the gain we apply
   float gain;
   // our envelope
   float env = self.env;
   // our threshold
   float threshold = self.threshold;
   // absolute value of sample
   float sampleAbs;
   // how much the envelope decays each sample
   // after the input volume has fallen below threshold
   float decay = exp(-1.0 / (api.getSampleRate() * self.release));
   // main loop
   for(unsigned i = 0; i < sampleFrames; ++i) {
      // loop for each channel
      for(unsigned x = 0; x < numChannels; ++x) {
         // catch the sample
         sample = in[x][i];
         // reset gain
         gain = 1.0;
         // calculate the absolute value of the sample (remove sign)
         sampleAbs = fabs(sample);
         // check if input volume is above our threshold
         if(sampleAbs > threshold) {
            // this is analogous to zero attack (because the envelope is
            // instantly set to the input volume)
            if(sampleAbs > env)
               env = sampleAbs;
         }
         // apply gain only if volume envelope is above zero,
         // else gain spins out of control
         if(env > threshold)
            gain = threshold/env;
         // apply release to envelope if input is falling below threshold
         if(sampleAbs < threshold)
            env *= decay;
         // lastly, gain the output sample!
         out[x][i] =  gain * sample;
      }

   }
   // save the envelope variable
   self.env = env;
   return status.ok;
}


Examples of included scripts
    resampling
    additive synthesis
    oscillators
    oscilloscope
    compressors
    bit- and samplerate reduction
    iir and fir filters, rms
    ringmodulation & tremolo
    pitch detection

APE comes as a complete package without any installers or dependencies - just move the folder into your VSTPlugins folder. Also included is an extensive manual that will teach you everything about the plugin and how to develop compilers for it.

About this release
I originally created this for own use in my education and for messing around in DSP code in a feasible manner. It quickly grew though, and i realized it might have potential / use for other people than me. I therefore continued to develop it to make a usable package for others.

About the code: It's a huge WIP, and to be honest, I'm not the best programmer in the world. Most of the code has to redesigned and rewritten, most have been written with the goal of getting a working product. As of such, it probably has multiple bugs and 'features', but it's now in a state where i want to share it.

APE's goal
I started doing small plugins but quickly realized it really was a hassle going through IDE's, project settings, dependencies and licenses, huge amounts of redundant code and compilations just to create a gain plugin... With this plugin, you can focus on writing that simple line:

Code: Select all
sampleOut *= self.gain;


and try it out instantly with a minimum of effort. So, besides being a tool for efficiently develop algorithms, it's designed to give a gentle introduction to people just entering this field: You have a huge package of example scripts working out of the box, and you can instantly get hands on with the code and try it out.

In the long run, I really hope people enjoy this program as much as i do, and ideally, I hope people will create interesting and cool scripts so this project can develop into a library of DSP code with a working implementation (instead of scattered example/pseudo-code on the internet).

Changelog
Alpha 0.3.2: 23-04-2014
    Missing examples for x64 vst target on OSx
Alpha 0.3.1: 14-04-2014
    Fixed config.cfg's for windows targets; tcc should now find missing headers
    Fixed a crash on host exit (directwrite disabled for now) on windows
Alpha 0.3.0: 08-04-2014
    Source code rewritten to support JUCE also, which is the primary target platform now.
      this affects several things; notably the editor is switched from Scintilla to JUCE's inbuilt code editor now
      this has the welcome side effect of hotkeys working again
      syntax highlight only for C++ and friends for the moment.
    x32 and x64 builds on both Windows and OSx as AudioUnit and VST 2.4
    Countless bug fixes / code rewrite
    project recall now implemented
    autosave now implemented
    support for high dpi display
    an actual threading- and multi-instance model is now implemented; it should be completely safe to run multiple plugins in the same or other processes
    fix of fpu exceptions
    improved header support for other compilers than tcc
    improved graphics on some points, degraded on others

Alpha 0.2: 10-02-2014
    fixed uninitialized variable 'Engine::clocksPerSample'
    scilexer now properly adds filenames to project struct even in case of singleString-compilation
    scilexer now properly sets amount of files in the project
    the console should now properly print strings with linebreaks in them, this affects the core, api and scripts.
    fixed a bug where newlines will crash the console code.
    output logging of console now properly contains newlines.
    due to larger amounts of info being printed to the console, it is now scrollable and has a longer history
    added new compiler: syswrap. syswrap allows to interface to installed system compilers.
    fixed a bug where closing the editor would not reset the editor button in APE
    pressing the editor button now properly restores the window if user had minimized it before
    fixed a memory leak in the TCC compiler (early return caused no deallocation of plugin data)
    fixed a wrong return value in CInterface.h
    added a new knob function: api.createRangeKnob(). This knob formats it's display value based off a minimum, maximum and a callback function.
    fixed a bug where knobs initially would have the wrong format
    to enhance c++ compability, 'this' is now an illegal identifier
    CInterface.h:
      added new valuestruct: scale
      added f_mod() and f_sin()
      added pi values
      compatized header with various compilers
      updated the example scripts to reflect these changes.
Alpha 0.1
    Initial release.

What's new?
The 0.3 release marks a complete overhaul of the program. Notably crossplatform support with both 32-bit and 64-bit, but also focus on making APE into a working plugin. Features that support this are project recall mostly, but improved saving and autosave is probably a welcomed feature.
Vast amounts of bugs have been eliminated, mostly cause of using proper threading safety and static data security. However, I'm still new to developing with both JUCE and OSx, so dont expect it to be bug free :) I will continue to work on it, of course.

I hope you enjoy this version, it for sure took a while to create.

Known bugs / peculiarities / missing stuff
    The transition to JUCE has brought along a pretty massive CPU load when rendering the graphics (mostly due to some weird osx redrawing of png's). This can be remedied by modifying the config.cfg. I'm trying to fix it, though. It is only a problem when the GUI is showing
    Some 32-bit hosts on OSX have shown some hanging behaviour on exit, however it might be the hosts - unfortunately, i dont have a suite of 32-bit hosts to test the plugin with on osx
    syswrap is not yet available on OSx, I'm working on it.
    TCC has a hard time coping with system headers on OSX, if you experience problems prefix system includes with tcc: #include <tcc/stdio.h>
    Legacy OSx support: I only have 10.8 atm, but it should work for 10.7 (i think? maybe lower) - im keeping an eye on it
    If ape is initiated, but the editor never is opened, project recall will cause the message "Cannot open file 'Untitled'" to appear.


Last words
It could be so cool to create a community project out of this, as of such all of my code is released open-source under GPL 3 (and the whole package is free, of course) - it's in the package. I'm inviting everyone to contribute to the project with ideas / code examples for the library, whatever really. Helping out writing on the plugin itself is very welcomed, I don't have much time myself (full time studies, other projects etc.).

I hope you take some time to check out the project :)

Notes
I'm not sure if this is the right forum (or if you're even allowed to post your plugins), but this is IMO definitely where i think it belongs (as a tool for DSP and plugin development) - but please move it or contact me if anything is wrong.

Download link
Version 0.3 download link (Mac OSX + Windows 32/64bit VST + AudioUnit)
Version 0.2 download link (Windows 32-bit VST)
Version 0.1 download link (Windows 32-bit VST)
Last edited by Mayae on Tue Apr 22, 2014 11:32 pm, edited 9 times in total.
mystran
KVRAF
 
4611 posts since 11 Feb, 2006, from Helsinki, Finland

Postby mystran; Tue Jan 28, 2014 11:35 am Re: Audio Programming Environment - a small IDE inside your

The forum is certainly correct and I suspect there's a bunch of us that have been wishing for such a tool, but haven't found the time to write one.

I wonder, did you use some off-the-shelf compiler or did you write your own from scratch? More specifically, what kind of optimizations one can expect, or can one interface to a system compiler?
Image <- plugins | forum
arakula
KVRAF
 
1940 posts since 16 Aug, 2004, from Vienna, Austria

Postby arakula; Tue Jan 28, 2014 11:43 am Re: Audio Programming Environment - a small IDE inside your

TCC

Nice documentation, BTW!
"Until you spread your wings, you'll have no idea how far you can walk." Image
Mayae
KVRian
 
520 posts since 1 Jan, 2013, from Denmark

Postby Mayae; Tue Jan 28, 2014 11:48 am Re: Audio Programming Environment - a small IDE inside your

mystran wrote:The forum is certainly correct and I suspect there's a bunch of us that have been wishing for such a tool, but haven't found the time to write one.

I wonder, did you use some off-the-shelf compiler or did you write your own from scratch? More specifically, what kind of optimizations one can expect, or can one interface to a system compiler?

Well, i was unemployed last year, so...

The included compiler uses TCC, so no, dont expect huge optimizations (i've looked at the assembly). It is possible to write inline assembly, though. It does the job, for now, though.

The thing is, the compilers (in the included case, a wrapper around libtcc) is loaded dynamically based on settings - while it's certainly very easy to write a wrapper for a compiler creating code in-memory (headers and guidelines included), it's possible write wrappercompilers, that supports 'system'-compilers.

With that said, i've been looking for a compiler ala GCC that compiles directly into executable memory (so you dont have to work with seperate processes).

arakula wrote:TCC

Nice documentation, BTW!


Thanks! :)
arakula
KVRAF
 
1940 posts since 16 Aug, 2004, from Vienna, Austria

Postby arakula; Tue Jan 28, 2014 12:20 pm Re: Audio Programming Environment - a small IDE inside your

First impression: nice. Especially nice that it runs on XP, too.

Little bug: if I close the editor window directly, the APE "editor" button stays down; to get the editor window again, I have to click on it twice.
"Until you spread your wings, you'll have no idea how far you can walk." Image
mystran
KVRAF
 
4611 posts since 11 Feb, 2006, from Helsinki, Finland

Postby mystran; Tue Jan 28, 2014 12:34 pm Re: Audio Programming Environment - a small IDE inside your

Yeah, TCC is single pass, but I do understand your choice, just wondering. ;)

[sadly it compiles to x87 so one is stuck with all the denormal handling :(]

One interesting possibility also be convincing TCC to link in an external .lib for including optimized primitives for the calculation heavy stuff (eg. FIR processing code, FFTs, matrix factorization etc.. stuff that useful as stock building blocks and more performance critical than the 90% glue-code one usually has).

I'll take a closer look in a moment and see if I can make some cool test-cases.
Image <- plugins | forum
mystran
KVRAF
 
4611 posts since 11 Feb, 2006, from Helsinki, Finland

Postby mystran; Tue Jan 28, 2014 1:09 pm Re: Audio Programming Environment - a small IDE inside your

Reading through the docs, the knob type mapping could use a tiny bit more flexibility.

Basically, most of the typical cases for continuous controls would be covered if one could specify upper/lower limits (eg print from -20dB to +20dB) and whether to use linear or exponential curve (eg. 20Hz to 20kHz exponential curve). A simple rule to convert [0,1] to an exponential range is simply vMin*pow(vMax/vMin, value) and this can cover a lot of typical situations. Right now I don't see the .ms and .hertz types as very useful and the decibel type is quite limited by the fixed range.

edit: otherwise seems reasonable enough :)
Image <- plugins | forum
deastman
KVRAF
 
6474 posts since 6 Aug, 2003, from San Francisco Bay Area

Postby deastman; Tue Jan 28, 2014 1:28 pm Re: Audio Programming Environment - a small IDE inside your

Very nice work! I'll definitely take a look as soon as I have time.
Incomplete list of my gear: 110V AC to 12V DC 1.5A power supply (+ tip)
arakula
KVRAF
 
1940 posts since 16 Aug, 2004, from Vienna, Austria

Postby arakula; Tue Jan 28, 2014 1:34 pm Re: Audio Programming Environment - a small IDE inside your

Second impression: not so nice that it uses language constructs and headers that aren't available in Visual Studio 2010 and earlier versions. Makes it impossible to generate APE on an XP machine.
"Until you spread your wings, you'll have no idea how far you can walk." Image
Mayae
KVRian
 
520 posts since 1 Jan, 2013, from Denmark

Postby Mayae; Tue Jan 28, 2014 2:11 pm Re: Audio Programming Environment - a small IDE inside your

arakula wrote:First impression: nice. Especially nice that it runs on XP, too.

Little bug: if I close the editor window directly, the APE "editor" button stays down; to get the editor window again, I have to click on it twice.


So glad to hear! I'll fix that.

mystran wrote:Yeah, TCC is single pass, but I do understand your choice, just wondering. ;)

[sadly it compiles to x87 so one is stuck with all the denormal handling :(]

One interesting possibility also be convincing TCC to link in an external .lib for including optimized primitives for the calculation heavy stuff (eg. FIR processing code, FFTs, matrix factorization etc.. stuff that useful as stock building blocks and more performance critical than the 90% glue-code one usually has).

I'll take a closer look in a moment and see if I can make some cool test-cases.


Yeah, right now it's mostly proof-of-concept. Possibilities are endless, though! As said, it's just a matter of writing a wrapper around a better compiler :)

I did consider that - I was thinking about making it a part of the API (thereby making it language-independant), but the other thing is surely possible too.

mystran wrote:Reading through the docs, the knob type mapping could use a tiny bit more flexibility.

Basically, most of the typical cases for continuous controls would be covered if one could specify upper/lower limits (eg print from -20dB to +20dB) and whether to use linear or exponential curve (eg. 20Hz to 20kHz exponential curve). A simple rule to convert [0,1] to an exponential range is simply vMin*pow(vMax/vMin, value) and this can cover a lot of typical situations. Right now I don't see the .ms and .hertz types as very useful and the decibel type is quite limited by the fixed range.

edit: otherwise seems reasonable enough :)


Yes, it is limited, i just couldn't think of a bright way to implement it so i decided just to wait. Because of that, tho, there's two way of making your own controls (by index-labels or formatting the string manually using control events (not very convenient though, but possibly!).

deastman wrote:Very nice work! I'll definitely take a look as soon as I have time.
Please do!

arakula wrote:Second impression: not so nice that it uses language constructs and headers that aren't available in Visual Studio 2010 and earlier versions. Makes it impossible to generate APE on an XP machine.

Are we talking about the source code? That is true to some extend, however most of it can be reverted. I really do think that the newer C++ constructs like standardized threading is really convinient. It is not necessary, however.
User avatar
Bronto Scorpio
KVRAF
 
5563 posts since 13 Feb, 2006, from Wiesmoor, Germany

Postby Bronto Scorpio; Tue Jan 28, 2014 2:20 pm Re: Audio Programming Environment - a small IDE inside your

Wow! This is really nice! Great work!
arakula
KVRAF
 
1940 posts since 16 Aug, 2004, from Vienna, Austria

Postby arakula; Tue Jan 28, 2014 9:51 pm Re: Audio Programming Environment - a small IDE inside your

Mayae wrote:
arakula wrote:Second impression: not so nice that it uses language constructs and headers that aren't available in Visual Studio 2010 and earlier versions. Makes it impossible to generate APE on an XP machine.

Are we talking about the source code? That is true to some extend, however most of it can be reverted. I really do think that the newer C++ constructs like standardized threading is really convinient. It is not necessary, however.

Yes, that's precisely what I mean. I tried to compile it in VS2010, and after a bit of rewriting the project files, the compiler complained in APE about some
Code: Select all
#include <thread>
#include <chrono>
and associated threading / timing code which definitely isn't available before VS2012 and some code areas looking like
Code: Select all
      auto copy_lines = lines;
      for(auto line : copy_lines) {
- this "auto" stuff is a construct that isn't available in earlier versions.
I agree that these are nice and convenient features, though.

Tcc4APE, however, could be compiled in VS2010 (again after having adapted the project files a bit, of course).
"Until you spread your wings, you'll have no idea how far you can walk." Image
Mayae
KVRian
 
520 posts since 1 Jan, 2013, from Denmark

Postby Mayae; Wed Jan 29, 2014 12:24 am Re: Audio Programming Environment - a small IDE inside your

Bronto Scorpio wrote:Wow! This is really nice! Great work!

Hey, thanks a lot!

arakula wrote:
Mayae wrote:
arakula wrote:Second impression: not so nice that it uses language constructs and headers that aren't available in Visual Studio 2010 and earlier versions. Makes it impossible to generate APE on an XP machine.

Are we talking about the source code? That is true to some extend, however most of it can be reverted. I really do think that the newer C++ constructs like standardized threading is really convinient. It is not necessary, however.

Yes, that's precisely what I mean. I tried to compile it in VS2010, and after a bit of rewriting the project files, the compiler complained in APE about some
Code: Select all
#include <thread>
#include <chrono>
and associated threading / timing code which definitely isn't available before VS2012 and some code areas looking like
Code: Select all
      auto copy_lines = lines;
      for(auto line : copy_lines) {
- this "auto" stuff is a construct that isn't available in earlier versions.
I agree that these are nice and convenient features, though.

Tcc4APE, however, could be compiled in VS2010 (again after having adapted the project files a bit, of course).

auto is a standard keyword.
this construct:
Code: Select all
for(auto line : copy_lines)

can be directly translated into either
Code: Select all
for each(auto line in copy_lines) (vs 2010+ specific)
or
for(auto it = copy_lines.begin(); it != copy_lines.end(); it++)
auto line = *(it);

They were convenient because of being typeless, while i changed alot of the types around.
<chrono> is afaik only used in misc::delay as a standardized sleep() function. <thread> is used for threading two places (event handler in the gui and asynchronous messageboxes) - you should be able to replace it directly with a createthread call.

e: i will consider rewriting it to conform to C++98.
arakula
KVRAF
 
1940 posts since 16 Aug, 2004, from Vienna, Austria

Postby arakula; Wed Jan 29, 2014 2:09 am Re: Audio Programming Environment - a small IDE inside your

Mayae wrote:e: i will consider rewriting it to conform to C++98.

If you put it on Github, I could as well create an "Oldtimer" fork 8-)
"Until you spread your wings, you'll have no idea how far you can walk." Image
Doogle
KVRist
 
129 posts since 20 Apr, 2004, from Brighton, UK

Postby Doogle; Wed Jan 29, 2014 3:08 am Re: Audio Programming Environment - a small IDE inside your

This looks ace and I'll be giving it a go soon.

Just wanted to say though.... if you wanted to move away from TCC, take a look at clang, part of the LLVM project. It takes up a bit more disk space, but it'll build SSE* code and can do auto-vectorisation. The entire LLVM system is a joy to use and incredibly powerful, whether being used as command line tools or integrated libraries.
We can conclude that the DCT of a pizza doesn’t resemble anything edible.
Next

Moderator: Moderators (Main)

Return to DSP and Plug-in Development