YMMV, but generally speaking exp(x) is faster than pow(x, y).juha_p wrote:WouldCode: Select all
2.0 / (1.0 + 2.71828182845904^(-2.0 * x)) - 1
be equal fast withCode: Select all
2.0 / (1.0 + exp(-2.0 * x)) - 1.0
?
Variable tanh() for saturation
- KVRian
- 519 posts since 12 Apr, 2010 from The Netherlands
-
- KVRian
- 1379 posts since 26 Apr, 2004 from UK
Actually pow usually calls ln and then exp AFAIK.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
Thanks for the info. After digging this a while I found this def:Miles1981 wrote:Actually pow usually calls ln and then exp AFAIK.
Code: Select all
pow(a, b) = exp(log(a) * b)
- KVRian
- 519 posts since 12 Apr, 2010 from The Netherlands
Yeah, that's what I use. The log(a) part is usually a constant, e.g. log(2) or log(10), so you end up with just a single multiplication and exp(x).juha_p wrote:Code: Select all
pow(a, b) = exp(log(a) * b)
- Banned
- Topic Starter
- 4021 posts since 7 Sep, 2002
No, I mean, computation time-wise, these formulas are equivalent to modern CPU:juha_p wrote:If you mean combining two or more different length formulas ... I would do it by switching the 'best' formula by size of the input x:Also note that modern CPUs calculate two parts of the equation in division in parallel, so it's not very useful to make one part shorter than the other.
x=>0:x<=2 (formula 1),
x>2:x<=n (formula 2) and
x>n (formula 3)
1. a/(b*c+d)
2. (a*b+c)/(d*e+f)
While compiler can't optimize these out, CPU performs calculation in out-of-order manner using several arithmetic units.
By the way, branching is a bit expensive, so combining formulas for different ranges may be expensive, too.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
To try understand what you're doing there, I plotted this method for few m values and the 2tanh() version (mentioned in post #1) with m=0.5 (x range [0;20]). Don't know the x range you use there but when m = 0.05, x=16.9 returned 1.000000000000000 (OOCalc accuracy).Aleksey Vaneev wrote:Things turned out to be fabulously simple. Here is the so far "perfect" variable tanh() function. No further search is probably needed.
Code: Select all
inline double tanhm( const double x, const double m ) // "m" in range [0; 1) ...
Does this plot (in range [0:4]) look familiar?
- Banned
- Topic Starter
- 4021 posts since 7 Sep, 2002
Not sure what you are trying to understand, but the plot is correct. The orig_f() function is however just an intermediate attempt, and on your plot it has some discontinuity which is unexpected.juha_p wrote:To try understand what you're doing there, I plotted this method for few m values and the 2tanh() version (mentioned in post #1) with m=0.5 (x range [0;20]). Don't know the x range you use there but when m = 0.05, x=16.9 returned 1.000000000000000 (OOCalc accuracy).Aleksey Vaneev wrote:Things turned out to be fabulously simple. Here is the so far "perfect" variable tanh() function. No further search is probably needed.
Code: Select all
inline double tanhm( const double x, const double m ) // "m" in range [0; 1) ...
Does this plot (in range [0:4]) look familiar?
-
- KVRist
- 51 posts since 16 Mar, 2014
No need to branch, just clamp the argument to min(8.8,max(-8.8,x)) as in my code above. For arguments outside that range tanh(x) is essentially +-1 anyway.Aleksey Vaneev wrote:By the way, branching is a bit expensive, so combining formulas for different ranges may be expensive, too.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
Well, this area of DSP is new to me so, I'm just trying to understand why would you call tanh() with input values as like 100 (your guitar amp overdrive example) or using even higher input values (by other similiar threads here in forum) when the -1:1 range is reached with much much lower input values as martinvicanek confirms.Aleksey Vaneev wrote: Not sure what you are trying to understand, but the plot is correct. The orig_f() function is however just an intermediate attempt, and on your plot it has some discontinuity which is unexpected.
EDIT: Could that discontinuity come from data 'accuracy' ... x's are given as 0.05 steps?
Last edited by juha_p on Sat Jun 25, 2016 2:51 pm, edited 1 time in total.
- Banned
- Topic Starter
- 4021 posts since 7 Sep, 2002
Convenience and efficiency. Branching is not efficient. Clamping is a little more efficient if SSE is used, but it's still 2 instructions on top of the overall formula.juha_p wrote:Well, this area of DSP is new to me so, I'm just trying to understand why would you call tanh() with input values as like 100 (your guitar amp overdrive example) or using even higher input values (by other similiar threads here in forum) when the -1:1 range is reached with much much lower input values as martinvicanek confirms.Aleksey Vaneev wrote: Not sure what you are trying to understand, but the plot is correct. The orig_f() function is however just an intermediate attempt, and on your plot it has some discontinuity which is unexpected.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
How would this
type of system compare with the tanh() based system? Usable/unusable?
EDIT: This (spreadsheet) formula is improved version of the one seen in above image:
, which allows setting slope (1/1 and lower ... are usable values) and adjust y limit. Maybe not very efficient by OP's standards but hopefully usable for someone.
type of system compare with the tanh() based system? Usable/unusable?
EDIT: This (spreadsheet) formula is improved version of the one seen in above image:
Code: Select all
=IF(x>=0;IF(x<=m;x;IF(m+(k*(x-m))>ylim;ylim;m+(k*(x-m))));IF(x>=-m;x;IF(-m+(k*(x--m))<-ylim;-ylim;-m+(k*(x--m)))))
, which allows setting slope (1/1 and lower ... are usable values) and adjust y limit. Maybe not very efficient by OP's standards but hopefully usable for someone.
Last edited by juha_p on Mon Jun 27, 2016 6:42 pm, edited 4 times in total.
-
- KVRian
- 1379 posts since 26 Apr, 2004 from UK
Not great for the pipeline, as Aleksey said.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
Actually, I didn't mean the formula (one seen in plot is just for spreadsheet use and I'm sure it can be improved when brought into implementation) but the shape (linear vs sigmoid).Miles1981 wrote:Not great for the pipeline, as Aleksey said.
-
- KVRian
- 1379 posts since 26 Apr, 2004 from UK
The thing is that you need something like a sigmoid if you want to be efficient. Nothing piecewise linear, because you can't improve it when grought into implementation. There is no way you get rid of the 3 different paths and not trash the pipeline without a cost that is equal or higher to either the trashing or the sigmoid.
-
- KVRist
- 76 posts since 5 Nov, 2015