Variable tanh() for saturation
-
- KVRian
- 1379 posts since 26 Apr, 2004 from UK
That's the definition of tanh() written differently, right? It should be equal to tanh() up to machine precision.
But Aleksey said that his version is faster than taking an exp().
But Aleksey said that his version is faster than taking an exp().
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
Yes, that definition is derived from tanh : logistic function relation.Miles1981 wrote:That's the definition of tanh() written differently, right? It should be equal to tanh() up to machine precision.
But Aleksey said that his version is faster than taking an exp().
Is speed equal with tanh() as well?
Edit: But don't OP's final method include a tanh() call?
-
Another rational approximation to tanh(x) for speed/accuracy comparisons:
Code: Select all
function ri_tanh(x){
return (-.67436811832e-5+(.2468149110712040+(.583691066395175e-1+.3357335044280075e-1*x)*x)*x) / (.2464845986383725+(.609347197060491e-1+(.1086202599228572+.2874707922475963e-1*x)*x)*x);
}
Error comparison between ri_tanh() and fasttanh_ultra2() (against tanh())
http://oi64.tinypic.com/1zex3q.jpg
Last edited by juha_p on Sat Jun 25, 2016 9:44 am, edited 8 times in total.
- KVRian
- 519 posts since 12 Apr, 2010 from The Netherlands
While we are sharing tanh approximations, I mostly use this:
https://varietyofsound.wordpress.com/20 ... -fraction/
https://varietyofsound.wordpress.com/20 ... -fraction/
-
- KVRian
- 1379 posts since 26 Apr, 2004 from UK
It could be twice as fast, depending on the implementation.juha_p wrote:Is speed equal with tanh() as well?It should be equal to tanh() up to machine precision.
- Banned
- Topic Starter
- 4021 posts since 7 Sep, 2002
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
That R. Israel's design (ri_tanh()) isn't adjusted for that type of use (as you see the error growth is quite linear above certain point between 3.5 and 4).Aleksey Vaneev wrote: I would also do a comparison in the range up to 100, because guitar amp overdrive can go there easily.
Accuracy of Tale's fast_tanh()
Code: Select all
((((x^2 + 378.0) * x^2 + 17325.0)*x^2 + 135135.0) * x) / (((28.0 * x^2+3150.0)*x^2+62370.0)*x^2+135135.0)
BTW, did you try if replacing the tanh() call (in your method tanhm()) by the tanh() defined through logistic function relation (function one_exp_tanh() I posted earlier) gives any speed increase (as Miles1981's conception is) ?
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
By spreadsheet I'm using for those plots, tanh( 17.8 ) returns 1.0000000000000000 and all the higher values returns 1.0 as well (OO Calc uses 16 digits precision ... higher precision may change the point where 1.0 is exceeded) ... so, doesn't this mean you're playing with the error of your own tanh() implementation when calling it with values out of this range (i.e. shouldn't the guitar amp overdrive be implemented differently or are you talking 'bout some other value used in your implementation ?)?Aleksey Vaneev wrote: I would also do a comparison in the range up to 100, because guitar amp overdrive can go there easily.
Based on OOCalc results, if the range overdrive uses is x = +/-100 as you mentioned and the usable tanh() range is +/- 1.0 which is reached by x = +/-17.8 then shouldn't you need to scale the x using 0.178x ... (for tanh() call)?
Last edited by juha_p on Thu Jun 23, 2016 1:34 pm, edited 3 times in total.
-
- KVRist
- 51 posts since 16 Mar, 2014
Rational functions approximate tanh(x) pretty well. For instance this one has an error < 2e-7 and does not blow up for large x:
Code: Select all
streamin x;
streamout tanhx;
float a1=1272167.74 ;
float a2=170307.8954;
float a3=4456.746782;
float a4=26.39563706;
float a5=0.017267359;
float b1=1272167.842;
float b2=594363.4418;
float b3=32955.97311;
float b4=419.6573937;
float xmax=8.8;
float xmin=-8.8;
/*
a1*x + a2*x^3 + a3*x^5 + a4*x^7 + a5*x^9
----------------------------------------
b1 + b2*x^2 + b3*x^4 + b4*x^6 + x^8
*/
movaps xmm2,x;
minps xmm2,xmax;
maxps xmm2,xmin;
movaps xmm1,xmm2;
mulps xmm2,xmm2;
movaps xmm0,a5;
mulps xmm0,xmm2;
addps xmm0,a4;
mulps xmm0,xmm2;
addps xmm0,a3;
mulps xmm0,xmm2;
addps xmm0,a2;
mulps xmm0,xmm2;
addps xmm0,a1;
mulps xmm0,xmm1;
movaps xmm3,xmm2;
addps xmm3,b4;
mulps xmm3,xmm2;
addps xmm3,b3;
mulps xmm3,xmm2;
addps xmm3,b2;
mulps xmm3,xmm2;
addps xmm3,b1;
divps xmm0,xmm3;
movaps tanhx,xmm0;
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
But, is this faster than exp() based solution?martinvicanek wrote:Rational functions approximate tanh(x) pretty well. For instance this one has an error < 2e-7 and does not blow up for large x:Code: Select all
streamin x; streamout tanhx; float a1=1272167.74 ; float a2=170307.8954; float a3=4456.746782; float a4=26.39563706; float a5=0.017267359; float b1=1272167.842; float b2=594363.4418; float b3=32955.97311; float b4=419.6573937; float xmax=8.8; float xmin=-8.8; /* a1*x + a2*x^3 + a3*x^5 + a4*x^7 + a5*x^9 ---------------------------------------- b1 + b2*x^2 + b3*x^4 + b4*x^6 + x^8 */ movaps xmm2,x; minps xmm2,xmax; maxps xmm2,xmin; movaps xmm1,xmm2; mulps xmm2,xmm2; movaps xmm0,a5; mulps xmm0,xmm2; addps xmm0,a4; mulps xmm0,xmm2; addps xmm0,a3; mulps xmm0,xmm2; addps xmm0,a2; mulps xmm0,xmm2; addps xmm0,a1; mulps xmm0,xmm1; movaps xmm3,xmm2; addps xmm3,b4; mulps xmm3,xmm2; addps xmm3,b3; mulps xmm3,xmm2; addps xmm3,b2; mulps xmm3,xmm2; addps xmm3,b1; divps xmm0,xmm3; movaps tanhx,xmm0;
-
- KVRist
- 51 posts since 16 Mar, 2014
On my system the rational approximation is significantly faster than exp. You can trade speed for accuracy though. For a saturator you could get away with fewer terms and still have reasonable accuracy.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
Would
be equal fast with
?
Code: Select all
2.0 / (1.0 + 2.71828182845904^(-2.0 * x)) - 1
be equal fast with
Code: Select all
2.0 / (1.0 + exp(-2.0 * x)) - 1.0
?
Last edited by juha_p on Thu Jun 23, 2016 10:12 am, edited 1 time in total.
-
- KVRian
- 1379 posts since 26 Apr, 2004 from UK
It all depends on the order of the polynomial and the CPU. Aleksey version is faster than tanh on his boxes, I saw that any approximation of exp is slower than exp on mine. It's a compromise.martinvicanek wrote:On my system the rational approximation is significantly faster than exp. You can trade speed for accuracy though. For a saturator you could get away with fewer terms and still have reasonable accuracy.
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
This implementation is excellent up to 10 as input value but after that 'error' starts to grow quite fast. Antto's formula from this forum:martinvicanek wrote:Rational functions approximate tanh(x) pretty well. For instance this one has an error < 2e-7 and does not blow up for large x:
...
Code: Select all
(1.0 - 1.0 / (1.0 + x + x^2 + 0.58576695 * x^3 + 0.55442112 * x^4 + 0.057481508 * x^7));
http://oi63.tinypic.com/9zyold.jpg
Combining these two ...
Last edited by juha_p on Sat Jun 25, 2016 9:45 am, edited 1 time in total.
- Banned
- Topic Starter
- 4021 posts since 7 Sep, 2002
-
- KVRian
- 832 posts since 21 Feb, 2006 from FI
After reading this reply: http://www.kvraudio.com/forum/viewtopic ... 0&start=52 , I assume that the reason for those 'spikes' seen in linked plots relates with the error. Correct if I'm wrong.Aleksey Vaneev wrote:It's not very correct to look at absolute difference with tanh(). Relative difference is more important.
If you mean combining two or more different length formulas ... I would do it by switching the 'best' formula by size of the input x:Also note that modern CPUs calculate two parts of the equation in division in parallel, so it's not very useful to make one part shorter than the other.
x=>0:x<=2 (formula 1),
x>2:x<=n (formula 2) and
x>n (formula 3)
Tried this in OOCalc and got quite good results against original tanh() in input range 0 - 100 (though, in OOCalc it looks like 0 - ~17.8 is the 'usable' input range for tanh()).
http://oi66.tinypic.com/1opug8.jpg
Last edited by juha_p on Sat Jun 25, 2016 9:46 am, edited 2 times in total.