Plug-ins, Hosts, Apps,
Hardware, Soundware
Developers
(Brands)
Videos Groups
Whats's in?
Banks & Patches
Download & Upload
Music Search
KVR
   
KVR Forum » MeldaProduction
Thread Read
Downloads - odd behaviour
DarkStar
KVRAF
- profile
- pm
PostPosted: Thu Aug 23, 2012 12:14 am reply with quote
Yesterday I downloaded the v7.02 updates - using Firefox 3.6.xx and DownThemAll

The downloads ran through to about 97-98% then paused. Firefox/DTA thrashed around for as while, using 99% CPU; eventually the downloaded completed.

For example. for the Creative bundle, DTA paused at 97% - it then took over 3 minutes for the remaining 4 MB to be downloaded. For most of that time there was no data transmission, no disk activity, just CPU load.

I tried a couple of big downloads from other sites - they ran OK, no pauses, no heavy CPU load.

I think I recall something similar with previous downloads.
----
DarkStar
Interesting, if true ...
^ Joined: 02 Sep 2003  Member: #8767  Location: Surrey, UK
xplo
KVRer
- profile
- pm
PostPosted: Thu Aug 23, 2012 1:02 am reply with quote
Sounds like you're running some kind of internet security suite that snoops your HTTP traffic and withholds the last few bytes of the download while it runs the downloaded file through its antivirus scans.
Were those other large downloads you tested the same file type? (.exe?) Same Protocol? (HTTP/HTTPS)
^ Joined: 27 Aug 2010  Member: #238333  
DarkStar
KVRAF
- profile
- pm
PostPosted: Thu Aug 23, 2012 1:29 am reply with quote
^^^^ Could be.

NIS 2012 - but I had switched it into "Silent Mode" to prevent it running any background activities and I did not it using the CPU in Task Manager - it was Firefox running at 99%. The other downland files were .zips.
----
DarkStar
Interesting, if true ...
^ Joined: 02 Sep 2003  Member: #8767  Location: Surrey, UK
MeldaProduction
KVRAF
- profile
- pm
- www
PostPosted: Thu Aug 23, 2012 2:24 am reply with quote
Ojojojoj, Norton??? Run away!! I had this personally (because I got it for free Smile ) and it lasted 2 days here... This is a complete CPU overkill, which slows down everything and considers everything "potential threat"... Now I have AVG, and I'm fine Wink. (no advertising, just experience, one needs to be careful about antiviruses, they can do more harm than the viruses Very Happy )
----
Vojtech
MWobbler | MTotalBundle...
^ Joined: 15 Mar 2008  Member: #176122  Location: Czech republic
DarkStar
KVRAF
- profile
- pm
PostPosted: Thu Aug 23, 2012 2:45 am reply with quote
^^^^
Maybe Wink

But "Silent Mode" prevents it doing anything in the background. And this is not my DAW PC (which is not Interwebz-connected) - it's my general-purpose PC (which happens to run a DAW or two anyway).
----
DarkStar
Interesting, if true ...
^ Joined: 02 Sep 2003  Member: #8767  Location: Surrey, UK
xplo
KVRer
- profile
- pm
PostPosted: Thu Aug 23, 2012 3:00 am reply with quote
DarkStar wrote:
^^^^ Could be.

NIS 2012 - but I had switched it into "Silent Mode" to prevent it running any background activities and I did not it using the CPU in Task Manager - it was Firefox running at 99%. The other downland files were .zips.


The way some of these products do the mentioned download stalling is that they intercept and artificially delay the API calls the downloading application send to the OS to read the incoming packets. Some downloaders don't expect that they do not get an answer from their API calls in a specified amount of time and freak out, in your case by crunching at 99% presumably.
^ Joined: 27 Aug 2010  Member: #238333  
MeldaProduction
KVRAF
- profile
- pm
- www
PostPosted: Thu Aug 23, 2012 3:42 am reply with quote
xplo wrote:
DarkStar wrote:
^^^^ Could be.

NIS 2012 - but I had switched it into "Silent Mode" to prevent it running any background activities and I did not it using the CPU in Task Manager - it was Firefox running at 99%. The other downland files were .zips.


The way some of these products do the mentioned download stalling is that they intercept and artificially delay the API calls the downloading application send to the OS to read the incoming packets. Some downloaders don't expect that they do not get an answer from their API calls in a specified amount of time and freak out, in your case by crunching at 99% presumably.


Didn't know that! What's the purpose? I mean, most requests have like 30s time limit, if that exceeds it is considered disconnected...
----
Vojtech
MWobbler | MTotalBundle...
^ Joined: 15 Mar 2008  Member: #176122  Location: Czech republic
xplo
KVRer
- profile
- pm
PostPosted: Thu Aug 23, 2012 4:12 am reply with quote
MeldaProduction wrote:
Didn't know that! What's the purpose? I mean, most requests have like 30s time limit, if that exceeds it is considered disconnected...


What happens is essentially that most downloaders nowadays request a nonblocking socket from the OS to read available data from and do their own timeout handling.
Now when the security suite actually hooks into the API it will prevent the read() call on that nonblocking socket from returning anything at all until it did its background scanning.
The application however expects read() on a nonblocking socket to return immediately, even if no new data has arrived since the last call. If it doesn't, things can go badly.

I'm guessing the reason why this method is preferred by the AV companies over filtering the last packets on the network/packet-level is that they don't need to worry about TCP resends or other automatic lowlevel error correction that may kick in if they simply delay the last packets by using network filter drivers.

Now this is only the basic idea behind it, I'm sure some of these security suites have additional tricks to avoid above scenario.

Of course a more proper way would be making read() act as if the final part of the received file wasn't received yet until after the scans are done. But then they risk that the downloader application decides that the connection is dead after X secs of no new data.
^ Joined: 27 Aug 2010  Member: #238333  
All times are GMT - 8 Hours

Printable version
Page 1 of 1
Display posts from previous:   
ReplyNew TopicPrevious TopicNext Topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Username: Password:  
KVR Developer Challenge 2012