Speeding up get URL
shari at gypsyware.com
Mon Aug 4 02:52:07 CDT 2008
Good suggestions, Sarah. Thank you! I've settled on a solution
that's going to partly go in the back door (retrieving their XML data
via their affiliate door) and partly go in the front door (get or
load url). So I'll parse what I can from their affiliate XML files
and do the rest the other way. That should cut down on the load to
them, speed it up, and still give me all the data I'm seeking. It
should cut the pages for get/load url down to a small percentage of
the total retrieve.
I'm excited for the end result as this is going to really be
beneficial to my business :-)
Two more days to hopefully find out if I sold Herman Silver on their
own copy of Rev... It kills me to do things in Excel there that could
be done sooo much better with Rev... Excel is so limiting when you
have too many sheets tying in to other sheets I've discovered. And
all day long I'm thinking gee, this would be so much better with
>Shari, I think the delay will be due to the connection to the server,
>not your script, so there may not be a lot you can do about it.
>I did have one idea: can you try getting more than one URL at the same
>time? If you build a list of the URLs to check, then have a script
>that grabs the first one on the list, and sends a non-blocking request
>to that site, with a message to call when the data has all arrived.
>While waiting, start loading the next site and so on. Bookmark
>checking software seems to work like this.
>Would opening a socket and reading from the socket be any faster? I
>don't imagine that it would be, but it might be worth checking.
>The other option is just to adjust things so it is not intrusive e.g.
>have it download the sites overnight and save them all for processing
>when you are ready, or have a background app that does the downloading
>slowly (so it doesn't overload your system).
Dogs and bears, sports and cars, and patriots t-shirts
WlND0WS and MAClNT0SH shareware games
More information about the use-livecode