How can I scrape text from a web page that is generated by javascript?

[Mike Kerner]

I always start my scrapes manually, so I am no help there with this
plugin.  They all take many hours, so I'm not starting them constantly.

On Mon, Jul 24, 2017 at 5:03 PM, Roger Eller via use-livecode <
use-livecode at lists.runrev.com> wrote:

> So... I can make it scrape, then save to a CSV, then read the CSV into LC
> and use the data.  But how to automate the scraping and CSV saving at a
> given interval, I'm not there yet.
>
> ~Roger
>
>
> On Mon, Jul 24, 2017 at 4:53 PM, Mike Kerner via use-livecode <
> use-livecode at lists.runrev.com> wrote:
>
> > The developer was also helpful when I was running into difficulty writing
> > some of the parameter sets.
> >
> > On Mon, Jul 24, 2017 at 4:27 PM, Roger Eller via use-livecode <
> > use-livecode at lists.runrev.com> wrote:
> >
> > > Scratch that.  It's amazing what watching the video or reading the docs
> > can
> > > do to enlighten the unenlightened.
> > >
> > >
> > > On Mon, Jul 24, 2017 at 2:59 PM, Roger Eller <
> > roger.e.eller at sealedair.com>
> > > wrote:
> > >
> > > > I installed the plugin, but am unfamiliar with how to use it.  I
> didn't
> > > > see an obvious way to filter the page down to the current prices I
> want
> > > to
> > > > grab.
> > > >
> > > > ~Roger
> > > >
> > > > On Mon, Jul 24, 2017 at 1:34 PM, Mike Kerner via use-livecode <
> > > > use-livecode at lists.runrev.com> wrote:
> > > >
> > > >> Don't know what to tell you.  I scrape a few hundred thousand pages
> > at a
> > > >> time from multiple sites with this plugin.
> > > >>
> > > >> On Mon, Jul 24, 2017 at 12:12 PM, Bob Sneidar via use-livecode <
> > > >> use-livecode at lists.runrev.com> wrote:
> > > >>
> > > >> > No workie. I create a new sitemap with a URL of
> http://192.168.1.87
> > > and
> > > >> > it tells me it's not a valid URL!!! That is strike 3 for me when I
> > > >> > encounter that kind of problem. Even if I load the full url
> default
> > > page
> > > >> > for that device, then copy paste the URL it STILL tells me it's
> > > invalid.
> > > >> >
> > > >> > Bob S
> > > >> >
> > > >> >
> > > >> > > On Jul 24, 2017, at 06:13 , Mike Kerner via use-livecode <
> > > >> > use-livecode at lists.runrev.com> wrote:
> > > >> > >
> > > >> > > Roger,
> > > >> > > I use a chrome plugin called "Web Scraper", and then I process
> the
> > > >> CSV in
> > > >> > > LC - then again, I'm usually scraping many many many pages, and
> > this
> > > >> > > automates the process.
> > > >> > > _______________________________________________
> > > >> > > use-livecode mailing list
> > > >> > > use-livecode at lists.runrev.com
> > > >> > > Please visit this url to subscribe, unsubscribe and manage your
> > > >> > subscription preferences:
> > > >> > > http://lists.runrev.com/mailman/listinfo/use-livecode
> > > >> >
> > > >> >
> > > >> > _______________________________________________
> > > >> > use-livecode mailing list
> > > >> > use-livecode at lists.runrev.com
> > > >> > Please visit this url to subscribe, unsubscribe and manage your
> > > >> > subscription preferences:
> > > >> > http://lists.runrev.com/mailman/listinfo/use-livecode
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> On the first day, God created the heavens and the Earth
> > > >> On the second day, God created the oceans.
> > > >> On the third day, God put the animals on hold for a few hours,
> > > >>    and did a little diving.
> > > >> And God said, "This is good."
> > > >> _______________________________________________
> > > >> use-livecode mailing list
> > > >> use-livecode at lists.runrev.com
> > > >> Please visit this url to subscribe, unsubscribe and manage your
> > > >> subscription preferences:
> > > >> http://lists.runrev.com/mailman/listinfo/use-livecode
> > > >>
> > > >
> > > >
> > > _______________________________________________
> > > use-livecode mailing list
> > > use-livecode at lists.runrev.com
> > > Please visit this url to subscribe, unsubscribe and manage your
> > > subscription preferences:
> > > http://lists.runrev.com/mailman/listinfo/use-livecode
> > >
> >
> >
> >
> > --
> > On the first day, God created the heavens and the Earth
> > On the second day, God created the oceans.
> > On the third day, God put the animals on hold for a few hours,
> >    and did a little diving.
> > And God said, "This is good."
> > _______________________________________________
> > use-livecode mailing list
> > use-livecode at lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
> >
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



-- 
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."