Using eValid as a web crawler tool

Use and application of eValid's site analysis (site scanning) features.

Using eValid as a web crawler tool

Postby EllieD » Thu Aug 15, 2019 7:17 am

Good morning.

Why can't eValid be a basis for web spidering?

I want to index my company's entire website and eValid might be just the tool for the job.

Thanks
EllieD
 
Posts: 1
Joined: Thu Aug 15, 2019 7:12 am

Re: Using eValid as a web crawler tool

Postby eValid » Fri Aug 16, 2019 12:14 pm

EllieD wrote:Good morning.

Why can't eValid be a basis for web spidering?

I want to index my company's entire website and eValid might be just the tool for the job.

Thanks


Interesting question EllieD.

Evidently you want to build some kind of index for a domain, kind of like what the search engines do when their robots visit your pages.

This is possible, but to the fully disclosive, eValid is not efficient enough for this kind of spidering or web crawling.

There are much more efficient ways to generate a searchable table for a website that downloading hundreds or thousands of pages fully, just to extract text and/or to identify static links.

There are many good references on the web that should point you in the right direction.


-- eValid Support
eValid
 
Posts: 1952
Joined: Tue Jan 01, 2008 12:48 pm
Location: USA


Return to Website Scanning and Analysis

Design Downloaded from free phpBB templates | free website templates | Free Web Buttons