[olug] Spider

[Sean Edwards]

I do not know of any Spider software, but dynamic
content is something of which you need to be aware.

Unless you maintain some control or influence over the
data you are trying to collect (internal web servers,
intranet, corporate web ring, etc), you may not get
the results you expect.

(from
http://www.webmasterworld.com/forum48/604-1-10.htm)
An example is using php to pass a variable to
javacript to display eMail addresses:

<?php $email="email at email.com"; echo $jscript;?>

In the above example, the contents of $email is
generated by PHP when the page is accessed, $jscript
would read the $email variable and combine it with the
javascript.  The contents then only displays within a
client that can execute javascript.

Here is another javascript example (from 
http://www.js-x.com/javascript/?view=159)

<script type=text/javascript>
var _u   = "someone";
var _d = "somewhere.com";
var _l   = _u + "@" + _d;
var _m = "Click Here to E-mail Me";
document.write("<a href='mailto:"+_l+"'>"+_m+"</a>");
</script>

If you are trying to collect data from sites on the
internet, there is no guarantee that the data you are
trying to collect is displayed in a manner which is 
retreivable with Spider software.  Chances are, the
Spider software will need to have some robust browser
client functionality to get data displayed from dyamic
content engines such as ColdFusion, JavaScript, and
Flex.  The first place I would go to search for such a
thing would be in the PERL world.

-=Sean Edwards=-
cybersean3000 at yahoo.com

--- John Dickson <jman at neonramp.com> wrote:

> Does anyone have any suggestions on a Spider server
> that collects site (web servers) information on
> predefined criteria? By that I mean I would like to
> have a server hit sites to look for terms or collect
> terms and filter later.
> 
> John
> > _______________________________________________
> OLUG mailing list
> OLUG at olug.org
> http://lists.olug.org/mailman/listinfo/olug
> 



	
		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail