WEBSITE DESIGN & DIGITAL MARKETING

Menu

Robots, spiders and crawlers

In order to protect our customers privacy, copyright and the performance of our websites, we have a number of automatic and manua systems that actively monitor robots, spiders and crawlers accessing websites that we host, and block those that don't comply with a few common sense rules:

In general, all webcrawlers, robots and spiders will be blocked unless they:
  1. Follow the robots exclusion standard
  2. Properly identify themselves in their user-agent.  Robots that misidentify themselves with a web-browser user agent string (eg ""Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1") or no user agent string at all are generally assumed to be malicious spiders and blocked
  3. Don't attempt to submit forms, crack passwords, engage in web application attacks or display an access pattern consistent with known malicous tools
  4. Don't cause undue load to the website, eg by accessing pages more than twice a second

About Zeald

Zeald was formed in 2001 by three young guys from the small New Zealand town of Mangawhai Heads. Now, Zeald is the largest website design and digital marketing agency in New Zealand and has recently made moves into Australia. This is the Zeald story …

read more
 

Connect With Us

Keep up to date with industry news, stay ahead of the competition with our helpful articles

Google Plus Facebook Youtube Twitter Linkedin
 
 

Contact Us

New Zealand
09 415 7575
Australia
1800 224 032

42 Tawa Dr, Albany, Auckland 0632

Support