WEBSITE DEVELOPMENT & RADICAL DIGITAL TRANSFORMATION

Menu

Robots, spiders and crawlers

In order to protect our customers privacy, copyright and the performance of our websites, we have a number of automatic and manua systems that actively monitor robots, spiders and crawlers accessing websites that we host, and block those that don't comply with a few common sense rules:

In general, all webcrawlers, robots and spiders will be blocked unless they:
  1. Follow the robots exclusion standard
  2. Properly identify themselves in their user-agent.  Robots that misidentify themselves with a web-browser user agent string (eg ""Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1") or no user agent string at all are generally assumed to be malicious spiders and blocked
  3. Don't attempt to submit forms, crack passwords, engage in web application attacks or display an access pattern consistent with known malicous tools
  4. Don't cause undue load to the website, eg by accessing pages more than twice a second

About Zeald

Zeald was formed in 2001 by three young guys from the small New Zealand town of Mangawhai Heads. Now, Zeald is the largest SME website design and digital transformation agency in New Zealand and has clients throughout both New Zealand and Australia. This is the Zeald story …

read more
 

Connect With Us

Facebook Youtube Twitter Linkedin
 
 

Contact Us

New Zealand
0508 932 748
Australia
1800 224 032

 

Support