|
Design Tip:
Hide From Email Spiders
by Larisa Thomason,
Senior Web Analyst,
NetMechanic, Inc.
What's one spider you DON'T want on your site? An email spider that visits with one purpose in mind: collect as many email addresses as possible so the owner can spam you with email later. Learn how to keep your email address available to humans but invisible to email spiders.
All Email Isn't Spam
Opt-in email is mail that you either requested or agreed to receive. Think about online newsletters you subscribe to, emails from companies with whom you do business, breaking news emails from CNN.com, or personal email from family and friends. That's legitimate email.
Opt-out email is completely unsolicited mail that appears in your inbox offering you "services" like university diplomas, easy credit, or mail order medicines guaranteed to jump start your personal life. To get off these lists, you have to petition the sender to remove you, hence the term "Opt-out." Of course, sometimes even that doesn't work: it just tells the spammer that your email address is valid.
Many people wonder how they possibly get on these lists. Often, the culprit is an email spider that ignores robots.txt files. They crawl through message boards, Web sites, and any online forum that might contain email addresses. The spiders search the Web until they have extracted a certain number of email addresses. Then they triumphantly crawl back to their evil masters with a list of victims.
Don't Be An Email Spider Victim
Before you know it, your inbox is flooded with spam email and you're spending valuable time sorting out the junk mail from legitimate email. That's time you could be spending promoting your site or providing service to your real customers.
Many people protect their personal email by maintaining at least two email addresses: one that only friends and family have and another email address used for online orders or message boards posts. Online services (like Yahoo Mail and Hotmail) that offer free email addresses make that easy to do.
The email addresses on your Web site are more problematic. You can password-protect your site, but that option keeps out both human visitors (including directory editors) and all spiders (not just email spiders). The human editors at Web directories expect to see contact information on the site's home page. Other visitors expect it too.
The trick is keeping the lines of communication open to human visitors, but closed to email spiders. Here are three reliable methods. Select the one that fits your level of Web and programming expertise.
1. Use Special HTML Characters
This is the easiest way to thwart the email spiders, but it's also the easiest method for the spiders to detect and compensate for: simply use the "@" character in place of the @ sign in your email addresses.
Put it in your code inside a link or just as a text email address:
Link: <a href="mailto:news@netmechanic.com">Email Us!</a>
Text email: Email us at: news@netmechanic.com
Since the HTML code doesn't look like a proper email address, the email spider doesn't harvest it. But the browser understands the code and displays it on the page properly. Visitors will be able to send you mail from the linked address if they have their mail client configured to do so. Otherwise, they can cut and paste from your Web page into their mail client to email you.
2. Hide Email Addresses With JavaScript
The special HTML character alone may not do the trick though. Before long, email spiders may be configured to look for it and replace it with the @ symbol. Then you're back with a full inbox of unwanted mail.
So consider using JavaScript to parse the email address components. Be sure to change the variable names to reflect your site and email address! Also, remember to keep the document.write statement on one line in your HTML code to avoid getting a JavaScript error message on your Web page.
Place the script in the BODY section of your document.
We've included the code in this newsletter story here:
Send your comments and questions to our
This works great if you only have a couple of email addresses on your page. A long list of contacts is harder to easily manage and maintain. But including the script for all your email addresses probably takes less time than wading through hundreds of spam email messages!
Note: Please be aware that this code is client-side JavaScript and screen readers for blind and low vision users cannot parse it. We welcome any ideas for an accessible method for hiding email addresses on Web pages.
3. Use A Comment Form With CGI Script
Installing a CGI script isn't difficult, but seems intimidating to many novice webmasters. If you can get over that fear though, a CGI form processing script is a foolproof way to ward off email spiders. It's also easier to maintain in the long run because you don't have to change a lot of JavaScript variable names when you need to update addresses.
Note that we use one for newsletter comments. When a visitor clicks on the author's name at the beginning of the story, the link opens a new browser window with a comment form. This hides the email address from email spiders, but still allows visitors to send questions and comments.
Most Web hosts have a set of common CGI scripts for their customers to use. If not, you can easily search the Web for free CGI form processing scripts. It's safer to use your host's scripts or a remotely hosted CGI script though, because poorly written CGI scripts can get you into trouble with your neighbors who share server space with you.
If your host doesn't give you access to CGI scripts or to your own CGI bin, then consider moving your site to a new host who does.
Make Time For Customer Queries
Before you use our handy comment form to complain, let us stress that NetMechanic isn't criticizing all forms of marketing email!
Unscrupulous email marketers who use email spiders to collect thousands of email addresses to send bulk email to disinterested, unsuspecting parties are the issue here. They waste your time and clog up mail delivery systems - slowing all email. Hiding your email address from spiders helps you respond to visitor and customer queries more quickly because you don't waste time separating them from the garbage email.
Your customers and site visitors need to communicate with you. Spam emailers do not.
|
