<div id="popup_box_thanks" style="display:none" onClick="close_popup_thanks('popup_box_thanks', 'ts')"><br>Thanks for submitting your tip! All submissions are moderated by an editor before appearing online. We've reset the form so you can enter another tip. Or you can close the tip submission box. <div class="x_close" id="thanks_upper_right"><a href="javascript:void(0)" onmousedown="close_popup_thanks('popup_box_thanks', 'ts'); return true;">Close</a></div></div>
<div class="tbf_row"><div class="tbf_wide_extra_top not_bold">Please submit only technical tips that will help other TidBITS readers better use their Macs, iPhones, and related software and hardware. All product announcements should be sent to <a href="mailto:releases@tidbits.com">releases@tidbits.com</a>.</div></div>
<div class="tbf_left">URL</div><div class="tbf_right"><input type="text" value="" name="tip_link_url" tabindex="3"><span class="tip_description"><br>Enter the URL to a Web page that supports your tip.</span></div>
</div>
<div class="spacer"></div>
<div class="tbf_row">
<div class="tbf_left">Linked text</div><div class="tbf_right"><input type="text" value="" name="tip_link_label" tabindex="4"><span class="tip_description"><br>Enter the name of the page linked above.</span></div>
<div class="tbf_wide"><input type="submit" value="Preview Your Tip" name="preview_tip" onClick="fill_preview('tipbits_enclosure_preview', 'ts', this.form); return false;" tabindex="7"> <input type="submit" value="Send Us Your Tip!" name="submit_this_tip" onClick="handle_tip_submission('ts', '', this.form, 'tip'); return false;" tabindex="8"></div>
</div>
<div class="spacer"></div>
<div class="tbf_row">
<div class="tbf_wide"><span class="fine_print">When you submit a tip, you give us permission to use it. Read <a href="javascript:void(0)" onClick="generic_show_hide('tip_terms')">our terms</a> for more details. All submissions are reviewed before publication.</span></div>
<div class="tbf_wide"><span class="fine_print">Our terms: By submitting a tip, you agree to assign TidBITS Publishing Inc., a non-exclusive, worldwide, perpetual license to reproduce, publish, and distribute your tip in connection with the TidBITS Web site and associated products in any media. You agree that you created the content you submitted, and that you have the right to assign us this license. You give us permission to use your name, but your email address won't be publicly displayed or shared. We review all submissions before publication, and reserve the right to select which submissions we feel are appropriate for our readers and to edit those we publish.</span></div>
<div id="comment_thanks" style="display:none" onClick="close_popup_thanks('comment_thanks', 'comm')"><br>Thanks for submitting a comment! Please check your email for a link that, when clicked, will verify that you're a real person and cause your comment to appear immediately. <div class="x_close" id="comment_upper_right"><a href="javascript:void(0)" onmousedown="close_popup_thanks('comment_thanks', 'comm'); return true;">Close</a></div></div>
<div class="tbf_wide"><span class="fine_print">Our terms: We reserve the right to edit or delete any comment, so please post thoughtfully. We use your email address <i>only</i> to send you a one-time verification message confirming that you posted this comment. We also store your address to allow you to verify using other Web browsers in the future. For more info, see our <a href="http://db.tidbits.com/privacy.html">privacy policy</a>.</span></div>
<li><a href="/feeds/tidbits.rss" title="Subscribe via RSS" class="gettb">RSS <img src="/images/feed-icon-12x12.gif" width="12" height="12" border="0" class="nav_img" alt="Subscribe via RSS"></a></li>
<li><a href="http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=276986548" title="Subscribe to the podcast" class="gettb">Podcast <img src="/images/feed-icon-12x12_podcast.gif" width="12" height="12" border="0" class="nav_img" alt="Subscribe to the postcast"></a></li>
<li><a href="http://www.twitter.com/TidBITS" title="Get Article Updates via Twitter" class="gettb">Twitter <img src="/images/feed_icon_12x12_twitter.png" width="12" height="12" border="0" class="nav_img" alt="Get Article Updates via Twitter"></a></li>
<li><a href="http://www.facebook.com/pages/TidBITS/195314925519" title="Go to the TidBITS Page at Facebook" class="gettb">Facebook <img src="/images/feed_icon_12x12_facebook.gif" width="12" height="12" border="0" class="nav_img" alt="Go to the TidBITS Page at Facebook"></a></li>
<li><a href="javascript:void(0)" title="Sections" class="tabhead" onClick="return showhide('articleslist')">Sections <span id="articleslist_triangle"><img src="/images/nav_triangle_open.gif" width="9" height="9" border="0" class="navtriangle" id="articleslist_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="javascript:void(0)" onClick="return showhide('stafflist')" title="Staff" class="tabhead">Staff <span id="stafflist_triangle"><img src="/images/nav_triangle_closed.gif" width="9" height="9" border="0" class="navtriangle" id="stafflist_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="javascript:void(0)" title="Issues" class="tabhead" onClick="return showhide('issuelist')">Weekly Issues <span id="issuelist_triangle"><img src="/images/nav_triangle_closed.gif" width="9" height="9" border="0" class="navtriangle" id="issuelist_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="javascript:void(0)" onClick="return showhide('abouttidbits')" title="About TidBITS" class="tabhead">About TidBITS <span id="abouttidbits_triangle"><img src="/images/nav_triangle_closed.gif" width="9" height="9" border="0" class="navtriangle" id="abouttidbits_tri_image" alt="Click to show or hide the contents of this section."></span></a></li>
<li><a href="http://www.thedatarescuecenter.com/">The Data Rescue Center</a></li>
</ul><div class='sponsor_sidebox_bottom'> </div>
</div>
<!-- end sponsor_sidebox -->
</div> <!-- end leftcolumn div -->
<!-- end left column -->
<!-- begin centercolumn_border -->
<div id="centercolumn_border">
<div class="center_top">Thoughtful, detailed coverage of the Mac, iPhone, and iPad, plus the best-selling <a href="http://www.takecontrolbooks.com/?pt=TB-TAGLINE" style="color:yellow">Take Control</a> ebooks.</div>
<!-- begin centercolumn -->
<div id="centercolumn">
<!-- begin rightcolumn_container -->
<div id="rightcolumn_container">
<!-- begin rightcolumn -->
<!-- rightcolumn is embedded within centercolumn so featured text wraps around it -->
</div><!-- end tearoffbox_wide_container for watchlist items -->
<!-- begin tearoff box wide -->
<div class="tearoffbox_wide_container">
<div class="tearoffbox_wide_tips">
<div class="tip_display">
<div class="tips_sponsor_logo">
</div>
<h6>Expose Shortcut for Arrange All Windows</h6>
<p><p>In Expose in Snow Leopard, with all windows visible, press F9 (or the Expose key [F3] on recent Mac laptops), then press Command-1 to arrange the windows by name or press Command-2 to arrange them by application.</p></p>
</div>
<div class="tearoffbox_wide_bottom_tips">
<div style="padding-bottom:35px"><div class="tip_display" style="float:left"><p><br><a href="/tipbits/192">Link to this tip</a></p></div><div class="tip_display" style="float:right; width:150px">
<div class="tbf_wide_80" id="hc_rc_2594">To help us avoid automated posts and misuse of our site, please enter the words below.</div><div class="x_close_row" id="hc_upper_right2_2594"><a href="javascript:void(0)" onmousedown="HidePopupContent('hc_2594', 'hc', '2594'); return true;">Close</a></div>
<div id="article_box_2594"><P>Search engines and searching tools have become ubiquitous on the Internet. People flock to search engine sites in order to find information quickly, and the information available comes with startling breadth and depth. (See Kirk McElhearn's article in <A HREF="http://www.tidbits.com/tb-issues/TidBITS-333.html">TidBITS 333</A>).</P><P>For instance, I just searched AltaVista for "watermelon." I've barely scratched the surface of my search results, but I've already read about the status of the Texas watermelon crop, scanned an article about preparing watermelon (along with nutritional information), and visited a Web page devoted to Cezanne's painting, "Still Life with Watermelon and Pomegranates."</P><P><STRONG>Indexing Robots</STRONG> -- Search engines acquire much of their information through robots. Also known as spiders or crawlers, robots traverse the Web, looking for and recording information. Robots typically start with URLs that seem like a reasonable starting spot, such as a URL submitted by a user, a page having lots of links, or the top level of a site. A robot accesses the initial page and then recursively accesses all pages linked to from that page. The robot might also check out all pages that it can find on a particular server. After accessing a page, the robot works with the search engine to index portions of the page, perhaps the title, some or all of the text, specific keywords, or other tagged elements.</P><P>One topic that deserves attention, however, is how to prevent search engines from indexing individual Web pages or Usenet news postings. Conventions exist to keep robots out of specially-marked Web pages or news postings, though whether individual robots comply to these standards is purely voluntary. So far, mainstream searching engines appear to respect these conventions.</P><P><STRONG>Hey You, Get Out of My Site</STRONG> -- Using the Robots Exclusion Protocol, you can ask robots to ignore Web pages that you don't want indexed. For example, you might want to store club meeting minutes on the Web without having those minutes show up in search engines. You could, of course, set up a password system, but that might be a more complicated solution than you wish to implement. You might also have a site whose pages change so frequently that there's no point in a robot attempting to index them.</P><P>To tell robots to go away, you place a robots.txt file on the local root level of a Web site. Using a specific syntax, this file tells robots that they should keep out of certain (or all) sections of the server. If you want to set up such a file, I recommend reading the World Wide Web Robots, Wanderers, and Spiders page:</P><P><<A HREF="http://info.webcrawler.com/mak/projects/robots/robots.html">http://info.webcrawler.com/mak/projects/robots/ robots.html</A>></P><P>As a brief example, though, to ask all robots to keep out of a directory called watermelon, your robots.txt file might look like this.</P><P> User-agent: *<BR> Disallow: watermelon/</P><P>If you don't have enough control over your server to set up a robots.txt file, you can try adding a META tag to the head section of an HTML document. For instance, a tag like this:</P><P><META NAME="ROBOTS" CONTENT="NOINDEX"></P><P>tells robots not to index that particular page. Or, a tag like this:</P><P><META NAME="ROBOTS" CONTENT="NOFOLLOW"></P><P>tells robots not to follow links on the page. Support for the META tag among robots is more sporadic than the Robots Exclusion Protocol, although most of major Web indexes currently support it. Information on the robot META tag can be found in the Spidering BOF (Birds of a Feather) Report:</P><P><<A HREF="http://www.w3.org/pub/WWW/Search/9605-Indexing-Workshop/ReportOutcomes/Spidering.txt">http://www.w3.org/pub/WWW/Search/9605-Indexing- Workshop/ReportOutcomes/Spidering.txt</A>></P><P><STRONG>Private News</STRONG> -- To keep the fingers of search engines out of your Usenet news postings, you can create an "X-no-archive" line in of your postings' headers:</P><P>X-no-archive: yes</P><P>Although common news clients, such as NewsWatcher, permit you to add an X-no-archive line to the headers of your news postings, you aren't completely out of luck if your client doesn't permit you to do so. At least one engine, Deja News, will ignore your posting if you make the following text the first line of text in the body of your message:</P><P>X-no-archive: yes</P><P>In addition, if you inquire personally, Deja News will remove your posts from their archive; to ask, send email to <<A HREF="mailto:comment@dejanews.com">comment@dejanews.com</A>>.</P><P><STRONG>Assumption of Non-Privacy</STRONG> -- The source of confusion regarding privacy and Internet indexing systems usually stems from the assumption (made by most search engines) that <EM>all</EM> information they find is public unless marked otherwise.</P><P>Many Internet veterans have no problem with the search engines' assumption that all information is public, since much of the material has always been available one way or another. However, some new Internet users find the practice startlingly invasive. For these Internet users, it's like being told every phone call they made during the last year was recorded by a private company, who's now giving away those conversations to anyone who asks.</P><P>The long-term memory of these search engines makes the ramifications of their behavior larger than ever. Though Digital's AltaVista search engine currently only remembers the last few months of Usenet, Deja News has archives going back to early 1995, and repeatedly claims that it wants to index all the way back to Usenet's inception in 1979, where possible. In 1979, how many Usenet users could have known about the X-no-archive tag? Furthermore, though the robot and archive exclusion standards may help keep your material out of major, high-profile indexes, there are indexing and archiving systems out there that respect no such rules.</P><P>If you're highly concerned about the privacy of your email and Usenet postings, check out anonymous remailers and PGP, a controversial strong encryption program from Phil Zimmerman. Both topics are beyond the scope of this article.</P><P><<A HREF="http://www.well.com/user/abacard/remail.html">http://www.well.com/user/abacard/remail.html</A>><BR><<A HREF="http://www.io.com/~combs/htmls/crypto.html">http://www.io.com/~combs/htmls/crypto.html</A>><BR><<A HREF="http://world.std.com/~franl/pgp/">http://world.std.com/~franl/pgp/</A>></P><P>If you're not particularly concerned about privacy, still remember that your words on the Internet may become immortal - anything you write on Usenet will be archived somewhere for eternity, anything you publish on the Web will be indexed somewhere. Choose your words with care - you may have to stand behind them in a future situation that you cannot currently imagine.</P><P>In the future, as privacy becomes a larger issue on the Internet horizon, we can probably expect commercial and consumer newsreaders and publishing tools to tout "privacy compatibility" as a feature. No doubt newsreaders will soon come pre-configured to insert X-no-archive headers by default, and Web authoring programs will come with preferences to insert robot META tags and create robots.txt files automatically. However, these features will not alter the fundamental assumptions of Internet indexing tools: everything is public.</P><!-- Keeping Robots Out of Your Corner of the Net Tonya Engst --></div>
<!-- end article text -->
<!-- PayBITS -->
<p> </p><div class="sponsorbox">
<div class="sponsortext"><A HREF="http://www.smilesoftware.com/"><IMG SRC="http://db.tidbits.com/images/badges/SmileLogo2010-50x50.gif" ALT="" HEIGHT="50" WIDTH="50" BORDER="0" ALIGN="left"></A>Get more productive with software from Smile: PDFpen for<br />editing PDFs; TextExpander for saving time and keystrokes while you<br />type; DiscLabel for designing CD/DVD labels and inserts. Free demos,
<br />fast and friendly customer support. <<a href="http://www.smilesoftware.com/">http://www.smilesoftware.com/</a>></div>
