Apple recently announced that in order to submit a product review of any app in the iPhone App Store, the reviewer must own the app in question, whether through free download or purchase. When you attempt to write a review for a particular product, iTunes searches your Mobile Applications folder for the app in question. If found, iTunes opens the review form for that product, and you can enter your review. If iTunes can't find the app, a dialog informs you, "You must own this item to write a Customer Review" and refuses to let you review the app.
[image link] [1]
[image link] [2]
Prior to this change, many developers felt their products were being unfairly rated and reviewed by people who hadn't even tried them. Numerous reviews (often highly negative ones) were based on price, screenshots, or product descriptions alone. Since the App Store is the only place to acquire apps for the iPhone/iPod touch, it seems more than reasonable that you would be allowed to review only those apps you actually own.
While the new policy may not actually improve the depth or quality of reviews - people who have downloaded apps may still glibly trash them - it will at least assure readers that the opinions are in fact based on firsthand user experience.
Unfortunately, the new limitation may eliminate a small portion of credible opinions. For example, if I try out an app on a friend's iPhone, I can't share my thoughts on it through the App Store - despite my opinion being based on firsthand experience. Yet such cases are certainly in the minority, and owning the product would definitely give you a better sense of it than briefly tinkering with it on someone else's phone. The tradeoff seems entirely worthy.
How does Apple's new policy situate the App Store reviews in comparison to customer reviews on other sites, like Amazon.com? (Amazon is the gold standard for customer reviews - even people who don't end up buying at Amazon use their customer reviews when evaluating possible purchases).
Apple's ownership requirement would seem to raise the bar of review quality, since sites like Amazon and VersionTracker have no way of determining whether or not a reviewer owns the product in question. However, in practice, the App Store is likely just attaining equal footing. While few sites restrict reviewing to people who have purchased or downloaded through the site, the assumption is that anyone reviewing the products has some experience with them. For instance, books, DVDs, home goods, and other items sold through Amazon are also widely available through many other retailers, so reviews can reflect firsthand experiences no matter where the product was purchased.
Requiring commentators to own the apps being reviewed is just one more thing the App Store does to improve the quality of reviews. Most importantly, potential buyers can say whether or not a particular review is helpful, and sort reviews by those that received the most helpful votes. Unfortunately, Dan Frakes of Macworld notes that the App Store's Most Helpful sort order keys off the number of helpful votes, not the ratio of helpful to not-helpful. As a result, an older review that has a 20 (helpful) of 40 (total) ratio will sort well above a newer review that has 9 of 10 helpful votes. Obviously, it would need some threshold, since 1 of 1 helpful votes has a great ratio but not necessarily much relevance. It's also possible to sort by the reviews with the highest ratings, and to see which other products a reviewer has evaluated (since someone who writes one good review is likely to write others).
Perhaps the main thing that's missing is the way Amazon shows the most helpful reviews with low ratings; that feature makes it easy to find well-received - and likely legitimate - criticism for a generally liked product, something that's tricky on the App Store now. Amazon's bar chart showing the distribution of different ratings is also especially welcome.
All in all, Apple's new ownership requirement is a wise move, and one that marks a subtle but important difference in the way the App Store does business.
[1]: http://www.tidbits.com/resources/2008-10/App-Store-review-form.png
[2]: http://www.tidbits.com/resources/2008-10/rejection-dialog.png
[3]: http://www.tidbits.com/resources/2008-10/poor-review.png
[4]: http://www.tidbits.com/resources/2008-10/Amazon-review.png