I created my Yelp profile in early 2010. I'd used the site for years before then, but for whatever reason, I didn't feel compelled to start writing reviews until a few years ago. Yelp, in the off chance you've never heard of it, is a massive community-driven site which primarily focuses on offering honest customer reviews for all sorts of businesses. It's equally useful to find a place to get lunch, somewhere to get your hair cut, a cool shoe store, or a million other things. Taking a step back, there's tons of similarities between Yelp and the App Store, but Yelp does two things that make a massive difference.

Just like how you can find listings for anything from a roller-skate repair shop to a studio to learn fencing, you can equally come up with just about anything on the App Store. Both feature an insurmountable amount of content that's hard to even imagine, and both Yelp and the App Store orbit around an equally important customer review system. Sure, sites like ours provide a more "official" venue for long-form reviews, just like a proper newspaper columnist would offer restaurant reviews, but I'd argue that a platform for legitimate customer reviews is just as useful.

"Legitimate" being the key word here. Developers on our forums have encouraged players to leave reviews for their games since, well, the birth of the review system. This sort of vague motivation generally results in reviews which I'd describe as both thoughtful and extremely useful for fellow App Store shoppers. Whether the particular review has a positive or negative slant usually doesn't matter, as people who are leaving reviews just for the sake of leaving reviews often form at least a semi-intelligent argument for why you should (or shouldn't) download something.

Then two things started happening.

Developers started putting these sorts of annoying pop-ups in game, interrupting gameplay to pressure players into leaving a review. Also, they forged this strange ultimatum of sorts, dangling the potential promise of future updates in exchange for five star reviews. In my opinion, both of these things taint the entire review process to the point of bordering on complete uselessness.

If you're already into the second world of Ragdoll Blaster 3 [99¢] and you get this confusing pop-up asking if you like the game, why would you tap "no"? So, you hit "yes," but at this point your head isn't in a "alright I'm going to sit down and write something useful" space, you're thinking, "I just want to get back to the game." Similarly, if you've got a game you even vaguely enjoy and you notice the all-too-common update text that mentions something along the lines of "your five star reviews keep updates coming!" you're not leaving a review because you want to assist in the purchasing decisions of the iOS gaming community, you're doing it because you like free stuff.

I can't really fault developers for this behavior either. It's difficult to keep your head above water on the App Store, especially when there's only space on the top lists for around couple hundred apps/games to be even making a decent amount of money at a time. Deciding you're going to take the moral high ground and not beg for reviews could make the difference between keeping the lights on in your studio or not.

So, anyway, switching gears back to Yelp, if you've used it at all you've likely noticed that nearly all reviews you come across are at least somewhat useful. You almost never see reviews like these on Yelp:

You might stumble across the occasional funny but vague review, or reviews that are as simple as "Try the tacos!" but I'd argue that both of those are more useful than "love it" or "hate it" with an accompaniment of one to five stars.

Yelp accomplishes this in two interesting ways.

First off, there's a real motivation to build your own profile up on Yelp. They've done an outstanding job of making your Yelp profile something you'd link your friends to show them what sort of things you enjoy locally, or maybe places you went to on vacation. It features all the social networking tropes that make it feel like home, complete with a basic avatar system and space for superfluous personal details such as "favorite thing." As a Yelp user, you don't want to leave a useless review because the profile creates a feeling of ownership to your reviews, and provides a record of all the places you've been. Comparatively, this is about the closest thing you have on iTunes.

There's no ownership to that page. No customization, and no reason at all why you'd ever link it to someone or include it on the links on the side of your blog, or anywhere else that you'd normally put links you care about. There's more to it than that. Even with a profile, some people will just flat out never get invested enough into things to put forth that effort to produce good reviews. Similarly, automated review spammers don't care at all that they can have an avatar. This is where the truly ingenious Yelp review filter comes in to play.

Check out this fantastic video which details how and why it works:

The crazy part about the Yelp review filter is it works so well you don't even know it's there. As mentioned, I've been using Yelp for years. Yelp explained how the filter worked around the same time I officially joined the site. I discovered that it existed only a few weeks ago when I noticed that I could solve a CAPTCHA at the bottom of a restaurant's review listing. Doing so revealed reviews of the same caliber of App Store reviews, what have systematically been deemed worthless by the Yelp review filter.

Apple has the resources to make the App Store incredible, and make app reviews just as useful as Yelp reviews for a new restaurant. They've already dabbled in building a low-level social network inside of iTunes via the Ping music service. A similar feed or apps that friends of mine are buying (and hopefully reviewing) would be immensely useful. From there, you're only an avatar, a tidy URL, and a few silly profile data points to having something that people would genuinely want to link people, and in the process, pour effort into maintaining beyond "★☆☆☆☆ sux" or "★★★★★ ownz".

Genius for apps already exists, and provides great recommendations for things you should try based on your previous purchases. I can't imagine it being much effort to massage that same algorithm into flagging drive-by low-content reviews from people who can't be bothered to spend more than 10 seconds typing out a coherent thought, review bots up voting, or people mudslinging with one star reviews of apps/games they don't like- Especially if it falls out of line with what this customer would typically enjoy per Genuis.

Can you even imagine how different the purchase experience would be on the App Store if reading customer reviews was closer to reading product reviews on Amazon? Comparatively, just think for a second how much a wild west style review system like the App Store currently has would completely ruin sites like Amazon, Yelp, and others that are dependent on honest and thoughtful customer reviews for purchase decisions.

And no, the "Was this review helpful?" system is not a solution. All that seems to have proven is that, in some strange way, the App Store hive mind believes that of the 11,546 reviews for the current version of Angry Birds, this is the most useful:

This is one of the best games I've ever played! Plus it extends your time on the toilet by a good 10 hours."

I rest my case.

The Yelp system is by no means perfect. Yelp has been the the target of all sorts of accusations both from business owners and internet conspiracy theorists. However, the fact remains: When I read Yelp, I almost without fail genuinely feel like the reviews are useful to me. App Store reviews, on the other hand, generally just result in a whole series of face palms.

I'd love if that changed.

  • http://profile.yahoo.com/MO6U7PYMQPXQYV5XTEZLV32X6M Travis

    I don't give a flying jay jay the jetplane F

  • Adams Immersive

    Great idea! (Although to save people time: the video doesn’t actually explain how or why it works. It simply says there IS a filter. Understandable: if people knew how it worked, that could help them game the system.)

  • Derek Traver

    * * * * *

    Great article.

  • Anonymous


  • http://twitter.com/michael_one Michael Silverwood

    Awesome article Eli, completely agree. And if they fix reviews like this, and make them more relevant, they could even start letting the star ratings reflect chart placement in a small way as well. Obviously the charts still need to be timely, so they can't be exclusively about review score, but if the system was able to filter out these 'best' reviews and use them to nudge good apps higher on the charts and bad apps lower I think it would help a lot.

  • Travis Ray

    The one thing you failed to mention that I feel stymies the iTunes review process is the fact that it wipes out your written reviews so easily. 
    As an app enthusiast I've downloaded thousands of apps.  That leads to tons of updates which need to be re-reviewed.  This is fine but if I just want to change my star rating, for example, from a four to a five star, it replaces my written review with a blank five star review. Something has to change.

    • http://toucharcade.com Eli Hodapp

      I sort of think this was a needed change though. The reason this was implemented in the first place was because you'd have situations like this:

      -Pretty good game gets released, has fairly wide-spread crash bug.
      -iTunes page gets hammered with 1 star reviews, all mentioning that it crashes.
      -Update lands quickly after, game retains solid 1 star rating and all the reviews mentioning it crashes (which are no inaccurate) are still all over the place.
      -Game sort of bombs because no one wants to download anything with 1,000 single star "Crashes constantly!" reviews regardless of whether or not it was fixed.

      It'd be cool to have a better "review console" for lack of a better thing to call it where you'd see some log of existing reviews you have out for apps that have since been updated, at which point you could either re-publish or otherwise amend your review.

      The point is, there's tons that can be done to improve the whole process.

      • Travis Ray

        Thanks for the reply Eli. 
        I see your point and for the developers it's a good change.
        I was pointing out how pointless it makes giving a full review on iTunes feel as a consumer because it erases what you previously wrote.  I like your idea of a "review console" that would keep your review text and maybe add a search feature too.
        Thanks for the great article.

      • http://fromonesurvivortoanother.tumblr.com Chungyen Chang

        The appstore (and itunes in general) still feels like something that was programmed by the guys who made IE 6 a decade ago. Have you ever tried to navigate the wishlist? It's absolutely horrendous. And importing playlists into iTunes is still completely ridiculous. They've had a bug in it for at least 3 versions where Windows users can't import M3U playlists unless they convert all of the entries into Mac OS style filepaths. Little (and sometimes hugely inconvenient) bugs like that are still packed into the tenth version of one of the most popular and yet, most bloated music players ever made.

  • Anonymous

    Historically the "Please review this game" popup was something devs started doing to combat the "Negative Review Generator" effect of the old review on delete popup that appeared in iOS 3 (and fortunately was eventually removed by Apple).
    From the developer point of view, the one thing a lot of devs (myself included) would like to see would be an option to be able to send feedback to reviewers (via Apple so we don't get their contact details) as so many players tend to use the reviews as a forum to ask questions about the game or report issues that we'd love more information on if they feel like talking to us. However unless you're Zynga, it's generally regarded as bad form to post your own reviews to reply to them, so having some way to be able to send them a polite reply would be handy.

  • http://players.openfeint.com OFErOne (Erwin Ocampo)

    Maybe Apple could sponsor a competition where money is awarded to the group who can come up with a better algorithm like Netflix did a while back

  • Anonymous

    Excellent article. Just last night I was looking at Pinball HD and. came across many one star reviews, simply because only one pinball machine was playable;  you actually had to  purchase the rest. I'm sure many of you visualize a very smack-able human who leave these type of reviews. How therapeutic would that be!

  • Aazlain

    My apologies. Double post. Please delete.

  • Anonymous

    Bravo and thank you. I officially rate this article "★★★★★ ownz"

    I basically gave up on checking reviews on the app store and rely solely on sites like this one to get an honest opinion.

    On a side note, if you think the app store reviews are bad, you should see the movie store. It feels like 95% of all movies are rated either 4 or 4.5 stars. Every movie is a masterpiece!… right.

  • http://twitter.com/starksimon Stark

    The whole App Store is a horrible experience, especially on the iPhone. I mean seriously, where's the freakin' "add to wishlist"?

  • Anonymous

    Oh my gosh that review that said: 

    This is one of the best games I've ever played! Plus it extends your time on the toilet by a good 10 hours."

    was hilarious!

  • http://pulse.yahoo.com/_2CUJHSQSQYTYT4DPZSKTVESYNQ B

    Then of course there's this problem:


    I agree... when it comes to games I can look here, but for other kind of app I'm pretty much lost.  Wading through the store trying to figure out which of various apps are ACTUALLY good is a pain in the rear.

    (Though I will say that is you find an app with lots of good-well written reviews, it's usually safe to assume it's a winner, if it inspired enough people to care enough to write useful reviews.)

  • nicholas valenti

    another app review i cant stand are those made by the app devs themselves. happens alot especially when the app first drops and the fist amount of reviews are glowing.

    • Anonymous

      I don't mind that--if they use their real names.

  • http://pulse.yahoo.com/_FQ6MZYN2IZPGA5FBSFNMIPQFZU Paco

    My brother, who owns a nightclub, was appraoached by Yelp. They offered to skew reviews (move negatives further down the list or remove them entirely) if he would pay them. There are others making this claim.

    Any system can be manipulated, but ya...the app store needs serious attention, especially the iPad version.

    • Anonymous

      Yes, there is a dark side to Yelp. It is still the best of its ilk.

  • http://pulse.yahoo.com/_WPMZXGUE4MJYAKW47IRGWIPIL4 Sam Crown

    App Store reviews are also a great place to see 'Add me - J0aZx-Pwn3r'

  • http://twitter.com/spongefile Tina Aspiala

    In short, the reputation of a reviewer matters, and you can gather that by (for example) giving more weight to reviewers whose reviews across various apps get the most "useful review" votes.

  • Anonymous

    Almost all internet postings should be monitored by their own community. This insures that the community gets worthwhile information. I hate when a site is lazy and just uses facebook as their login. I usually don't want my real name tied to internet postings. At the same time, there has to be a way to keep posts from idiots littering the landscape.

    One useful thing from iTunes app reviews: When a new version of an app comes out, some of the reviews will be useful in figuring out if the update breaks any functionality that you need.

  • Matthew Carnevale

    You know what would REALLY REALLY be great? Don't make me leave the damn app / game to write a review!!!!

    This article nails it.  I'm in the mood for playing a game when the 'Why don't you leave a review?' dialog pops up.  I'd leave so many reviews if it just let me review the game right then and there WITHOUT EXITING THE GAME AND POTENTIALLY HAVING THE GAME KILLED BECAUSE MY IPAD NEEDS TO LOAD THE APP STORE and wait and wait.....

    Here's what they could do: You write your review, right then and there in the dialog.  Then it sends your review *in the background* to Apple, so you get to go back to your game straight away.  I would leave so many reviews that way.  As it is now I hardly ever leave reviews, even if I really like the game.  It's just unnecessarily cumbersome. 

  • http://www.facebook.com/profile.php?id=1600586419 Jay Alan Babcock

    Is it a big deal for the app to keep track of how much time you spend using it? Imagine that, in addition to a giant-ass list of highly subjective and easily falsifiable reviews and star ratings, the app store told potential purchasers how many hours per day the average user spends with the app in the first week after downloading it, or some rating based on that kind of collected data. High score means addictive game/useful productivity app, low score means sucky game, etc.

  • https://me.yahoo.com/rekzkarz#a0df5 rekzkarz

    AppShopper could get funding from Apple to continue it's amazing job catagorizing and letting people post ratings.  I do wish Appshopper had a "block developer or game" setting.
    Appstore IMO is horribly flawed -- but does Apple want to fix it b/c it's a billion dollar profit engine???
    Why don't we users get to grade/review Apple's apps & give feedback/dev ideas to Apple & fix their apps?
    Many "Mac & Apple lovers" like saying how great the design is -- but the iron hand controlling who can say what is also a part of their design control.  And it's frustrating!