New ranking algorithm on the App Store?

Discussion in 'Public Game Developers Forum' started by fairlady, Feb 3, 2010.

  1. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    This would explain why my competitor, who gathered many reviews while being free, is now ranked higher than me. Wonder if one of these classic apps went free today, and collected 10,000 dls and ratings, they'd suddenly jump further up. If true, that would be extremely bad.
     
  2. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    MikeSz_spokko- The more I think about it, you may be partially correct here! You obviously had massive downloads of Save Kitty while free, probably where a large percentage of your ratings came from.

    So let's say for sake of argument it is lifetime sales instead of ratings. If Apple is adding a weighted modifier to the normal sliding sales window calculation based on "lifetime unit sales" of an app, then any app that had tremendous success in the early days of the store, or those that have switched between free and paid once or twice will have an advantage in the paid store due to the system apparently counting all downloads equally. A few hundred thousand free (or paid) downloads back then could boost your paid ranking now!

    And yes LittleWhiteBear, your competitor wins in either case here, whether they are simply counting ratings/reviews, or lumping paid and free sales into a lifetime unit sales figure. Scary stuff. Although you may be in an interesting position to see what happens when you switch Compression back to paid! At that point you'll have collected more "unit sales" than most paid puzzle games have made in the last year.
     
  3. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    Yes, I've considered that. However, Compression has only been out two months, so it may not get the "classic" bump at all.
     
  4. ThunderGameWorks

    ThunderGameWorks Well-Known Member

    Nov 14, 2009
    692
    0
    0
    So ... wait a minute ... are you guys suggesting that boosting downloads and reviews by going free might somehow be loaded into the rankings now?!? Dear Lord! Has Apple lost their minds?

    It's a little hard for me to believe that being the case. Apple makes piles and piles of cash off the AppStore ... but none off of free games. Would seem counter-productive to reward free apps.

    Thoughts?
     
  5. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    I think that "classic bump theory" may simply be due to the fact that older apps generally had better sales and long term visibility. They were available when the app store was much smaller in size, but there was a very rabid intake of apps at that point. Each app available back then got a bigger piece of the pie so to speak. And ones that flipped between free and paid back then got even bigger, especially back when there was no ranking penalty for switching. And let's not forget the old days of the App Store allowed you to rate and review WITHOUT BUYING, so if it is actually keyed to ratings rather than sales, very old apps would also benefit even if they were never free.

    Don't get me wrong, I'd love it to be some sort of classic boost that kicked in once you were around for more than a year. That would mean I could get bumped up in a mere 2 weeks from now. But I don't believe that's what is happening here.
     
  6. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    /facepalm

    Ratings/reviews to sales ratio is often skewered. If an app is horrible but sells well, it will get more ratings/reviews, because more people deleted it or felt the need to complain about it. With such an unstable system why would Apple implement it? They wouldn't. I don't know why you guys are basing your math on that. You should focus on another possibility.

    With sales reports messed up, the developer forums down, and the rankings scattered, I'm sticking with the glitch theory.

    Apple reviving old apps that were once popular is bad for revenue, because a lot of users already saw the app, then either bought it already or decided they didn't like it. Giving old apps exposure when they are no longer selling tremendous amounts is a horrible idea. People stopped buying it for a reason -- obviously.
     
  7. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    Other than Apple hand picking the rank placements for these classic apps, or this all being a big glitch, I don't see any other way to explain how classic apps with much lower sales, but huge reviews from free days, got a much bigger jump than other classic apps.
     
  8. Foursaken_Media

    Foursaken_Media Well-Known Member
    Patreon Indie

    Wow this looks really lame. Those old apps had their chance. In fact, they had a BETTER chance when they were out with less apps in the total market. They ALREADY MADE THEIR CHEESE -- people already bought them, or didn't. If anything, they should redo the top-grossing category to be more like the current model (this would allow you to see older, top selling games), and leave the regular top list as it was... kind of like a "whats hot right now in the past couple of weeks" type thing. Looking at the top 50 right now it looks like its practically entirely filled with older games just popping up from nowhere. Sigh.

    How is a new game supposed to fight its way into the 25 now? It was hard enough as it was...
     
  9. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    I don't think it's a year limit. It may scale by time on store. One of my apps is only 11 months old, and saw a reasonable jump. Not enormous, but noticeable.
     
  10. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    Good apps die because of saturation and/or too much competition. Maybe this is Apple's way of keeping them in the store for the newer audience. Maybe they want to give the big companies some assurance that they'll make their money back.

    Or it's all a glitch. :)
     
  11. CommanderData

    CommanderData Well-Known Member
    Patreon Indie


    The hand picking argument loses with me when I look at all the Point Packs in RPGs that were boosted up. Why would anybody intentionally promote all of those limited functionality apps, and not the main game itself? But the swapping between free and paid, that tells me that the total unit sales or ratings DO play into it. :)



    Well, our theory fits the visible data. Do you have any possible theory to contribute other than "it's a glitch" and "Apple wouldn't do that"? Apple does a lot of things we don't understand. Rate on delete is a long-standing example they still have not acknowledged as a bad idea.

    Free apps are bad for Apple's revenue. Why do they allow those?

    My point is, just keep an open mind. We'll know soon whether this is permanent or not. :p
     
  12. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    Very interesting info, but there's actually a lot more bumped apps out there.

    Apple may have "tweaked" the algorithm at some point yesterday or last night, as people monitoring rankings saw things sag and rise again (LittleWhiteBear and Nattylux both noticed that, for reference). Note that those apps listed right now are ones that gained over 50 places today, 2/4. Yesterday there were many more apps and games that jumped. Look at Tap Tap Classic, iMob 30 respect points and others as examples, those jumped hundreds of places yesterday and are not to be found on the jump list today, because they are now very near the top of their categories.

    Anyone who did not benefit from this hopes it is a glitch. I certainly did not benefit from it. The RPG category is a disaster now, even worse than usual. I'd love it to be a glitch too. But if not, I'm determined to know how it works so that I can alter my upcoming games and strategies to take advantage of it.

    I think I'll examine some of the big losers and see if there's any clues hidden in there :)
     
  13. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    See post #20. I already proposed the only legitimate theory aside from a glitch.

    Free apps aren't necessarily bad for Apples revenue. For two reasons:

    1.) People are more willing to open the app store knowing they can leave without spending a dime, and still have new stuff to play with; therefore, they increase traffic to the App Store.

    2.) Many free apps include advertisements for Paid Apps; therefore, they increase revenue of paid apps.

    The reason others have said the change you're proposing is false is because this would give an advantage of free apps OVER paid apps, which in turn, will have a negative impact on overall revenue.
     
  14. er98052

    er98052 New Member

    Feb 4, 2010
    2
    0
    0
    Joining into the discussion

    Hi guys,

    Great thread. I found toucharcade googling around for the change on Feb 2. It's really nice to have a forum like this where experienced app developers can talk - if you have other pointers to similar forums I'd really appreciate it.

    So here's a data point. My paid app's rank dropped from 9 to 26 in its category in the US and suffered pretty heavy losses around the world - it was #1 in several countries. Sales down to half.

    I've been poking around at Apps whose rankings I had been following and there doesn't appear to be a clear pattern. Anyway, let me start by summing up what people's theories are:

    1. (apparently most favored so far) The time window over which downloads are considered has expanded. The arguments I have seen so far don't appear to be compelling - a number of all-time-best-hitting apps actually lost rankings - e.g. ColorSplash (everyone's seen that one) dropped and is no longer in the US top 50. Also, I know a few apps that have done well in the past but that stuck around the same ranking.

    2. Ratings now play a role in the ranking. Possible, but would be braindead to do this - as if ratings weren't enough of a menace. I see 1-page long reviews for my app every other day written via some account with a random name that has only 1 review. The potential for misuse is too high, so personally I rule this out.

    3. In-app purchases adding up in your favor. Not true - I've seen a number of apps e.g. WritePad in productivity which went up something like 50 places.

    4. All of the apps in your account now being considered. My free apps are downloaed about 20x more than my paid apps. No dice.

    5. A magic 'Apple' factor is added to apps the App Store team likes. It appears that this is not the case - a number of my competitors who were previously featured under 'What's Hot' and such have stayed down.

    6. Another theory I have - basically based on my data is that rankings now only depend on App Store turnouts. Meaning that if you advertise heavily then that doesn't help. Most of my sales are conversions from my LITE app and it might be the case that these are not getting counted towards the rank.

    Anyway, just brainstorming. If the answer is #1 then that wouldn't be all that bad since I'm assuming they will have put in provisions for brand new apps to get an advantage in the beginning.

    Also, has anyone seen changes to the rankings of their free apps?

    Zach
     
  15. bomber

    bomber Well-Known Member

    Nov 9, 2008
    942
    1
    0
    #56 bomber, Feb 4, 2010
    Last edited: Feb 4, 2010
    Aaah our new game Classic+ dropped 20 places without big decrease in the download number (was #12 in Role Playing games, now out of the top25).

    All those modifications in the app ranking are probably fun for Apple but they should check if it cannot be abused. Once updates didn't show in the release-date anymore some developers switched to release new apps each few days. And now the top charts are even more flooded by +1 respect point / love point / prestige point / loyalty point / legend point / favor point / honor point / master point / reward point apps.

    The main problem in the app store is that everything is based on visibility, so that games which are in the tops are downloaded often and also reviewed more often. New apps don't have any chance even if they are better. By adding the review number to the calculation this is only getting worse.
     
  16. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    Right you are! I forgot about the Point Packs.
     
  17. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    I re-checked your #20 post, and it is not in direct disagreement with what I have been saying. You stated that they may be taking average sales since the introduction of your app into the calculation. That is basically what we're trying to determine now... is it merely the ratings, or is the number of ratings just an indicator of the MASSIVE number of sales that some of these apps had, which then cause them to jump up.

    What I am proposing definitely would give free apps an advantage over paid, IF they switch from free to paid. That is what makes this whole thing scary. What if the Facebook app switched to 99 cents tomorrow? Even with zero paid sales it would probably be the #1 app instantly, because this calculation is either taking lifetime unit sales (both free and paid), or lifetime ratings/review, of which Facebook has almost 775,000. Either way it would dominate the charts before the first paid download hit Apple's coffers.

    So it seems there's no need to argue about it, we're pretty much on the same page ;)



    Zach,
    What's your app? I'd be curious to analyze it and how it fits in the puzzle here.

    Re: ColorSplash... Never heard of it until you mentioned it. I see it's #62 in the Top 100 US now, and has been out of the top 50 for several weeks now. Looking at the last two days data-points here:

    http://www.topappcharts.com/304871603/app-details-colorsplash.php

    Shows me that it too has actually benefitted from this ranking change/glitch.



    Bomber, you were partially affected by the same thing I was. Rogue Touch was already outside the Top 100. It too was pushed further down despite actually increasing sales during this disturbance. It is easy to see how ALL apps outside the top 3 RPGS got displaced downward in rank by these point packs getting massive boosts... whether they are boosted due to ratings/reviews counts or lifetime download counts I am not sure. But they are there, and polluting the category worse than ever.
     
  18. MikeSz_spokko

    MikeSz_spokko Well-Known Member

    May 27, 2009
    594
    0
    16
    ok, gonna post the numbers for Save Kitty (from top of my head - quite accurate, but not 100% perfect). you can try to make sense out of it:

    released late january 2009
    total of 500.000 + downloads
    10.000-12.000 paid downloads
    was free sometime around march (IIRC), thats when it got most of the downloads. made it to top 10 free apps when we made the switch
    remained paid since then, averagin 10-20 downloads a day for months now (except for christmas of course)
    has 30.000 rankings, mostly a result of "rate on delete" - result of the free downloads I mentioned earlier
    average rating is 2.5-3 - many of the negative reviews are because the game features animal violence and some people cant stand it (we actually had death threats over it :D )
    it is in top 60 free apps since feb 2, was outside of any ranking before that

    so basically: a typical app, that was not a commercial success, doesnt have great rankings or anything. sales average over last months was very steady and insignificant (from rankings point of view)

    500k downloads and 30k ratings... coincidence ? dont think so

    ---

    if that is the case, then Im really upset about this change. yeah, it moves our old app higher in the charts but makes it much, much harder for any new app to reach high in the rankings. really bad idea on apple side
     
  19. M of IMAK

    M of IMAK Well-Known Member

    May 26, 2009
    199
    0
    0
    iPhone App Developer
    Austin, TX
    Thanks for these links. I was able to find my son's app he released the first day of the app store. Hold On! is now #97 in Productivity, up 95 spots:
    http://topappcharts.com/search.php?show=jump&start=190

    I also notice that Sky Voyager is up 61. Remember the few days that one was free?

    Also notable is WordsWorth, up 50, which has been on the store for over a year and has never been free.

    Another one of our apps, Brain Blaze Divide, jumped into the top 100 of educational games. BB Divide has a sum total of 11 reviews, so I think we can safely cross review count off the list. It does have 1388 ratings based on over 65K downloads, 99% of these when it was free. Hold On! has a similar profile with even more downloads. Certainly our data points to old free downloads having a higher weighting in the ranking algorithm, but apparently something else is also going on which we would need the developer of WordsWorth, or a similar app which has never been free, to provide some info to add to the discussion.

    Oh, and ColorTilt is still hanging out in the top 10 of ed games. It had a huge number of paid sales during the first month of the store that don't seem to have helped it a bit now.
     

Share This Page