# Identifying rating algorithm

I'm trying to identify the rating algorithm used to calculate the displayed average of a 1 star to 5 stars rating system. To analyze the data I collated the first and last 1000 ranks and added the real average.

Here is what I know about the underlying algorithm:

• It is symmetrical.
• It gives more power to the majority of votes.
• It is pretty simple and not (intentionally) based on any known (complicated) mathematical method.
• No information other than the respective number of votes is used for the calculation.
• It tries to give supposedly dishonest votes (those allegedly submitted just to force the rating up (5 stars) or down (1 star)) less weight, so not every vote is worth the same. For example: If the vast majority of votes concentrate around 1 star and 2 stars votes with a smooth decline and a sudden jump at 5 stars votes, those 5 stars votes will count less (not necessarily 3.00 or 4.00, it could also be something like 4.88).
• To detect outliers (suspected dishonest votes) a relative smooth rating curve is expected.
• Items with overwhelmingly favorable ratings get bonus points what gets them over 5 stars.

I guess the easiest way to determine the algorithm is to analyze the items with the fewest numbers of total ratings. For example:

rank 5 stars 4 stars 3 stars 2 stars 1 star total displayed average real average
535. 15 1 3 0 1 20 5.42 4.45
9802. 2 3 10 0 0 15 3.11 3.47
9908. 2 1 3 2 2 10 2.86 2.90
9944. 0 0 3 0 1 4 2.71 2.50
9978. 0 0 3 0 2 5 2.26 2.20

You have not given some information on which your knowledge is founded. That might help to confirm those - or not.

Am I correct that you cannot create a response from the site by creating specific input for your own defined combinations?

Then I would try to attack the algorithm by

• selecting most simple combinations, and when displayed, and the real average is not the same. Like here:

• Or selecting combinations where only a single star rating is incrementing and the others keep the same

Edit 10.04.22

Meanwhile, I did a second approach: guessing an almost linear formula with a fixed factor for each star, so that the average is calculated by (number of 5 stars)*a+(number of 4 stars)*b+etc/total. Then I selected the ones with Zero 1 star, and Zero 2 stars and optimized the factors, resulting in

• 5 stars: factor 5.705
• 4 stars: factor 4.19
• 3 stars: factor 2.335

Total difference of deltas of that model (`abs(result - displayed average)` versus average model is reduced, but not Zero => guessed formula is not good enough. This is expected from the information you gave for the algorithm. However the biggest deltas are found for good scored results => a new model is to be guessed to represent that non-linear impact.

I think in that way you can come closer to a model that is good enough and get a final low delta, but it will never be the original algorithm. Alternatively, an "automated formula guesser" is needed, that combines the 5 variables to a result that comes close to the displayed average. However, this will also most probably not be the original algorithm.

• Thanks for your comment. - My knowledge is based on a survey of the author. Unfortunately, he didn't want to reveal more information. - Yes, you are correct. - That's an interesting approach. Have you come to any approximation with this? Apr 9, 2022 at 13:30
• Edited above answer with new input Apr 10, 2022 at 9:20
• Thank you very much for your considerations. - Another approach might be to put the respective number of the 1 star to 5 stars ratings in relation to the number of all ratings, somehow "smooth" these five ratio values and then calculate the number of the 1 star to 5 stars ratings from them to finally get the desired result. What do you think? Apr 13, 2022 at 12:21
• Yes, but I would not call that reverse engineering anymore because not exactly the same but only similar results are received. Methods I know for that are multivariate analysis or artificial neural networks. Apr 13, 2022 at 17:40
• I think, you got me wrong. My approach should serve as a possible way to discover the actual algorithm. But I don't really know how to do the "smoothing". May 22, 2022 at 13:10

I think what you are looking for is a statistical method for computing the formula. If you fit a least squares model to the five data values you've provided, you can get a prediction formula for this data that looks like:

`displayed_average = 3.6746154 + 0.162307923 * five_stars + 0.275384615 * four_stars -0.171548462*three_stars - 0.45 * one_star`

which predicts the displayed average as

Obviously, this is only based upon the five data values you provided. If you had 10000 values, you could come a lot closer to the actual formula by using those additional data points. For more information on fitting least squares linear regression models in R, see this tutorial.

• If you fit a linear regression model to the 2000 points provided (after normalizing the scores to a total of 1 for each observation) and look at residual plots you will see huge heteroskedasticity (especially for higher displayed averages.) The patterns of residuals are very good indication of the existence of non-linear features which the question indicates. In your example regression it's not surprising that you get a perfect fit as you've fitted a 5 parameter model to just 5 data points. Other than in degenerate cases this will almost always be the result. Sep 9, 2022 at 8:35
• I goofed and didn't realize the EtherCalc table contains the whole dataset. I might delete this answer. Sep 9, 2022 at 10:52