Identifying rating algorithm

Question

I'm trying to identify the rating algorithm used to calculate the displayed average of a 1 star to 5 stars rating system. To analyze the data I collated the first and last 1000 ranks and added the real average.

Please see my EtherCalc table.

Here is what I know about the underlying algorithm:

• It is symmetrical.
• It gives more power to the majority of votes.
• It is pretty simple and not (intentionally) based on any known (complicated) mathematical method.
• No information other than the respective number of votes is used for the calculation.
• It tries to give supposedly dishonest votes (those allegedly submitted just to force the rating up (5 stars) or down (1 star)) less weight, so not every vote is worth the same. For example: If the vast majority of votes concentrate around 1 star and 2 stars votes with a smooth decline and a sudden jump at 5 stars votes, those 5 stars votes will count less (not necessarily 3.00 or 4.00, it could also be something like 4.88).
• To detect outliers (suspected dishonest votes) a relative smooth rating curve is expected.
• Items with overwhelmingly favorable ratings get bonus points what gets them over 5 stars.

I guess the easiest way to determine the algorithm is to analyze the items with the fewest numbers of total ratings. For example:

rank	5 stars	4 stars	3 stars	2 stars	1 star	total	displayed average	real average
535.	15	1	3	0	1	20	5.42	4.45
9802.	2	3	10	0	0	15	3.11	3.47
9908.	2	1	3	2	2	10	2.86	2.90
9944.	0	0	3	0	1	4	2.71	2.50
9978.	0	0	3	0	2	5	2.26	2.20

Since I've been thinking about this for too long and can't come up with the solution, I kindly ask for your valuable help.

Answer 1

First, this is a comment at the moment, but I cannot add comments still not having a minimum of 50 reputations.

You have not given some information on which your knowledge is founded. That might help to confirm those - or not.

Am I correct that you cannot create a response from the site by creating specific input for your own defined combinations?

Then I would try to attack the algorithm by

• selecting most simple combinations, and when displayed, and the real average is not the same. Like here:

• Or selecting combinations where only a single star rating is incrementing and the others keep the same

Edit 10.04.22

Meanwhile, I did a second approach: guessing an almost linear formula with a fixed factor for each star, so that the average is calculated by (number of 5 stars)*a+(number of 4 stars)*b+etc/total. Then I selected the ones with Zero 1 star, and Zero 2 stars and optimized the factors, resulting in

• 5 stars: factor 5.705
• 4 stars: factor 4.19
• 3 stars: factor 2.335

Total difference of deltas of that model (abs(result - displayed average) versus average model is reduced, but not Zero => guessed formula is not good enough. This is expected from the information you gave for the algorithm. However the biggest deltas are found for good scored results => a new model is to be guessed to represent that non-linear impact.

I think in that way you can come closer to a model that is good enough and get a final low delta, but it will never be the original algorithm. Alternatively, an "automated formula guesser" is needed, that combines the 5 variables to a result that comes close to the displayed average. However, this will also most probably not be the original algorithm.

Answer 2

I think what you are looking for is a statistical method for computing the formula. If you fit a least squares model to the five data values you've provided, you can get a prediction formula for this data that looks like:

displayed_average = 3.6746154 + 0.162307923 * five_stars + 0.275384615 * four_stars -0.171548462*three_stars - 0.45 * one_star

which predicts the displayed average as

Obviously, this is only based upon the five data values you provided. If you had 10000 values, you could come a lot closer to the actual formula by using those additional data points. For more information on fitting least squares linear regression models in R, see this tutorial.