I have been getting numerous emails complaining about the map ratings and how they can change so drastically overnight sometime dropping a full 2 or three points. Please stop asking, I will explain why it changes in a little bit. When I built the rating system into the website I knew from the start that it was never going to be a good indicator of the relative importance of how much the “community” liked a map. People here and in other forums told me I was dead wrong that it was the only way to get a feel for how good a map was and it was important that the map archive have a rating system. I can safely report that after accumulating over 107,908 data points they were wrong. The voting record is so far off statistical norms that it is close to meaningless.

Although I do not have a degree in statistics I do have 15 plus years experience in new product marketing and customer feedback analysis and the one thing I found is that it is more meaningful to know what people do than what they say. In the case of the Halo CE file archive what they do equates to the number of downloads, what they say shows up in the map rating.

In most fair and impartial rating samplings there is a statistical normal that look pretty much like a bell curve as shown below. This can be skewed to some degree by various factors but it is the basis that is used to determine if the actual results are valid. The second chart shows the actual rating curve that the Halo Maps users have created by their voting habits. As the chart shows over 68% of the users have handed out the highest rating, only 2% an average rating where 11% have given the lowest rating.

Chart 1


Chart 2


This is a clear indication that the rating system is being artificially skewed and that others are trying to correct by giving the lowest ratings. In other words, as I predicted, the rating system is not showing what the community actually thinks of the maps.

So to answer the question as to why a rating can drop several points in one night: It is because there were a number of high ratings and one or more low ones. This then makes a large shift in the average rating.

Example: if there are 7 ratings of 10 , and 3 of 9 the average is 9.7. Introduce a low ratings of 1 and the average drops to: 8.9 overnight. Why overnight? The data is collected during the day and at night we compile it and update the database.

So how do you fix this issue: Rate the maps more fairly reserving 10 for only the very, very best and 1 the very, very worst.

i agree with dennis, dont rate the files just because u really like it. and its better than u. rate it by how you would think it would affect you....or something

Dennis, Do you mind if I make a suggestion? Well anyways, I think people should be able to view what votes occurred. Like pull up a chart like above that lists the votes for that particular file/map

Having that information available will skew the ratings even further which is why we don't post how many people voted for each map. There is a number of votes that is required before a rating is tabulated and shown but beyond that additional information will skew the results even more.

You will notice that I did not speculate on any "reasons" why the results are so far off and that is because the audience would not be open to them being related to age group and motivations.

I think you should mix in some of the other data like downloads and create a whole new rating. Like downloads change the rating slightly. Voting does another chunk. And maybe you can give your own rating when you upload it. But that would be not such a good position to place yourself in.

This is a little stupid: "OMGZ DENNIS! WTF, MAI MAP ISNT A 10 BECUZ I THINKZ IT SHOULD!" That's just plain ridiculous if people complain about map ranks. No map has a perfect 10. If they think their map should be higher, then that's their problem. It's the creators opinion and it is extremely different from anyone else's. But are you serious, people were complaining that the map rank of their map was lower than what they thought it should be?

Edited by DarkHalo003 on Aug 17, 2008 at 05:29 PM

That is exacty the point however 68% of the votes are rated 10.

The "how people would normally vote" is based too much off a mathematical model, but the way we vote seems to be simple: good or bad.

I've given up rating maps a long time ago. I just look for what pleases me and don't care what everyone else says about it.

i dont ever look at the rateings. if i were to see how good it was i would look at how many people downloaded the map. that gives u a true rateing. the more ppl that downloaded it, the more people like it. the less people download it, the less people like it. its just common sens.... y would u download something if u dont like it. unless some1 downloads something and then hates it and deletes it. but thats only a really small percent of the downloads.

#1 kaos: Old post is old.

#2 Good find.

Here's my suggestion, Dennis: You have a server detector thing right? Why not detect how many people are playing what maps for so many hours and have that count for a star of the ratings? Then if the ratings on certain maps are too high for their not being played online at all (excluding ai maps) then have something that counterracts the votes until it reaches no lower than 2 stars?

May be complicated but I think it would work.

thats acctually a good idea.
altho it is kindov like what i had sayd. instead of downloads, more accuratlly, playing hours.
good idea tho itl be quite hard to set the whol thing up