The MTC is a great competition. It gives interesting building challenges each month and allows builders to receive criticism on their works. It instills competition and encourages builders to not only build outside their comfort zone but to focus on building up their skills as well. After all, you don’t win an MTC unless you’ve proven that you can make a good map. The MTC also gives players a chance to voice their opinions on what makes a good map by opening up the voting to the public. Add some comments to a score, and builders know what to change up for their next creation. There’s a flaw in this system, though; because the maps are publicly available before the voting period even begins, there is potential for accidental bias to occur. By noting that some maps already have some amount of awards while others do not, it is entirely possible that we unconsciously grade certain maps better than if we did not know this information in the first place. This is called the Anchoring Effect.
I’d like to start talking about this by recommending that you watch
this video before reading further; while it talks about the Anchoring Effect in terms of video game review scores, the same principles clearly apply to other areas (such as MTC judging). If you’re skipping the video, the Anchoring Effect is essentially this: if the first bit of information on something you see is positive and seems reasonable, then your opinion on that something will likely be more positive than if you had not been given the info (and vice versa). The Anchoring Effect doesn’t invalidate the judging process; it’s just something to think about and possibly try to mitigate. As a builder, I want the most honest opinion of my map a judge can give. As a judge, I want to make sure I’m giving the least biased judgement I possibly can. The Anchoring Effect is just one more hurdle on the road to removing this bias.
I think there’s a relatively easy solution to help with this, and we already have a system that could be expanded upon to make this work. I talk about the map pools; while they currently apply to only KO maps, it would not be infeasible to expand the system’s functionality and apply it to a broader array of maps. To start, this doesn’t need to be a public system. Keeping map pools to being created/managed by moderators would be fine. These pools could be extended from KO pools to include grouping MTC maps together as well as other types of maps (like tournament maps) if desired.
Pools as they stand won’t solve the problem, though; they still need a few more functions for what I’m suggesting. First and foremost, the ability to lock/hide awards from the public until a certain date or within a certain time frame. In terms of the MTC, this just means that submitting a map through that month’s map pool will prevent people from awarding it until after the voting period is over (or would at least hide the number of awards from the public). By doing this, there’s no chance that a submission would look better than another by the number of awards it has. Another potentially helpful function would be preventing updates of a map, again either after a date or within a timeframe (just to guarantee that a map doesn’t change versions during the voting period), but that’s just an aside by me that doesn’t relate to the Anchoring Effect.
Working on the Anchoring Effect isn’t a pressing concern, of course. The system as it stands is perfectly fine. It’s just something to think about doing, and I understand if it’s not something that can or will be done. Anyways, that’s the end of my essay. If you have any thoughts on this, feel free to discuss it here!
