Thursday, February 13, 2014

Gaming the system

Introduction
I'm of the opinion that most systems have been effectively gamed. When introducing an idea, it is important to define the idea and so let me try and characterize a gamed system. To me, a gamed system is one where a participant is able to pass the system's tests but not through the intended pathway, thus allowing participants without the skill being tested (with or without some other skill) to get credit for passing the system.

Example scenario Uno: Indian schools
To illustrate this idea, take the case of exams in Indian schools. For my 12th grade exams, I was exceptionally weak in some subjects (read: Physics), yet I managed fine in the exams. Along with me, I know quite a few other kids who understood little to nothing on the conceptual level yet received high marks in the exam. Here, a pass equates to scoring high marks in the exam, the test is the examination and the intended pathway would involve understanding the concepts at a deep level and the ability to think, analyze and understand (the skills being tested). The problem here lies with the limited pool from which questions for the examination appear. At most a few numerical values are changed, and questions are slightly rephrased, but they all boil down to a few thousand questions at most.

Missed pathways
And the aid/missed pathway that lets you solve this problem are the guidebooks (Pradeep's, anyone?) that have a whole bunch of questions and the solutions to these questions. It is important to mention the presence of solutions, as without that, the guides would just stimulate thinking. The ease with which solutions are obtained have led to a huge reduction in this thinking ability, that now when we (most Indian students, I generalize) have to, we find it really hard to, or just forsake the logical route in exchange for some random numerical substitutions, hoping the answer pops up. (Another aid/missed pathway are the coaching centres/tutions that students go to, where any questions you may have missed are drilled into you, with discipline (having to solve problems within a given time period) being forced upon you.)

Example scenario Dos: College exams
Schools in India offer precious little in terms of intellectual development and that can be scarily exposed in college. Unfortunately, examination systems in colleges are gamed too. The test, pass, the intended pathway and the skills being tested remain the same, but the problem is a slightly different one. The questions still repeat, but because, I assume, teachers are lazy or don't have the time (which doesn't really seem reasonable?) to think of new questions. So, they fall back upon questions they asked the previous year. This system is even easier to game, since it takes a few hours at most to go through a few question papers and solve those questions or learn only those concepts. It's tougher to game this system across the board, as a single teacher who sets original, innovative questions that test concepts learned through the course can hit your overall GPA/score by a startling amount.

Example scenario Tres: Interviews
Last semester, I was surprised(?) to and thrust into realizing that the interview system had been completely gamed too. Again, the problem rises from a limited number of questions, though the reason for that here is quite honestly, tough to gauge. Surely, companies can come up with new questions - specifically, assign people to the task - but even easier, for software jobs, they can just come up with questions to be programmed - for elimination rounds. Sites like Geeksforgeeks and Careercup, with books like Cracking the Coding Interview, however, have swept all possible question categories to the extent that going through just the interview sections of those sites and looking at previous questions asked will put you at a significant advantage. There really is no need to brush up your concepts, or read textbooks, for just a few hours/days of browsing through those sites should land you a cushy job. If it seems obvious that that should help, because, "Hey! it seems like that is preparation, right?", then you are missing the point of this post. The skill being tested is intelligence, not preparation/hard work, I would hope and companies that miss this point will end up getting hard workers, but no real innovators. If that is what they were going for in the first place, then it should be noted that there is no guarantee that the people selected are hard working by nature, just for the interview process, whereas if the system had not been gamed, it seems only natural that the selected candidates would be intelligent by nature, and not just "intelligent for the interview"(I guess the only way that would be possible is if the interviewee had a really good day on the day of the interview). One solution(?) adopted by companies is to ask interviewees to sign NDAs. With sites mentioned above, NDAs just don't work, especially when not enforced and when you consider that people who don't get through the interviews really have precious little to lose in this scenario.

Thinking about effective solutions to these problems is a good exercise, but due to the excuses listed above, I cannot think right now and will hopefully post to this later. [I actually do have a solution to point tres, but I will trackback to this in the future, with a consolidated list of solutions].

No comments:

Post a Comment