On the first day, I tried to take in all the amazing complexity of the grading experience. I was very impressed with the procedures put into place to safeguard an accurate fair reading. There were five problems on the APP1 exam. No fewer than five people read each student's exam and possibly as many as ten. About seventeen people read Question 4 of the operational exam. The number of people working on each question depended upon the leadership team's estimate of the bottle necks. For example, question 2 had 80 readers. Question 2 was the laboratory design question and in past years the laboratory design problem had been the bottle neck. As the reading progressed the leadership team monitored the statistics and adjusted the allotted readers moving readers from team to team to make sure that everything was optimized so that we would finish in time. In fact they did such a good job of pre-planning and adjustments that we finished many hours early.
I had worked out question 4 before attending and thought that it was relatively straight forward. I predicted that it would be difficult for students due to the fact that it requires students to be able to imagine a traveling wave given only a word description and a static picture. Coming into the reading I didn't think there would be much uncertainty in the rubric. It turned out that I was right about many students finding the problem difficult (I estimate the average to be 2.5/7) and my estimate on what the rubric would be was correct too. However, due to student responses the interpretation of the rubric was challenge. Since this was only a seven point problem, the numerical part received credit only if the answer was correct. This meant that if students used incorrect physics and yet their answer was the correct number they earned credit. Although this was rare it did happen. More commonly students incorrectly added such that their final answer was incorrect but their logic was correct. They received no credit. Additionally, we were forced to interpret the vector drawings. Sometimes we were too strict but I feel that more than likely we were too lenient. I think this could have been avoided if a grid would have been overlaid on the wave form. If this would have been done I estimate that the average would have been lower.
The most surprising thing for me was how at least 40% of the students left at least one item blank. I would estimate that 20% of the students I graded didn't even attempt the problem either leaving it completely blank, drawing pictures, telling jokes, writing life stories, complaining, apologizing, or pleading. The reason why this is surprising is that I believe that these students could have earned a few points even if they didn't know how to answer the question. It seemed like either the students were trying to make a statement by performing academic harakiri or were frozen into in action by the challenging material. I remember one student writing an apology after attempting the problem saying that they did their best and that they hadn't studied wave behavior. She followed the directions and made some educated guesses and that earned her 3/7 point. Not bad. Much better than the students who complained and made absolutely no attempt. It is true that there were students who fully attempted the problem and who earned a 0 but I still think that is better than leaving it blank or using it as a diary entry.
The other surprising thing is how many students used ink pen. Incredible! I would say of those answering the question at least 60% used an ink pen. It was so sloppy and hard to read after all the student scribbles and re-drawings that I have no doubt that they scored much lower than the pencil users. It definitely cost them points. Some students even wasted time in redrawing graphs in the margin due to their sloppy scratch marks. How much time did that cost them I wonder. I now resolve to make the following announcement and on my syllabus, "students are required to always use pencil (never ink). Every graded item in which ink is used will receive no points.".