Coding Surface Errors

So what is a surface error, anyway? It's usually the kind of thing a student would call a "grammar mistake." It might be a misspelling, a comma splice, or a problem with number agreement. When I coded my data, I decided the best way to standardize my coding was to code those things that handbook listed as an error. Since the department required us to use the sixth edition Diana Hacker's "A Writer's Reference," it was the obvious handbook choice. I knew the handbook well, and I knew that it was a required text for the classes I was studying. I read through each paper marking errors according to the section Hacker covered them in. This method isn't perfect. Some "errors" are actually stylistic choices. But interviewing each participant about each error was out of the question, so I marked them as errors anyway.

Then I counted the errors in each paper. But I figured it was the concentration of errors that was important, not the absolute number. That is, if a student wrote a single paragraph which contained three errors, that translates into a higher concentration of errors than a student who wrote a ten-page paper with the same number of errors. So I calculated each student's average number of errors per 100 words.

The really tricky part is that I decided to divide the errors into "cognitive errors" and "performance errors." Cognitive errors follow a certain mental logic and are fairly consistent. A student may consistently write "a lot" as "alot." Performance errors are slip-ups. A student may type "they" instead of "the." I hypothesize that procrastination should show stronger correlation to performance errors than to cognitive errors. Performance errors may pop up when a student is rushed and doesn't proofread. But cognitive errors are difficult for students to catch in their writing, since they don't usually realize that they are making a mistake at all. So proofreading is unlikely to help, unless the student gets outside help from a friend or tutor.

So how exactly can I look at a grammatical mistake and know what caused it? How do I know if the student merely made a mistake or whether they misunderstood the rules of writing?

Two ways. One, I looked for patterns. If a student always puts a comma before prepositions, that's a cognitive error. They're clearly using a rule, even if it's not the right rule. On the other hand, if a student only once fails to capitalize the word beginning a sentence, there's no pattern. It's not likely to be a cognitive error.

The second way I determined if errors were cognitive was based on my experience as a writing tutor and teacher. I knew that certain errors were more likely than others to be cognitive errors. For instance, students who violated punctuation rules when joining independent clauses were often obeying the "comma when you take a breath" rule. Knowing that certain types of error were quite likely to be cognitive allowed me to classify an instance as cognitive if a pattern wasn't totally clear. But if I couldn't tell, I classified errors as performance errors.

I also calculated cognitive and performance errors per 100 words.

I Remember Coding Data

So I'm working on my methods section. But, you see, it's been a couple of years since I actually conducted my study. I've moved twice since then. So my memory is fuzzy and my records are poorly organized. It took me over a week before I tracked down the questionnaire I used. Now my next goal is to write about how I actually coded the data. Luckily, I put a great deal more thought into coding than I did in formulating the questionnaire. And luckily I recorded at lot of those thoughts on this blog. So I don't have to reconstruct quite as much.

Basically, I looked at three things in the students' papers I collected. Length, surface errors, and use of evidence. So let's start with the simplest one, length.

Length is pretty straightforward. I counted the number of words in the papers. Some of the students who wrote the papers probably weren't familiar with this method of determining length, since they'd enlarged their margins and font size to make their papers physically larger. I did the word counts by hand, since I had hard copies of the papers. I didn't count words that were part of citations or headers. Only the actual body text. I did count words that were part of quotations.

I felt that length might not compare well across classes, because even though the three instructors were working with a pretty standardized syllabus, they might have had very different ideas of what length requirements meant. An instructor who automatically failed any paper which didn't meet a minimum length would likely receive longer papers than an instructor who considered length a guideline to help students figure out how much depth to go into. If such differences existed between the instructors of the classes I studied, I didn't want them to overshadow differences that might be associated with procrastination. So I found the median length of paper in each class in addition to an overall median length for all the papers. I considered a paper short if it fell in the first quartile of length in its class and long if it fell in the fourth quartile.

Does cutting time mean cutting revision?

Believe it or not, I'm actually working on my thesis again. I'm only producing a couple of pages a week, but that's more than I produced for the entirety of the summer.

What I want to look at in the next week is the issue of time. In composition, the main criticism of procrastination is that it leaves the student with too little time to write. "Don't wait to the last minute to start your papers," we advise our students. When procrastination is mentioned in an article, it is usually followed with a reference to abbreviated revision time. Donna Gorrell says that students use procrastination to motivate themselves to write, but this creates the problem that "ideas which are essentially rough-hewn must remain that way, since there is no time for revision" (645). Ronald A. Sudol suggests (in 1985) that limited access to computer labs will prevent students from procrastinating, since they can only write when they can get in the computer lab, and that "the time spent reducing procrastination now can be better spent revising later" (332). This is the composition party line--procrastination is bad because it reduces time writing which eliminates revision.

First off, let's question the assumption that procrastination will necessitate chopping off some element of the writing process. When I delay a writing project, I have in mind how long I think it will take me to churn something out based on my personal writing process. I don't cut down my writing process, I just move it as close to the deadline as possible. I prewrite, I draft, I prewrite, I draft, I revise, I revise, I research, I revise, I revise. This is the same writing strategy I used as an undergrad. I may compress my writing process based on time, but I don't eliminate chunks of it.

Notice that we assume that revision is the part of writing that will be cut--not prewriting or drafting. The obvious reason is that revision occurs at the end of writing, and if we've managed our time poorly, we run out of time before we get to that part. But is that even true? I think Gorrell's example suggests that students use procrastination instead of prewriting, not revision. She writes that "Having a paper due the next morning is a marvelous incentive for the flow of ideas. Thoughts which earlier would not come now suddenly spring forward" (645). Isn't she suggesting that students procrastinate as an invention strategy, a kind of prewriting technique? Maybe where they're cutting time is actually on the front end of writing, not the back end.

And anyway, do we buy this "front end"/"back end" business? I thought we agreed that writing was recursive, not linear. Remember my writing process from above? Certainly revision can't occur at the beginning (unless one is reworking an old project), but it isn't a set amount of time at the end of a writing session. "Okay, done with drafting. Now time for revision." Some people may write that way, but certainly not all people. Even if the back end of writing is what's being cut off when students procrastinate, that's no guarantee that revision is skipped entirely.

My advisor has also helpfully suggested that I look at recommendations to "set aside" writing for an hour or a day or whatever. This is another prescription that revolves around time at the back end of a project for revision. But we should remember that time on the front end of a project can also be useful. We wouldn't likely label it procrastination, but we can let our ideas "incubate" before we start writing, adding time to the front end of the writing process. Donald Murray recommends this in "Write before Writing," and Maxine Hairston suggests that writing teachers can tackle their own procrastination by accepting is as part of a productive writing process and allowing it to pass after a few days (68). But we rarely give this kind of advice to students.

Works Cited

Gorrell, Donna. "A Comment on 'Toward Irrational Heuristics.'" College English 44.6 (1982): 644-645.

Hairston, Maxine. "When Writing Teachers Don't Write: Speculations about Probable Causes and Possible Cures." Rhetoric Review 5.1 (1986): 62-70.

Murray, Donald. "Write before Writing." College Composition and Communication 29.4 (1978): 375-381.

Sudol, Ronald A. "Applied Word Processing: Notes on Authority, Responsibility, and Revision in a Workshop Model." College Composition and Communication 36.3 (1985): 331-335.

Traditional Process Studies Had to Omit Procrastination

My study goes against the grain of process pedagogy, I think. You see, procrastination is a process issue, even if no process theorist has ever talked about procrastination in an article. But I'm really studying the written product! I'm not analyzing protocols of the steps the participants went through as they wrote their papers. I'm circling and categorizing errors. Sure, I'm also looking at questionnaires in which the participants gave some information about their writing process, but this isn't how process studies traditionally go.

And that's got to be part of the reason that procrastination wasn't studied during the heyday of process studies. If you're doing the cognitive protocol-based study, you're in a controlled environment watching the participant go through the whole process of writing. And while you can get useful information that way, you're missing a huge part of the picture of writing. The huge part that allows you to procrastinate for hours, days, weeks, or more. Time.

If you take writing out of a laboratory setting, a lot more goes on. You can't very well follow a freshman around for the whole semester to jot down every time she thinks about her term paper out loud. So while technically procrastination is a process issue, it didn't fit into the model for empirical studies that process theorists were using.

This is bad. What it did was allow our profession to have strong feelings about procrastination as an important issue since it apparently represented a dysfunctional writing process without research to support our beliefs. If we're teaching process, we're not going to just let our students wait until the last minute to start working on their papers--they need to be thinking and prewriting and drafting and revising over the course of time. Not just churning something out the night before the paper is due.

But none of this is based on empirical research. There weren't any studies done in our field to suggest that procrastination led to bad writing--and how do we know it's part of a bad process if we don't know it leads to a bad product?

"Uneducated"

Someone I know personally has recently been slinging the insult "uneducated" toward people she disagrees with politically. She, however, did not graduate eighth grade and has a low level of literacy. I don't really know how to react to this. Even if people are uneducated, does that make them bad people? Does it make them unable to have reasoned opinions on political issues? And how does an uneducated person assert her superiority to others by suggesting that they are uneducated? Is the insult that entrenched that she can say it without blinking?

"Writing" is self-paced, but a narrow view of writing

In "Writing as a Mode of Learning," Janet Emig indicates that writing is self-paced, and that "one writes best as one learns best, at one's own pace" (12). I'm reading and rereading this in context to try to determine whether this has anything to do with procrastination. Out of context, I could jump up and say that if writers ideally write at their own pace, we must be careful of pushing deadlines and labelling writers "procrastinators" for not meeting those deadlines.

But I hesitate to say that's what Emig was really getting at. She elaborates on the sentence in question by saying that "to connect the two processes [writing and learning], writing can sponsor learning because it can match its pace" (12). Now this may still imply that students have to write at their own pace, but I don't think that is Emig's point here. I think she's saying that students learn through writing because they only go as fast as they put words on paper. In fact, I think she's using the narrow view of writing I mentioned in my last post--actually sitting down and putting words on paper, not considering an assignment, thinking about a topic, doing research, and so on.

This reading of Emig is supported when she cites Sartre, who said he couldn't write anymore because listening to himself on tape was no kind of way to revise. He might need to review his words slowly, or he might want to skim through, but where writing and reading allow for that kind of self-pacing, talking and listening to a recording do not (13).

Emig, Janet. "Writing as a Mode of Learning." 1977. Cross-Talk in Comp Theory: A Reader. Ed. Victor Villanueva. Urbana IL: National Council of Teachers of English, 2003. 7-16.

Writing vs. Doing Homework

It just occurred to me as I am reading old process literature, that it was quite easy to skip over procrastination based on certain approaches to studying process.

A certain segment of process researchers were looking for a scientific way to study writing. They used tape recorders and coding systems. They wanted to be objective and to have replicable results. These are essentially lab-based studies, then. The researchers would set up the lab, give the student an assignment, and study them on the spot, likely asking the student to talk through their thoughts and actions as they wrote.

This kind of research is useful and even important, but by being lab-based, it loses the ability to study procrastination at all. I mean, I suppose some students would still delay in a lab environment, but it's a rather different kind of situation than being given an assignment to turn in within a week, let alone by the end of the semester.

This makes some sense, in that the researchers were more interested in writing itself, not "doing homework." On the other hand, psychology researchers have tended to be more interested in the "doing homework" side, and this explains why I have found more of their research to be directly relevant to my study.