D7_+Peer+Review+of+Draft+Research+Plan

PEER 1: Sheila Blocher http://8484wikisdb.wikispaces.com/

//1. Did the author define the key terms used in his/her research problem or question?// Yes, she defines, CAI in the literature review, citing a recognized definition of the term.

//2. Did the author group his/her relevant articles based on similar ideas in support of his/her research plan?// Yes, and the articles flow towards the research question. The lit review is shaped by ideas.

//3. How did the author write his/her literature review? As a synthesis or as an annotation?// She wrote her lit review as a synthesis and included article summaries beneath it for reference. For the final draft, I think the summaries can safely be removed.

//4. Did the literature review provide a substantive arguments to support the author's research problem or statement?// The lit review does a good job of leading into the research question.

//5. Is the data appropriately answering the research problem or question?// As it currently sits, I don't think so. But rather than change the data, I actually think some changes need to be made to the research question, which would dictate a change in methodology (and data collection and analysis). I think the research question ought to be more specific about the type of CAI, the setting, and the population under study. As it currently sits, it sounds as if you are trying to do a large-scale study on CAI and math. It sounds like any population and any CAI intervention will do the trick.

//6. Is the data collection strategy free of contamination of extraneous variables?// In terms of variables that could affect the outcome, if different teachers are delivering the instruction, then that may be the reason for measurable effects. Could the extra 10-15 minutes for 8 weeks just be "extra practice" that improves the treatment groups' scores? It might have less to do with whether it was the CAI and more to do with having more practice and exposure.

//7. Is there enough step-by-step description of how the author plans to analyze the data collected?// Yes, nicely done!

//8. Is the research problem or statement aligned with the research methodology or approach as described by the author?// I don't know that it is. If the author sticks with the same research question, then I don't see a need in the methodology for the qualitative pieces on perception.

//9. Did the author include or substantially described the data collection instrument or strategy?// The collection of data is described in non-specific terms. No instrument is described.

//10. Did the author identify possible limitations or challenges in implementing the research plan?// I don't believe limitations have been described.

//11. Will you be able to implement the research plan with the amount of information provided? Why? Why not?// I will need to know more about the CAI treatment as well as the pre and post tests being used. Also, I don't know what to ask in the qualitative pieces (the interviews and surveys).

//12. What would you suggest in order to improve this research plan?// I would double-check my research question and make sure it's specific enough and then just make sure everything around it is aligned.

PEER 2 Checkwood http://checkwoodresearch.wikispaces.com/

//1. Did the author define the key terms used in his/her research problem or question?// I don't believe so. The RQ is this: " what affect would Web 2.0 applications (in particular Blogging) have on High School student’s writing (peer editing, collaboration, et cetera) and how the use of these new virtual applications could or would prepare such students for their college English classrooms (collaboration)? " So, I think there should be a definition of //Web 2.0 applications//, //Blogging//, and //collaboration//. I realize that you discuss some of these ideas at length, but it's not always clear what the definition is. I would recommend changing //affect// to //effect// unless you mean affect as in emotion or feeling, which it doesn't seem you do.

I think your RQs could use some narrowing. As it is, you have too much to explain and too much to study. If you narrow the tool and the type of writing you want to impact, perhaps it would help. Maybe narrowing it to something like this: To what extent are high school English students' journaling abilities enhanced when using a blog? This way you could set up a simple study where one group (the control) uses a more traditional journaling approach (the good ole spiral) and another group (the treatment) use a blogging platform.

If you don't narrow your RQ a bit, you've got many tomes to write to address it all, I think. :)

//2. Did the author group his/her relevant articles based on similar ideas in support of his/her research plan?// The article seem somewhat grouped. For instance, the wiki articles were together at the top. But if your RQ is about blogging, then I would say, for instance, that wikis deserve no more than 1 paragraph (if even a sentence) of your lit review, unless you need to explain "why blogs instead of wikis." Oh, wait a second. I see now that your RQ involves Web 2.0 and not just blogs. Your intro seemed to suggest that it was all about blogs, but I may have been reading uncarefully, too! Okay, so I would recommend in your lit review that you deal with each tool that you have researched and offer insights into why it's a candidate for English classroom writing instruction (and why not, to offer contrasting views).

//3. How did the author write his/her literature review? As a synthesis or as an annotation?// The lit review is written as an annotated bibliography at first where all articles are summarized, and then at the end, he synthesizes things. If you can, you might chop it down to about 5 paragraphs that use the literature to build the argument that your research question is worth pursuing. (I feel like a complete hypocrite: I need to take my own advice! Your lit review is way better than mine!).

//4. Did the literature review provided a substantive arguments to support the author's research problem or statement?// The literature might point to your research question. I think it's difficult for any of us to say since we've just scratched the surface of our respective literatures. I do think if you cut out about 70% of your lit review and focus on presenting it as almost an argument leading toward your research question, it might it tighter. Easier said than done!

//5. Is the data appropriately answering the research problem or question?// It is unclear to me whether or not the data appropriately answers the research question. One data pile is writing pre assessment. Another pile is the writing post assessment. However, I don't really know what that means and whether it can answer the question.

//6. Is the data collection strategy free of contamination of extraneous variables?// I'm not sure, because the strategy is not yet revealed in this draft.

//7. Is there enough step-by-step description of how the author plans to analyze the data collected?// The timeline offers some step-by-step info, but there isn't a lot there yet.

//8. Is the research problem or statement aligned with the research methodology or approach as described by the author?// I would say you can probably cut out the first paragraph in your research methodology, since it doesn't tell use about your methodology. So far, you haven't settled on a research methodology, and I don't blame you! My own RM is unclear to me.

//9. Did the author include or substantially described the data collection instrument or strategy?// The instrument and/or strategy are mentioned in the timeline, but not described.

//10. Did the author identify possible limitations or challenges in implementing the research plan?// None have yet been identified.

//11. Will you be able to implement the research plan with the amount of information provided? Why? Why not?// I need to know more about what I'm trying to do. The methodology section needs elaboration.

//12. What would you suggest in order to improve this research plan?// In your introduction, you introduce your topic of using Web 2.0 (specifically, blogs) to enhance student writing. I think in this section, there is too strong an assumption that technology will help improve writing. Before your article makes that assumption, it may be appropriate to offer some research points to support that idea. A lot of folks (and a lot of English folks on college campuses) do not agree with this. So, I might re-frame that section with skeptics in mind. .... You might use Lanier's anti-Web 2.0 stance in your intro to present an opposing view.

So as not to make you read what I've already written, I will just add that in numbers 1-11 above, I make suggestions for improvement. :)

PEER 3 Morgan http://morgan-medt8484-research.wikispaces.com/

1. Did the author define the key terms used in his/her research problem or question? The RQ is this: "What effect will integrating technology have on 9th grade students’ writing in both gifted and heterogeneous literature and composition classes?" I will hope to see //integrating technology//, //writing//, //gifted//, //heterogeneous//, //literature class//, and //composition class// defined (or some other arrangement of these terms).

A little later in my review of your draft proposal, I see that in the Data Collection section you have changed your RQ to this: "What effects do wikis have when using them to teach writing?" You have narrowed the type of tech integration to wiki use (which is good, I think), but you have in a sense broadened the population (I don't know what level you're studying this at). Also, you might try and narrow the type of writing with which you be using the wikis.

//2. Did the author group his/her relevant articles based on similar ideas in support of his/her research plan?// //3. How did the author write his/her literature review? As a synthesis or as an annotation?// //4. Did the literature review provided a substantive arguments to support the author's research problem or statement?// The articles in the literature review are written more as an annotated bibliography than a synthesis (mine are like yours, too!). Perhaps in the next draft, you can look for the key elements that dovetail into your research question and support its need and relevance.

//5. Is the data appropriately answering the research problem or question?// The type of data collected will be via observation, and I am not sure that it addresses an RQ that seeks to measure "effect." To add to this, your Planned Research Approach page introduces your intention to use quantitative pre and post test measures to measure a baseline and the effects your treatment may have had on it.

//6. Is the data collection strategy free of contamination of extraneous variables?// If you're measuring the effects of a treatment (wikis), how will you know what the baseline was without a pre-measure of some kind?

A little later in my reading, I am at the Planned Research Approach where you talk about doing a pre-test post-test type of structure to establish a baseline. So, I guess if you add this quantitative data to your qualitative, then you're going the Mixed Methods route.

On another tack, your choice to not use a treatment/control structure concerns me. How will you be able to say that improvements or changes are caused by technology integration and not just the instruction itself without a treatment and control? What if your instruction during this experiment is just spot-on and it also happens to include wikis. But what if it's the instruction and not the tech tool (the wiki). What if the instruction + traditional media (pencils on papers) could have had the same positive effect during that period? (Oh wait, on one page, Research Plan, you clearly state you will not use treatment and control structure. Then, on another page, Strats and Tools for Data Collection, you appear to be intending to use the treatment/control structure. So, I'll leave it at that. I am not sure which you're intending to do.)

//7. Is there enough step-by-step description of how the author plans to analyze the data collected?// There's quite a bit of step-by-step info in the timeline section of the plan. I think with some questions here and there and some specific instruments, I could replicate this study.

//8. Is the research problem or statement aligned with the research methodology or approach as described by the author?// I am not sure about this piece. Honestly, my cautious gut instinct is to avoid the word "effect" in educational research (at least until I fully grasp how effects are measured well). See my response to number 6 (third para.), since I would repeat it here.

//9. Did the author include or substantially described the data collection instrument or strategy?// Data collection is described pretty well. I might like a little more description about the nature of the instruments used.

//10. Did the author identify possible limitations or challenges in implementing the research plan?// I don't believe so.

//11. Will you be able to implement the research plan with the amount of information provided? Why? Why not?// I could come to a close approximation. I would need more info on instrumentation and analysis, and I might need to contact Dr. Morgan for clarification. :)

//12. What would you suggest in order to improve this research plan?// Your research question seems to point to a need for quantitative data. You ask "what effect does X have on Y," which seems to point to a causal-comparative (quantitatively-measured) design.

Later on, I see that you update your methods to include quantitative data.

Overall, I have made recommendations in 1-11 above, so rather than repeat here, please see above. :)