D5_Evaluation+Reports+and+Field+Work

D5 Prompt Prog Eval Spg 11

1. Now that you are well into the idea and process of an evaluation, what things are suprising to you and why? What are you finding the most challenging or difficult with the project implementation especially your data collection and analysis? Your response to these questions will be your first initial post. Subject heading should be last name + Challenges.

Huett + Challenges As I told my Wimba group last week, I was really surprised at how long it took to prepare the solicitation for participation. In my case, I planned to interview 13 people. To get 13 to participate, I email-solicited approximately 70 people. Preparing the email messages (there were about 4 versions, due to there being 4 different types of people), along with the attached informed consent document and the attached schedule of interview times proved to take what can only be described as an insane amount of time. At the end of that Sunday, I felt very disappointed in myself and I felt doomed to failure of some kind. I wondered, why would these people want to speak with me about this? No one is going to grant me an interview. Within 3 days of sending out those solicitations, however, I had all my participants lined up. So, that was a surprise. A low point followed by a higher point. In the end, it teaches you that to conduct research, you have to keep moving forward, and you have to believe in what you're doing, because no one else will.
 * Now that I am well into the idea and process of an evaluation, what things are surprising to me and why?**

The first thing that is hard is that given the lack of time and resources, I want to do a good job, and yet, I have to realize that I may not do as good a job as I wish. I want to take it seriously, but at the same time, it may not be the most amazing program eval my client has seen. I have to accept that and just get the job done as best I can.
 * What am I finding the most challenging or difficult with the project implementation, especially my data collection and analysis?**

In terms of data analysis, I am conducting qualitative analyses of my interviews. I have some amazing qualitative data analysis books at my disposal, but I don't have the time to read them, understand them, and then implement what's in them. This drives me a little nuts, but I have to live with this. Last weekend, I conducted some qualitative analyses for another project, and my eyes were opened to 1) how time-consuming this approach is and 2) how uncertain I am if the story that I see emerging is "valid." (probably not the right word). I understand that no two qualitative analysts going at the same mountain of data would arrive at the same story. But, you still want to handle the data properly and draw your conclusions in some kind of sound fashion. I feel uncertain about how I'm doing in all of this.

Dr. Baylen recommended that with my 13 interviews that I don't transcribe each one, but instead use "selective transcription". And another professor told me that for every 1 hour of interview data, you've got seven hours of work (transcribing, coding, etc.). I know he's right! Even with selective transcription, I continue to be somewhat uncertain as to the outcome. I know that once it's finished, I'll look back at it and say, "That wasn't so bad after all."

2. Your second initial post should focus on lessons learned from the "evaluation" experience. In 20/20 hindsight, what would you have done differently now that you are at the end of the experience? Subject heading should be last name + Lessons.

Huett + Lessons


 * What lessons have I learned from the evaluation experience? What would I have done differently?**

I have learned....
 * I used to think I was a qualitative researcher. Now, I'm not so sure that I am going to so naively make this statement (to myself). Undertaking qualitative research is a huge commitment.
 * In program evaluation, as in all things, it's good to obtain multiple perspectives. While I appreciate the Kaufman et al. textbook for its philosophy of results-based program evaluation and its emphasis on the connection to Mega, I was relieved to read Stufflebeam's monograph and learn about many other acceptable approaches to evaluation. Further, I obtained a copy of a qualitative methods book, with a chapter on program evaluation. The philosophies of evaluation described there did not resemble the Kaufman Way, but they were still acceptable methods of evaluation. As many research professors will tell us, each educational problem deserves a unique approach to its study and possible solution.
 * No matter what, get the course readings done on time! Reading empowers us so much. A professor friend told me recently about how she caught her students not doing the course reading, and when they arrived at their face to face meeting a couple of weeks ago, it was so irritating to have to change the plan for the class session. These were doctoral students. I am sometimes guilty of not doing the readings on time, but I always try to get them done. My advice to myself or the how-to-do-it-differently tip of the day is this: at the beginning of the semester, get ahead and do as much reading as you can before the real work starts!
 * In research, there is no time for emotional procrastination. I don't know about the rest of you, but I sometimes indulge in this practice. Research takes enough time without our wasting time worrying about how we're going to do it. I think sometimes that I think myself into a corner. I need to tap into my "doer" self, the one that jumps in and tries things, even if all Ps and Qs are not 100% accounted for, even if there are risks involved, and even if I might make a mistake.
 * Many other things. I don't want to reveal too much here. After all, what will I talk about in my final course reflections?

3. You are expected to actively interact and respond to your peers’ postings. You can do this by using the following interactive strategies ---

• Sharing additional evidence that support the position stated by another student (A) • Giving an example of the concept being discussed (B) • Asking for clarification (C) • Restating a position in different words (D) • Introducing a nuance that enriches the original formulation (E) • Offering a different opinion (F)

You are expected to do one of each response strategy. Make sure you change the subject heading.

4. Finally, post a third initial post about the status of your evaluation project plus your reflections thus far of your growth and development as an Ed.S. student and soon-to-be teacher-leader given your context. What knowledge and skills have you acquired? What else do you need to be successful?

Status of Project I have written my plan and rewritten it. I have collected my data. I have begun selectively transcribing my interview data. I need to finish this process, then organize the data into patterns. Finally, I need to write it up and offer suggestions. I have a lot of work to do, to be sure, but plan to work on this diligently every day till it's due!

Growth as an EdS Student I have been really pleased with the EdS, and I am glad I took the time to pursue this degree. Several people had told me to skip it and go straight to a doctorate. They were being pragmatic. But I knew that I needed to learn more, even if I wasn't always sure what that was.

There are many routes to becoming a teacher. I took the Master's plus certification route at a regional university in Texas. I was dissatisfied with my learning experiences. After having gone through a bachelor's in English and in Spanish lit, I felt under-stimulated and under-challenged by my Master's, which seemed to expect so little of me. Then, I entered the classroom, where I had to scramble to figure things out. Six years of teaching secondary English and Spanish turned out to be very stimulating, challenging, and rewarding, but I felt like I wasn't able to connect the dots to my satisfaction. I should have studied up on education (the politics, the philosophy, etc) more, but in hindsight, I really doubt there was time. I was so exhausted all the time.

Anyways, long story short, I have felt like I have begun the process of filling in some gaps in my education on education. More important than gaining discrete knowledge, this EdS has given me a set of tools that I can use to be a better problem-solver. For instance, the Diffusions of Innovations course taught me about change theory and important considerations for brining about change. That's an important tool for every educator to have. These two research courses have been amazingly eye-opening, and I think that what I have learned here will serve me well to make better-informed decisions.

To be successful, I need more formal training in research techniques. I need to study the literature in the field that interests me (k-12 online learning), and I need to attempt to add something of value to this conversation. The EdS program has been an important phase of my professional development, and I highly recommend it to all teachers I talk to (the EdS route in general).