17 septembre 2005

#$@&^@#$*&^@!#!!!

60 questionnaires that i can throw in the trash!! that hurts! i find it ironic, though, that it's iu's responses. in july, when i still throught we could do a pilot, i brought 100 copies of the student questionnaire to that school. later, it became impossible to use them because of their !@#*&@#!! irb, and i had to revise the questionnaire for the real study. i made a LOT of revisions: cut 1/3 of the questionnaire, changed the order of the questions and the sections, changed the negatively worded questions to word them all positively, etc. then iu's program director said she'd still help me. it was very fishy, though, for me to use them again, because they are on a quarter system instead of a semester system and consequently couldn't participate in the second stage of the project and a few other things too. but oh well. i sent them the new questionnaires with all the translations... and then the director wrote me an email asking if she could use the english version only, since her students were advanced english learners and i said of course (makes it easy for me when i enter the data).

got 60 responses yesterday... and you've probably guessed what the problem was. they used the old version of the questionnaires, the copies i had brought in july for the pilot! yep. darn! i thought of entering the questions that were exactly the same in both versions of the questionnaire, but statistically, that's a really bad thing to do, because it's been proven that the order of the questions will influence the answers. also, responses to negatively worded questions are not the same as when the items are worded positively (it's not just a matter of changing a 5 to a 1 and a 2 to a 4; it's been proven that students would not have responded the same way to negatively or positively worded questions because they would be influence by the negativity or positivity of the question; plus negatively worded questions are a mess for international students. if you say "my teacher doesn't make a lot of grammar mistakes when she speaks," some students will answer "no, i agree, she doesn't" and others will say "yes, i agree, she doesn't"). so there, completely useless! i guess it's probably better, given the above-mentionned problems, but still... it hurts! i REALLY hope umbc doesn't do the same thing...

other than that, i got an interesting email from one very kind program administrator (one of the first schools to enthusiastically say they'd participate in the project) who told me that she and her teachers had some comments about the teacher and administrator questionnaires. they said my questions implied that there WAS a difference between native and nonnative english speakers. i guess it's true, because i never imagined that there could be NO difference... but it's an interesting perspective and i should have thought about it when writing those questionnaires. oh well, i must remember that this study is A PILOT in a sense, and CAN'T BE PERFECT! i have 173 useable questionnaires as of today, and already noticed so many problems with the questionnaires and other things that i am scared of what i'll discover when i get more responses...

... i have to admit one thing, though: i'm still kind of having fun with this project! it's REALLY cool to get new envelopes every day, to take note of all the problems and all the things i want to remember to write about, to get comments and questions from some participants, to see the list of participants getting longer, to receive emails from schools saying that they just put 150 questionnaires in the mail...

ps. got another 44 questionnaires tonight and one of my articles was accepted by the INTESOL Journal. yeay :)

2 commentaires:

At 9/18/2005 10:51 AM, Blogger kiara a dit:

Just a thought: you're learning a lot through errors and feedback from people to who you've sent questionnaires, etc. Could one chapter of your thesis being about what you've learnt, like writting a detailled protocol with what not to forget and what not to do/what to do when you want to do that kind of study ? In science we sometimes publish technical papers with detailled protocol, and those are the most usefull publications... well, just an idea, don't know if it's relevant for you though !
Anyway, congrats for your paper ! :)

 
At 9/18/2005 11:27 AM, Blogger lucie a dit:

kiara: yeah, i'll do that in a section called "suggestions for future research" or something like that. i also want to write an article about getting irb approval and working through those different protocols (my summer's hell). it's true that it's the most useful sections, in general :)

i still hope that no one will EVER try what i'm doing, because it's just insane!!!

 

Enregistrer un commentaire

<< à la maison