3 home truths about research evidence in TESOL

Research doesn’t affect anything except practice among some university teachers or a small number of school teachers. It does not affect materials to a degree significant enough to affect teachers’ general classroom practices. This is OK; those of us with lab coat envy can feel good about our principles while others can feel good about being in touch with teaching realities.

However slavishly you believe that you follow a method or approach, you’re probably following your interpretation of it rather than something the developer would regard as a practice they prescribed. This is fine because it is also informed by judgment of how the people in the classroom would respond to your methods.

We can talk ourselves blue in the face about IATEFL, but I have met approximately two members of that organisation in the time I have been teaching. For most teachers, IATEFL has as much bearing on our professional life as Paris Spring Collection has on the clothes sold in Topshop. You can choose to enthuse or dissent about points of view expressed there there but your enthusiasm or dissent is irrelevant to most teachers. It is your own, and it affects mainly your own practice.

Advertisements

The Personal Side of Evidence-Based Practice

20170918_121002_Pano.jpg
My classroom after laying Post-Its to track where I went and who I monitored.

“Evidence based”. It’s so trendy, it’s science-tastic, you the forensic teacher in CSI (Classroom Sciencey Instruction). How many research papers we have found Open Access, through Sci Hub or requests hashtagging #ICanHazPDF!

It’s not all about reading research. Sometimes it’s more about doing very small-scale research to see what happens. Sometimes using recording media, sometimes just paper and pens.

 

A lot has been written about Action Research by much better brains than me. Anyway, this is a guide to what I might do in a personal action research project.

Gather thoughts

Research for the sake of it is just making work for yourself. What might be better is having a think about what you’d like to understand more about in your classroom(s). Write it down, and keep asking yourself probing questions, for example:

What about this could be a problem?

Is there simply a difference in personal values?

What would do I think is happening? Is this ideal? Is it definitely true?

This is likely to make your findings more compelling to you because they’ll be less superficial and you will understand already what the connections may be to other aspects of your practice.

Design your evidence capture

How you gather your evidence depends upon your classroom and the people in it. You might ask learners to help, or not. You might track your movement, or not. You can use post-it notes to stick in places, on items, etc. You can tally things on paper. You can record yourself on video, audio, or even log your steps taken with a pedometer. What and how you capture it is an important thing and you want accuracy but also ease of use if you don’t have a team (or peer) to help.

Check it before you forget it

You need the time to check your gathered data. Can it be interpreted in more than one way? Which way has most significance for you? It’s likely to suggest further action/intervention or continuing the action you were already doing. If it’s something different, you might need time to prepare and read up on how to do this, or get advice from someone who already does it. Also, keep your information somewhere you can find it. If your new action gets challenged, you want to be able to say why you’re doing it.

More data

As you take your new action/intervention, you may want to write down what happens when you do it, both positive and negative. It may be that any information is not strongly suggestive of anything: rather than stop, give it time or tweak it according to your intuition but write down what you did differently. You might find that the first way was the best way (or not). You might find that this intervention is not as good as what happened before. This is fine, because at least you know that this does not work for me/this class/this situation.

Decide what happens next

This could be a repetition of the same cycle, it could be that you feel you’ve finished it, it could be a return to the status quo. Keep your findings, though. It might be grist for the mill if you or a colleague have a similar train of thought in the future.

Stats analysis for research

Huh? Marc, the blog title is Freelance Teacher Self Development, not High Faluting Would-Be Academic Development!

OK, OK, I get it, really. But what about your CV that has no research on it since graduation and you’re looking for jobs constantly, what with being freelance or serial part time? See, it would be good. Perhaps. If you have time.

So, SPSS – expensive to buy yourself, and a pain in the arse to install cleanly in Windows 10 (from my experience). What can you do?

Well, there’s JASP, a nice but of software from the Netherlands. It has the look of Excel but with bits on the top that do stats magic. You can do ANCOVA and ANOVA analyses (comparing populations, i.e. experimental group and control group), and t-tests (analysing a population, I think but I dropped psychology at university, so quite likely I could be wrong) and regression analyses and make correlation matrices (compare two sets of values; in my case I compared scores for top-down listening teaching with scores for bottom-up listening teaching). I did the last one for my dissertation and it was kind of easy.

One word of warning – keep your data clean. The column heads are heads, below that is data – don’t do averages and standard deviation calculations at the bottom of your columns or JASP won’t read it properly.

You need: data as a .csv file (which Excel and Google Sheets can both make).

Choose the data to analyse. JASP will tabulate or plot it and you can copy and paste it, or output to HTML. It’s not super full of features, it’s a bit experimental apparently, but instead of buying SPSS you can buy coffee, beer, children’s shoes, or whatever you fancy.

And I promise the next post won’t be about stats software!

All subjects were undergraduate students… 

Or, ‘Really, can we only have research based in universities and nothing from other settings?‘, set in motion by a Twitter conversation (DM) with Louise B

The ELT profession is a confounding one. On the one hand we are the lovely, nice people who help communication, while on the other we are cultural-imperialist scum destroying minority languages in our wake. We preach respect for all but turn a blind eye to the precarious work conditions across the sector. I’ve ranted on these themes before. Today, I want to go on about professionalism, and who gets to be professional in ELT. 

While we might call ourselves teachers, how many of us have job titles of ‘Instructor’? How many of us get time to ensure our assessments of learning are as thorough as they could be? How many of us can take findings from our classroom practice and make them known to other teachers, to expand the collective body of knowledge in our field? 

University-based teaching staff (but usually only full-time staff) get to do this, but rank-and-file teachers in schools don’t often have the opportunity to do it. The International House organisation of private language academies has a journal that I suppose its DELTA/DipTESOL graduates publish in (as well as staff with MAs). University teachers have various journals such as ELTJ among others.

Who does research in young learners (YL) settings? There must be a number of people who focused on this setting in DipTESOL and DELTA courses, not to mention Masters courses. Why does it never get seen? 

Well, SLA-type stuff needs ethics approval. This means giving parents approval slips to sign. How confident are you that your organisation or school would be willing to cooperate in this? Maybe not, I’m guessing. However, research doesn’t have to be just based on how or what learners learn. Case studies might focus on what happens in one class, one course, in one department of a school. Action research and exploratory practice may be teaching focused, with a problem to solve or a theme to explore. 

Sharing it widely might be a problem, but it might not be. If you’re in a teaching association you might publish in its journal. If not, do their special interest groups have journals? If not, are there other avenues to explore? 

What might be a good idea, and I would love to help set something up like this (though I am aware that I have no time right now) would be an open-peer-reviewed blog, like Hybrid Pedagogy. It would also be nice if it were (voluntary) funded so that reviewers and writers could get paid for work done in the absence of grants and so that hosting costs would be covered. 

This would be awesome if it were not one of my ideas that are utopian but too much like hard work. The means of getting it done are there, except cash. Anyone who has ideas, links or support, the comment box is screaming for you! 

The Benefits of Being a Guinea Pig

Recently it feels like I have been taking part in a lot of research projects. I don’t think I have, not when I sit and count, but it feels like it. It could be a massive pain in the bum but I actually like it. 
For one thing, other people’s research about teachers and/or teaching is unlikely to align exactly with one’s current wondering about teaching. This is likely to cause one to consider one’s classroom practice, beliefs or CPD process more deeply than before. Another thing is that when you are stating practices or beliefs, you’re making it coherent because you don’t want to be “Well, I don’t know why.” Although I am sure that I have screwed up my face on Skype while searching for answers in the back of my mind. 

Next time you see a survey circulating on social media, why not have a go. Help build our body of knowledge. And help find the figurative lost bit of jigsaw puzzle of your mind from under the sofa. 

The Line Between Hare-Brained and Useful

notebook picture

I’ve had this post going on in my head for a while and probably the catalyst for getting it out of my head and into pixels is Sandy Millin’s Incomplete Thoughts post.

I was having a chat with a colleague yesterday and he said, “I don’t know where you get the time for all your ideas.”

“It’s a massive pain in the arse,” I replied, “because I can’t concentrate on other things when something pops up.”

I don’t know if this leads to a condition of not following things through properly, or even just dilettantism but a few things that have got me going all over the internet are:

Open Badges for Accreditation of Some Kind

This blog being about development (ostensibly, though probably more my own), actually having evidence-based accreditation for continuing professional development (CPD) would be a good thing in a landscape of expensive qualifications, cheap qualifications that mean nothing (20-hour internet TEFL courses) and absolutely nothing at all. ITDi provides this, with certificates available and whatnot, too. However, something that can also contribute to teacher-centred, teacher-led teacher development has bugged me for too long. Open Badges seem to sort if fill a gap in that people sit in webinars for certificates but there’s no real proof that they didn’t just leave the laptop on and play games on their phone. How about an open-peer-reviewed bit of writing that helps contribute to the community? Keep your eyes open at #TBLTChat.

Modular Materials

Again, with my Task-Based hat on (which is a beautiful purple crushed-velvet and Kevlar deerstalker), and my ‘I hate coursebooks‘ T-shirt on, how better to address a gap in materials availability than to actually get cracking and make some through refining them. Think less of a Minimum Viable Product than a ‘actually see if students react positively’ approach.

A Co-Op (ad)Venture

I am still investigating the possibility of sorting out a Tokyo/Kawasaki/Yokohama-based co-op of language teachers. Yes, inspired by Serveis Linguistics Barcelona. Viability? Time, Marc? It’s more the client liason that’s a problem but still something I’m looking into. Sometime in 2047.

Gritty Politti: Grit, Growth Mindset and Neoliberal Language Teaching

Over the summer, while everyone else was enjoying themselves I was ruing the day I decided to look at grit (Duckworth, 2007) for my Master’s dissertation. I decided it’s unworkable so you get to read about it here.

What is Grit?

Grit is so difficult to define that it takes Duckworth (2016) the best part of a book to describe it adequately. Grit is the orientation of short-term goals toward one’s passions and long-term goals. It has also now mutated, taking on Dweck’s (1996) Growth Mindset, Csikszentmihalyi’s (1990) Flow, Ericsson and Pool’s (2016) Deliberate Practice and become a behemoth. One is left with the impression that only Duckworth truly understands what she means by the  term ‘grit’. Grit has also been criticised for not being substantially different to conscientiousness (Kamenetz, 2016). Even Duckworth and Quinn (2009) acknowledge that:

Grit is similar to one Conscientiousness facet in particular, achievement striving, which is measured with items such as “I’m something of a ‘workaholic”’ and “I strive for excellence in everything I do” (Costa & McCrae, 1992a). We believe grit is distinct from achievement striving in grit’s emphasis on long-term goals and persistence in the face of setbacks. However, further research is needed to determine the relationships between grit and other facets of Big Five Conscientiousness.
(Duckworth and Quinn, 2009, p. 173.)

Grit does appear to be lauded, particularly in the USA, for being a commonsense approach to teaching and learning. I do not think this is a good thing. The Grit Score, a short, subjective, Likert-scaled questionnaire, is being used by teachers and learners to assess learners’ grit. One problem here is the weight of the scoring and the disparity between teachers’ and learners’ perceptions. When searching the literature I found nothing. The fact that grit is being used as a magic bullet for the school system’s ills is also worrisome when one might consider the least advantaged actually attending school being a gritty act in itself. Yet, it is these children with poverty-caused cognitive overload (Mani et al, 2013) who will be labelled ‘least gritty’. Duckworth has bemoaned this, much like Frankenstein died at his own monster’s hand.

What negative effects are present? Well, we have teachers buying in to a new orthodoxy dressed up as science, where learners score themselves on how they tend to stick with certain habits. If the teacher is scoring them, so much the worse due to the likelihood of other factors making their way into the scores, such as warmth toward the learners of even empathy towards them. This grit then is used to blame learners or anything but the system, including curricula and syllabi.  To me, grit is simply another tool for attacking the poor and the other. They might not be as gritty and it provides an excuse for not targetting financial resources in the classroom, such as teaching assistants, smaller class sizes or taking time to reflect on whole-school issues such as homework, which can effect some learners who may be primary carers. As Thomas (2013) states, “the dirty little secret behind ‘no excuses’  and ‘grit’ is that achievement is the result of slack, not grit.” However, if a lack of achievement can be put down to intrinsic factors within learners, then it provides grounds for ignoring them. 

In language teaching, progress is known not to be cumulatively acquired (Lightbown, 1985) yet when language is assumed to be a skill acquired like the learning of facts, it may appear that there is not enough work being done by learners. In adult EFL, language learning may not be the most important part of a learner’s life; even in ESOL, when learners may have to adjust to living in a new country, community and culture, integrative motivation (Gardner &  Lambert, 1972) may be lacking. To assume that everyone in a community shares the same values of utility is unwise; to assume language learners progress through stages at the same rate, with the same goals and motivational orientations is to ignore literature and evidence in the classroom. 

All of this grit and Growth Mindset then, seems to lend itself to a laissez-faire attitude in the classroom, where what is taught is what is learned, with learners not progressing at the same rate somehow deficient and lacking gumption to catch up. This characterises the grammar syllabus of textbook ELT. However, if one eschewed that, and taught learners English as opposed to teaching English as a subject, one need not abide by or pay lip service to such ideas. 

Grit, Growth Mindset and SLA

There is, so far, no research linking grit nor Growth Mindset and SLA. There is research linking motivation, particularly long-term motivation, to SLA. Dweck’s work thus far has looked at problem solving in puzzles yet it may prove to be useful in decreasing the effects of the frequently observed slump in intermediate-level language learners’ development (find and cite).

Dörnyei’s (2009) work on motivational selves has elements of learners orienting themselves toward long-term goals, expectations of others (such as schools, parents, bosses) and visualisation of what would be a possible worst-case scenario if they did not study. This is, in my opinion, compatible with grit but has the benefit of not being burdened with the unnecessary accoutrements of highly subjective data from questionnaires for groups; it is personal.

Duckworth (2016) also talks about building gritty communities yet what if one is alienated from the community of sees oneself as unintegrated in the target community? Seeing as grit is passion and perseverance for a goal, essentially personal, grit may not be the best paradigm to apply to the EFL/EAL/MFL classroom in a compulsory school setting.

Furthermore, developing grit may be an irrelevance. As Norton Peirce (1995) demonstrates in her work with ESL learners in Canada, learners are driven toward communication when needs arise, whether there may be negative affective factors in the immediate environment. This need can drive learning, with Martina, one of Norton Peirce’s learner-correspondents providing anecdotal evidence regarding problems in a part-time job:

In the evening I asked my daughter what I have to tell the customer. She said ‘May I help you’ and ‘pardon’ and ‘something else.’ When I tried first time to talk to two customers alone, they looked at me strangely, but I didn’t give up. I gave them everything they wanted and then I went looking for the girls and told them as usually only ‘cash’. They were surprised but they didn’t say anything.
(Norton Peirce, 1995, p.247)

Rather than nebulous ideas of how to teach or foster grit, self-regulative strategies (Oxford, 2013) could provide a jumping-off point for more effective self study or use of self-access facilities. Such strategies may foster greater motivation by providing learners with alternative ways of working than their current preferences.

One caveat here is that I think grit should be researched; I just don’t think there is a place for it to be brought into classes on a whim due to TED talks bringing a superficial informative lacquer to a few minutes of distraction.

If you disagree, by all means comment. I would love to know more, having spent ages with this.

References

Csikszentmihaly, M. (1990) Flow: The Psychology of Optimal Experience”, New York: HarperCollins ebooks.

Denby, D. (June 21st, 2016) The Limits of Grit. The New Yorker. Retrieved August 22nd, 2016 from: http://www.newyorker.com/culture/culture-desk/the-limits-of-grit

Dörnyei, Z. (2009). The L2 Motivational Self System. In Z. Dörnyei & E. Ushioda (Eds.), Motivation, language identity and the L2 self (pp. 9-42). Bristol: Multilingual Matters.

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087-1101.

Duckworth, A. L. & Quinn, P. D. (2009). Development and validation of the Short Grit Scale (Grit-S). Journal of Personality Assessment, 91(2), 166-174.

Duckworth, A. L. (2016) Grit: The Power of Passion and Perseverance. New York: Simon & Schuster.

Dweck, C. (2006). Mindset: The new psychology of success. Random House

Ericcson, A. & Pool, R. (2016) Peak: Secrets from the New Science of Expertise. Boston: Houghton Mifflin, Harcourt.

Gardner,  R. C.  & Lambert, W. E. (1972) Attitudes and Motivation in Second Language Learning. Rowley, MA: Newbury House. 

Kamenetz, A. (May 25th, 2016) MacArthur ‘Genius’ Angela Duckworth Responds To A New Critique Of Grit. Retrieved August 22nd, 2016 from: http://www.npr.org/sections/ed/2016/05/25/479172868/angela-duckworth-responds-to-a-new-critique-of-grit

Lightbown, P. M. (1985) Great Expectations: Second language acquisition research and classroom teaching. Applied Linguistics, 6(2), 173-189.

Mani, A. et al (2013) Poverty Impedes Cognitive Function. Science, 341,(6149), pp. 976-980

Norton Peirce, B. (1995) Social Identity, Investment and Language Learning. TESOL Quarterly 29/1: 9-31. In Seidlhofer, B (ed.) (2003) pp. 236-255. Oxford, New York: OUP.

Oxford, R (2013) Teaching and Researching Language Learning Strategies. Abingdon: Routledge.

Thomas, P. L. (November 10th, 2013) The Poverty Trap: Slack, Not Grit, Creates Achievement. Retrieved August 22nd, 2016 from: https://radicalscholarship.wordpress.com/2013/11/10/the-poverty-trap-slack-not-grit-creates-achievment/

Further Reading

Sheppard, R. (2016) Grit, Resilience and Conation in Adult Esol

Sheppard, R. (2016) Grit, Resilience and Conation in Adult Esol Part II