Monthly Archives: October 2017

Research update #34: Learning little things

It’s been way too long since I’ve posted about my research and that’s not great. I seem to be finding a lot of legitimate seeming reasons to do other things – also getting sick – and things feel a little out of control.

There is an overall plan – I’ve booked in December to write the first draft of my research proposal, including most importantly the lit review and I think I’ll spend most of November getting ducks in a row for that. The SOCRMx MOOC is helping me to understand research concepts and language a little better and I feel like it will give me enough to get through.

One thing that I have been excited to find is what p means in statistical tables. As I’ve surely mentioned, I’ve not spent a lot of time studying research methodology and pretty well no time at all working on stats. I managed to work out that n is the number of participants in a study – however it was only this afternoon that I learnt that this is referred to as the frequency. But I’ve always been baffled by what the p column meant.

Turns out that it is the probability that the two associated variables in the table have no relationship – also known as the null hypothesis. So a p value of 0.623 means there is a 62.3% chance that the relationship between thing in the column and the thing in the row just happened by coincidence. I still have no idea how that score is calculated but little steps. 

This information came from a particularly user-friendly guide to research concepts at https://saylordotorg.github.io/text_principles-of-sociological-inquiry-qualitative-and-quantitative-methods/index.html (Chapter 14) 

This I found through the SOCRMx MOOC, which is still kicking goals. I’m getting into some of the assessable work now. Given that I’m not formally taking the MOOC – as in not being accredited for it – I’m toing and froing about the next piece of work, reading a 17 page paper about food and culture and then taking a quiz about my understanding of the research methodologies but I’m here now so I guess I might as well.

Reading some of the other student work – several people here are doing this as part of their coursework – I’m highly conscious of the fact that I still really don’t write in an ‘academic’ style and I worry that this makes me look dumb. For the most part it’s a political or ideological decision – I read something by John Ralston Saul a long time ago about how language is used by technocrats to exclude people and it’s been important to me ever since to be accessible in my communication. I realise that there is a need to be mindful of one’s audience as well and I know that when I do formal academic writing, I will be more ‘proper’ but for the here and now, I really just prefer natural language.

Week #4 SOCRMx – Reflecting on methods

This week in the Social Research Methods MOOC we take a moment to take a breath and consider the approaches that we currently favour.

One of the activities is to reflect in our blog – so I guess this is that. I’m looking at surveys because I still need to get my head around discourse analysis, not having really used it before.

Reflecting on your chosen methods

Choose one of the approaches you’ve explored in previous weeks, and write a reflective post in your blog that answers the following questions. Work though these questions systematically, and try to write a paragraph or two for each:

What three (good) research questions could be answered using this approach?

I’m fairly focused on my current research questions at the moment and I would say that using surveys will help me to start answering them, but I certainly wouldn’t rely solely on surveys. The questions are: How do education advisors see their role and value in Tertiary Education? How are education advisor roles understood and valued by teachers and institutional management? What strategies are used in tertiary education to promote understanding of the roles of education advisors among teaching staff and more broadly within the institution.

What assumptions about the nature of knowledge (epistemology) seem to be associated with this approach?

The main assumption is that subjective or experience based knowledge is sufficient. I don’t believe that this is the case. Clearly, a survey can be useful in collecting broad data about the attitudes that people claim or even believe that they hold however people can have a tendency to want to see themselves in the best possible light – the heroes of their own story – and responses might be more indicative of what people would like to think they believe than what their actions show them to believe.

What kinds of ethical issues arise?

This would depend on the design of the research. Assuming there is no need for participants to be subsequently identifiable, anonymity should enable respondents to express their opinions freely and without concern for consequences. Questions should be designed in a way that is not unnecessarily intrusive or likely to influence the way that respondents answer. I’d also assume that good research design would ensure that the demographics of survey participants is reflective of that community.

What would “validity” imply in a project that used this approach?

I would say that ‘validity’ would require addressing some of the issues that I’ve already raised. Primarily that the survey itself could be relied upon to collect data that accurately reflects the opinions of the survey respondents without influencing these opinions or asking ambiguous questions that could be interpreted in different ways. My overall preference would be for the survey to be one part of a larger research project that provides data from different sources that can be used to provide greater ‘validity’.

What are some of the practical or ethical issues that would need to be considered?

The survey would need to be anonymous and the data kept securely. Questions should be designed to be as clear and neutral as possible and a sufficiently representative sample of participants obtained. Given the number of surveys that people get asked to complete these days, ensuring that people have a clear understanding of the purpose and value of the research would be vital. For the same reason, I’d suggest that we have a responsibility to ask people only for the information that we need and nothing more.

And finally, find and reference at least two published articles that have used this approach (aside from the examples given in this course). Make some notes about how the approach is described and used in each paper, linking to your reflections above.

Mcinnis, C. (1998). Academics and Professional Administrators in Australian Universities: dissolving boundaries and new tensions. Journal of Higher Education Policy and Management, 20(2), 161–173.

Comparison of two surveys, one of academic staff (1993) and one of administrative/professional staff (1996). Analysis of results, some additional questions were added to the second survey

Wohlmuther, S. (2008). “Sleeping with the enemy”: how far are you prepared to go to make a difference? A look at the divide between academic and allied staff. Journal of Higher Education Policy and Management, 30(4), 325–337.

Based on an anonymous online survey of 29% of all staff – academic and professional at her institution, which included questions about demographics, perceptions of the nature of their roles, the ‘divide’ and the value of different types of staff in relation to strategic priorities.

Both surveys related to workplace issues and attitudes, which meant that privacy was a significant factor. I was less impressed with the approach taken by Wohlmuther, which I felt was overly ambiguous in parts.

“Survey respondents were asked what percentage of their time they spent on allied work and what percentage of their time they should spend on allied work. The term ‘allied work’ was not defined. It was left to the respondent to interpret what they meant by allied work” (p.330)

I do still think that I’ll use surveys as a starting point but expect to then take this information and use it to help design interviews and also to inform analysis of other sources of data.

Week #3 SOCRMx – Discourse Analysis

When I first stumbled across Foucault in some paper since cast to the depths of my mind, my immediate response was that it was wanky and unhelpful theoretical tosh. I’ll admit that I struggled to get my head around it but my broad takeaway was that it sat too far in the whole post-modern create your own reality school that has since brought us ‘fake news’ and Donald Trump.

Imagine my surprise then as I worked through the resources relating to Discourse Analysis – and particular five different theoretical approaches to doing it – only to find the Foucauldian Discourse Analysis might in fact be the closest thing to what I need in exploring the language used around Edvisors to see if and how it shapes their status and identity in tertiary education institutions. The other option is Critical Discourse Analysis, which kind of works in the same way but seems slightly angrier about it. Maybe not angrier but you seem to need to start from the position that there is an existing problem (which there probably is) and then dig into what you’re going to do about it. Both are on the table for now anyway.

The great news is that from what I knew of this a week ago – that it existed and a couple of people had mentioned that it sounded like what I wanted to do – I now think that can see why and how it might be valuable. Not that I know how to do it yet but that will come with time.

So once again the EdinburghX SOCRMx MOOC is coming through for me. I had hoped to have explored 2-3 additional topics by now but came down horribly sick late last week and am barely just functional again now.

For what it’s worth, here are my other scratch notes on Discourse Analysis taken from the course so far:

Qualitative approach to the study of language in use – spoken or text.

Covers diverse sources from interviews/focus groups to secondary material such as archival material, policy documents, social media and so on.

Various ways of doing it from the micro (sentence by sentence) to the macro (overall impact of how language is used) depending on the theoretical framework chosen.

References: Discourse – David Howarth and Analysing Discourse – Norman Fairclough (more practical)

Common criticisms of DA – it’s idealist (the world is just a product of our minds) and relativist (anything goes). Also that Discourse Analysts confuse changing the way that we talk about a thing with actually changing the thing itself. Maybe, maybe not.

“Critical discourse analysis is actually really interested in the ways in which systems of representation have actual material effects and asymmetrical effects on the distribution of burdens and benefits on particular social groups, access to resources and so on and so forth” (MOOC video introduction)

There are many different types of discourse analysis, including conversation analysis, which analyses talk in detail (see Charles Antaki’s excellent web site for a good introduction to conversation analysis), and critical discourse analysis, which pays particular attention to how relations of power and domination are enacted through discourse. “”

An important aspect of discourse analysis, for our purposes, is that it treats language as action. As Gee puts it, language “allows us to do things and be things… saying things in language never goes without also doing things and being things” (Gee, 2011, p.1). It also places importance on context: “to understand anything fully you need to know who is saying it and what the person saying it is trying to do” (ibid, p.2).

Not Conversational Analysis for my work

Critical Discourse Analysis – about power relationships and social issues. Almost seems too loaded? Documents that seek to present particular political positions

Foucauldian Discourse Analysis might be relevant – how language shapes identity

There was also an assignment for us to try it out with. One of my major interests is job advertisements, which is perhaps not the best place to start given how formalised the structures of these things are but I did it all the same. Outlaw Country!

This is the sample text:

*This is a new open-ended, part-time (0.5 FTE) post in the E-Learning Development Team, which has been created to support the development of the University’s online distance learning provision. The role holder will provide application management support to academic programme teams for the delivery of fully online courses. In the performance of these duties the role holder will coordinate the registration of courses, students and staff on the University’s Canvas learning management system (LMS).

The post will provide first-line user support to staff and second-line support to students, responding to queries on the Canvas LMS. The post requires a combination of good technological skills, awareness of course and user administration processes and expertise in delivering training and support services. Creative approaches to problem solving and the ability to learn and apply new skills quickly will be necessary, as well as good organisational skills, excellent interpersonal skills and above all, a strong commitment to customer service.

The role forms part of a small team working to the highest standards and best practices for online learning. You will be expected to work on your own initiative, leading staff training and user support services, as well as working effectively within a team.*

These are my responses.

1.Significance: The nature of the text is highly specific and directive. The requirements expected of the reader are made explicit with the use of terms like “The post requires”, “will be necessary” and “ you will be expected”. As a job advertisement this is fairly standard language. The use of “and above all” gives extra weighting to the need for a “strong commitment to customer service”

2. Practices: This text is being used to describe a recruitment process

3.Identities: This text describes in detail the characteristics that the (suitable) reader should possess and explicitly states their relationships with other people and groups described. This positions the writer very much as the person holding the power

4.Relationships: The text defines the relationship between the reader (if successful) and stakeholders in the university and also the relationship between the reader and writer (employee/employer)

5. Politics: The nature of a job advertisement is to describe ‘how things should be’. It broadly pushes a line that the institution cares about quality teaching and learning and also quality customer support.

6. Connections: Everything is relevant to everything else in this piece of text because it has a singular focus on the specific goal of recruiting the right person.

7. Sign systems and knowledge: Some of the language used assumes that a certain type of knowledge relating to technology enhanced learning is possessed by the reader. It is heavily factual and not supportive of different interpretations of what is written.

 

I don’t know if I’m ‘doing it right’ particularly but it did make me think a little more about the nature of the power relationships expressed in job ads and the claims that they make to reflect an absolute truth in reality. So that seems like a thing.

I haven’t taken a look at the discussion posts for the other topics but the fact that there are only 3 other posts about Discourse Analysis in this MOOC after 3 weeks makes me wonder whether it’s simply a topic that people are engaging with or whether people aren’t really engaging with the MOOC overall. Hopefully it’s the former, because I’m getting a lot out of this.

Week #2 SOCRMx – Surveys

Week 2&3 of the EdinburghX Social Research Methods MOOC sees us starting to dig into a couple of methods from a list of about 8. Being a nerd who really wants to get my head around 4 or 5 of them (surveys, discourse analysis, interviews, focus groups and social network analysis) I think I’ve already over-committed but the readings and the activities are great.

For surveys, I now need to design a simple survey of 6-8 questions exploring some aspect of the use of social media by a specific group of people. Big surprise, I’m going to delve into how TEL advisors (academic developers, education designers, learning technologists) use social media as part of their participation in a community of practice. Given the nature of the participants, I am assuming a reasonable level of understanding of the concepts.

I think some of these questions might be more complex than I need them to be but I figure they’re a work in process. (And now I’m wondering if WordPress has some kind of cool survey building tool that I can put them into. Ok, looking for plugins is a rabbit hole – text is just fine.)

  1. Do you use social media platforms as a part of your professional community of practice as a TEL advisor?

[ ] Yes  [ ] No

2. If yes, which of the following social media platforms do you use to participate in your professional Community of Practice (CoP). (Choose as many as are applicable)

[ ] Twitter  [ ] Facebook [ ] LinkedIn [ ] Google+ [ ] Instagram [ ] Tumblr [ ] Wechat [ ] other ___________ (please list)

3. Of the social media platforms that you use in your professional CoP, rank them from 1 (most useful) to n (least useful) where n is the final option

[ ] Twitter  [ ] Facebook [ ] LinkedIn [ ] Google+ [ ] Instagram [ ] Tumblr [ ] Wechat [ ] other ___________ (please list)

4. Approximately how long have you used social media as part of your professional CoP

[ ] 0 years (I don’t) [ ] Less than 1 year [ ] 1 – 2 years [ ] 3-4 years [ ] 5 or more years

5. Approximately how many people are you connected with in the social media platforms that you use for your professional CoP? (Including people that you follow and those that follow you)

[ ] Under 20 [ ] 20 – 50 [ ] 51- 99 [ ] 100 – 500 [ ] 501 – 1000 [ ] More than 1000

6. Rank in order of importance to you from 1 (most important) to n (least important) the reasons why you use social media with your professional CoP

[ ] To get help [ ] To promote your work [ ] To belong to a community [ ] To keep up to date [ ] To share ideas [ ] Other ________________ (please list)

7. How important is it for you to separate your professional life from your personal life when you use social media platforms?

[ ] Highly important [ ] somewhat important [ ] neutral [ ] somewhat unimportant [ ] highly unimportant

 

Ok, overall I’m reasonably happy with these questions – they’re possibly a little wordier than I’d like but I’m trying to be pretty specific. Bringing ranking in is possibly also more complex than it needs to be, particularly when I’m not asking people to rank all the options, only those that they selected or find relevant. May be overreaching there.

All in all though, I think this could result in some pretty rich data. Not sure what to do about people who don’t use Soc med – maybe that’s a screening question? Though it would kind of be useful to get a sense of the proportions

Can I get a method: The EdinburghX SOCRMx Social Research Methods MOOC Week #1

MOOC Week #1 question responses

Making a blog post is part of the participation in the MOOC. I’m just going to put my answers here at the top so people don’t need to read the rest of my post about the MOOC and methods etc.

I’ve been working on this PhD for a little under two years now, so most of these questions I’ve covered in previous posts but will answer for the sake of the exercise.

  • What kind of topics are you interested in researching?

The relationships between edvisors (academic developers, education designers, learning technologists etc) and academics and institutional management

  • What initial research questions might be starting to emerge for you?

What strategies are used in HE to promote understanding of the roles and value of edvisors among academic staff, and more broadly within the institution? Which among these strategies are effective and why?

How do edvisors see their role and value in Higher Education institutions?

How are edvisor roles understood and valued by academics and HE management?

  • What are you interested in researching – people, groups, communities, documents, images, organisations?

People, groups/organisations, documents

  • Do you have an initial ideas for the kinds of methods that might help you to gather useful knowledge in your area of interest?

Currently leaning towards survey/interview and document analysis – job advertisements and organisational structures

  • What initial questions do you have about those methods? What don’t you understand yet?

Is this the best way to do what I want to do? Are there better alternatives?

  • Do you perceive any potential challenges in your initial ideas: either practical challenges, such as gaining access to the area you want to research, or the time it might take to gather data; or conceptual challenges; such as how the method you are interested in can produce ‘facts’, ‘truths’, or ‘valuable knowledge’ in your chosen area?

Not sure yet. I’m conscious that there might be sensitivities and politics to deal with.

Ok, so that’s the end of the ‘homework’ part of the blog. This next bit is what I’d already started writing about why I’m here and so on. 

One of the nice things that comes up from time to time when I discuss my research with academics is that they’ll excitedly start telling me about the methods and methodology that might be helpful. It’s a shame that no single suggested approach to data collection or analysis has been the same and that I don’t have a rich enough understanding of all the options to be able to make a comparison. It absolutely all gets noted down though and I will give all of the options extra attention as I come to some conclusion about what I plan to do.

A couple of things strike me about this variety of opinions – chief of which being that it can seem almost ideological in some ways. I’ve had people that I’ve barely finished describing the broad research question to swear up and down that their magic potion is the only one that will possibly cure my ailments. This is before I’ve even gotten down to what kind of data I think will be helpful or what my underpinning theories are.

Now I don’t question the sincerity of these people for a second and I even find it slightly touching that they can be so supportive and encouraging to a complete stranger. I’m sure that they’ve worked through any number of methods and learnt hard lessons about what works and what doesn’t and are keen to see other people spared some of those difficulties. It does seem though overall that once you’ve landed on a methodological tribe, that’s where you live. (But honestly, this is definitely supposition, I’m sure there’s more nuance than that – or at least I hope so).

If this is the way that things work, I can see positives and negatives. On the positive side, I would hope that pretty well any method or methodology can be valid if you can make a strong enough case for it. On the negative side, if there is an ingrained tribalism to a method and your reviewer lives in a different tribe, will you get the fairest hearing? Scholarship is meant to be grounded in objectivity but if a scholar has sunk part of their soul into a particular theory or a particularly approach to scholarship, might you not have to work a little harder if you choose a different angle?

Working out the angle in the first place is my real challenge. I have some ideas about where I’m going and what I want to explore, and I think there are some theories will inform this but I still feel that I’m very much in the unknown unknowns territory when it comes to methods. There was a mandatory research methods unit when I did my Masters way back when but at the time I had no intentions of moving into further research so I left it until last. Without seeing any particular application for the unit, I did the base level of work needed to finish it – actually I’m being ungenerous there, I still managed a Distinction – and promptly forgot everything.

There are research training opportunities available at my current uni but they are virtually entirely catered for on-campus, full-time students so it’s up to me to find my own way. It’s only recently that I’ve felt that I had a reasonable grasp on my topic so I’ve been happy to stay focused on the literature and put the how-to-research part on the back-burner. Which is all a very long-winded way of talking about why I’ve started the EdX EdinburghX SOCRMx Social Research Methods MOOC. From what I can see, this offers the overview of options that I need – they seem to favour creating one’s own bespoke set of methods, which suits my personal approach – and I’m hopeful that this will give me the focus that I’ve been lacking. I’ll obviously be keeping an eye out for the approaches that have already been commended to me, hopefully I’ll get a better picture of where exactly they sit on the map.

There’s a couple of other things that I’m already liking about this MOOC – there seems to be a manageable number of participants (~94 posts in the introduce yourself forum) and the MOOC moderators seem quite keen on the use of our own blogs for reflections and communication.

Oh and now I’m completely sold – I know this is pretty basic tool but this is essentially exactly what I’ve been looking for. They’ve used a multi-choice quiz to provide detailed feedback about methods that might suit particular research preferences. (Kind of like a buzzfeed quiz that isn’t harvesting your personal data for fun and profit). (All the answers are right)

mooc methods questions

There was also a nice video explainer of Epistemology – which I kind of knew was essentially about ways of knowing of but wasn’t clear why it mattered and perhaps also the nature of the different ways of knowing (e.g getting information from an authority figure vs experience/logic/science/tradition etc).

So yes, pretty happy with what I’ve seen so far here

Research update #33: Making it my own

There’s been something about the updated research questions that I’ve been working with that just hasn’t been sitting right. These are the questions:

What strategies do edvisors in HE use to promote understanding of their role and value(s) among academic staff, and more broadly within their institutions?

What are the roles and value(s) of edvisors? (as seen from their perspective)

How are those roles and value(s) seen from the academics’ and institution’s perspective?

Which among these strategies are particularly successful?

It’s only a small thing perhaps and maybe it’s important in sharpening focus but it bugs me that there is an implicit assumption that it is the sole responsibility of edvisors to make academics and the institution value them and their work. I can understand that this isn’t the job of academics – though it would be nice if some of them made more of an effort – but surely the institution itself, and by that I guess I mean institutional management, has a part to play as well. After all, why provide expert support if you don’t intend for it to be used and for it to work as effectively as possible.

So I’m changing the question. This is partially also because I think it will be valuable to gather some data about how different institutions organise their edvisor support units and what impact this has on their efficacy. With the old questions, there isn’t really room for this.

I’ve also found the sub-questions a little clunky and while I think that the value/values issue is interesting, I can still cover that in the survey and interview questions.

Which brings me to this.

What strategies are used in HE to promote understanding of the roles and value of edvisors among academic staff, and more broadly within the institution?

How do edvisors see their role and value in Higher Education institutions?

How are edvisor roles understood and valued by academics and HE management?

Which among these strategies are the most effective and why?

The ordering still seems slightly odd and while it’s been suggested to move the main question (what strategies…?) to question 3, this seems to miss the main point of the research. (Which is a worry in itself but maybe that indicates that I need to communicate a little more with my supervisors)

All of this brings me to the Pat Thomson journal question suggestion du jour – “The best advice I’ve been given about the PhD was…” that it’s My PhD and I need to own it. This doesn’t mean that I won’t change things based on advice but I need to believe in what I’m writing and I didn’t believe the promoting understanding was the sole responsibility of edvisors.

I attended the ASCILITE Spring in Research Excellence School this week – 2 solid days of workshopping and discussing research ideas. I was hoping to pin down a methodology – I think I know what I want to do but I’m not sure if it’s the best way because I don’t know what all of the options are and what the language is surrounding my approach. I now have two well regarded books on the matter though – Creswell’s Research Design: qualitative, quantitative and mixed methods approaches (2014) and Saldana’s The coding manual for qualitative researchers (2015) so I’m hoping that between the two of these, I can figure it out.