|
Scope of my researchTo begin it is helpful to provide an overview of the different phases of activity. It should be noted however that a flexible research design allowed me to move back-and-forth between the different stages outlined below, refining my approach as it unfolded.
|
The nature of my research was necessarily small-scale in order to focus on meaning within the context of the DE programme. My intention was to achieve depth over breadth, reflected in the methodological approach outlined in Figure 1. This helps to explain why my research concerns the experiences of tutors within a single programme of study rather than attending to a wider range of courses. Similarly, while there would have been value in interviewing DE students (particularly in relation to the factors that promote or discourage a multimodal approach during assessment) additional insights would have been achieved at the expense of critical depth into tutor experiences of multimodally. Furthermore, whereas there is a clear gap within the literature concerning the experiences of tutors concerned with multimodal assessment, there exists research that examines student experiences of digital writing (see Literature review). From a practical perspective meanwhile, discussion with my Dissertation Supervisor identified potential ‘interview fatigue’ amongst DE students as a result of previous research exercises that could have made recruitment of interview participants problematic.
Epistemology, theoretical perspective and methodology
My research design demonstrates a flexible, qualitative approach that allows for a focus on meaning, rather than the generalisability or creation of rules commonly associated with a quantitative approach (Robson 2011). Using Crotty’s proposed four elements of the research process (1998), the overlapping components of my research design can be seen in Figure 2:
|
Crotty defines constructionism as an epistemological position where meaning is constructed by the realities of the world around us, whilst recognising that different people construct meaning from the same phenomenon in different ways (1998, p8). There is no objective ‘truth’ waiting to be discovered, Crotty argues, but instead meaning is constructed through our interaction with the realities of the social world. My constructionist position was aligned to an interpretivist theoretical perspective. Crotty sees interpretivism as being concerned with interpretations of the social-life world that are historically situated and culturally derived (1998), while Robson points to the suitability of this theoretical perspective within a constructionist epistemology through its focus on how individuals make sense of their world (2011). Applied to my own research, this constructionist, interpretivist approach placed value on the attitudes and experiences of tutors within the specific social context of the DE programme. The epistemological and theoretical foundation described here supported a broadly ethnographical methodology. My understanding of ethnography is based upon the work of Hammersley and Atkinson (2007) who look beyond the contested understanding of this methodological approach to construct their definition around the practical approaches undertaken by ethnographers. While not every part of my methodological approach sits neatly within the five features that Hammersley and Atkinson propose – the use of interviews could be seen as overly structured, for instance – my wider approach shares a number of common features with their understanding of the work of ethnographers. This includes the use of ‘in the field’ research through the observation exercise, the collection of data from a range of sources (interview, observation, online material), the small scale of the exercise in order to achieve depth of understanding within the context of the DE programme, and an interpretive approach to analysis that focuses on the interpretation of meaning over a quantitative strategy. |
Semi-structured interviews with course tutorsDiscussion with my Supervisor identified seven DE tutors involved with multimodal assessment. These individuals were subsequently approached by e-mail and agreed to participate in an interview. The confirmation of arrangements included a request to look at two selected multimodal artefacts prior to interview. The interviews took place over a 3-week period and were conducted on a synchronous, one-to-one basis, using Skype text. This was an appropriate medium as the same resource is used within the DE programme itself. A separate motivation for using Skype text was the desire to gather interview data without the need for transcription of audio. Whilst acknowledging the disadvantages of adopting a method of data collection that dispenses with the need for transcription (Braun and Clarke 2006, Ross 2010), I felt these were outweighed by the benefits of an approach that suited the preferences of tutors, whilst removing the time consuming and problematic task of transcription (Robson 2011).
A semi-structured approach was taken during interviews, combining a small number of pre-designed questions alongside space for tutors to share their ‘assessment stories’. Robson points to the suitability of semi-structured interviews within a flexible research design, where: |
Interviewers have their shopping list of topics and want to get responses to them, but they have considerable freedom in the sequencing of questions, in their exact wording, and in the amount of time and attention given to different topics. (2011, p 285)
|
In the context of my own research, the ‘shopping list of topics’ was prepared during the Literature review, with questions designed to elicit data relevant to the research questions. The encouragement for tutors to ‘tell their assessment stories’ meanwhile saw the emergence of ideas not considered within the pre-designed questions (or the literature) as we will see in due course.
Bearing in mind the sensitive nature of discussion surrounding assessment practices, tutors were reassured data would be anonymised. I also emphasised that the purpose of my research was to explore experiences of multimodal assessment, not to critique individual’s practices. |
Observation of multimodal assessment in practice
This collected interview data confirmed the potential value (previously identified within the Literature review) of observing multimodal assessment within a ‘traditional’ campus setting. By approaching observation in a way that Robson describes as a ‘supplementary method’ I was able to complement and set in perspective data collected during earlier stages of my research (2011). Approaches were made to Heads of Department of two subjects where students look beyond text as significant representational mode within assessment. The decision to approach two departments was taken with a view to increasing the likelihood of seeking approval and establishing suitable activities and dates for observation (a decision vindicated when one contact withdrew due to other commitments).
The observation exercise thus comprised two visits to the School of Architecture and Landscape Architecture (ESALA) at The University of Edinburgh. The first visit involved observing undergraduate Architecture students as they participated in a semi-formal ‘crit’ exercise, followed by a second visit where tutors discussed and graded the final submissions. On each occasion, data collection comprised making written observations and taking photographs (with permission sought) of students, tutors and examples of work. Additional context was gained by consulting course documentation and speaking directly with tutors. Again, aware of the sensitive nature of assessment practices, tutors were made aware in advance that the purpose of the exercise was to observe multimodal assessment in practice, not to judge their own strategies. Furthermore, prior to the beginning of each observation exercise I addressed the assembled students and tutors to explain my presence. Analysis of the dataMy approach to the analysis of data - which took a theoretical thematic approach - is similar to that described by Braun and Clarke (2006), albeit with the omission of a stage dedicated to the transcription of interview data. Within my theoretical thematic approach, the coding of data and identification of themes was informed by my review of the literature, whilst also being driven by my research questions. Whilst acknowledging that this approach can potentially engender an overly narrow analytic field of vision (ibid) - conscious of the scope of this exercise, my intention was to achieve depth of insight into themes directly relevant to the research questions.
Bearing in mind the small-scale of this research exercise, it was agreed with my Supervisor that the use of complex data analysis software (as originally proposed) was not merited. Instead, coding was undertaken using Microsoft Word, alongside an initial manual approach (written annotation on transcripts, hand-drawn thematic maps). Having pasted all of the interview data into Word documents, I spent time familiarising myself with the data, reading and then re-reading the transcripts. This was followed by writing codes onto the transcripts, a process that provided further immersion within the data. The process of coding was followed by the identification of themes that simultaneously reflected patterns in the data, whilst helping to address the research questions (an approach driven by my theoretical thematic approach). Individual items of data were then collated under any theme to which they applied. This was followed by further refinement of themes, a process that would continue during the process of ‘writing up’. The collected data allowed for the creation of a thematic framework directly related to the three research questions: |
Figure 3. Interactive thematic map
Reflecting on my 'insider' statusIt is important to reflect on my status as a participant and former Research Associate and Guest Tutor on the DE programme. Robson (2011) highlights a number of opportunities and challenges that arise when a researcher undertakes a study within the setting of her own work, a number of which applied directly to my own dissertation. I was able to exploit my existing, positive relationship with members of the DE team to ensure their participation within interview. Furthermore, an existing level of trust contributed to the sharing of experiences in a frank and open way. My understanding of course context also meant that I could focus on eliciting insights from tutors during interviews, with little need to spend time clarifying course details. Finally, I was able to exploit existing professional relationships to gain permission and practical support for the observation exercises within Architecture.
My ‘insider status’ was not without its drawbacks however. Within a number of interviews reference was made to my own use of the multimodal form, a situation I had not anticipated and found unsettling – thus enacting Robson’s (2011) observation that interviewing those with whom you have an existing working relationship can be difficult. |