Last Friday, after I had confirmation that Ako would be funding us to measure the effectiveness of our open education work, I invited Shelagh Ferguson from Otago University's Faculty of Marketing, and Russel Butson from the same university's Educational Development Centre, to help start some thinking about a good method with which to measure open education. Shelagh has a growing interest in social media marketing (in the broader sense of the word) and Russel has an established interest in informal learning. The conversation that followed between us 3 revealed that Kathy was right! The two fields do have a lot to offer each other! Especially informal learning and marketing research methods.
The meeting was to gauge Shelagh and Russel's level of interest in the project. Going from the conversation, I reckon they are interested as long as it doesn't draw on too much of their time. I put it to them that I would like to spend a 1/2 a day with them and a few other researchers from other fields such as open and distance learning, to draw up a research method for measuring our open education work.
Russel and Shelagh agreed that there would be 2 aspects worth measuring:
- Usage is the stuff that managers are typically interested in, it speaks directly to the bottom line and is relatively quick and easy to measure. It involves data such as numbers of page views, response rates, enrollment numbers, completion rates, costs, savings, etc.
- Value has more to do with people's sensibility to the work, how they perceive its importance, usefulness, effectiveness, etc. Value based research would be broken up according to stakeholders such as staff, students, managers probably with a structured survey targeting each group.
Shelagh recommended that a project with such a small amount of funding and with such a tight time line would be better to be based on a convenience sample. We would first identify a sample of open education work we wanted to measure, and then identify individuals from each of the stakeholder groups for the value based research.
Russel has been implementing videography (as in video-ethnography) to gain incites into how students at the university interface with university services etc. It could be that such a method of data gathering could be useful for our convenience sample.
My next steps:
Identify a sample of open education work we want to measure.
Formulate a plan and method for gathering usage data.
Find a date for the half day (or full day if possible) meeting where research experts come together to formulate a method for measuring the value of the open education work
This work is licensed under a Creative Commons (Attribution) license.