Category Archives: education design

A hierarchy of digital badges – level 4 Classroom

The final level of digital badges (in education at least) is Classroom badges. Now in keeping with the let’s-not-get-caught-up-on-semantics theme of this series of posts, it applies equally to the training room, the tutorial group and so on – the name ultimately doesn’t matter, it’s the function that counts.

Classroom badges I would consider to be the most informal of all badges, used primarily to add fun to learning and to give recognition to learn progress through a subject as well as to acknowledge notable contributions to class. This might be in an online forum or class discussion, for punctuality or courtesy or in dozens of other intangible ways.

These aren’t generally going to be badges that learners would attach to their e-portfolios or online presence but they can still be valuable tools to enhance motivation and engagement.

Classroom badges are closely tied to gamification, which is simply about taking game mechanics (e.g. instant feedback, competition/leader boards, collection quests, unlocking levels) and applying them to new contexts. Gamification is facing a not-unjustified backlash because it is possible to doing it quite badly and many gamification evangelists take an oversimplistic approach that involves copy-pasting concepts that work in advertising.

Used in education, gamification can drive intrinsic and extrinsic motivation. Extrinsic motivators take the form of external rewards – physical prizes obviously but also unlocking access to new content and particularly peer respect. This can be incredibly effective in the short term but you run a serious risk that learners engage more with the rewards than the learning and when the rewards dry up, motivation plummets. Intrinsic motivators tap into a learner’s own desires and their reasons for undertaking the study. These often focus on recognition of progress and achievement, curiousity and personal interests. These can be more difficult to design but are far more valuable in sustained engagement.

It’s certainly possible to find the right balance of both extrinsic and intrinsic motivators in a gamified approach using classroom badges, it just requires a little more consideration.

I’m currently working with the head of our Year 12 program (final year of secondary education) on a badge based approach to encouraging at-risk youth to complete their studies. It is currently largely driven by extrinsic motivators – get enough badges during the year and we will put on an end-of-year BBQ that you can come to – but we will also include some subtler drivers.

Lee Sheldon, in his fantastic “The Multiplayer Classroom” book notes some examples of teachers that also got learners to design and issue classroom badges (a limited amount to increase their value) to their peers for certain achievements such as explaining a concept in class in such a way that they were able to understand something for the first time. Peer based badging opens a whole new door to this approach that is well worth taking further.

So this is what I consider to be the four levels of digital badging in education. Maybe I’ve missed some, maybe the terminology needs some work and maybe creating a hierarchy is redundant (as different people have different needs of badges) but I think this is a decent start.

I’d really quite like to hear your thoughts on this – and particularly where we go next.

A hierarchy of digital badges – Level 2 Work skills

In my last post on this topic, I discussed why I think it helps to identify different types of digital badges and looked at more formal badges that are linked directly to accredited qualifications.

The next level of badges I’d say would be those that denote particular skills or knowledge in the badge-holder without necessarily having the same degree of accountability or rigour in the evidence gathering process.

These badges should certainly still be designed around specific, well defined capabilities/competencies that a badge issuer needs to evaluate but by disconnecting this level of badges from formal institutional systems and processes, we can support a wider range of badge issuers and support more flexible and responsive badge programs operating in much shorter time-frames.

Two particular examples of these kinds of badges spring to mind.

The Insignia Project at the Australian National University (ANU), driven by Dr Inger Mewburn, Dr Kim Blackmore, Dr Katie Freund and Emily Rutherford (all people I know and respect) was created last year to explore the use of Open Badges in Research Education. It ties to ANU’s “compulsory, yet non credit bearing, research integrity course.”

So here we have a training program that is designed to equip students with vital skills that should serve them throughout their studies and into careers in academia but which isn’t considered a part of formal study. The skills addressed by these badges include Research Integrity, Library Searching and the use of Endnote, valuable additions to a CV but not necessarily something that you would receive a qualification for.

Similarly, the Mozilla Foundation has a huge open access education program designed to teach people web development skills. The Webmaker project covers skills including html, javascript, web development and digital literacy and offers badges both to people developing these skills and to those teaching them.

The conditions for these badges are fairly clearly set out and they would certainly enhance the online presence of someone that you might be looking for with these skills – whether for work or collaboration.

Neither of these projects tie to formal qualifications but depending on the provider / badge issuer, it’s easy to see that these may hold more value to badge readers that accredited ones. This is clearly a question of validity and credibility, which is one of the greatest issues with digital badges and one deserving its own discussion.

Rejigging professional development training for teachers – Sarah Thorneycroft

Sarah Thorneycroft (@sthcrft) from the University of New England (Australia) often impresses with her thoughtful presentations about dragging academics and teaching staff into the 21st century when it comes to professional development in online teaching and learning.

This paper that she presented at Ascilite 2014 showcases her work in shifting from conventional workshops to webinars and “coffeecourses”. Teachers being teachers, the results were mixed (why can’t we learn about online teaching in a face to face workshop) but the signs are encouraging nonetheless.

The video runs to 19:23 but is well worth checking out.

ePortfolio grading rubric

Here’s a useful assessment rubric created by the University of Wisconsin – Stout that can be applied to ePortfolios. I would consider adding links within the criteria to exemplars of best practice but I think it provides a solid basis for evaluating student work.

https://www2.uwstout.edu/content/profdev/rubrics/eportfoliorubric.html

Final thoughts on DDLR / DDeLR

It feels like I’ve been banging on about this subject forever now but with assessments now finalised, it seems like a good time for a final wrap up.

In broad terms, I was a little disappointed with my students. It might have been a bad time of year to run this subject, with its demanding workload, but the majority of them seem to have only put in the absolute barest effort needed to pass. Assessment instructions which I thought were pretty clear weren’t followed and most of the reflections lacked any great insight. I had to ask many of them to rework and resubmit their assessments just to meet the minimum requirements.

What this does make me ask is whether this is the fault of my students or me.

As I haven’t taught formal classes for more than a decade, there are a lot of things that I haven’t had to deal with in teaching an ongoing subject with rigorous formal assessment. I also have a tendency at times to over-complicate things because it just seems like it makes them better. This probably also extends to my communication with my students and my expectations of them.

Fortunately, I am still keen to try this again.

Even during the marking process, as I had to walk away from the computer and swear at the walls, I was constantly reshaping the course structure, the assessments and the class activities in my mind to help avoid some of the issues that were arising. The fact that a handful of the “good” students were able to understand and follow my instructions suggests that I’m on the right track at least and am not entirely to blame but the fact that more than a few got things quite wrong does tell me that there is more work to be done.

I need to make it clearer that when students are creating draft learning resources, they actually need to be resources – things, objects – rather than broad and loose activity plans for a class. I need to explain clearly that the final learning resources should be the same as the draft learning resources but improved based on testing and feedback. To be honest, these things seemed so self evident to me that I couldn’t conceive of anyone not getting it but there we are.

I tried to put into practice a number of ideas that I’ve encountered in the education design community about getting students more involved in designing parts of their own assessments but this really just confused more people than it helped. (Which was a shame as I do believe that it is a valid and valuable approach)

I tried to give my learners freedom to follow their particular learning needs and interests but for the most part this ended up just giving them the opportunity to follow the path of least resistance and allowed for some fairly lazy work. I also should’ve factored into my thinking that the first week of a class is often going to be plagued by technical (logins not working) and administrative hassles and try to make allowances for this in not expecting too much work to be achieved in the first week. (That said, we have a strong need to demonstrate engagement in class activities to receive funding for students that later drop out and I was certainly able to prove that)

I think next time around there will need to be a little less freedom, a bit more structure and lot more clarity and simplicity.

On the whole I am happy that I have managed to get these teachers doing things they haven’t done before and I think they have developed useful skills and knowledge. I’d just like to do more.

Delivering DDLR & DDeLR: Reflections

Life got quite busy in the last few weeks, so screenface had to go on the back-burner for a little while. I think it’s worth taking a look at what happened with the DDLR & DDeLR (Design & Develop Learning/eLearning Resources) subjects and what I might do with them next time.

What happened?

The majority of teachers taking the DDLR subjects have a reasonable expectation that this is a class where they will be able to develop some rich skills in using our eLearning platform to make new things for their students.

The units and elements of competency however are heavily focused on a design and development process for learning resources. The assessments for the teachers (who are the students in this case) hinge on providing evidence that they have considered the characteristics of their cohorts and mapped out a plan for whatever resource they are building. (This should include documenting necessary materials, sources of support and risk planning possible contingencies that may arise). They then need to create the resource, test it with peers or students and make refinements to it before final implementation.

All in all, sensible practice and (I assume) something that most teachers already do as a matter of course in their teaching practice. (Whether or not they formally name the steps in the process is another matter)

What the units and elements of competency don’t particularly care about is what the teachers learn about in terms of usability, readability, general design principles and, of course, the use of a range of new technological tools to get it all done. (Which is what they are most interested in addressing)

So we already have tensions built into the subject in the conflict between what the teachers want and need and what they have to demonstrate and be assessed on.

While we started with a full house of 14 people on the first day, numbers quickly dwindled to a dedicated core of 6 – 7. (A number of factors came to play here including personal issues for a couple of the cohort and running this subject at the very end of semester, when these teachers are themselves inundated with their own grading and teaching responsibilities)

For those that remained, we were able to provide what I hope was an engaging range of activities and training in design principles for usability, copyright and the use of our eLearning platform. (I was well supported by a member of my team – Jo – who also kindly filled in for me when I was away).

Assessment items have been slow to come in – possibly due to the onerous nature of evidence requirements for the subjects. Learners are required to provide 4 draft learning resources (with accompanying design documents and student group profiles) of which 2 are then tested and refined into final learning resources.

I tried to streamline this process in the first week by having the class work on a draft learning resource in the first week – a checklist that might be used to test the quality of their other learning resources. There has been a fair amount of confusion about this and I need to consider whether it is worth trying again and also how I go about explaining the concept.

The idea was to get the class thinking about important qualities in their learning resources and also to get some more buy-in in their own assessments, by effectively designing part of their grading tool. (This is not a graded subject but my intention was that by having them use their learning resource checker on their other resources, they would be more mindful of issues relating to pedagogy, content and technology.

What have I learned?

I need to lower my expectations of what can be achieved in the first lesson. We were beset with technical and enrollment questions that disrupted my carefully planned series of tasks and activities.

I had also put too much faith in the technical skills of the cohort and their ability to effectively use our LMS. I tried to do too many clever things – setting up conditional release on activities so that the learners could only access certain activities or resources after completing others.

I didn’t provide sufficient information about how the class might submit assessment items which were from their own development courses in our LMS. The assessments were set up as an assignment dropbox to receive files. I ended up telling people to create a word document with some screenshots and a link to the resources that they had created but this should have been explicitly stated in the assessment instructions.

I am happy that I was able to be flexible enough with the course to ask the learners what tools they were most interested in learning about and reshaping the course to accommodate this. A core principle of adult learning is that adults need to see the value in what they are being taught and this was an easy way to achieve this.

I’ve been able to speak to the previous teacher of this subject and she also struggled with a number of these issues – hopefully input from a wider group of colleagues might offer some solutions.

Designing DDLR & DDeLR – (Over)Structuring activities

I’ve been a bit caught up preparing for this course and consequently this post has been sitting in the draft section for a while now. I ran the first class last Friday (17/10) and it seems like a good idea to share some reflections.

I’m going to leave the pre-class post up as an interesting contrast.

(Before running the class)

As I continue to work on the Design and Develop Learning Resources and Design and Develop eLearning Resources subject (can anyone explain why an eLearning resource should not just be folded into an expanded definition of Learning resource?), I am now at the point where I need to work out what we will do each week.

Previous work on this has led to the development – well, adaptation really – of an assessment structure that should hopefully work well. I’m trying to incorporate as much assessment into in-class activities as possible and also get the learners to take ownership of some of their assessment by having them design the assessment criteria (while still ensuring that all the necessary assessment items are addressed). This also lets us get a flying start on the process of learning about designing and developing resources by working together on one in class. I’m thinking that using a TPACK (Technology, Pedagogy and Content Knowledge) framework to evaluate learning resources seems like a solid base at this point.

The course as it has been delivered previously seems like a very rich opportunity for our teachers to learn about using our LMS (Moodle – called eLearn here) but the more I look at the elements of competency, the more I have to wonder how relevant some of the material really is. Refocusing the course on designing and developing learning resources will have to be a priority. Topics on designing assessments and forum activities and using our learning object repository are undoubtably valuable but not relevant in this specific instance.

(After the class)

One of the things about having a more theoretical approach to teaching is that it can be very easy to get excited about trying a load of new things and using a lot of ed tech (Moodle to be precise) without really thinking through the limitations of the class.

I spend a lot of time researching approaches to teaching with technology and providing 1-to-1 support for teachers at their desks. I also run semi-regular workshops for small groups of teachers about using specific tools. What I haven’t done is taught a full subject in a proper class setting over a number of weeks – well not in the last ten years anyway.

The first week is always going to be a little bumpy – learners turning up to class who haven’t enrolled yet (or properly) and thus have no access to our eLearning platform. The other thing I sometimes forget – but really shouldn’t – is that few teachers have the same level of skill, enthusiasm or experience in using our LMS as I do. So designing the lesson for Week 1 as primarily a series of sequential activities in Moodle in the first week is probably not the ideal approach. Actually, there’s no probably about that.

Furthermore, getting learners to use new online tools that seem perfectly straightforward (Padlet) can and will take much longer than anticipated.

On top of this, I decided that it would be fun to try to gamify the course. Not hugely but using a 12 sided die to randomise the process of calling on learners to answer questions and making use of the activity restriction function in Moodle (you can’t see one activity until you complete the previous one) really does complicate an already messy session unnecessarily.

Something else that I’d decided (based on sound pedagogical principles) was that getting the students to create a resource that can be used to identify criteria in their assessment would be a useful way to engage them with the content and get them to think more meaningfully about what is important in designing and developing learning resources. On reflection, I guess creating a resource that can be used to measure the quality of other created resources gets a little meta and might be overly complicated. I should’ve also considered that these teachers would be far more interested in developing workable resources for their own students and not for themselves and their classmates.

All in all, I think I tried to do too much, too cleverly and expected far more of the students than I should’ve. I should’ve made more allowances for lower levels of e-learning and digital literacy and factored in the necessary messiness of getting everyone started.

So now I need to simplify and streamline this course. Several of the activities were successful and we did have a reasonably meaningful and deep discussion about what is important to consider in the process of designing learning resources, so I don’t consider the class to be a total wash. We also were able to identify specific learning resources that the students are interested in learning about – several of which (marking rubrics) were nowhere on my list of things to cover in this course.

So it’s back to it, I guess.

Designing DDLR – More work on assessment

Now the focus of this project on Designing the Design & Develop Learning Resources course is on pinning down the assessments. J’s assessments for DDLR 3&4 seem strong but I just want to see whether it’s possible to streamline them slightly – largely to allow learners to knock over the analysis (and design) components quickly. (Given that they should presumably have a decent idea what their students are already like and already design resources with this in mind)

After a couple of hours of looking over this, I’m wondering whether it mightn’t have been better to try to write up my own assessment ideas first and then look at J’s for additional inspiration. It’s quite difficult to look past the solid work that has already been done. I’m still mindful of the fact that the amount of documenting and reporting seems a little high and am trying to find ways to reduce this while still ensuring that the learner addresses all of the elements of competency.

One of the bigger hurdles I face with this combined subject is that the elements of the units of competency are similar but not the same. For the analysis and design sections, they match up fairly well, with only mild changes in phrasing but the development, implementation and evaluation components start to differ more significantly. Broadly speaking, both of these units of competency appear to be targeted more at freelance education designers than practicing teachers – the emphasis on talking to the client and checking designs with the client (when the teacher would clearly be their own client) requires some potentially unnecessary busy work for the teacher wanting to be deemed competent here.

I’ve tried to address the differences between the elements of competency by clustering them with loosely matching ones from the other unit of competency in this fairly scrappy looking document. I’ve also highlighted phrases that look more like deliverable items.

This made it much easier to look over the existing assessment documents and resources to firstly check that all of the elements were addressed and secondly to feel confident that I am sufficiently across what is required in this subject.

Broadly speaking, the existing assessment items cover these elements of competency pretty well, I only needed to add a few extra questions to the design document template to address some aspects that it might be possible for learners to overlook.

These questions are:

How does the learning resource address the element or unit of competency?
What equipment, time and materials will you need to develop your learning resource?
Where will you source content for your learning resource?
Who can/will you contact for support in developing your resource?
How will you review your work as it progresses?
Describe the type of learning design that your learning resource uses

So as it stands, I think I’ll be largely sticking to the existing assessment plan with only a few minor changes. (Largely because my predecessor knows her stuff, which has been tremendously helpful). I am still keen to find ways to address as much of this assessment as possible in class activities – being mindful of the fact that learners may not make every class and there needs to be a certain amount of flexibility.

Overall though – and clearly the dates will need to be changed, this is what the assessments look like.

The next step is to update the subject guide and add my amendments to the existing documents. I do also need to devise a marking guide for the learning resources themselves – there is something appealing in the idea of having the learners create this as one of their draft resources as the unit of competency document does stretch to define learning resources as including assessment resources too. This seems like a great opportunity to get the learners thinking more critically about what makes a good learning resource.

Designing DDLR & DDeLR – Assessments

Today is all about pinning down the most appropriate types of assessments for this subject. Yesterday I think I got a little caught up in reviewing the principles of good assessment – which was valuable but it might also be better applied to reviewing and refining the ideas that I come up with.

For what it’s worth, these are the notes that I jotted down yesterday that I want to bear in mind with these assessments. DDLR Assessment ideas

Looking over the four versions of this subject that my colleague J has run in the last 2 years has been particularly enlightening – even if I’m not entirely clear on some of the directions taken. The course design changed quite substantially between the second and third iterations – from a heavily class-based activity and assessment focus to more of a project based structure. (For convenience I’ll refer to the subjects as DDLR 1, 2, 3 and 4)

DDLR 1 and 2 provide an incredibly rich resource for learning to use eLearn (our Moodle installation) and each week is heavily structured and scaffolded to guide learners through the process of developing their online courses. The various elements of the units of competency are tightly mapped to corresponding activities and assessment tasks – moreso in DDLR 2. (Image from the DDLR subject guide)

I have to wonder however whether the course provides too much extra information – given the relatively narrow focus on designing and developing learning resources. Getting teachers (the learner cohort for this subject) to learn about creating quizzes and assignments in Moodle is certainly valuable but are these truly learning resources? This may well be one of the points where my approach to this subject diverges.

The shift in approach in DDLR 3 and DDLR 4 is dramatic. (As far as a diploma level course about designing learning resources might be considered dramatic, at least.) The assessments link far more closely to the units of competency and all save the first one are due at the end of the subject. They are far more formally structured – template based analysis of the target audience/learners, design documents, prototypes and finished learning resources, as well as a reflective journal.

It does concern me slightly that this subject has a markedly lower rate of assessment submission/completion that the two preceding ones. That said, this subject is often taken by teachers more interested in the content than in completing the units of competency and that may just have been the nature of this particular cohort.

This new assessment approach also seems far more manageable from a teaching/admin perspective than the previous ones, which required constant grading and checking.

My feeling is that this is a more sustainable approach but I will still look for ways to streamline the amount of work that is required to be submitted.

The next step was to map the various elements of competency to assessment items. The elements for both units of competency are written differently enough to need to be considered separately (unfortunately) but they both still broadly sit within the ADDIE (Analyse, Design, Develop, Implement, Evaluate) framework. ADDIE seems like a useful way to structure both the course and the assessments so I have mapped the elements to this. I have also highlighted particular elements that are more indicative of outputs that might be assessed. Working through the analysis process will be quite dry (and could potentially come across as slightly patronising) so finding an engaging approach to this will be important.

(I’m also quite keen to bring digital badges into this process somehow, though that’s a lower priority at the moment)

Finally, I had a few ideas come to me as I worked through this process today that I might just add without further comment.

DDLR / DDeLR ideas

Get the class to design and develop a (print based? ) checklist / questionnaire resource that might be used to address DDLR 1 and DDeLR 1 UoCs. Get someone else in the class to use it to complete their Analysis phase.

Can I provide a range of options for the forms the assessment/resource pieces might take?

Try to develop a comprehensive checklist that teachers can use on the resources that they produce to raise the quality overall of resources at CIT. (Again, this could be a student led tool – the benefit of this is that it makes them think much more about what a good resource requires – does this meet any UoCs??)

Convert the print based Analysis document into a web resource – book tool or checklist maybe? Also possibly fix the print based one first – from a deliberately badly designed faulty version. (Lets me cover some readability / usability concepts early)

How much of this subject is leading the learners by the hand? How much is about teaching them how to use eLearn tools?

Could one of the learning resources be about developing something that teaches people how to use a particular eLearn tool???

Need to identify what kinds of resources teachers can make. Good brainstorm activity in week 1.

Think about the difference between creating a learning resource and finding one and adding it to your course. (Still important but tied to the UoC?)

If I give teachers the option to use previously developed resources (authenticity issues??), they should still provide some kind of explanatory document AND/OR edit the resource and discuss what changes they made and why.

Need to consider the relative strengths and weaknesses of the various types of tools.

In-class feedback of learning resources to better support the evaluation and implementation based elements of competency.

One activity (possible assessment) could be for learners to gather information needed to do an analysis from a partner in the group. (and vice versa) Might lead to a more critical examination of what information is being sought. Learner might even provide suggestions for design/development?

Sample resources?

Designing DDLR and DDeLR – a live blog

In three weeks time (Friday 17/10) I’ll start teaching the Design and Develop Learning Resources (DDLR) and Design and Develop eLearning Resources (DDeLR) subjects for the Diploma of Vocational Education and Training. (Dip VET)

It’s been ten years since I last taught a formal subject. (But I’ve run a bucket-load of workshops and provided a lot of 1-1 training and support in that time)

I thought it might be a useful process to document my process as I continue to design and develop this course over the next three weeks. Obviously I’ve already spent a fair amount of time looking over the units of competency (linked above) – the holy documents within VET that define exactly what a learner needs to be able to demonstrate at the end of the course. These also outline the types of evidence that can be used to demonstrate competency and provide additional information about suggestions about interpreting the elements that make up the units of competency.

(As a side note, I know a number of people in Higher Education – the university sector – that shudder when they hear competency mentioned but it has been interesting to note how frequently it does seem to be coming up in discussions of the future of adult learning lately)

I’ve also spent a decent amount of time looking over the courses designed and delivered by my colleagues and dug down into the approaches that they have taken – as well as having long chats with them and other people on my team. (The logical thing to do would be to just tweak and re-deliver their old course but where’s the fun in that?)

There is also a practical consideration in refreshing my own course design and development skills. I have even toyed with the idea of trying to gamify the entire course but that seems unnecessarily over-ambitious. Maybe next year, when I have a better sense of how this subject runs in a conventional form.

So I started by blocking out exactly what it is that I need to accomplish in these three weeks. (Clearly other things will arise that will take priority but it is currently holidays so I have two weeks – boss free also – to get stuck into things like this with hopefully minimal disturbance) The final week is left free for feedback, editing and contingencies.

Assessment seems the logical place to start as I know what I need to cover, I just need to make sure that the learners provide enough evidence that they know it as well.

As with all good lists, the first item is to make the list. Nice to get a quick win on the board. From there it seemed prudent to revisit the subject I recently took as a student about assessment for some inspiration and that has given me some handy tools and processes that I might have eventually arrived at myself but not nearly as quickly. Once I’ve designed the assessments I think I’ll come back to this to make sure I haven’t missed anything vital.

Ensuring that the assessments are targeted at the appropriate Australian Qualifications Framework (AQF) level – 5 in this instance – and that every element of both the units of competency are addressed are the key factors here.

Taking another look at the assessments that the previous teacher of this subject – Jo – designed comes next. She’s run this subject four times now and so has had a good opportunity to refine her assessment tools. Being part of the Education Design and Technology team, we all maintain high quality online courses and Jo has invited me to make use of anything I find in her courses. (Thanks Jo – with four courses and subject guides to pore over, I might be some time)

Screenface

Better teaching through technology