Category Archives: ePortfolio

Thoughts on “Academic Leaders’ perspectives on adopting ePortfolios for developing and assessing professional capabilities in Australian Business Education” (Holt et al, 2016)

While I’m still letting the sociomaterial stew simmer in the back of my mind, it seems like a good time to dig in to something that feels slightly more practical (not to mention incredibly relevant to my day to day work).

This paper is about the first stage of research in a major Australian project looking at the use of (and attitudes towards) ePortfolios in Higher education business schools. As someone working in a business school and advocating the use of ePortfolios, it is unsurprisingly of some interest. Now that we’ve had a real live semester of ePortfolios actually being used in teaching – rather than speculated upon – it’s particularly nice to be able to come to this with something more than a theoretical viewpoint. (I’ll freely admit that it was used only in two subjects and in one, the lecturer only picked up the tool in week 6 of a 13 week semester, but what they’ve already learned and the success that they have had has been incredibly encouraging)

As you might expect, the paper dives into a detailed explanation of the context of using ePortfolios in Higher Ed – it notably hasn’t been as commonly used in business disciplines, something that the authors attribute in part to the diversity of kinds of disciplines in this field. Some (e.g. accounting) have distinct pathways with clearly articulated accreditation leading to specific careers while others contribute more generally to a student’s ability to work in ‘business’. The authors suggest that professions including medicine, law, engineering, teaching and architecture might be more suited to tools and practices supporting the collection of evidence that can be used in external accreditation processes and this might be why ePortfolios have been reported on less frequently in business. All the same, they argue that there are still many compelling reasons that ePortfolios should be used in business schools.

This paper focuses particularly on attitudes towards ePortfolios amongst academic leaders in business schools – Associate Deans (Education/ Teaching & learning), program/course directors and subject/course/unit conveners. (It does seem that survey responses from teachers were also accepted though the details on this are a little hazy).

A lot of reasons (excuses?) are given for why ePortfolios aren’t being used widely which align with those that I’ve come across commonly in the literature (and day to day practice) about the use/non-use of ed. tech in general. Seeing these has helped me to reframe my research question – though I still need to run this by my supervisors – to How can (and do?) Education (or TELT?) Advisors help universities overcome barriers to adopting TELT practices? (Previously – How can Higher Education better support TELT practices?). This is no small thing for me, as it feels like I’m narrowing the focus of the research to something more achievable and personally meaningful.

Anyway, there are a few points of particular interest that this paper covers – reasons for using ePortfolios in business ed., perceived strengths amongst management and the beginnings of a framework for effective implementation of ePortfolios in this space.

Reasons to use ePortfolios

The use of ePortfolios across Higher Ed. is tied very closely to the development of professional capabilities (a.k.a competencies) and employability skills. I’d suggest that the technology can do more than this in terms of offering new opportunities for content management, creation and publishing that might finally enable us to move beyond linear text heavy essays into student construction of richer resources that more adequately reflect the world that we now live in. That said, the portfolio has been a vital tool for demonstrating one’s skills to prospective employers for centuries and the ePortfolio is simply the latest iteration of this.

The authors identify a common set of professional capabilities that universities aspire to equip students with via threshold learning outcomes and program and graduate attributes. ( I think we include something about being a global citizen and thought leader but the rest all seem fairly common and laudable)

1. Professional judgement: Use knowledge and skills to solve novel business challenges.
2. Problem solving: Use knowledge and skills to identify and solve common business problems.
3. Communication: Demonstrate oral, written and visual communication skills appropriate to the needs of different business stakeholders.
4. Teamwork: Demonstrate skills in working collaboratively with colleagues in undertaking complex and varied work tasks.
5. Leadership: Demonstrate skills in constructively influencing the work of colleagues individually and in teams towards mutually agreed goals.
6. Digital literacy: Use knowledge and skills in ICT to frame, analyse and report on business problems and their solutions.
7. Self-management: Demonstrate skills in self-initiative, self-motivation and self-directed learning in business studies and practices.
8. Creativity and innovation: Demonstrate the capacity to generate new ideas to meet customer needs, and in the understanding of how good ideas become marketable products.
9. Entrepreneurship: Appreciate how new businesses are created, grow and adapt to changing market conditions.
10. Social responsibility: Develop a critical awareness of businesses’ obligations to the societies within which they operate, and to those parties who directly contribute to their viability.
11. Cultural awareness: Demonstrate knowledge and skills in working effectively with cultural diversity as related to global and international business practices.
12. Sustainability as applied to business organisations: Develop a critical awareness of businesses’ need to evolve and adapt to the imperatives of an economically, environmentally and socially sustainable world in the service of future generations.
13. Ethics: Develop a personally meaningful set of values to guide professional practice which reflect honesty, fairness, respectfulness, loyalty, composure and competence. (p.5-6)

The academic leaders were asked which of these they rated as most important and which they were most satisfied with the current development of in their students. Communication and problem solving were rated as most important with entrepreneurship at the end of that list. (Accountants tended to rate creativity and innovation lower than others). In terms of how well they think their colleges/schools are doing, academic leaders felt most confident about problem solving, digital literacy and communication and least about leadership and creativity/innovation. (However, confidence in the teaching of all 13 capabiliities ranged from 2.53/5 to 1.68/5, so across the board, there’s work to be done in this space)

There’s also an interesting breakdown of the pedagogical approaches that academic leaders in business colleges saw as being valuable in supporting these varying capabilities – I’m not sure if they were provided with a set list because the choices seem limited. Exams are nowhere to be seen though, which I find heartening.

1. Problem solving: Ranked in order of decreasing importance, respondents indicated: 1. Case studies; 2. Projects; 3. Work-placement; and 4. Simulations, might be effective ways to assess a student’s ability to solve problems.
2. Communication: Case studies, projects, presentations and written work appear to be the most favoured ways to assess a student’s ability to communicate.
3. Teamwork: Group projects and work placements appear to be the most favoured ways to assess a student’s ability to work in a team.
4. Leadership: Projects appear to be the most favoured ways to assess a student’s ability to lead a team.
5. Digital literacy: Projects, simulations and assignments appear to be the most favoured ways to assess a student’s digital literacy.
6. Self-management: Projects and work placements appear to be the most favoured ways to assess a student’s ability to self-manage.
7. Creativity and innovation: Projects appears to be the most favoured way to assess a student’s creativity and innovation.
8. Entrepreneurship: Projects perhaps within a business context appears to be the most favoured way to assess a student’s entrepreneurial skills.
9. Social responsibility: Projects and Case studies appear to be the most favoured ways to assess a student’s sense of social responsibility.
10. Cultural awareness: International study and work tours appear to be the most favoured ways to assess a student’s cultural awareness.
11. Business Sustainability: Case studies, work placements and projects appear to be the most favoured ways to assess a student’s understanding of sustainability in a business context.
12. Professional judgement: Case studies, work placements and simulations appear to be the most favoured ways to assess a student’s professional judgement skills.
13. Ethics: Case studies and projects (p.9-10)

The perceived drivers of the implementation of ePortfolios offers some interesting insights into the value that academic leaders ascribe to ePortfolios and perhaps some selling points to stress when having the discussion about using them. (I have to admit at this point though that I’ve noticed that a lot of research seems to centre around people’s perceptions of things rather than the concrete realities of them. Maybe this is just the nature of education as a social science and there is certainly value in understanding why people make the decisions that they do in this space but just because people feel a certain way about a tool or a practice doesn’t necessarily make it so. Unless we are also looking at what is needed to shift perceptions, I’m not altogether sure what we hope to achieve by simply cataloguing them.)

The best reasons seen for using ePortfolios included “Improve student reflective learning”, “Enhance Student work placement experience”, “allow students to better demonstrate the achieve of learning outcomes to others” and “improve student understanding of learning outcomes.” (p. 11) Considered least important was “the imperative to use technology given the nature of the institution, i.e. mission, vision etc” (p.11) – which I’m fairly ok with and I’m also pretty happy that a teaching and learning goal was most highly valued. (But are people saying this because it seems like the right thing to say?)

In examining which kinds of support were most useful when using ePortfolios (and I suspect any other ed. tech), there was no clear preference for any of the following:

guidance on the purpose of the ePortfolio; guidance on how to use the ePortfolio; a workshop alongside to support the ePortfolio process; tutor/mentor support; IT helpdesk support for the learner; and support for producing media files. (p.12)

At least this seems like a solid checklist for developing a support plan and an implementation strategy.

The paper draws a few implications from the survey results as well as the existing literature, these align pretty well with my own feelings about ePortfolios and unfortunately they tend to make implementation projects harder, because taking a whole of program/degree approach to the use of ePortfolios in a siloed institution tends to get put quickly into the ‘too-hard basket’.

Housego and Parker (2009) and Woodley and Sims (2011), reflecting on their respective investigations, conclude that ideally ePortfolios should be integrated appropriately into the whole curriculum. With this in mind, ePortfolio implementation for assessing professional capabilities can be seen in the context of: whole-of-program-based curriculum designs; the major disciplinary studies allowing specialisation in business degrees; key areas of the curriculum dealing with work-integrated learning (WIL) (see Papadopoulos, Taylor, Fallshaw, & Zanko, 2011), foundational core and capstone studies, and those dealing directly with managerial capabilities like intrapersonal and interpersonal intelligence. (p.14)

I do note with some interest though that perhaps my college’s biggest success story with ePortfolios came in a fairly reflective unit related to management (post-grad), which aligns well with the final line of the paragraph.

The final section of the paper and perhaps the one that gives me the most to unpack (and which also excites me the most about the future work of this team) is a preliminary guidance framework for the implementation of ePortfolios in undergraduate business programs.

ePortfolios framework Holt et al 2016

This is absolutely something that I’m going to take to the powers that be at my institution to explore further. All in all this is a rich paper and I greatly look forward to future reports from this project.

Papadopoulos, T., Taylor, T., Fallshaw, E., & Zanko, M. (2011). Engaging industry: Embedding professional learning in the business curriculum final report. Sydney: Australian Learning and Teaching Council.

Research update #8 – Shame and progress

Sometimes posting a research progress update can be like jumping on the scales after a weekend of eating cake – it’s important to do to maintain accountability but you know it’s not going to be pretty. This is one of those times.

As you can tell by my recent posting history, it’s been a while since I read and reflected upon anything. Since my last update however, I have had a PhD progress review where the panel was satisfied with how I’m going and took me off probation and I also attended the ePortforum conference in Sydney, two days of talking and learning about what is being done in Higher Ed. with ePortfolios.

I also read a chapter of a book my supervisor (hi Peter) wrote about teacher attitudes towards education technology which got me thinking much more about methodology than I have been to date. There’s a strangeness to reading (and particularly writing) about one’s supervisor’s writing – a lot of different conflicting feels. Am I obliged/expected to fall into line with his ideas and/or particular areas of interest? (I don’t think so – he’s been remarkably chilled about what I’m doing. Offering thoughts and suggestions, of course but I’ve never felt pressured). Is it ok if I disagree with something that he’s said in his writing? (Again, I think that if I was able to present a solid argument, it would be fine. That said, I’ve not come across anything yet that hasn’t been eye-opening, as you would hope for from a mentor/supervisor). If I read too much of his work, does it get weird or obsequious?

On the one (rational) hand, you approach a supervisor because you think that their interests/methods will inform yours and presumably align (or vice versa) so why wouldn’t you but on the other (emotional) hand, have I had some kind of need to explore the other literature first to come to some of my own conclusions before being shaped too much by his take on things? (In the same way that a filmmaker on a remake might go back to the initial novel but not watch the first film that came from it?). Even Peter said that I didn’t necessarily need to read this particular book as it’s from 2002 and attitudes to ed tech have no doubt shifted since then. He suggested more that I look at who has cited it.

I’m really glad that I did read it though as, as I mentioned, the methodological ideas gave me a lot to think about – largely in getting tutors to describe their grading process as almost as stream of consciousness in real time which was also recorded so that they could watch the recording and add a layer of reflection later. This may well be a common methodology but it’s not one that I’ve come across in the reading that I’ve done to date. I’ll post something about this chapter soon anyway.

I’ve also been working away on an application to upgrade myself from Associate Fellow of the HEA to Senior Fellow. This requires a lot of reflective writing (around 7000+ words) and has been useful in thinking in greater depth about my own professional practices and ‘learning journey’. (I always feel a little bit hippy using that expression but I haven’t come across a better one). So this application has taken up a decent chunk of my time as well.

I have also – because clearly I have a lot of free time on my hands – been slowly nudging forward the formation of a Special Interest Group through HERDSA (but not solely for HERDSA members I think) that is focused on Education Advisors. (a.k.a Education Support Staff – academic developers, ed designers, learning technologists etc). We had a great lunchtime conversation (vent?) about some of the issues that we face which aligned particularly with many of the papers that i have posted about here in the last couple of months. I suspect that one of the trickiest parts will be explaining to teaching academics that this isn’t a group for them. I guess this is one of the things that we’ll need to pin down in the formation of it. It’s far from a new idea – there are a range of city and state based parallels in varying states of activity – but having a national (transnational to include NZ) body isn’t something I’ve seen before. The funny thing is that while this is important to me, some of the issues/ideas that came up in the conversation yesterday, I felt like I have already moved on from in pivoting to research academic staff now and their issues and concerns. But I’m pretty sure I can walk and chew gum at the same time.

Trying out Mahara – an ePortfolio adventure

Managing your online identity – particularly your professional identity – is arguably now one of the core digital literacies. This is why I’ve long taken an interest in the use of ePortfolio tools.

As the “Education Innovation Office” in a college at a major Australian university, it’s my job to keep us moving forward and to find the best ways to do so. I’d previously dabbled with ePortfolios before coming here but had never really used them for a project and thus had no direct evidence to support a case that they are worth pursuing.

A few months ago I came across CMALT – Certified Member of the Association of Learning Technologists – useful seeming accreditation and connection to a community of practice of my peers. The application process lends itself very well to the use of ePortfolios so I decided to take the Mahara ePortfolio platform out for a spin.

Our IT/Web team was kind enough to install an instance on a local server – A Windows server sadly (more on that later) – and off I went.

Using Mahara 

Mahara enables users to curate a range of files, text, links and other resources into highly customisable pages. A range of page layouts are available and content can easily drag/dropped into the page.

mahara edit content screenshot

These pages can then be gathered into collections and private URLs generated so that the user is able to choose which pages are shared with whom.

In terms of ease of use, so far so good. My biggest concern at this stage was in finding a way to provide clear connections between the evidence that I was providing and the reflective posts that I was making to respond to the selection criteria for the CMALT application. Utlimately I decided on footnote style supertext annotations that referred to numbered sections containing files at the side of the text.

mahara footnotes

The good parts

Using Mahara was a highly intuitive process that made it very easy to quickly produce a professional looking page. It certainly helped that Mahara is based on Moodle code (to some extent) as I have used this for a number of years, but I feel confident even a user without Moodle experience will pick it up quickly.

The file management system is similarly quick to pick up, with a simple space that can be used to upload files and organise them into folders.

The range of themes that can be used to style a portfolio (or the whole site) offer a reasonable degree of personalisation and I suspect that it is possible to do a lot more if you are happy to dive into the CSS and tinker.

The lesser parts

As I mentioned earlier, our Mahara instance was installed on a Windows server rather than the recommended platform and this generated a number of back-end error messages. Broadly the system seems to be working fairly well and when it is rolled out officially at a university level, I’m sure that it will be done according to the recommended specs.

For this reason, it’s hard to know whether some things don’t work – most notably the Open Badges plugin – because of our non-compliant server configuration or because of other issues. This was more of a nice-to-have in any case so it hasn’t been a major headache.

One thing that did cause a few more headaches though was the fact that when adding a folder of files to a page, the process for correctly selecting the folder (so that it actually displays the files) is fairly unintuitive in the latest release. A user needs to click the white space next to the folder name – but not the name itself or the folder icon – to select it. This caused me a good few hours of frustration but I have been told it will be addressed in future versions.

The user manual is fairly rich and detailed and there is also a growing community of users, so it isn’t generally too hard to find an answer when you strike a problem.

Wrapping up

I did spend a fair while tweaking my ePortfolio to get it just right but this was never labourious. My ePortfolio probably doesn’t make full use of the range of tools on offer but I think it has done the job reasonably well.

Feel free to take a look at it at http://mahara.cbenet.anu.edu.au/view/view.php?t=vex1phdEaZm2QsXRUGc9 and if you have any thoughts or suggestions, please let me know.

A hierarchy of digital badges – Level 1 Accredited

Part of me thinks it’s a really dumb idea to try to identify a hierarchy for digital badges and particular to try to name them. Because the people out there that don’t get badges are often the same kinds of people that get fixated on names for things and let the names blind them to the function or purpose of the thing. (This is why we start getting things called micro-credentials and nano-degrees. Personally I would’ve called them chazzwozzers but that’s just me)

Maybe hierarchy isn’t even the right term – taxonomy could work just as well but I do actually believe that some badges have greater value than others – determined by the depth and rigour of their metadata and their credibility with an audience. (Which isn’t to say that some educators mightn’t find classroom/gamified badges far more valuable in their own practice).

In the discussions that I’ve seen of digital badges, advocates tend to focus on the kinds of badges that suit their own needs. Quite understandable of course but it does feel as though this might be slowing down progress by setting up distracting side-debates about what a valid badge even is.

Here is a quick overview of the badge types that I have come across so far. If I’ve missed something, please let me know.

Level 1 – Accredited 

Accredited badges recognise the attainment of specific skills and/or knowledge that has the highest level of accountability. The required elements of these skills are identified in fine detail, multiple auditable assessments are conducted (and ideally reviewed) and supporting evidence of the badge recipient’s skill/knowledge is readily available.

I work in the Vocational Education and Training (VET) sector in Australia, where every single qualification offered is built on a competency based education framework. Each qualification is comprised of at least 8 different Units of Competency, which are generally broken down into 4 or 5 elements that describe highly specific job skills.

VET is a national system meaning that a person undertaking a Certificate Level 4 in Hairdressing is required to demonstrate the same competence in a specific set of skills anywhere in the country. The system is very tightly regulated and the standards for evidence of competence are high. Obviously, other education sectors have similarly high standards attached to their formal qualifications.

Tying the attainment of a Level 1 badge to an existing accredited education/training program seems like a no-brainer really. The question of trust in the badge is addressed by incorporating the rigour applied to the attainment of the existing qualification and having a very clearly defined set of achieved skills/knowledge offers the badge reader clarity about the badge earner’s abilities.

E.G. A badge for Apply graduated haircut structures could easily be awarded to a hairdressing apprentice on completion of that Unit of Competency in the Certificate III in Hairdressing. It would include the full details of the Unit of Competency in the badge metadata, which could also include a link to evidence (photos/video/teacher reports) in the learner’s ePortfolio.

I use a VET example because that’s what I know best (and because it seems a natural fit for badges) but obviously, any unit in a formal qualification would work just as well

Next post, I’ll look at Level 2 – Work skills

 

ePortfolio grading rubric

Here’s a useful assessment rubric created by the University of Wisconsin – Stout that can be applied to ePortfolios. I would consider adding links within the criteria to exemplars of best practice but I think it provides a solid basis for evaluating student work.

https://www2.uwstout.edu/content/profdev/rubrics/eportfoliorubric.html

screenshot of eportfolio assessing rubric

Using Feedback in Moodle for more than student evaluations & Using Padlet

The first in our series of CITFLN TeacherNet Show and tell sessions went well with Jo Whitfield sharing some ideas about using the Feedback tool for more than student evaluations and I presented Padlet, an embeddable interactive wall.

Here are the videos from these presentations