Trends Unpacked: Organizational Challenges and Learning Analytics (Part 4)

This is a guest post by Lindsay Pineda and Patrick Lynch. Their bios are at the end of the article.

What organizational challenges have you identified at your institution in regards to implementing learning analytics? Do you wonder if they are they similar to those of other institutions? Do you worry that you are alone in facing those challenges? We are here to assure you that, not only are you not alone, but that those challenges can be overcome.

In March 2017, the first installment of the “Trends Unpacked” series was posted. The article focused on the first two organizational trends Patrick and I observed while onsite at several institutions in the UK. In today’s article, we will expand upon the next three organizational challenges and trends discussed in the original article, “Learning Analytics Adoption and Implementation Trends: Identifying Organizational and Technical Patterns.”  We will also offer potential solutions and recommendations to solve these challenges, based on institutional feedback.

This article will focus on the below aspects of organizational challenges and trends:

• Organizational infrastructure
• Policy, process, and practice management
• Ease of integration within the existing organizational structure

Organizational Infrastructure

Some of the challenges and trends we observed while onsite at various institutions were related to organizational infrastructure:

• Most institutions we visited did not have the organizational infrastructure to support the implementation and adoption of learning analytics technology.
• Several institutions did not have formalized organizational structures. Many did not know what other departmental staff did on a daily basis or how their job duties affected each other.
• Institutions were very concerned about issues of redundancy, additional workload, and time management as it related to staff requirements.

The following examples illustrate the types of organizational infrastructure challenges most often expressed at the institutions:

• “Is that what your department does? I had no idea.” – Surprisingly enough, this was a common statement made when larger groups were gathered to discuss organizational infrastructure and institutional operating procedures. At several institutions, this was the first time they had met as a group to learn how each department/ role functioned and how this would likely affect a larger-scale implementation such as learning analytics. One institution told us, “We obviously need better sharing mechanisms to ensure we are working holistically toward the students’ success.”
• “Development of a collective vision is vital.” – The notion that departments and individuals were often “working on their own thing without an understanding of how it impacts another department or individual” was a collective concern. At one institution we were advised, “Senior leadership needs to be part of this collective vision. Without their support, it will certainly fail.” We agree with this statement.

Institutions shared with us their ideas regarding potential solutions and recommendations that they feel would be beneficial:

• Create a shared vision – “Creating a shared vision also helps develop a strong institutional rationale for doing a project like learning analytics. It will also likely help with other areas such as resourcing, budget planning, and task allocation.” This quote was taken from an institution that understood the importance of developing a collective/ shared vision of where the institution wanted to go and creating a plan to get there. Working with various departments, and representatives from multiple departments, will be necessary when developing the collective/ shared vision. As one institution told us, “It cannot be successfully done in a vacuum.” Sharing the vision and understanding the roles each department, or individual, plays in achieving success reduces duplication of effort. In addition, it enhances the feeling of learning analytics as a collaborative institutional endeavor, thus increasing the chances of a successful adoption.
• Sort out the fundamentals – Sorting out, upfront, what an institution really wants to gain from a larger-scale initiative such as learning analytics is very important to help continue the journey toward a successful implementation. As one institution put it, “We need to know, what do we have and what don’t we have? I would think we’d need to know this upfront before we even start.” This statement can apply to which resources are available, what financial allocations are necessary to set aside, and what additional tasks might be required from staff. Having a clear picture of the fundamentals helps to further define the details as the initiative progresses.
• Create an implementation plan – This is likely something that will need to be done with the support of senior leadership, along with the staff responsible for executing the tasks required. One institution advised us, “Senior leadership has a bigger picture view of the institution so their input will be necessary when developing any type of project or implementation plan.” This is absolutely true. Within the implementation plan, it is necessary for senior leadership to set the tone for the overall initiative. Aspects such as the value and importance of using data to drive decision making; creating a shared significance for the goals of the initiative; and establishing the importance of continuous, transparent communication within the institution are a necessity.

Policy, Process, and Practice Management

Policy and practice management is another challenge we observed while visiting institutions.

• Overall, the institutions we visited did have policies, processes, and practices in place that were currently being managed. However, there were discrepancies evident in most institutions regarding how the staff and leadership perceived the oversight along with the implementation of those policies, processes, and practices.
• Several institutions were concerned about issues including micromanagement; current inconsistencies; the execution and accountability of those policies, processes, and practices; and whether learning analytics would add to their already overloaded requirements.
• Several institutions identified a gap between policies set by the institution, different processes adopted across the institution to fulfill those policies, and individual practice, on occasion, being completely at odds with both.

The following examples illustrate the types of policy, process, and practice management challenges most often expressed:

• Standardization is necessary, but not always welcome – One institution we visited stated that they “Would like to have some standardized practices and guidelines across all departments while still keeping some amount of individuality. We don’t have that now.” Standardization can be difficult for larger institutions as they have many individuals on staff. Most institutions did not want to have standardized policies without considering the importance and value of individuality when implementing those policies and practices. An individual at one institution advised, “We don’t want to become a machine that doesn’t teach our staff how to think critically. Part of the joy of working within an institution of higher learning is the importance of thinking. That cannot be removed.”
• “No one is monitoring the institution’s policies, processes, and practices” – We found that this concern was shared across several institutions. As one institution put it, “We have lots and lots of policies and we do our best to enforce them, but when they aren’t done properly, no one is held accountable or taught how to [enforce them].” If no one is monitoring or upholding the policies, processes, and practices at an institution, how do you know if they are being implemented properly? The answer is: You do not. One institution brought to light a comical, but quite important question, “Sometimes I feel as though I’m aboard the Titanic. Is anyone manning this ship? Or are we all slowly and quietly going down with it?” This is an essential insight to start considering, and to ask, does my staff feel this way as well?

Institutions shared with us some ideas regarding potential solutions and recommendations that they feel would be beneficial:

• Create a communication strategy – Part of creating a shared vision, collective path forward, and common set of goals is to create a strategy based on information and communication. This type of plan needs to involve staff and their input in such areas as training plans, communication dissemination strategies, and task responsibility. Once the strategy is developed, it then needs to be upheld and accountability needs to be established. What will happen if someone does not implement a policy, process, or practice in the agreed upon way? How will they be further trained to avoid the same situation again? As one institution told us, “If we aren’t sure if our staff is providing accurate advisement to students, how can we know where they need more training to improve?” Being aware of what your staff knows, does not know, and is or is not doing well is essential to helping define a communication strategy. It is also fundamental in providing proper training for support and holding individuals accountable.
• Empower staff – In balance to the approach expressed above, another recommendation was to empower staff. This concept was summed up by an individual who said, “Information given to staff isn’t just about improving their students, but also [about] helping to further development themselves”. Having staff that feel they are empowered to make decisions and have input in what the institutional strategies are is a formidable motivator. Involving them in important activities such as the development of policies, processes, practices, along with strategies regarding implementation and accountability are ways that staff reported feeling empowered. Checking in with the staff regularly to collect their thoughts and concerns also helps drive home how essential their input is to the overall mission.
• Develop a centralized policies, processes and practices repository – We have discussed creating a centralized repository for data, such as a data warehouse or learning records warehouse. This works well for the technical side of things, but what about for the organizational side? One institution advised us, “Having a centralized place for all of the institution’s policies, processes, and practices would be very helpful. One that is updated regularly and communicated often would be extremely beneficial for staff to understand what they are required to do and where they can go for a central point of reference.”

Ease of Integration within the Existing the Organizational Structure

The ease of integration of technology like learning analytics within the existing organization structure is a challenge on the forefront of many institutions we visited.

• We heard many concerns expressed about how learning analytics would actually work in practice. All the institutions visited liked the theory and the idea behind it, but were uncertain about what it would mean to actually put it into practice.
• Most of the current organizational structures at the institutions did not appear to support a learning analytics initiative in present form.

The following examples illustrate the types of integration challenges most often expressed:

• “Who is responsible for what?” – Most institutions have many roles within them and those roles may or may not easily accommodate the implementation of learning analytics. For example, many institutions were divided about their use of personal tutors and instructors/ lecturers. Others did not have the personal tutoring role at all. Determining who would be responsible for implementation and interventions was a topic of much debate. Many questions arose, including, which department “owns” interventions and the responsibility for delivering them to students? Should there be a new tutor/ advisor policy to incorporate the changes in the requirements of the role?  How will things be enforced? These are all very valid questions to ask and consider.
• “We already know who is at risk. Why do we need a formal process or solution to tell us what we already know?” – This is a sentiment we received from more than one institution. Many of the institutions visited expressed concern about how another process or technology solution was going to be implemented when tutors/ advisors already know who is at risk and who is struggling. One institution advised, “This seems redundant. We have the experience to know who is having trouble. Why do we need a piece of software to tell us? Shouldn’t that be part of our jobs already?”

Institutions shared with us some ideas regarding potential solutions and recommendations that they feel would be beneficial:

• Setting clear expectations – Clearly outlining and setting expectations early on for anyone who will be responsible for the interventions related to learning analytics becomes very important in many ways. Receiving full participation and buy in from staff is an essential element for a successful implementation. Staff knowing what their responsibilities are and what accountability looks like also help increase empowerment and ownership. An institution expressed to us, “It seems we will need to revisit our tutoring expectations to ensure our staff is provided with the right kinds of guidance and training to help them better assist their students. After all, the goal is to provide a better student experience. How can we do that if our staff isn’t provided the same opportunity?”
• “Proactive, not reactive” – All institutions expressed the desire to be “proactive, not reactive” when it came to students. One institution told us that a learning analytics initiative would “Share what’s already in our heads with others and will allow us to provide more targeted support to students along with creating a better use of our time.” Learning analytics is not just “another piece of software,” nor is it intended to replace human thought and action; it is meant to support what tutors and advisors already know and do. Students look to their tutors and advisors to help guide them on their journeys, to provide them with well-informed support, and to have necessary information to help with decision-making. Another institution advised us that “Learning analytics seems like a tool we as tutors are given to help students build their houses. We aren’t doing it for them, but with more information and the right tools, we can help them build a stable house to grow in.” This is an excellent way of illustrating the purpose and goal of learning analytics information.
• Standardize the level of support – Patrick and I met with student groups while onsite to gain their perspectives on how learning analytics might look or benefit them. Quite often, students reported that the quality of support they received varied between staff. One student told us, “When I call in to have a chat with a personal tutor, I expect that they would all respond the same way to a question, but they don’t. I will often get two or three different answers to the same question. And when I call back, it appears that no one knew I had already spoken with someone else. It seems to me like they should all know the same information about me and the institution in order to help guide me.” Creating a standard as it relates to areas such as advisement, support, and guidance would greatly benefit multiple institutions in the quality of the overall support they are providing to their students.

While organizational challenges within institutions are prevalent, they are still manageable and should not stand in the way of maximizing the benefits of learning analytics. There is an underlying desire to create consistency in advisement and support, along with a need to standardize policies, procedures, and practices. Institutions seek a proactive culture in order to ensure an equivalence of experience for all involved. Constant and consistent communication regarding learning analytics initiatives is the most important aspect among institutions. To help with this effort we recommend institutions involve stakeholders from all relevant departments. The collaboration resulting from this involvement is paramount to the overall initiative.

Please be on the lookout for our next article in September, which will offer real, practical activities that can be used at your institution to achieve the suggestions and recommendations outlined above.

Here’s to continuous growth and improvement!

Useful Reading:

• Pineda, L. (2017). “Learning Analytics Adoption and Implementation Trends.”
• Pineda, L. & Lynch, P. (2017). “Trends Unpacked: Organizational Challenges (Part 1).”
• Pineda, L. (2017). “Trends Unpacked: Four Things Senior Leaders Need to Know About Their Role in a Learning Analytics Initiative (Part 2).”
• Pineda, L. & Mason, A. (2017). “Trends Unpacked: Technical Challenges and Learning Analytics (Part 3).”
• Pineda, L. (2017). “Learning Analytics Adoption and Implementation Trends: Quantitative Analysis of Organizational and Technical Trends.”