Learning analytics: what types of product are available?

Every educational institution wants its learners to reach their full potential.  Learning analytics can help us to measure and predict student success using data relating to engagement, grades, retention, graduation and employability.  But what products are out there to enable institutions to improve on the indicators of success, and to help visualise and analyse the increasing amounts of data being collected around our students? How well do the various tools facilitate interventions with learners and help us to monitor the effectiveness of those interventions?

I’ve been reviewing some of the main analytics solutions for Jisc, and visiting some of the institutions who are beginning to use them – I’ll be discussing my findings in a series of blog posts.

There’s a diverse range of learning products available, most of which are still at a very early stage of development.  The marketplace is developing rapidly though, and I’m told that universities and colleges are being bombarded by vendors to try to get them to buy into their nascent analytics products.

Picking the wrong solution is likely to be an expensive mistake. On the other hand those institutions who seem to have got it right are already providing significantly enhanced information about their students to managers, academics, librarians and the learners themselves.  Crucially, they’re attributing initial improvements in some of the indicators of student success to the use of these tools and the interventions being taken as a result of the improved information.

In order to carry out comprehensive analytics, data is generally extracted from various institutional systems, in particular the virtual learning environment (VLE/LMS), the student information system (SIS – also known as student records system) and a variety of library systems.  So the development of analytics products is complicated by the range of underlying software being used by educational organisations.

To make things slightly simpler, in recent years the VLE market has consolidated, with most further and higher education institutions in the UK deploying either the commercial market leader Blackboard Learn or its open source competitor Moodle.  Meanwhile though some universities have in-house developed student information systems there is a limited number of commercial products  in use such as Tribal SITS:Vision and Ellucian Banner.   In further education these products tend to be called Management Information Systems (MIS) and include Capita UNIT–e Student Management Information System.

Vendors of VLEs and SISs are rapidly developing products to help institutions make sense of the large quantity of data that is being accumulated as students undertake their learning, drawing conclusions about their likely success by adding this to already established information such as ethnicity or prior educational attainment.  To try to make sense of the range of products out there I’ve classified them in four categories, based on the software they originate from:

1. VLE-based engagement reporting tools – which sit within the VLE, generally look at VLE data only, and provide simple indications of a student’s progress.  Examples: Blackboard Retention Centre, Moodle Engagement Analytics plug-in.
2. VLE-centric analytics systems – developed by VLE vendors, combining data from the VLE with data from the SIS to enable more extensive analysis.  Examples: Blackboard Analytics for Learn, Desire2Learn Insights.
3. SIS-centric analytics systems – which sit alongside the SIS but may also draw in data from the VLE, providing learning analytics alongside wider business intelligence. Example: Ellucian Student Retention Performance, Compass promonitor.
4. Generic business intelligence systems – developed to provide better analysis in any business, not specifically for education, sitting outside both the VLE and SIS but drawing data from those and other systems, often in conjunction with a data warehouse. Examples: QlikView, Tableau, IBM Cognos, HP Autonomy.

Most of the VLE-centric and SIS-centric solutions are themselves developed on top of one of the generic business intelligence (BI) solutions so these categories are not mutually-exclusive.

I discussed this classification with Richard Maccabee of the University of London Computer Centre, which hosts Moodle for around a hundred institutions.  He suggested a fifth type of learning analytics system: one which examines data from across a range of institutions and provides analysis back to the organisations.  It might be that there is more to learn by comparing your modern language students with those in another institution than there is by comparing them with your own engineering students.

Assuming institutional agreement, appropriate anonymisation and compliance with data protection legislation, the sum of big data from multiple organisations may indeed be greater than the parts.  This concept has not been lost on companies which provide hosted VLE services such as MoodleRooms and Desire2Learn.  Richard and I concluded though that rather than a separate type of analytics system, this is more of a dimension or addiional set of services which could be applied to any of the above categories.

An institution’s choice of learning analytics products will be limited to some extent by the systems it has already deployed.  Thus if you’ve invested significantly in Blackboard Learn as your sole institutional VLE you’ll not be deploying analytics tools based on Moodle or Desire2Learn Brightspace.

Institutions can carry out limited analytics at low cost by deploying one of the VLE-based engagement reporting tools.  The choice for more sophisticated analytics based on multiple data sources may be whether to buy into an analytics system developed by a trusted vendor of your institutional VLE or SIS – or to purchase a generic BI tool.  Some of the latter can already be integrated relatively easily with underlying data sources and come pre-populated with education-specific dashboards.

A number of the products available are components of wider analytics offerings which cover activities such as student recruitment and fundraising (though some of these systems are currently quite US-specific). The learning analytics solution you choose may be best considered as part of a broader exercise to develop institutional business intelligence.  Some of the learning analytics initiatives report that they’ve prompted institutions to re-examine and clean-up a wide range of data sources and structures with positive knock-on effects for other aspects of the business.

The institution will also need to decide whether it has the expertise to go it alone or requires assistance from one of the increasing number of vendors eager for business in this area or consultancy from a third party.  Meanwhile communities of practice are springing up around some of the tools, which can provide excellent support and examples of successful implementations.