learning based innovation
evaluating design-led approaches
to research and innovation
BACKGROUND
This project is an evaluation for complex research and innovation initiatives such as ‘Experience Labs’. Experience Labs are a new way of working within the health and social care context in Scotland, and bring together academic, business and civic partners. Within each lab, the Experience Labs team work collaboratively with participants to understand and investigate their experience. Using design research methods, Experience Labs explore possible future alternatives in order to tackle the complex health challenges facing Scottish society.
PROJECT AIM
This project aims to explore ways of evaluating design-led approaches to research and innovation. We reviewed the nature of ‘labs’ and identified them as part of a changing understanding of innovation. This led us to identify the limitations of traditional evaluation models that connect objectives to tangible outputs. Instead, we aimed to develop an evaluation approach which reflects the complex character of innovation, and can capture the impact and value of Experience Labs meaningfully. This approach centres around the concept of ‘Learning Based Innovation’. It aims to demonstrate how the emergent, nonlinear and complex nature of Experience Labs is vital to understanding its value - and to show how this can be taken forward into practice.
METHODOLOGY
desk research
We reviewed many examples of innovation ‘labs’ from around the world and compared and contrasted their methods and setup against Experience Labs.
We located Experience Labs within a wider shift in how innovation is understood, practiced and evaluated with a new focus on complexity and systems.
We conducted desk research on evaluation methods. This ranged from traditional evaluation approaches towards the emerging field of evaluating in the context of new understandings of innovation and complexity.
visual mapping
We created and continuously revised visual maps of the activities, outcomes, impacts, structures, and context that make up Experience Labs and their context. This helped us to translate our desk research on evaluation principles for complex innovation into a working methodology.
Visual mapping helped us to implement an embedded and participatory approach. The maps gave frequent opportunities to include the Experience Labs team in the development of an evaluation framework, and to expand and adapt it according to their reflections.
interviews
Semi-structured interviews with project partners sense-checked and enriched the evaluation framework further, and explored how learning based innovation might be put into practice.
The interview analysis aimed to trace Experience Labs’ key potential across diverse contexts and projects, rather than evaluating isolated labs or activities.
The analysis also focused on identifying what allows a project to work well, and to build on the interdependent, emergent, and complex nature of Experience Labs. Or what in turn prevents it from engaging in learning based innovation.
FINDINGS
WICKED PROBLEMS HAVE CHANGED HOW WE UNDERSTAND INNOVATION AND EVALUATION
The language of ‘labs’ is currently used by many organisations to describe everything from brand development to marketing which can cause confusion.
This project defines innovation labs as teams using experimental innovation methods and collaboration to address social and public challenges and discover or design new ways of working.
The kind of issues and contexts that such labs aim to address have been increasingly recognised as wicked problems. Wicked problems have many interrelated causes, no definitive boundaries, and are characterised by increasing complexity and change. This means they are difficult to describe and impossible to simply solve.
The need to address wicked problems has caused a shift in how innovation is understood. There has been increasing recognition that wicked problems require joined up action, creativity, and an ability to embrace the unknown. Labs can provide environments where this new way of working is encouraged. They act on the new understanding of innovation as interactive, emergent, and occurring within a wider social context of relationships and networks.
We identified four key factors all labs have in common:
Characterized by problem searching and critically engaging with questions rather than delivering solutions
Based on a different concept of success which values the ability to experiment and to embrace risk. Labs embrace the idea that innovation involves risk, surprise, and sometimes failure, rather than aiming for reliability, replicability, and productivity
Based on an emergent strategy rather than strategic planning. Outcomes are not planned but what is created and learned drives the process and leads to unexpected opportunities
Part of the system which they aim to address. Labs take into account the interconnected and complex nature of their context and influence it in indirect and non-linear ways
The nature of labs means they are ideally placed to work in complex contexts and to address wicked problems. It also means they cannot be completely controlled or predicted in advance. Traditional evaluation approaches however focus on assessing specific impacts against a set of predetermined outcomes. Yet applying this model to labs is inherently problematic. This project therefore aimed at developing a more flexible evaluation approach and method which takes into account the dynamic and interactive nature of lab approaches to research and innovation.
experience labs practice learning based innovation
Experience Labs applies and develops design-led approaches to research and innovation within health and care. Like many other innovation labs, Experience Labs consists of a multitude of diverse and evolving perspectives. Their aim, research team, participants, location, activities, and outcomes change from lab to lab and project to project.
Individual labs might for example work at various ends of the research or innovation process and with various multi-disciplinary methods. These change as the team learns from early labs and adapts to what emerges.
What remains constant is that with each lab, a design process is used to create spaces where a diverse set of people can collaborate and learn. These three elements of process, space, and people adapt to and shape each other, and value emerges through their interaction and interdependence.
Experience Labs do not only consist of the space, people, and process that make up a lab on a certain day. They also include the wider activities of researching, engaging with stakeholders, designing, analysing before and after each lab, and producing outputs. The boundaries between Experience Labs and the Digital Health and Care Institute, which it is part of, are in fact often blurry and not strictly defined. Moreover, Experience Labs cannot be understood in isolation from its wider context of Health and Care.
In order to evaluate Experience Labs, it is important to understand how these parts interact and influence each other, how learning emerges, and how learning moves through this system. To shed more light on this dynamic, the ripple effect provides a useful analogy:
Experience Labs creates value for and with the people present at a Lab through what has been defined above as space, people, and process. This is the most direct and most easily identified way in which Experience Labs create value. However, this more direct form of value, might not be the most impactful - where most ripples follow. For example, one patient understanding himself as active participant in their own care is a great success. However, experimenting with and then sharing how to make such a change happen, could have much greater impact beyond the Experience Lab itself. Fostering collaboration and exchange between academia, healthcare, and partners thus generates a greater ripple effect, where actionable knowledge can spread amongst a variety of actors beyond the individual labs and lead to a range of benefits.
We define Experience Labs’ flexible and emergent approach which allows it to work positively within its complex context and to address wicked problems meaningfully as Learning Based Innovation. Practicing learning based innovation means building on the interdependence of all stakeholders, co-generating learning, translating this into evidence for change, distributing it throughout the context, and thereby building capacity for innovation.
learning based innovation as Evaluation framework
Summarising and building on these insights, we developed a visual evaluation framework based on learning based innovation.
Key stakeholders and their aims and expectations are arranged into four interdependent impact areas. An Experience Lab project might, for example, deliver value north to healthcare as a wider context, west to partners who can take insights forward into implementation, east to the field of academic research around design and innovation and, south to participants who take part in labs and might carry learnings into their daily life. The value of learning-based innovation resides in its ability to build on this interdependence. If the value delivered south to participants lies in being involved in re-imagining health services, this cannot happen without working towards shaping preferable future health services together with the Digital Health and Care Institute and NHS Scotland. This in turn can only happen when west, east, and south are involved.
RECOMMENDATIONS
Based on evidence and reflections from literature, visual mapping, and the analysis of partner interviews, we conclude that the biggest potential for Experience Labs lies in engaging in and focusing on learning-based innovation. Findings from interview data offer a way forward to take these insights into practice. Three key themes emerged from these findings that provide guidelines and directions to work positively with complex contexts and wicked problems:
Balancing diverse aims, expectations, and aspirations
Translating learning into evidence for change and innovation
Building synergy between different stakeholders
Below we include some of the themes and interview quotes to illustrate these findings:
THEME 1: Translating learning into evidence for change and innovation
Partners stress that it is very unclear how to move forward to “the next stage” after an Experience Lab project, and how to use valuable learnings from Labs further.
Partners see Experience Labs as evidence, and as giving their project credibility and direction. Different partners value different elements as evidence for change including reports, presentations, the Lab experience itself, and design ideas.
“The outcomes that come out of the Experience Lab
helps make more concrete decisions.”
“Labs present one form of evidence.”
“It’s kind of like, “Here’s your Experience Lab, here’s
what we found, good luck [...] I think it just left a lot of people in limbo and they weren’t sure where to go after that. I certainly have some colleagues who experienced that. They were like, “Okay, what do I do now?”, and they really weren’t sure what the next step was.”
THEME 2: Balancing diverse aims, expectations, & aspirations
Partners point to Experience Labs’ ability to create a common language as one of the greatest strengths of Experience Labs. Examples included the ability of shifting the conversation from problems towards exploring preferable futures, or the use of visual tools.
Partners frequently consider aims and expectations of other stakeholders involved and the need for balance. For example, the need to innovate the wider system and participants being engaged and having a voice was frequently mentioned.
Partner interviews point to a lack of clarity regarding the project setup, roles and responsibilities as an element that affects the success of the project.
Partners point to a lack of consideration for their position and needs affecting the success of the project. As examples: a disregard for their commercial need, the stage of their project, their existing knowledge base were listed.
“Experience Labs showed how difficult it sometimes is to balance the needs of different groups.”
“Moving away from complaints and looking at the bigger picture kind of brought everyone together and gave more common ground to work with.”
“You need to develop a common language and a common understanding before you can get outputs.”
“While I respect and appreciate the process, and I think it has great value, I think where it fell apart was right at the very beginning. It just wasn’t set up properly in the first place.”
“You need to get your partners working more closely together and if you’re going to engage at all with small businesses, you need to be much more sensitive to their commercial sensitivities.”
THEME 3: Building synergy between different stakeholders
Partners describe project context as a “system of planets” and “parallel universes” when illustrating problems with creating synergy.
Partners feel that this would be valuable on a wider level, that Experience Labs could act as bridge and they see a potential for Experience Labs to strategically rethink the current system.
Partners see value in Experience Lab ability to break down barriers between different stakeholders. This is articulated in relation to both the workshops itself and on a wider level.
Partners acknowledge that the type of challenges the health and care context faces cannot be addressed in isolation by one group of stakeholders alone.
Partners point to the value of Experience Lab to bringing together diverse knowledge and experience, allowing what emerges to drive the process.
“There have been some changes in people’s attitudes throughout the lab sessions because it gave us an opportunity to explain why certain things were done in a way [...]”
“Experience Labs’ approach could be very beneficial to promote a radical rethinking and starting to look at the broader issues.”
“[...] bringing the disciplines together to try and solve problems through the lenses of different specialities has to be the way forward.”
“I think there is the bigger picture of working creatively, and I think most of us value working creatively with people who can take you through that process in a comfortable environment which doesn’t threaten you.”
“They had clearly thought, ‘This is what we want to get from the day, these are the ingredients that we have. We know how to help mix, and cook and bake this’ [...] you felt very much looked after and it was an environment of exploration and open minded [...]”
Authors
Ute Schauberger, Daniela Quacinella and Jay Bradley
FURTHER INFORMATION
For further information please contact:
Dr Jay Bradley | j.bradley@gsa.ac.uk
Ute Schauberger | u.schauberger@gsa.ac.uk