UNHCR’s Innovation Service has been undertaking an experiment. We created an Innovation Index; our first attempt at measuring how we\u2019re doing when it comes to innovation in our field operations. We never released the product. This piece outlines our Index story looking at the whys, the hows, and most importantly the lessons we learned from the experience.<\/p>\n
The Index is a product that aims to provide a framework to analyse diverse approaches to innovation. It applies this to nine pre-defined UNHCR country operations to determine how they approach innovation across four key pillars: Community Engagement, Data Driven Implementation, Innovation Capacity and Space, and Access to\u00a0 services. For each \u2018pillar\u2019 there are a number of indicators made of up publicly and internally accessible data. Operations are then given a score based on this data and ranked. The product includes anecdotes of specific activities and initiatives within operations and outlines some case studies that demonstrate good practice.<\/p>\n
The Index was a product of an innovation process in and of itself and started somewhat by accident. With the topic of innovation being featured prominently in the High Commissioner\u2019s Strategic Directions, the Innovation Service understood that a new degree of rigour was required in respect to how UNHCR is collectively reforming itsprogramming and incorporating innovative approaches. We needed to understand better how the Innovation Service could have an added value regarding the support we provided to country operations.<\/p>\n
Like many other units in UNHCR, the Innovation Service has to prioritise its activities and interventions. How could we do this? We realised that it was hard to determine with the information we had on hand where opportunities lay, and how we might be able to support, without having a clear understanding of how innovation is being approached in different field operations.<\/p>\n
While we didn\u2019t know precisely what might\u00a0come out of this process, we realised when we started that creating an Index was something we needed to embark on. The Index acted as a container for some of the more significant questions marks we had as a team. By bringing them together we felt we could tackle them together and link them to UNHCR\u2019s strategic\u00a0vision of innovation, and how it contributes to making a positive impact on the lives of refugees. We\u2019d seen other indices such as the Big Max Index, or Corruption Perceptions Index and saw how powerful they could be at telling stories. By doing this ourselves maybe we would be able to learn more about ourselves as a Service.<\/p>\n
Ultimately, it also sought to improve how\u00a0we can hold ourselves to account. UNHCR has made commitments to innovate more, and we were resolute that we should be able to measure how we are, and how we are not achieving this.<\/p>\n
The accident of the Index started in the context of the Innovation Service looking to support a field operation. As part of this, we wanted to delve into UNHCR\u2019s programming data in the operation to find out whether there was anything that might be construed as innovative, but there were limits to what we could infer from the data available. We then expanded on this to see whether we could discern anything from other operations\u2019 data in this regard; it was complicated. There was a\u00a0 considerable variation from country to country which also meant it was hard for us to compare; we needed to do this to help us prioritise where action was needed, and where opportunity lay.<\/p>\n
This first assessment led us to realise that we needed to define better variables or \u2018indicators\u2019 for innovation, as well as the limitations of our existing programmatic data in terms of measuring these. The idea of developing an \u2018index\u2019 evolved thus, requiring the relative assessment of innovation across a number of operations. The approach was to start small and to expand the exercise if successful.<\/p>\n
Internally, the Innovation Service had some debate about the indicators, and content of the Innovation Index. Every member of the team was enthusiastic about its potential, and everybody had their own idea of what the Index should be doing. To be completely candid, it was challenging to build something that resembled consensus. One team member stated: \u201cEach member of the team needs to be able to stand behind each of the indicators and defend them wholeheartedly.\u201d<\/p>\n
The team made its best effort to make that the case. Variables included quantifiable \u2018objective\u2019\u00a0variables, such as the number of Innovation Fellows within an operation, as well as more \u2018qualitative\u2019 variables, such as the quality of UNHCR\u2019s coordination of Communicating with Communities initiatives. Once these were determined, an exercise was undertaken to populate the variables, with data from a range of sources. During this phase, two issues were evident: there wasn\u2019t data available for many of the variables, and the Innovation Service didn\u2019t have consensus on what was being measured. The lack of consensus was particularly the case with regards to output focused variables.<\/p>\n
While the concept of creating \u2018Innovation Indicators\u2019 developed, it was recognised that\u00a0they could also be used for internal advocacy – to communicate both good practices as well as areas of improvement. One other key learning from this phase, is that innovation cannot be measured by outputs, although these are inherently less subjective they do not enable an assessment of the result of an\u00a0intervention. It was at this point that we recognised the Index was inherently subjective.<\/p>\n
After a number of revisions, the Innovation Service produced a draft copy of the Index, which\u00a0subsequently was tested amongst a number of UNHCR staff of different genders, backgrounds, length of tenure in agency, and levels of knowledge around humanitarian innovation.<\/p>\n
Ultimately the Index was never published. It was clear from the conversations that we needed to revisit the\u00a0objectives and the purpose of the Index. With so many things on the agenda that were exciting both for us and for those we tested the Index on, we realised we were trying to do too much with one product.<\/p>\n
Innovation is difficult enough to define as it is, and will probably remain a porous concept. Blurring certain elements of monitoring and evaluation, with looking at enablers\u2019 to innovation, as well as stories of good practice: who knew what our admittedly loose target audience was meant to take away.\u00a0Feedback showed us that consensus amongst the Innovation Service itself wouldn\u2019t\u00a0currently build the framework we need for innovation.\u00a0There are too many projects and initiatives either inherited, politically driven or disassociated from\u00a0operational delivery that feedback told us shouldn\u2019t be incorporated into a set of criteria applied to operational delivery. They are too specific to the\u00a0Innovation Service itself; we couldn\u2019t see the wood for\u00a0the trees.<\/p>\n
There was no aspect of success. This was a dismal failure.<\/p>\n
When we confront failure, we need to understand that\u00a0failure derives from both external and internal factors. We must to be able to ask ourselves the difficult questions like how engaged were we to make this a success, were we diligent enough, or at what point did we stop believing in the process, and why did we pivot late? So as much as the failure of the Index to \u2018launch\u2019 was due to the very complex nature of\u00a0measuring innovation, it was also due to the project\u00a0team; perhaps the setup or resourcing of team was wrong, maybe the skillsets weren\u2019t right to make this work. As we asked these questions, and examined our roles in this failure, one of the surprising things\u00a0about receiving feedback from UNHCR staff on the Innovation Index was how positive they were about the prospects. We were told not to stop working on this and to keep reevaluating its place in our organisation.<\/p>\n
While we didn\u2019t publish the Index, the Innovation\u00a0Service learnt a lot about the work thematic of innovation – particularly, how different parts of UNHCR Headquarters view innovation and UNHCR\u2019s Innovation Service. Most excitingly, this helped bring a degree of clarity and understanding to the strategic\u00a0direction of the Innovation Service. New products and directions were identified that can help support more consolidated objectives of how the Innovation team services the field, including tools to support UNHCR operations improve their practices. We want to be transparent about this and are open to comments and suggestions about how\u00a0we can help. In due course, we\u2019ll revisit the Index but most likely learning from this process by breaking the product apart. You\u2019ll be able to find more examples\u00a0and case studies on our website.<\/p>\n
We want to start engaging with academic institutions and others to look at how we can develop a better understanding of\u00a0enablers of innovation. And before the Innovation Service starts looking into actions of field operations, we want to start at the beginning: helping our field operations understand how they can be innovative through innovation capacity building sessions and toolkits. If there are things you\u2019d like to see from us, please get\u00a0in touch and let us know.<\/p>\n","protected":false},"excerpt":{"rendered":"
The Innovation Index Experiment UNHCR’s Innovation Service has been undertaking an experiment. We created an Innovation Index; our first attempt at measuring how we\u2019re doing when it comes to innovation in our field operations. We never released the product. This piece outlines our Index story looking at the whys, the hows, and most importantly the […]<\/p>\n","protected":false},"author":3,"featured_media":28577,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[544,545],"tags":[],"class_list":["post-28574","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-how-to-resources","category-innovation-process"],"acf":{"author":"","authors_title":"UNHCR Innovation Service"},"yoast_head":"\n