Construction of composite indices is an important method for measuring multi-dimensional concepts by combining large number of indicators in a standardized way into a single index. Many indices have been developed, such as the Human development index, the OECD Better Life Index, Corruption perceptions index and others. Composite indices are a useful tool in policy analysis and public communication, because they allow simple comparison of countries and identifying trends. Nevertheless, there are challenges in constructing composite indices. Here we will point out seven issues that we deem important for the current JustJobs Index.
Lack of data may damage the link between the indicators and the conceptual framework
It is important to select indicators that are compatible with a conceptually sound theoretical framework. Data are available for many of countries. Nevertheless, data availability and collection is a big challenge. To construct a composite index, a complete data set of indicators for all the included countries during the study period must be prepared. When data are not available, conceptually important aspects of the index may have to be omitted.
In the JustJobs Index imputation of missing data is used to fill in any gaps, but there are several limitations. Imputation is only possible when there is at least one data point available in the country to be included during the study period for each indicator. When data is completely missing for one indicator for all years, the country cannot be included in the JustJobs Index. Imputation is more accurate when there are relatively few missing data points for a country. Imputation is also more accurate in the countries with relatively stable indicators than for indicators that are unstable. For the developing countries with important reforms, the indicators could have changed rapidly in certain periods than other periods. It is difficult for the large scale imputation to make corresponding adjustments unless one looks closely at each country.
Bad data may distort results
The quality of indicators can distort the results. Quality differs between countries, but is generally higher in developed countries than in developing countries. For example, the available data on the cases of occupational injury show very high incidence in European countries, such as Luxembourg, Spain; and rather high incidence in Northern European countries, such as Denmark, Norway and Sweden; but very low incidence in Asian and African countries. The challenge could be the difficulty in collecting such information in developing countries where a system to collect or report such cases is not yet in place, while such systems have been rather sophisticated in developed countries. Thus, the inclusion and interpretation of such indicators can be more misleading than beneficial to the JustJobs Index.
Some indicators may be difficult to interpret
The contributions of some indicators are difficult to interpret in the whole system of JustJobs Index. While the status in employment is conceptually considered as important indicator to explore the opportunities for work, the interpretation on the prevalence of wage and self-employed workers is not straightforward. While the main purpose of constructing JustJobs Index is to compare across countries, individual indicators need to be combined with own clear contribution to the composite index. It is, therefore, not possible to include indicators with controversial interpretations. Alternatively, the prevalence of vulnerable employment which has clearer interpretation can be used so as to explore employment stability.
Fourth, the value of JustJobs Index needs to be interpreted with caution. The composite index is constructed by combining number of indicators in a standardized way, and the main purpose is to compare between countries. The composite index itself is not meaningful, and the rank of the index over a list of countries in each year is the main aim of the practice. That makes it meaningless to compare the JustJobs Index over years for certain country. The comparison of countries’ ranks should be made instead.
Improvement in data collection may lead to changes in the index
The JustJobs Index is vulnerable to changes when data collection is improved. As discussed earlier, data missing is one of the main challenges in composite index construction. The effort made on data collection can contribute to adjustment in index construction, which may result in shifting of countries in their rankings among the list of countries. It is, therefore, a challenge to properly understand the shifting of country rankings in JustJobs Index by studying the new data included.
It is not trivial how indicators should be weighted
The current version of JustJobs Index has constructed based on equal weighting of all the indicators included. Equal weighting is preferred by many. Still, the construction of the index hides away some inequality of weighting: If several indicators are used to cover one aspect, that aspect receives more weight. For example, when employment are measured with the labour force participation rate, unemployment rate, and youth unemployment rate; then employment is counted thrice. Some will argue that the weight should.
Robustness and sensitivity is important
While we have tested the index with respect to sensitivity and the consequences of normalization methods, the work in this regard is far from finished. There are many indicators that we would like to include that cannot be included because of lack of data, and this limits the possibility of sensitivity tests.
Main lesson: Data must be improved
The high level panel for the post 2015 development goals made the point that a data revolution is needed, because “Too often, development efforts have been hampered by a lack of the most basic data about the social and economic circumstances in which people live”. That conclusion is equally valid in the case of Just Jobs. Monitoring progress towards better working conditions and equity at work is seriously hampered by lack of good data.
