3.1 An update to the classification of goods and services in the CPI presents several challenges that must be addressed to ensure a smooth and accurate transition. It is important to have a clear strategy for data management, analysis, comparability and resource allocation to ensure the transition does not disrupt the ongoing accuracy and reliability of the CPI (see Chapter 2). This chapter focuses on the implementation of COICOP18 in the CPI but is also relevant if other classifications are used.

3.2 Section 3.2 is dedicated to the comparison between COICOP18 and COICOP99. The comparison is described in detail in the document Classification of Individual Consumption According to Purpose (COICOP) 2018. The section focuses on new features introduced in COICOP18 and the correspondence table between COICOP99 and COICOP18. Section 3.3 discusses how to redistribute elementary aggregates according to the new classification. Section 3.4 gives guidance on reviewing the sample of goods and services and the aggregation of goods and services into broader groups using available expenditure data.

3.3 The bridge to move from the old to the new version of classification is a correspondence table that establishes the links between the old and the new classification codes. The United Nations Statistical Division (UNSD) has released the correspondence table between COICOP99 and COICOP18, which covers the 2- to 5-digit categories of COICOP18 and describes how these are related to the categories in COICOP99.

3.4 In general, a correspondence table should begin at the lowest hierarchical level of the classification to link detailed categories in one classification to the most detailed category possible in the other classification. This means that if the new version of the classification does not introduce a new disaggregated level, the correspondence can be of 4 different types:

3.5 If a new disaggregated level is introduced, it is mapped to the lowest disaggregated level of the old classification. In the case of COICOP99 and COICOP18, the lowest level of aggregation in COICOP99 is that of classes (4-digit level) and COICOP18 is subclasses (5-digit level). The UNSD correspondence table maps the COICOP18 subclasses to the appropriate COICOP99 classes.

3.6 COICOP18 has introduced many changes compared to COICOP99 which are summarized in Box 1. The correspondence between the two versions of the classification can be complicated in some cases. As shown in Box 1, one of the new features introduced by COICOP18 is the more detailed 5-digit subclass level, with a proposed further breakdown at the 6-digit level for food products.

 

Box 1 Main difference between COICOP99 and COICOP18

The main differences between COICOP99 and COICOP18 can be summarized as follows:

The following table compares COICOP99 and COICOP18.

3.7 Countries that have used COICOP99 for the CPI most likely introduced further breakdowns below the 4-digit level class level. Regional versions of COICOP99, such as ECOICOP for the European Union (EU) countries and PACCOICOP 2012 developed by the Secretariat for the Pacific Community, also include level of details below the class level. ECOICOP includes a customized 5-digit subclass, which was introduced for the EU countries in 2015.

3.8 To introduce a new classification, countries will have to link the most detailed categories in the old classification, down to the elementary aggregates, to the detailed categories in the new classification. Table 1 presents an example of a detailed correspondence table, in which country specific 5-digit and 6-digit levels used in COICOP99 are linked to the 5-digit subclass level of COICOP18.

 

Table 1 Example of detailed correspondence table between COICOP99 and COICOP18

3.9 Elementary aggregates represent the most detailed level of groups of goods and services for which price indices are compiled. Higher-level price indices are compiled by weighting together elementary aggregate price indices using expenditure shares as weights (CPI Manual, 1.143). In this way, the elementary aggregates form the building blocks of the CPI. When a new classification is introduced, the elementary aggregates must be redistributed according to the new classification. This should be done carefully to maintain comparability over time to the extent possible. The redistribution of elementary aggregates according to COICOP18 can be carried out by considering the different aspects summarized in Box 2.

 

Box 2 Redistributing elementary aggregates to COICOP18

Review of COICOP18 and identification of the elementary aggregates that are affected by the new structure.

When transitioning to COICOP18, it will be necessary to make certain changes to existing elementary aggregates. Some aggregates may need to be expanded to include new products that have emerged since the last classification, others might need to be merged or split to better reflect the new classification structure. Additionally, new elementary aggregates may need to be introduced to accommodate new categories of goods and services.

Using the official correspondence tables (such as UNSD or that of Eurostat), as a starting point to customize the correspondence table for each respective country.

Countries are recommended to use correspondence tables provided by official sources, such as UNSD and Eurostat, as the starting point to customize a correspondence table for the respective country. This correspondence table will demonstrate the link between the elementary aggregates in the old classification and the elementary aggregates in the new classification.

Adjustment of data collection, compilation, and dissemination systems to align with the new classification.

Countries will have to adjust their systems of data collection, index calculation, dissemination, and reporting, including questionnaires, compilation software, press-releases, disseminated data tables, etc. to incorporate changes between the classifications. This process needs particular attention to ensure high quality of the statistical outputs.

Recalculation of historical CPI may be necessary.

Introduction of a new structure may result in non-comparability of certain elementary aggregates or even higher CPI aggregates through time. To avoid breaks in data series and to ensure sufficient international comparability, countries may need to rebuild historical CPI times series according to COICOP18. Each country should decide whether this reclassified series will replace the official, published indices, or if it is rather to assist users with the transition.

3.10 Elementary aggregates are formed at a relatively detailed level, capturing groups of goods and services consumed by households, typically at or below the lowest level of the international classification. For example, COICOP18 introduces a 5-digit subclass level as the most detailed level, which facilitates a more detailed recording of consumption expenditures. Countries may implement further breakdowns and coding below the 5-digit level for national purposes while maintaining the international classification at subclass level and above. This is illustrated in the example in Table 2.

 

Table 2 Example of a detailed national breakdown of COICOP18 5-digit level

3.11 While transitioning to COICOP18, applying the new classification could be achieved by reorganizing the CPI basket items in the new structure without adjusting the expenditure items that households have reported. Expenditure items reflect the level of detail typically collected via the HBS and allow the compilation of elementary aggregate weights. However, if the changes in classification are significant, a more detailed adjustment may be necessary to ensure that the CPI remains representative of consumption patterns. Failing to do so may lead to less accurate and less representative data due to potential discrepancies between the new COICOP structure and the old expenditure items.

3.12 To address this, a general recommendation is to derive the CPI item weights from Household Budget Survey (HBS) data, based on the new classification, which in turn requires conducting the survey according to the COICOP18 (see Chapter 2), or as a minimum, recoding of existing HBS expenditure items to the new classification. This approach enhances data accuracy and consistency, incorporates new goods and services, and minimizes the need for reclassification of expenditures. Weights for higher aggregates (such as sub-division and division) are in some countries derived from national accounts data, which also uses HBS information as the main input data. Therefore, harmonized, and consequent transition of HBS, national accounts and price statistics to the new classification is highly important for ensuring the high quality of the CPI (see Chapter 2). It is also important that updating the basket weights and the introduction of new elementary aggregates related to classification changes should be scheduled during the regular CPI weights update period to maintain adherence to the fixed weights approach throughout the year.

3.13 Deriving weights from the updated HBS may not always be possible, for several reasons such as a lower frequency of HBS (e.g. the HBS is carried out once every five years in some countries), time lags between HBS or national accounts data availability and the CPI compilation process, and constrained resources of NSOs. To overcome this, the weights from the HBS based on the older COICOP version can be adjusted to fit the newer COICOP18. The adjustment process includes mapping the elementary aggregates according to COICOP18 and identifying any elementary aggregates that need to be split into several new elementary aggregates. For the latter, either additional expenditure data (preferable) or expert judgment should be used to divide expenditures among the new detailed subcategories. Supplementary data sources may include:

3.14 CPI compilers should make sure these data sources are suitable for the purpose of CPI weights redistribution. Documentation describing the process of adjustment is also important, including the rationale for the distribution of weights and assumptions made. This transparency will be important for users of the CPI data (see Chapter 5).

3.15 The transition to COICOP18 presents additional challenges for NSOs utilizing transaction data in CPI compilation, such as scanner data. Specifically, the classification process for Global Trade Item Numbers (GTINs) must be adjusted to align with the new COICOP structure. Accomplishing this task may involve several approaches, including mapping (correspondence tables between old and new elementary aggregates), leveraging machine learning, as well as manual reclassification of the transaction data.

3.16 As mentioned, countries that used COICOP99 for the CPI may have implemented further national breakdowns below the 4-digit class level. For instance, in COICOP99, class 01.1.1 ‘Bread and cereals’ was the lowest level for classifying this type of product. Hence, countries may have recorded all products within this class by introducing a 5-digit subclass level, as illustrated in Table 3 and further breaking down the subclass by a 6-digit level category, as in Table 4 for ‘Bread’ (the example is taken from ECOICOP adopted in 2015 in the EU).

 

Table 3 ECOICOP 5-digit detail of COICOP99 classes. Bread and cereals

 

Table 4 ECOICOP 6-digit detail of subclasses. Bread

3.17 By introducing the 5-digit subclass level, COICOP18 allows NSOs to focus attention on the coverage of households’ consumption across the CPI basket at a consistent and detailed level. For instance, COICOP18 first changes the definition of class 01.1.1 from ‘Bread and cereals’ to ‘Cereals and cereals products’ and then further breaks this down into six subclasses (Table 5).

 

Table 5 COICOP18 5-digit detail of class Cereals and cereal products

3.18 In COICOP18, ‘cereals’ are now separated from ‘flour of cereals’, and a 6-digit level is proposed that, beyond providing further details, can be helpful to detect if there are missing products in the CPI basket. The 6-digit detail of the subclass ‘cereals’ is shown in Table 6. Looking at this detail, which was not available in COICOP99, it would be possible to detect insufficient coverage of the existing CPI basket of a country, understanding, for instance, that ‘rice’ should be complemented by ‘maize and quinoa’ because they are relevant in the households’ consumption or to better cover the subclass of ‘cereals’.

 

Table 6 COICOP18 6-digit detail of subclass Cereals

3.19 COICOP18 aims to reflect the development of information and communication services. Division 8 is now ‘Information and communication’, whereas, in COICOP99, it was ‘Communication’. Within this division, instead of having a class 08.3.0, ‘telephone and telefax services’, there is a group 08.3 ‘Information and communication services’, broken down into 6 classes, repeated at the level of a subclass, except for 08.3.9 ‘Other information and communication services’, which is split into two subclasses (Table 7). This presents the opportunity to increase its coverage in CPI. 08.3.4.0 ‘Bundled telecommunication services’ includes telephone/Internet/TV packages and any other combination of telecommunication packages, often combining goods and services, such as smartphones and Internet and TV subscriptions. Hence, COICOP18 now facilitates the introduction of such packages. The extent of these may vary by country, and compilers should be guided by expenditure data.

 

Table 7 COICOP18 4- and 5-digit detail of Group 08.3 Information and Communication Services

Introduction of new products in the CPI

3.20 As a result of changes in the classification, it may be necessary to add new products to the sample of goods and services. A new product can be introduced in the compilation of the CPI as a new elementary aggregate. When a new elementary aggregate is established, the elementary price index can be compiled and included in the calculation of higher-level indices (see Chapter 4.3). New products may also be introduced in existing elementary aggregates. In this case, there are two possibilities:

1) The new product enters an elementary index calculated without expenditure weights, i.e., as an unweighted average of observed prices.

2) The new product enters an elementary index calculated with the use of expenditure weights, i.e., as a weighted average of observed prices.

3.21 To illustrate, consider an example where an elementary index has to be calculated for subclass 01.1.1.1 ‘Cereals’ in Table 6. Assume that prices were already collected for ‘millet‘ (01.1.1.1.5) and ’maize’ (01.1.1.1.6) and that the NSO considers adding ‘quinoa’ (01.1.1.1.7) in the calculation of the elementary index for ‘Cereals’.

Calculation of an unweighted elementary price index

3.22 If an elementary price index is calculated without weights, prices for ‘quinoa’ can be combined with prices for ‘millet’ and ‘maize’ in an unweighted mean to obtain the elementary price index for ‘cereals’. While no explicit weights are used, the actual weight of ‘millet’ and ‘maize’ prices will be reduced when prices for ‘quinoa’ are included in the calculation of the elementary index.

Calculation of a weighted elementary price index

3.23 If an elementary price index is calculated with weights, prices for ‘quinoa’ must be assigned a weight to reflect the relative importance of ‘quinoa’ compared to ‘millet’ and ‘maize’ in households’ consumption expenditure.

3.24 If suitable data are available to estimate the weights of ‘millet’, ‘maize’ and ‘quinoa’, the elementary index for ‘Cereals’ can be calculated as the weighted average of the prices of these three product groups. In the case where the data source is the same, the derived weights will usually be comparable and suitable for use. If the weight for a new product group, in this example, ‘quinoa’, is based on other data sources, the NSO should ensure that the expenditure data and the derived weights are comparable.

3.25 If data sources are not available to estimate the weight of a new product, it should be added to the HBS. In the example, ‘quinoa’ should be added to the list of food items in the HBS. If the HBS is near to being carried out, it is recommended to check if ‘quinoa’ is already included. If the HBS is far from being carried out, a special survey or other data sources may help to obtain data to estimate the weights until weights from the HBS are available.

3.26 The inclusion of new products in the compilation of CPI due to an update of the classification should be carried out following the general rules and timing for updating the sample of goods and services and the weights of the CPI.