17. Serviceability and Accessibility of External Sector Statistics
- Eduardo Valdivia-Velarde, and Tamara Razin
- Published Date:
- December 2014
17.1 The focus of this chapter is on two important aspects of data quality assessment—serviceability and accessibility—and on their applicability to the external sector statistics. Serviceability of statistics refers to the extent to which datasets are kept in good working condition to be able to serve various users’ needs effectively. Accessibility refers to the extent to which statistics are made available to users in a clear and understandable manner; the adequacy of the forms of dissemination; the level of restriction in accessing the data; the extent to which pertinent metadata are made available and kept up-to-date; and the availability and promptness of support to data users. Serviceability and accessibility are two of the five dimensions of the IMF data quality assessment framework (DQAF), the Fund’s organizing model of the Report on the Observance of Standards and Codes (Data ROSC). Other dimensions of data quality are assurance of data integrity, methodological soundness, accuracy and reliability, along with the prerequisites of quality that relate to institutional aspects (see Box 17.1). These attributes of data quality are applicable in the assessment of external sector statistics.
17.2 To serve the various users’ needs effectively, external sector statistics must be in good working condition. This requires that data and metadata must be kept relevant, produced in a timely manner within the defined periodicity, be fully consistent within the dataset, and based on a clearly defined revision policy and practice. These attributes are elaborated upon ahead.
17.3 To remain relevant for effective use, external sector statistics must be closely monitored through a comprehensive and well-established process of consultation. The consultation process must involve both compilers and users of the statistics and, to the extent possible, must involve several stakeholders including representatives from the government, private sector, and the academia. In some economies, such consultation involves formally established advisory bodies or user groups that bring together both compilers and users. The advisory bodies provide an opportunity for the compiling agency to identify the extent to which user views are reflected in the ongoing work of external sector data developments, compilation, and analysis. Feedback received from such bodies play a crucial role in helping to prioritize developmental projects for external sector statistics.
17.4 Timeliness and periodicity of production are another important element in data quality assessment. Timeliness usually refers to the time lag between the reference period (to which the data pertain) and the date at which the data become available (including the time lag between the advertised date for release and the actual date of release). For example, the reference period may be the 2011 calendar year, but data may not become available for analysis until June 2012. These aspects are important and should be taken into consideration when assessing data quality, as lengthy delays between the reference period and data availability, or between advertised and actual release dates, can have implications for data accuracy or reliability, and the credibility of the statistical agency. If data collection is expected to be ongoing, the frequency of release is an important measure of data serviceability. Frequency includes information about the proposed frequency of repeated collections and when data will be released for subsequent reference periods. To assist in evaluating the timeliness and periodicity aspects of production of external sector statistics or products, the following questions could be helpful:
- What gap of time exists between the reference period, the time when the data were collected, and time when the statistics became available?
- Are there likely to be subsequent surveys or data collection issues for this topic?
- Are there likely to be updates or revisions to the data after official release?
- What is the gap between the advertised and actual release dates of the data?
17.5 Consistency within the dataset is another important element in assessing the extent to which external sector statistics are serviced. Consistency refers to the internal coherence of a statistical collection, product, or release, as well as its comparability with other sources of information, within a broad analytical framework and over a reasonable period of time. To be well serviced, external sector statistics must be coherent within the dataset, over time, and with other major datasets. The use of standard concepts, classifications, and target population promotes coherence, as does the use of common methodology across surveys. Consistency is an important element as it provides an indication of whether the dataset can be usefully compared with other sources to enable data compilation and comparison. However, this does not necessarily imply full numerical consistency, but consistency in methods and collection standards. Whenever the agency responsible for external sector statistics introduces changes in source data, methodology, or techniques, it is imperative that historical series are reconstructed as far back as reasonably possible. Quality statements of statistical measures must include a discussion of any factors that would affect the comparability of the data over time. To this end, the coherence of a statistical collection, product, or release can be evaluated by considering a number of key aspects:
- Changes to data items—To what extent a long time series of particular data items might be available, or whether significant changes have occurred to the way that data are collected and analyzed
- Comparison across data items—This refers to the capacity to make meaningful comparisons across multiple data items within the same collection. The ability to make comparisons may be affected if there have been significant changes in collection, processing, or estimation methodology that might have occurred across multiple items within a collection.
- Comparison with previous releases—The extent to which there have been significant changes in collection, processing, or estimation methodology in this release compared with previous releases, or any “real world” events that have impacted on the data since the previous release
- Comparison with other products available—This refers to whether there are any other data sources with which a particular series has been compared, and whether these two sources tell the same story. This aspect may also include identification of any other key data sources with which the data cannot be compared, and the reasons for this, such as differences in scope or definitions.
17.6 The IMF DQAF provides comprehensive guidance on consistency evaluation, and the following questions could be helpful in this regard:
- Is it possible to compile a consistent time series of a particular data item of interest over a number of years?
- To what extent can a user meaningfully compare several data items within this collection?
- Could any natural disasters or significant economic events have influenced the data since the previous release?
- Have these data been confronted with other data sources, and are the messages consistent from all data sources?
17.7 Finally, fully serviced external sector statistics must follow a well-articulated data revision policy and practice. The IMF DQAF specifies three attributes of good revision policy. Firstly, the revision policy must follow a regular, well-established, and transparent schedule. For example, revision policy should be clearly stated in balance of payments quarterly publications, as well as for users accessing the data online. Quarterly data could be revised on an ongoing basis, while data for previous years could be revised once a year. Secondly, preliminary data or first estimates must be clearly identified and users with direct access to such data informed accordingly. Because of the need for more information, the recent financial crisis has brought to the fore the need to provide clear guidelines for compiling “flash” estimates of datasets. Thirdly, whenever major revisions are carried out, it is important to explain the changes with, or before, first release of revised data.
17.8 Generally, accessibility of a statistical collection, product, or release can be evaluated by considering the extent to which data and metadata are made available to the general public and the level of assistance provided to users. Good access to external sector data requires that pertinent metadata are made publicly available in a clear and understandable manner; forms of dissemination are adequate; and statistics are made available on an impartial basis and are up-to-date; and prompt and knowledgeable support service is available to users. As regards the latter, pertinent information should be publicly available in appropriate formats and through appropriate delivery channels, and should be written in plain language adapted to the level of understanding of the main user groups. For many citizens, the news media provide their only exposure to official statistics; therefore it is very important that the statistical agency responsible for external sector statistics be able to communicate effectively with the media. The media uses many vehicles to reach their audiences, including newspapers, magazines, and other periodicals, as well as radio, television, and the Internet. Newspaper and other printed media can provide more detailed coverage of statistical information. The Internet has also become a prominent source of information for many data users; thus it is important to have a good Internet site to facilitate easy retrieval of data and dissemination of key messages to the media.
17.9 The statistical agency responsible for external sector statistics must ensure that its clients are able to access and correctly interpret the information on statistical methods, concepts, variables, and classifications used in producing statistical results. This means that external sector statistics must be presented in a manner that facilitates proper interpretation and meaningful comparisons. To promote proper interpretation, publication of external sector statistics should feature a section with comprehensive metadata providing key concepts, definitions, and data sources for the main accounts. If data quality assessment is made, any discrepancies should be adequately explained to the reader. For a major statistical release, it is often helpful for the statistical agency to organize a press briefing event.
17.10 Good access to external sector statistics requires that users be provided with adequate information on how and where to access key information, the contact person, and other services, including information on any charges. Where feasible, special data services could be provided, including special or nonstandard groupings of data items or outputs, and their usefulness and their costs. For example, the publication for each release should include tables and charts (if appropriate) as well as analysis of developments in the form of highlights. The highlights could be used to convey significant findings, comparisons, and trends to assist the media, and other users, in understanding and using the publications. This approach helps to demonstrate the relevance of external sector statistics to the general public and fosters informed decision making throughout society more effectively.
17.11 Users often have more confidence in the integrity of the statistics if they are released according to a published advanced release schedule giving the date (and preferably the time) when the figures will be available to all users. It is essential that the dates in the schedule are met.
17.12 The extent to which the agency responsible for external sector statistics can communicate effectively with and through the media has a large impact on how well it can achieve these objectives. Thus it is in the best interest of the agency responsible for disseminating external sector statistics to build a strong working relationship with the media, to make it easy for journalists to report on statistical information in an accurate, timely, and informative manner, and to take steps to increase media coverage as a way of reaching the broader society with important statistical information. The key to building strong working relationships with the media is to understand who they are and how best to meet their information needs in a manner that is both proactive and user friendly.
Box 17.1The Data Quality Assessment Framework
The DQAF covers five dimensions of quality and a set of prerequisites for the assessment of data quality. The coverage of these dimensions recognizes that data quality encompasses characteristics related to the institution or system behind the production of the data, as well as characteristics of the individual data product. Within this framework, each dimension comprises a number of elements, which are in turn associated with a set of desirable practices. The following are the statistical practices that are associated with each dimension:
- Prerequisites of quality—The environment is supportive of statistics; resources are commensurate with needs of statistical programs; and quality is a cornerstone of statistical work.
- Integrity—Statistical policies and practices are guided by professional principles, are transparent, and are guided by ethical standards.
- Methodological soundness—Concepts and definitions used are in accord with internationally accepted statistical frameworks; the scope is in accord with internationally accepted standards, guidelines, or good practices; classification and sectorization systems are in accord with internationally accepted standards, guidelines, or good practices; and flows and positions are valued and recorded according to internationally accepted standards, guidelines, or good practices.
- Accuracy and reliability—Source data available provide an adequate basis to compile statistics; statistical techniques employed conform with sound statistical procedures; source data are regularly assessed and validated; intermediate results and statistical outputs are regularly assessed and validated; and revisions, as a gauge of reliability, are tracked and mined for the information they may provide.
- Serviceability—Statistics cover relevant information on the subject field; timeliness and periodicity follow internationally accepted dissemination standards; statistics are consistent within the dataset, over time, and with other major datasets; and data revisions follow a regular and publicized procedure.
- Accessibility—Statistics are presented in a clear and understandable manner, forms of dissemination are adequate, and statistics are made available on an impartial basis; up-to-date and pertinent metadata are made available; and prompt and knowledgeable support service is available.