Table of contents
  1. Story
  2. Slides
    1. Slide 1 Why Doesn't EPA Have a Self-Contained Statistical Unit?: A Tribute to Doug Engelbart
    2. Slide 2 A Tribute to Doug Engelbart
    3. Slide 3 Preface
    4. Slide 4 Purpose
    5. Slide 5 Some of My Principles and Practices
    6. Slide 6 A Well-designed Spreadsheet
    7. Slide 7 A New, Innovative Way to Display Water Quality Information
    8. Slide 8 Scotland’s Environment: Homepage
    9. Slide 9 Scotland’s Environment: Trends and Indicators
    10. Slide 10 The Scottish Government Environmental Statistics
    11. Slide 11 “Drag and Drop” Onto a Tool
    12. Slide 12 Creates Statistics and Visualizations in the Public and Private Cloud
    13. Slide 13 Get a Data Story Idea
    14. Slide 14 Add Your Personal Experience
    15. Slide 15 Add Your Personal Opinion
    16. Slide 16 Bring In More Ideas and Data Sets
    17. Slide 17 EPA Scientists Used These Data Sets
    18. Slide 18 EPA Provides These Open Data Sets
    19. Slide 19 EPA Just Received Recognition For Their GeoPlatform
    20. Slide 20 US EPA Environmental Dataset Gateway Download
    21. Slide 21 EDG Well-Designed Spreadsheet
    22. Slide 22 EDG Visualizations: Bar Charts
    23. Slide 23 EDG Visualizations: Bar Charts
    24. Slide 24 Build a Knowledge Base in MindTouch
    25. Slide 25 Build a Knowledge Base Index in Spreadsheet
    26. Slide 26 Some Conclusions and Recommendations
  3. Spotfire
  4. Spotfire
  5. Research Notes
  6. U.S. Environmental Protection Agency Daily Digest Bulletin
  7. EPA Scientists Presented Open Science at White House
  8. Spotfire – a new, innovative way to display water quality information
    1. Search for Spotfire
      1. Leading-edge approach
  9. Principles and Practices for a Federal Statistical Agency Fifth Edition
    1. Highlights
      1. Principle 1: Relevance to Policy Issues
      2. Principle 2: Credibility Among Data Users
      3. Principle 3: Trust Among Data Providers
      4. Principle 4: Independence from Political and Other Undue External Influence
    2. Definition of a Statistical Agency
    3. Why We Need Statistical Agencies
    4. About the Committee on National Statistics
  10. Full Report
    1. Cover Page
    2. The National Academies Press
    3. The National Academy of Sciences
    4. Committee on National Statistics 2012-2013
    5. Dedication
    6. Acknowledgments
    7. Executive Summary
    8. Introduction
    9. Part I: Principles and Practices for a Federal Statistical Agency
      1. Definition of a Federal Statistical Agency
      2. Establishment of a Federal Statistical Agency
      3. Principles for a Federal Statistical Agency
        1. Principle 1: Relevance to Policy Issues
        2. Principle 2: Credibility Among Data Users
        3. Principle 3: Trust Among Data Providers
        4. Principle 4: Independence from Political and Other Undue External Influence
      4. Practices for a Federal Statistical Agency
        1. Practice 1: A Clearly Defined and Well-Accepted Mission
        2. Practice 2: Necessary Authority to Protect Independence
        3. Practice 3: Continual Development of More Useful Data
        4. Practice 4: Openness About Sources and Limitations of the Data Provided
        5. Practice 5: Wide Dissemination of Data
        6. Practice 6: Cooperation with Data Users
        7. Practice 7: Respect for the Privacy and Autonomy of Data Providers
        8. Practice 8: Protection of the Confidentiality of Data Providers’ Information
        9. Practice 9: Commitment to Quality and Professional Standards of Practice
        10. Practice 10: An Active Research Program
        11. Practice 11: Professional Advancement of Staff
        12. Practice 12: A Strong Internal and External Evaluation Program
        13. Practice 13: Coordination and Collaboration with Other Statistical Agencies
      5. Footnotes
        1. 1
        2. 2
        3. 3
        4. 4
        5. 5
    10. Part II: Commentary
      1. Definition of a Federal Statistical Agency
      2. Establishment of a Federal Statistical Agency
      3. Principles for a Federal Statistical Agency
        1. Principle 1: A federal statistical agency must be in a position to provide objective, accurate, and timely information that is relevant to issues of public policy
        2. Principle 2: A federal statistical agency must have credibility with those who use its data and information
        3. Principle 3: A federal statistical agency must have the trust of those whose information it obtains
        4. Principle 4: A federal statistical agency must be independent from political and other undue external influence in developing, producing, and disseminating statistics
      4. Practices for a Federal Statistical Agency
        1. Practice 1: A Clearly Defined and Well-Accepted Mission
        2. Practice 2: Necessary Authority to Protect Independence
        3. Practice 3: Continual Development of More Useful Data
          1. Introduction
          2. Multiple Data Sources
          3. The Role of Surveys
          4. Roles for Administrative Records
          5. Roles for Nontraditional Data Sources
          6. Integration and Synchronization of Data Across Agencies
          7. Longitudinal Data
          8. Operational Methods
        4. Practice 4: Openness About Sources and Limitations of the Data Provided
        5. Practice 5: Wide Dissemination of Data
        6. Practice 6: Cooperation with Data Users
        7. Practice 7: Respect for the Privacy and Autonomy of Data Providers
          1. Introduction
          2. Respecting Privacy
          3. Protecting and Respecting the Autonomy of Human Research Participants
        8. Practice 8: Protection of the Confidentiality of Data Providers’ Information
        9. Practice 9: Commitment to Quality and Professional Standards of Practice
        10. Practice 10: An Active Research Program
          1. Substantive Research and Analysis
          2. Research on Methodology and Operations
          3. Research on Policy Uses
        11. Practice 11: Professional Advancement of Staff
        12. Practice 12: A Strong Internal and External Evaluation Program
          1. Introduction
          2. Evaluating Quality
          3. Evaluating Relevance
          4. Types of Reviews
        13. Practice 13: Coordination and Collaboration with Other Statistical Agencies
          1. Introduction
          2. Role of OMB
          3. Forms of Interagency Collaboration
      5. Footnotes
        1. 1
        2. 2
        3. 3
        4. 4
        5. 5
        6. 6
        7. 7
        8. 8
        9. 9
        10. 10
        11. 11
        12. 12
        13. 13
        14. 14
        15. 15
        16. 16
        17. 17
        18. 18
        19. 19
        20. 20
        21. 21
        22. 22
        23. 23
        24. 24
        25. 25
        26. 26
        27. 27
        28. 28
        29. 29
        30. 30
        31. 31
        32. 32
        33. 33
        34. 34
        35. 35
        36. 36
        37. 37
        38. 38
        39. 39
        40. 40
        41. 41
        42. 42
        43. 43
        44. 44
        45. 45
        46. 46
    11. Appendixes
      1. A Legislation and Regulations That Govern Federal Statistics
        1. The 1980 Paperwork Reduction Act and Associated Guidance
          1. Background
          2. Paperwork Reduction
          3. Survey Clearance Process
        2. OMB Statistical Policy Directives
          1. Standards and Guidelines for Statistical Surveys
          2. Compilation, Release, and Evaluation of Principal Economic Indicators
          3. Release and Dissemination of Statistical Products
          4. North American Industry Classification System/Standard Industrial Classification
          5. Standard Occupational Classification
          6. Metropolitan Area Classification
          7. Definition of Poverty
          8. Data on Race and Ethnicity
        3. The 1997 Order Providing For The Confidentialy of Statistical Information
        4. The 2002 Confidential Information Protection and Statistical Efficiency Act and Associated Guidance
          1. Subtitle A—Protecting Confidentiality
        5. Subtitle B—Sharing Data
        6. OMB CIPSEA Guidance
        7. The 2002 E-Government Act, Section 208
        8. The 2002 Federal Information Security Management Act
        9. The 2000 Information Quality Act and Associated Guidelines
        10. 2004 OMB Peer Review Guidance
        11. 2010 Government Performance and Results Modernization Act
        12. 2009–2010 Guidance on Scientific Integrity
        13. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
          5. 5
          6. 6
          7. 7
          8. 8
          9. 9
          10. 10
          11. 11
          12. 12
          13. 13
          14. 14
          15. 15
          16. 16
          17. 17
          18. 18
          19. 19
          20. 20
          21. 21
          22. 22
          23. 23
          24. 24
          25. 25
          26. 26
          27. 27
          28. 28
          29. 29
          30. 30
          31. 31
          32. 32
      2. B Organization of the Federal Statistical System
        1. Overview
          1. Budget
          2. Value
          3. Structure
          4. Figure B-1 Organization of principal federal statistical agencies and programs, by department, 2013.
          5. Figure B-2 Members of the Interagency Council on Statistical Policy: Organizational location and relevant congressional appropriations subcommittee based on subcommittee jurisdictions in the 113th Congress.
        2. Office of Management and Budget
        3. Principal Statistical Agencies
          1. Bureau of Economic Analysis
          2. Bureau of Justice Statistics
          3. Bureau of Labor Statistics
          4. Bureau of Transportation Statistics
          5. Census Bureau
          6. Economic Research Service
          7. Energy Information Administration
          8. National Agricultural Statistics Service
          9. National Center for Education Statistics
          10. National Center for Health Statistics
          11. National Center for Science and Engineering Statistics
          12. Office of Research, Evaluation, and Statistics
          13. Statistics of Income Division
        4. Other Statistical Programs
          1. Health and Retirement Study
          2. Medical Expenditure Panel Survey
          3. National Agricultural Workers Survey
          4. National Automotive Sampling System
          5. National Resources Inventory
          6. National Survey on Drug Use and Health
          7. Panel Study of Income Dynamics
        5. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
          5. 5
      3. C Fundamental Principles of Official Statistics of the Statistical Commission of the United Nations
      4. D European Statistics Code of Practice for the National and Community Statistical Authorities
        1. Preamble
          1. The vision of the European Statistical System 1
          2. The mission of the European Statistical System
          3. The European Statistics Code of Practice
        2. Institutional Environment
          1. Principle 1: Professional Independence
          2. Principle 2: Mandate for Data Collection
          3. Principle 3: Adequacy of Resources
          4. Principle 4: Commitment to Quality
          5. Principle 5: Statistical Confidentiality
          6. Principle 6: Impartiality and Objectivity
        3. Statistical Processes
          1. Principle 7: Sound Methodology
          2. Principle 8: Appropriate Statistical Procedures
          3. Principle 9: Non-excessive Burden on Respondents
          4. Principle 10: Cost Effectiveness
        4. Statistical Output
          1. Principle 11: Relevance
          2. Principle 12: Accuracy and Reliability
          3. Principle 13: Timeliness and Punctuality
          4. Principle 14: Coherence and Comparability
          5. Principle 15: Accessibility and Clarity
        5. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
      5. E Selected Federal Statistical Websites, February 2013
        1. Web Sites
        2. Index of Federal Statistical Sites Listed
    12. References and Selected Bibliography

Principles and Practices for a Federal Statistical Agency

Last modified
Table of contents
  1. Story
  2. Slides
    1. Slide 1 Why Doesn't EPA Have a Self-Contained Statistical Unit?: A Tribute to Doug Engelbart
    2. Slide 2 A Tribute to Doug Engelbart
    3. Slide 3 Preface
    4. Slide 4 Purpose
    5. Slide 5 Some of My Principles and Practices
    6. Slide 6 A Well-designed Spreadsheet
    7. Slide 7 A New, Innovative Way to Display Water Quality Information
    8. Slide 8 Scotland’s Environment: Homepage
    9. Slide 9 Scotland’s Environment: Trends and Indicators
    10. Slide 10 The Scottish Government Environmental Statistics
    11. Slide 11 “Drag and Drop” Onto a Tool
    12. Slide 12 Creates Statistics and Visualizations in the Public and Private Cloud
    13. Slide 13 Get a Data Story Idea
    14. Slide 14 Add Your Personal Experience
    15. Slide 15 Add Your Personal Opinion
    16. Slide 16 Bring In More Ideas and Data Sets
    17. Slide 17 EPA Scientists Used These Data Sets
    18. Slide 18 EPA Provides These Open Data Sets
    19. Slide 19 EPA Just Received Recognition For Their GeoPlatform
    20. Slide 20 US EPA Environmental Dataset Gateway Download
    21. Slide 21 EDG Well-Designed Spreadsheet
    22. Slide 22 EDG Visualizations: Bar Charts
    23. Slide 23 EDG Visualizations: Bar Charts
    24. Slide 24 Build a Knowledge Base in MindTouch
    25. Slide 25 Build a Knowledge Base Index in Spreadsheet
    26. Slide 26 Some Conclusions and Recommendations
  3. Spotfire
  4. Spotfire
  5. Research Notes
  6. U.S. Environmental Protection Agency Daily Digest Bulletin
  7. EPA Scientists Presented Open Science at White House
  8. Spotfire – a new, innovative way to display water quality information
    1. Search for Spotfire
      1. Leading-edge approach
  9. Principles and Practices for a Federal Statistical Agency Fifth Edition
    1. Highlights
      1. Principle 1: Relevance to Policy Issues
      2. Principle 2: Credibility Among Data Users
      3. Principle 3: Trust Among Data Providers
      4. Principle 4: Independence from Political and Other Undue External Influence
    2. Definition of a Statistical Agency
    3. Why We Need Statistical Agencies
    4. About the Committee on National Statistics
  10. Full Report
    1. Cover Page
    2. The National Academies Press
    3. The National Academy of Sciences
    4. Committee on National Statistics 2012-2013
    5. Dedication
    6. Acknowledgments
    7. Executive Summary
    8. Introduction
    9. Part I: Principles and Practices for a Federal Statistical Agency
      1. Definition of a Federal Statistical Agency
      2. Establishment of a Federal Statistical Agency
      3. Principles for a Federal Statistical Agency
        1. Principle 1: Relevance to Policy Issues
        2. Principle 2: Credibility Among Data Users
        3. Principle 3: Trust Among Data Providers
        4. Principle 4: Independence from Political and Other Undue External Influence
      4. Practices for a Federal Statistical Agency
        1. Practice 1: A Clearly Defined and Well-Accepted Mission
        2. Practice 2: Necessary Authority to Protect Independence
        3. Practice 3: Continual Development of More Useful Data
        4. Practice 4: Openness About Sources and Limitations of the Data Provided
        5. Practice 5: Wide Dissemination of Data
        6. Practice 6: Cooperation with Data Users
        7. Practice 7: Respect for the Privacy and Autonomy of Data Providers
        8. Practice 8: Protection of the Confidentiality of Data Providers’ Information
        9. Practice 9: Commitment to Quality and Professional Standards of Practice
        10. Practice 10: An Active Research Program
        11. Practice 11: Professional Advancement of Staff
        12. Practice 12: A Strong Internal and External Evaluation Program
        13. Practice 13: Coordination and Collaboration with Other Statistical Agencies
      5. Footnotes
        1. 1
        2. 2
        3. 3
        4. 4
        5. 5
    10. Part II: Commentary
      1. Definition of a Federal Statistical Agency
      2. Establishment of a Federal Statistical Agency
      3. Principles for a Federal Statistical Agency
        1. Principle 1: A federal statistical agency must be in a position to provide objective, accurate, and timely information that is relevant to issues of public policy
        2. Principle 2: A federal statistical agency must have credibility with those who use its data and information
        3. Principle 3: A federal statistical agency must have the trust of those whose information it obtains
        4. Principle 4: A federal statistical agency must be independent from political and other undue external influence in developing, producing, and disseminating statistics
      4. Practices for a Federal Statistical Agency
        1. Practice 1: A Clearly Defined and Well-Accepted Mission
        2. Practice 2: Necessary Authority to Protect Independence
        3. Practice 3: Continual Development of More Useful Data
          1. Introduction
          2. Multiple Data Sources
          3. The Role of Surveys
          4. Roles for Administrative Records
          5. Roles for Nontraditional Data Sources
          6. Integration and Synchronization of Data Across Agencies
          7. Longitudinal Data
          8. Operational Methods
        4. Practice 4: Openness About Sources and Limitations of the Data Provided
        5. Practice 5: Wide Dissemination of Data
        6. Practice 6: Cooperation with Data Users
        7. Practice 7: Respect for the Privacy and Autonomy of Data Providers
          1. Introduction
          2. Respecting Privacy
          3. Protecting and Respecting the Autonomy of Human Research Participants
        8. Practice 8: Protection of the Confidentiality of Data Providers’ Information
        9. Practice 9: Commitment to Quality and Professional Standards of Practice
        10. Practice 10: An Active Research Program
          1. Substantive Research and Analysis
          2. Research on Methodology and Operations
          3. Research on Policy Uses
        11. Practice 11: Professional Advancement of Staff
        12. Practice 12: A Strong Internal and External Evaluation Program
          1. Introduction
          2. Evaluating Quality
          3. Evaluating Relevance
          4. Types of Reviews
        13. Practice 13: Coordination and Collaboration with Other Statistical Agencies
          1. Introduction
          2. Role of OMB
          3. Forms of Interagency Collaboration
      5. Footnotes
        1. 1
        2. 2
        3. 3
        4. 4
        5. 5
        6. 6
        7. 7
        8. 8
        9. 9
        10. 10
        11. 11
        12. 12
        13. 13
        14. 14
        15. 15
        16. 16
        17. 17
        18. 18
        19. 19
        20. 20
        21. 21
        22. 22
        23. 23
        24. 24
        25. 25
        26. 26
        27. 27
        28. 28
        29. 29
        30. 30
        31. 31
        32. 32
        33. 33
        34. 34
        35. 35
        36. 36
        37. 37
        38. 38
        39. 39
        40. 40
        41. 41
        42. 42
        43. 43
        44. 44
        45. 45
        46. 46
    11. Appendixes
      1. A Legislation and Regulations That Govern Federal Statistics
        1. The 1980 Paperwork Reduction Act and Associated Guidance
          1. Background
          2. Paperwork Reduction
          3. Survey Clearance Process
        2. OMB Statistical Policy Directives
          1. Standards and Guidelines for Statistical Surveys
          2. Compilation, Release, and Evaluation of Principal Economic Indicators
          3. Release and Dissemination of Statistical Products
          4. North American Industry Classification System/Standard Industrial Classification
          5. Standard Occupational Classification
          6. Metropolitan Area Classification
          7. Definition of Poverty
          8. Data on Race and Ethnicity
        3. The 1997 Order Providing For The Confidentialy of Statistical Information
        4. The 2002 Confidential Information Protection and Statistical Efficiency Act and Associated Guidance
          1. Subtitle A—Protecting Confidentiality
        5. Subtitle B—Sharing Data
        6. OMB CIPSEA Guidance
        7. The 2002 E-Government Act, Section 208
        8. The 2002 Federal Information Security Management Act
        9. The 2000 Information Quality Act and Associated Guidelines
        10. 2004 OMB Peer Review Guidance
        11. 2010 Government Performance and Results Modernization Act
        12. 2009–2010 Guidance on Scientific Integrity
        13. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
          5. 5
          6. 6
          7. 7
          8. 8
          9. 9
          10. 10
          11. 11
          12. 12
          13. 13
          14. 14
          15. 15
          16. 16
          17. 17
          18. 18
          19. 19
          20. 20
          21. 21
          22. 22
          23. 23
          24. 24
          25. 25
          26. 26
          27. 27
          28. 28
          29. 29
          30. 30
          31. 31
          32. 32
      2. B Organization of the Federal Statistical System
        1. Overview
          1. Budget
          2. Value
          3. Structure
          4. Figure B-1 Organization of principal federal statistical agencies and programs, by department, 2013.
          5. Figure B-2 Members of the Interagency Council on Statistical Policy: Organizational location and relevant congressional appropriations subcommittee based on subcommittee jurisdictions in the 113th Congress.
        2. Office of Management and Budget
        3. Principal Statistical Agencies
          1. Bureau of Economic Analysis
          2. Bureau of Justice Statistics
          3. Bureau of Labor Statistics
          4. Bureau of Transportation Statistics
          5. Census Bureau
          6. Economic Research Service
          7. Energy Information Administration
          8. National Agricultural Statistics Service
          9. National Center for Education Statistics
          10. National Center for Health Statistics
          11. National Center for Science and Engineering Statistics
          12. Office of Research, Evaluation, and Statistics
          13. Statistics of Income Division
        4. Other Statistical Programs
          1. Health and Retirement Study
          2. Medical Expenditure Panel Survey
          3. National Agricultural Workers Survey
          4. National Automotive Sampling System
          5. National Resources Inventory
          6. National Survey on Drug Use and Health
          7. Panel Study of Income Dynamics
        5. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
          5. 5
      3. C Fundamental Principles of Official Statistics of the Statistical Commission of the United Nations
      4. D European Statistics Code of Practice for the National and Community Statistical Authorities
        1. Preamble
          1. The vision of the European Statistical System 1
          2. The mission of the European Statistical System
          3. The European Statistics Code of Practice
        2. Institutional Environment
          1. Principle 1: Professional Independence
          2. Principle 2: Mandate for Data Collection
          3. Principle 3: Adequacy of Resources
          4. Principle 4: Commitment to Quality
          5. Principle 5: Statistical Confidentiality
          6. Principle 6: Impartiality and Objectivity
        3. Statistical Processes
          1. Principle 7: Sound Methodology
          2. Principle 8: Appropriate Statistical Procedures
          3. Principle 9: Non-excessive Burden on Respondents
          4. Principle 10: Cost Effectiveness
        4. Statistical Output
          1. Principle 11: Relevance
          2. Principle 12: Accuracy and Reliability
          3. Principle 13: Timeliness and Punctuality
          4. Principle 14: Coherence and Comparability
          5. Principle 15: Accessibility and Clarity
        5. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
      5. E Selected Federal Statistical Websites, February 2013
        1. Web Sites
        2. Index of Federal Statistical Sites Listed
    12. References and Selected Bibliography
  1. Story
  2. Slides
    1. Slide 1 Why Doesn't EPA Have a Self-Contained Statistical Unit?: A Tribute to Doug Engelbart
    2. Slide 2 A Tribute to Doug Engelbart
    3. Slide 3 Preface
    4. Slide 4 Purpose
    5. Slide 5 Some of My Principles and Practices
    6. Slide 6 A Well-designed Spreadsheet
    7. Slide 7 A New, Innovative Way to Display Water Quality Information
    8. Slide 8 Scotland’s Environment: Homepage
    9. Slide 9 Scotland’s Environment: Trends and Indicators
    10. Slide 10 The Scottish Government Environmental Statistics
    11. Slide 11 “Drag and Drop” Onto a Tool
    12. Slide 12 Creates Statistics and Visualizations in the Public and Private Cloud
    13. Slide 13 Get a Data Story Idea
    14. Slide 14 Add Your Personal Experience
    15. Slide 15 Add Your Personal Opinion
    16. Slide 16 Bring In More Ideas and Data Sets
    17. Slide 17 EPA Scientists Used These Data Sets
    18. Slide 18 EPA Provides These Open Data Sets
    19. Slide 19 EPA Just Received Recognition For Their GeoPlatform
    20. Slide 20 US EPA Environmental Dataset Gateway Download
    21. Slide 21 EDG Well-Designed Spreadsheet
    22. Slide 22 EDG Visualizations: Bar Charts
    23. Slide 23 EDG Visualizations: Bar Charts
    24. Slide 24 Build a Knowledge Base in MindTouch
    25. Slide 25 Build a Knowledge Base Index in Spreadsheet
    26. Slide 26 Some Conclusions and Recommendations
  3. Spotfire
  4. Spotfire
  5. Research Notes
  6. U.S. Environmental Protection Agency Daily Digest Bulletin
  7. EPA Scientists Presented Open Science at White House
  8. Spotfire – a new, innovative way to display water quality information
    1. Search for Spotfire
      1. Leading-edge approach
  9. Principles and Practices for a Federal Statistical Agency Fifth Edition
    1. Highlights
      1. Principle 1: Relevance to Policy Issues
      2. Principle 2: Credibility Among Data Users
      3. Principle 3: Trust Among Data Providers
      4. Principle 4: Independence from Political and Other Undue External Influence
    2. Definition of a Statistical Agency
    3. Why We Need Statistical Agencies
    4. About the Committee on National Statistics
  10. Full Report
    1. Cover Page
    2. The National Academies Press
    3. The National Academy of Sciences
    4. Committee on National Statistics 2012-2013
    5. Dedication
    6. Acknowledgments
    7. Executive Summary
    8. Introduction
    9. Part I: Principles and Practices for a Federal Statistical Agency
      1. Definition of a Federal Statistical Agency
      2. Establishment of a Federal Statistical Agency
      3. Principles for a Federal Statistical Agency
        1. Principle 1: Relevance to Policy Issues
        2. Principle 2: Credibility Among Data Users
        3. Principle 3: Trust Among Data Providers
        4. Principle 4: Independence from Political and Other Undue External Influence
      4. Practices for a Federal Statistical Agency
        1. Practice 1: A Clearly Defined and Well-Accepted Mission
        2. Practice 2: Necessary Authority to Protect Independence
        3. Practice 3: Continual Development of More Useful Data
        4. Practice 4: Openness About Sources and Limitations of the Data Provided
        5. Practice 5: Wide Dissemination of Data
        6. Practice 6: Cooperation with Data Users
        7. Practice 7: Respect for the Privacy and Autonomy of Data Providers
        8. Practice 8: Protection of the Confidentiality of Data Providers’ Information
        9. Practice 9: Commitment to Quality and Professional Standards of Practice
        10. Practice 10: An Active Research Program
        11. Practice 11: Professional Advancement of Staff
        12. Practice 12: A Strong Internal and External Evaluation Program
        13. Practice 13: Coordination and Collaboration with Other Statistical Agencies
      5. Footnotes
        1. 1
        2. 2
        3. 3
        4. 4
        5. 5
    10. Part II: Commentary
      1. Definition of a Federal Statistical Agency
      2. Establishment of a Federal Statistical Agency
      3. Principles for a Federal Statistical Agency
        1. Principle 1: A federal statistical agency must be in a position to provide objective, accurate, and timely information that is relevant to issues of public policy
        2. Principle 2: A federal statistical agency must have credibility with those who use its data and information
        3. Principle 3: A federal statistical agency must have the trust of those whose information it obtains
        4. Principle 4: A federal statistical agency must be independent from political and other undue external influence in developing, producing, and disseminating statistics
      4. Practices for a Federal Statistical Agency
        1. Practice 1: A Clearly Defined and Well-Accepted Mission
        2. Practice 2: Necessary Authority to Protect Independence
        3. Practice 3: Continual Development of More Useful Data
          1. Introduction
          2. Multiple Data Sources
          3. The Role of Surveys
          4. Roles for Administrative Records
          5. Roles for Nontraditional Data Sources
          6. Integration and Synchronization of Data Across Agencies
          7. Longitudinal Data
          8. Operational Methods
        4. Practice 4: Openness About Sources and Limitations of the Data Provided
        5. Practice 5: Wide Dissemination of Data
        6. Practice 6: Cooperation with Data Users
        7. Practice 7: Respect for the Privacy and Autonomy of Data Providers
          1. Introduction
          2. Respecting Privacy
          3. Protecting and Respecting the Autonomy of Human Research Participants
        8. Practice 8: Protection of the Confidentiality of Data Providers’ Information
        9. Practice 9: Commitment to Quality and Professional Standards of Practice
        10. Practice 10: An Active Research Program
          1. Substantive Research and Analysis
          2. Research on Methodology and Operations
          3. Research on Policy Uses
        11. Practice 11: Professional Advancement of Staff
        12. Practice 12: A Strong Internal and External Evaluation Program
          1. Introduction
          2. Evaluating Quality
          3. Evaluating Relevance
          4. Types of Reviews
        13. Practice 13: Coordination and Collaboration with Other Statistical Agencies
          1. Introduction
          2. Role of OMB
          3. Forms of Interagency Collaboration
      5. Footnotes
        1. 1
        2. 2
        3. 3
        4. 4
        5. 5
        6. 6
        7. 7
        8. 8
        9. 9
        10. 10
        11. 11
        12. 12
        13. 13
        14. 14
        15. 15
        16. 16
        17. 17
        18. 18
        19. 19
        20. 20
        21. 21
        22. 22
        23. 23
        24. 24
        25. 25
        26. 26
        27. 27
        28. 28
        29. 29
        30. 30
        31. 31
        32. 32
        33. 33
        34. 34
        35. 35
        36. 36
        37. 37
        38. 38
        39. 39
        40. 40
        41. 41
        42. 42
        43. 43
        44. 44
        45. 45
        46. 46
    11. Appendixes
      1. A Legislation and Regulations That Govern Federal Statistics
        1. The 1980 Paperwork Reduction Act and Associated Guidance
          1. Background
          2. Paperwork Reduction
          3. Survey Clearance Process
        2. OMB Statistical Policy Directives
          1. Standards and Guidelines for Statistical Surveys
          2. Compilation, Release, and Evaluation of Principal Economic Indicators
          3. Release and Dissemination of Statistical Products
          4. North American Industry Classification System/Standard Industrial Classification
          5. Standard Occupational Classification
          6. Metropolitan Area Classification
          7. Definition of Poverty
          8. Data on Race and Ethnicity
        3. The 1997 Order Providing For The Confidentialy of Statistical Information
        4. The 2002 Confidential Information Protection and Statistical Efficiency Act and Associated Guidance
          1. Subtitle A—Protecting Confidentiality
        5. Subtitle B—Sharing Data
        6. OMB CIPSEA Guidance
        7. The 2002 E-Government Act, Section 208
        8. The 2002 Federal Information Security Management Act
        9. The 2000 Information Quality Act and Associated Guidelines
        10. 2004 OMB Peer Review Guidance
        11. 2010 Government Performance and Results Modernization Act
        12. 2009–2010 Guidance on Scientific Integrity
        13. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
          5. 5
          6. 6
          7. 7
          8. 8
          9. 9
          10. 10
          11. 11
          12. 12
          13. 13
          14. 14
          15. 15
          16. 16
          17. 17
          18. 18
          19. 19
          20. 20
          21. 21
          22. 22
          23. 23
          24. 24
          25. 25
          26. 26
          27. 27
          28. 28
          29. 29
          30. 30
          31. 31
          32. 32
      2. B Organization of the Federal Statistical System
        1. Overview
          1. Budget
          2. Value
          3. Structure
          4. Figure B-1 Organization of principal federal statistical agencies and programs, by department, 2013.
          5. Figure B-2 Members of the Interagency Council on Statistical Policy: Organizational location and relevant congressional appropriations subcommittee based on subcommittee jurisdictions in the 113th Congress.
        2. Office of Management and Budget
        3. Principal Statistical Agencies
          1. Bureau of Economic Analysis
          2. Bureau of Justice Statistics
          3. Bureau of Labor Statistics
          4. Bureau of Transportation Statistics
          5. Census Bureau
          6. Economic Research Service
          7. Energy Information Administration
          8. National Agricultural Statistics Service
          9. National Center for Education Statistics
          10. National Center for Health Statistics
          11. National Center for Science and Engineering Statistics
          12. Office of Research, Evaluation, and Statistics
          13. Statistics of Income Division
        4. Other Statistical Programs
          1. Health and Retirement Study
          2. Medical Expenditure Panel Survey
          3. National Agricultural Workers Survey
          4. National Automotive Sampling System
          5. National Resources Inventory
          6. National Survey on Drug Use and Health
          7. Panel Study of Income Dynamics
        5. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
          5. 5
      3. C Fundamental Principles of Official Statistics of the Statistical Commission of the United Nations
      4. D European Statistics Code of Practice for the National and Community Statistical Authorities
        1. Preamble
          1. The vision of the European Statistical System 1
          2. The mission of the European Statistical System
          3. The European Statistics Code of Practice
        2. Institutional Environment
          1. Principle 1: Professional Independence
          2. Principle 2: Mandate for Data Collection
          3. Principle 3: Adequacy of Resources
          4. Principle 4: Commitment to Quality
          5. Principle 5: Statistical Confidentiality
          6. Principle 6: Impartiality and Objectivity
        3. Statistical Processes
          1. Principle 7: Sound Methodology
          2. Principle 8: Appropriate Statistical Procedures
          3. Principle 9: Non-excessive Burden on Respondents
          4. Principle 10: Cost Effectiveness
        4. Statistical Output
          1. Principle 11: Relevance
          2. Principle 12: Accuracy and Reliability
          3. Principle 13: Timeliness and Punctuality
          4. Principle 14: Coherence and Comparability
          5. Principle 15: Accessibility and Clarity
        5. Footnotes
          1. 1
          2. 2
          3. 3
          4. 4
      5. E Selected Federal Statistical Websites, February 2013
        1. Web Sites
        2. Index of Federal Statistical Sites Listed
    12. References and Selected Bibliography

Story

Why Doesn't EPA Have a Self-Contained Statistical Unit?

Slides

5thReportCover.png

 

In the 5th Principles and Practices For A Federal Statistical Agency, under Principal Statistical Agencies it says: This section provides information—primarily from agency websites (see Appendix E) and OMB publications—on 13 of the 14 members of the ICSP, excluding only the Office of Environmental Information in the Environmental Protection Agency, which is not a self-contained statistical unit. The information provided for the 13 agencies includes origins, authorizing legislation or other authority, status of head (presidential appointee, career senior executive service official), budget and full-time permanent staffing levels in 2012 (see U.S. Office of Management and Budget, 2012b:Table 1 and App. B), and principal programs. The agencies are discussed in alphabetical order.

I worked in EPA's Environmental Statistics Division and compiled a knowledgebase of their activities. Earlier I worked in the EPA Center for Environmental Statistics to try to become a Bureau of Environmental Statistics and produced an EPA Ontology State of the Environment Report.

While working in the EPA Center for Environmental Statistics, I helped produce the EPA Guide to Selected National Environment Statistics in the US Government and the Guide to Global Environmental Statistics. I received the EPA Bronze Medal for the former in 1993.

Since Congress never allowed EPA to have a bureau of Environmental Statistics and since the Office of Environmental Information in the Environmental Protection Agency would never allow the Environmental Statistics Division to become a self-contained statistical unit, I decide to spend the rest of my EPA career being a data scientist and applying my statistics and data architecture expertise to analyzing and visualizing as many EPA and government data sets as possible using the premier tool based on S-Plus and Spotfire called Spotfire by TIBCO.

This turned out to be very visionary because now the statistical agencies (e.g. Census) and OMB are actively looking to apply state-of-the-art tool to provide a lot of federal data to analysts and empowering them to use a visualization tool to derive new understandings. See:

http://semanticommunity.info/Data_Science/Free_Data_Visualization_and_Analysis_Tools

I wondered why this is so with EPA not having a self-contained statistical unit, so I asked Dr. Constance F. Citro, Director, Committee on National Statistics, and one of the principal authors of the 5th Principles and Practices For A Federal Statistical Agency, and she replied:

I don’t know the full history, but there were efforts (about 20-25 years ago, I believe) to create a statistical agency (Bureau of Environmental Statistics) within EPA, but these efforts never got anywhere.  I don’t know if the obstacles were in the executive branch, Congress, or both.  One concern, I understand, was that EPA as a regulatory agency was not the appropriate place to house a statistical agency, and there was some initiative to create a cabinet department including EPA and other relevant agencies, including a new statistical agency.  But again, these efforts came to naught.  The person who headed the OMB statistical policy division in the late 1980s and early 1990s decided to include EPA as a member of the ICSP, and his successor (Katherine Wallman) didn’t change that, but, otherwise, there is no unit in EPA that is recognized as a “principal statistical agency.”

MORE TO BE ADDED

Also see: http://semanticommunity.info/Data_Science/Free_Data_Visualization_and_Analysis_Tools/Tableau#Story

Slides

Slides

Slide 1 Why Doesn't EPA Have a Self-Contained Statistical Unit?: A Tribute to Doug Engelbart

http://semanticommunity.info/
http://breakinggov.com/author/brand-niemann/

BrandNiemann07082013Slide1.GIF

Slide 2 A Tribute to Doug Engelbart

http://www.dougengelbart.org/

BrandNiemann07082013Slide2.GIF

Slide 3 Preface

BrandNiemann07082013Slide3.GIF

Slide 4 Purpose

BrandNiemann07082013Slide4.GIF

Slide 5 Some of My Principles and Practices

BrandNiemann07082013Slide5.GIF

Slide 6 A Well-designed Spreadsheet

http://www.scotland.gov.uk/Resource/0040/00400791.xls

BrandNiemann07082013Slide6.GIF

Slide 7 A New, Innovative Way to Display Water Quality Information

http://semanticommunity.info/CNSTAT/Principles_and_Practices_for_a_Federal_Statistical_Agency#Spotfire_.E2.80.93_a_new.2C_innovative_way_to_display_water_quality_information

BrandNiemann07082013Slide7.GIF

Slide 8 Scotland’s Environment: Homepage

http://www.environment.scotland.gov.uk/default.aspx

BrandNiemann07082013Slide8.GIF

Slide 9 Scotland’s Environment: Trends and Indicators

http://www.environment.scotland.gov.uk/trends_and_indicators.aspx

BrandNiemann07082013Slide9.GIF

Slide 10 The Scottish Government Environmental Statistics

http://www.scotland.gov.uk/Topics/Statistics/Browse/Environment

 

BrandNiemann07082013Slide10.GIF

Slide 11 “Drag and Drop” Onto a Tool

BrandNiemann07082013Slide11.GIF

 

Slide 12 Creates Statistics and Visualizations in the Public and Private Cloud

BrandNiemann07082013Slide12.GIF

Slide 13 Get a Data Story Idea

http://semanticommunity.info/CNSTAT/Principles_and_Practices_for_a_Federal_Statistical_Agency#Principal_Statistical_Agencies

http://semanticommunity.info/CNSTAT/Principles_and_Practices_for_a_Federal_Statistical_Agency#E_Selected_Federal_Statistical_Websites.2C_February_2013

BrandNiemann07082013Slide13.GIF

Slide 17 EPA Scientists Used These Data Sets

http://epa.gov/comptox/

BrandNiemann07082013Slide17.GIF

Slide 18 EPA Provides These Open Data Sets

http://www2.epa.gov/open

BrandNiemann07082013Slide18.GIF

 

Slide 20 US EPA Environmental Dataset Gateway Download

https://edg.epa.gov/data/

BrandNiemann07082013Slide20.GIF

Slide 22 EDG Visualizations: Bar Charts

Web Player

BrandNiemann07082013Slide22.GIF

Slide 23 EDG Visualizations: Bar Charts

Web Player

BrandNiemann07082013Slide23.GIF

Slide 25 Build a Knowledge Base Index in Spreadsheet

http://semanticommunity.info/@api/deki/files/24897/EPAOpenGovernmentData.xlsx

BrandNiemann07082013Slide25.GIF

Slide 26 Some Conclusions and Recommendations

BrandNiemann07082013Slide26.GIF

Spotfire

For Internet Explorer Users and Those Wanting Full Screen Display Use: Web Player Get Spotfire for iPad App

Error: Embedded data could not be displayed. Use Google Chrome

Spotfire

For Internet Explorer Users and Those Wanting Full Screen Display Use: Web Player Get Spotfire for iPad App

Error: Embedded data could not be displayed. Use Google Chrome

Research Notes

Open Initiative Homepage
http://www2.epa.gov/open
What Is This?

Federal Facilities Search Results
http://www.epa.gov/fedfac/ffsep/index.htm
http://cumulis.epa.gov/supercpad/cur...753d5b67b1ff56
http://cumulis.epa.gov/supercpad/cur...qryrslt_ff.cfm

Data Inventory and Activities
http://www2.epa.gov/open/data-invent...and-activities
Contact the Open Government Web editor to ask a question, provide feedback, or report a problem.
crawford.tim@epa.gov

EPA's data.gov data set inventory (XLSX)(21 pp, 1 MB)
http://www2.epa.gov/sites/production...y2011q1_0.xlsx
Data Finder
http://www.epa.gov/data/
Is There Anything New Here Since I Did This?

EPA List of Substances
http://cfpub.epa.gov/ncea/iris/index...ist_type=alpha
Download Excel
Could Build LOD Tables of Rest

EPA's Environmental Dataset Gateway (EDG)
https://edg.epa.gov/metadata/catalog/main/home.page
Download EPA's Geospatial Data
https://edg.epa.gov/data/
What Are All of These?

CNSTAT’s 122nd meeting will be held October 24-25, 2013, in the NAS main building at 2101 Constitution Ave., NW.  This meeting will feature a public seminar and luncheon with statistical agency heads on the 25th.

Stephen Fienberg, the Maurice Falk university professor of statistics and social science at Carnegie Mellon University, will deliver the 23rd Morris Hansen lecture, Thursday afternoon, October 3, 2013, at the USDA Auditorium, Washington, DC. His topic is “Envisioning the 2030 Census,”

Please see the C-SPAN web site for announcements of upcoming C-SPAN Washington Journal sessions on “America by the Numbers,” which feature interviews with federal statistical agency heads and senior staff.  The programs highlight trends and allow the public to call in or email their views.  Links to videos of previous C-SPAN ABTN programs is available at the same site. 

http://www.c-spanvideo.org/topic/ABTN

CNSTAT Public Seminars and Other Symposia
http://sites.nationalacademies.org/D...3#.UdYjxDsY7Sh

U.S. Environmental Protection Agency Daily Digest Bulletin

Source: June 20, 2013 Email

The people have spoken: what green apps people want
06/20/2013

Ever wonder what environmental apps people want? We’ve done some market research so you understand where the demand is. It’s getting easier than ever to convert ideas into green apps.

We asked people to suggest ideas for green apps, then to “like” their favorites (see the favorites at http://www.epa.gov/mygreenapps/ideas.html and click on the button for “Show Popular Ideas”). For the top four ideas, we found related apps that already exist so you don’t have to reinvent the wheel (http://blog.epa.gov/data/2013/06/green-app-ideas/). Finally, we found EPA data you can use to convert these ideas into apps (http://www.epa.gov/mygreenapps/Ideas%20for%20Apps%201-11-13.xlsx).

So we’re challenging you to make some of these apps! You know what people want, what apps are already out there, and what data can be used to make the apps. If you need help, just contact us at greenapps@epa.gov.

Here are some events and opportunities you may want to know about:

My Air, My Health Challenge Winners are Announced: The objective of the challenge by the U.S. Department of Health and Human Services and the Environmental Protection Agency was to develop a low cost, real-time personal digital device that measures health effects of harmful air pollution. The team of David Kuller, Gabrielle Dockterman, and Dot Kelly won the $100,000 grand prize! See details at http://challenge.gov/HHS/372-my-air-my-health-challenge.

NIEHS-NCATS-UNC DREAM Toxicogenetics Challenge: The objective of this challenge is to obtain a greater understanding about how a person's individual genetics can influence cytotoxic response to exposure to widely used chemicals. It is being led and organized by scientists from Sage Bionetworks, DREAM, the University of North Carolina, the National Institute of Environmental Health Sciences and the National Center for Advancing Translational Sciences. The challenge will close on September 15, 2013, and the top-scoring team(s) will be announced at the November, 2013 DREAM Conference (www.iscb.org/recomb-regsysgen2013) in Toronto, Canada. http://online.wsj.com/article/PR-CO-20130611-901905.html

International Space Apps Challenge: Results: On April 20-21, over 9,000 people around the world participated in the known galaxy's largest mass collaboration ever - the International Space Apps Challenge. You can see the incredible work they created and check if they need help with coding or marketing. http://spaceappschallenge.org/

National Day of Civic Hacking: On June 1-2, over 11,000 individuals engaged in the National Day of Civic Hacking in 95 events, 38 states, 83 cities, and 2 territories. You can see the list of 26 challenges from federal, state, and local governments at http://hackforchange.org/challenges and see if you can coordinate with teams as they flesh out their ideas. http://hackforchange.org/

REST API Service is Available for Safe Drinking Water Data: The newly expanded REST Application Programming Interface (API) services for Envirofacts Safe Drinking Water Information System (SDWIS) data were recently made available. These new SDWIS services represent an expanding catalog of special Envirofacts data services that build upon the core Envirofacts REST API service. Get familiar with the Envirofacts API here: http://www.epa.gov/enviro/facts/services.html. See the REST API for SDWIS here: http://www.epa.gov/enviro/facts/otherdata.html.

EPA Scientists Presented Open Science at White House

Source: http://blog.epa.gov/science/2013/06/epa-scientists-presented-open-science-at-white-house/

My Note: There is Excel data here and they are looking at dashboards: http://epa.gov/research/priorities/d...fact-sheet.pdf

2013 JUNE 26

By Tina Bahadori

From weather forecasts, air quality advisories, and portable GPS navigation devices, to waterfowl migration, and the mapping of the human genome, the use of government and government-supported science and data have vastly improved our lives. They have also sparked countless new private businesses and industries leading to economic growth and opportunity for innovators and entrepreneurs in every region of the country.

Recognizing the power and potential of such Open Science, on June 20, 2013 the White House invited four EPA scientists—Drs. Richard Judson, Keith Houck, Matt Martin, and Ann Richard—to present research posters describing their efforts to provide public access to massive amounts of data from chemical safety studies. The scientists presented their posters after the White House’s “Champions of Change” award ceremony. The award ceremony recognized 13 Champions of Change for their efforts to provide the public access to innovative science.

In addition to the 13 Champions of Change, the White House selected 12 scientists (including the EPA researchers) to present posters describing their vision and commitment to Open Science.

EPA scientists at the White House poster session.

EPA scientists Ann Richard and Matt Martin at the White House poster session.

The select group of 25 was chosen from hundreds of nominations submitted to the White House’s request for innovative Open Science leaders. The White House event highlighted outstanding individuals, organizations, and research projects promoting and using open scientific data and publications to accelerate progress.

To exemplify Open Science work, the four EPA scientists presented how they are using advances in computational toxicology to provide open and accessible chemical safety data to help better protect human health and the environment. Each of the EPA scientists are working to harness the power of computer science and innovative new chemical safety assessment methods and tools to provide open, transparent public access to chemical information. For example:

  • Dr. Matt Martin leads a team of Agency scientists and partners who developed theToxicity Reference database (ToxRefDB). ToxRefDB contains 30 years and $2 billion worth of pesticide registration studies. The database allows scientists and others to search and download thousands of toxicity testing results on hundreds of chemicals that were previously only available on paper or microfiche.
  • Dr. Ann Richard is the leader behind another open, accessible database, theDistributed Structure-Searchable Toxicity Database (DSSTox). DSSTox provides open-access to information on the physical and structural properties of chemicals and links this information to toxicity potential. This is key information for assessing the potential risk of chemicals to human health and the environment.
  • Dr. Richard Judson leads a team of scientists who developed the Aggregated Computational Toxicology online Resource (ACToR). ACToR is EPA’s online warehouse of all publicly available chemical data aggregated from more than 1,000 public sources on more than half a million chemicals. ACToR can be used to query a specific chemical and find available public hazard, exposure, and risk assessment data as well as previously unpublished studies related to cancer, reproductive, and developmental toxicity.
  • Dr. Keith Houck is the driving force behind EPA’s Toxicity Forecaster (ToxCast), a research program advancing the use of automated, rapid chemical tests (called “high-throughput screening assays”) to screen thousands of chemicals in more than 650 assays for toxicity potential. This includes the development of the ToxCast Database (ToxCastDB) which provides publicly accessible, searchable, and downloadable access to all the screening data generated by ToxCast.

These four scientists have led the effort to democratize access to knowledge and information and level the playing field for all those involved and interested in protecting public health and the environment. By doing so, they exemplify the spirit of Open Science celebrated by President Obama’s Champions of Change program.

About the Author: Tina Bahadori, Sc.D. is the National Program Director for EPA’s Chemical Safety for Sustainability research program.  Learn more about her on EPA’s Science Matters: Meet our Scientists web page.

Editor's Note: The opinions expressed in Greenversations are those of the author. They do not reflect EPA policy, endorsement, or action, and EPA does not verify the accuracy or science of the contents of the blog.

Spotfire – a new, innovative way to display water quality information

Source: http://www.sepaview.com/2013/06/spotfire-a-new-innovative-way-to-display-water-quality-information

My Note: 

http://www.environment.scotland.gov.uk/

http://www.environment.scotland.gov.uk/trends_and_indicators.aspx

http://www.scotland.gov.uk/Topics/St...se/Environment

Key Scottish Environment Statistics 2012 – includes a set of Excel tables with data and charts for all years.

http://www.scotland.gov.uk/Resource/0040/00400791.xls (Excel)

A new, web-based application is transforming the way Scotland’s public bodies display their information about Scotland’s environment. 

SE-Web-Water-Application0 

SEPA Director David Pirie said: “Scotland’s Environment Web and Spotfire present a tremendous opportunity to make a real difference to the environment. It’s about bringing people together and working collaboratively. Displaying information from a number of sources, all on one map, is something we wouldn’t have been able to do in the past. Spotfire allows us to do that seamlessly, making the raw data available and allowing the community to choose how they want to use it.”

Scotland’s Environment Web project is leading the way in developing digital technologies to help the public discover more about Scotland’s environment. Scotland’s Environment Web, a partnership of public sector organisations in Scotland, is using Spotfire, a visual data analysis tool, to overlay multiple sets of environmental data that different organisations hold, and present that data in an accessible and useable form. The first Spotfire app to be released is on water quality, and is designed for users with varying levels of knowledge; from members of the general public, to education, local authorities, planners and professionals. Several more applications are planned over the coming months.

The water application will show the overall quality of all surface water in Scotland, and a summary of how many and which water bodies have improved or deteriorated over time. Users will find more detail on each individual stretch of water monitored. A map of Scotland will show what condition our waters are in. This can then be filtered, allowing the user to interrogate and analyse the data, which can be exported for use in reports and presentations, either in map, graph or table form.

Spotfire provides users with a platform which will make complex environmental data much easier to access and understand, allowing them to focus the information to their area or level of interest. Importantly for the user, it is all web-based and can be used on the Scotland’s Environment website without the need to download any additional tools.

There was extremely positive feedback on the “spotfire” tool and it helped delegates to see for the first time how their environmental data could be integrated with others and used to present some very powerful and understandable environmental information to a wide audience...It was also noted that the web site, and in particular the data presentation and analysis functionality, as demonstrated by “spotfire” had significant potential to be an extremely powerful education tool that would compliment a range of national curriculum objectives as well as increasing the awareness of young people on...

 

Scotlands_Environment_Web_Leading_Edge_Approach.pdf

These include ‘Spotfire’ and a new interactive map search tool...Through ‘Spotfire’, SEWeb provides users with a platform which will allow them to analyse and view multiple levels of data on the environment, allowing them to filter the information to their area or level of interest...As well as ‘Spotfire’ and the LIS, SEWeb is also developing tools and resources to help people measure and monitor their own environment...

Leading-edge approach

Source: http://www.environment.scotland.gov....e_Approach.pdf (PDF)

Hazel McFarlane is Corporate Communications Officer at the Scottish

Environment Protection Agency
By Hazel McFarlane

Putting Scotland at the forefront of sharing information, prioritising problems and involving citizens

Environment, Spring 2013 // connect // 41

Visit http://www.environment.scotland.gov.uk

 
‘It’s an opportunity to get understanding out of raw data and find the issues that matter the most’
 
In this digital age, Scotland’s Environment Web (SEWeb) is leading the way in developing digital technologies which will help the public to find out more about Scotland’s environment.
 
People expect more from information provision than simply reproducing books and publications on screen. They want interest and interactivity – to be able to visualise information spatially and in other ways to find out what is important. This is truer than ever when it comes to Scotland’s environment, and the scientific data and information which is collected, researched, analysed and used.
 
SEWeb’s innovative approach, providing access to data and information in a digital format, brings together information on Scotland’s environment so that it is easily available and in a useable form. Jointly funded by the European Union’s LIFE+ programme and the Scottish Environment Protection Agency (SEPA) and with input and support from a number of partner organisations across Scotland, the SEWeb project is committed to developing a modern and dynamic approach to presenting information. It aims to move away from static reports, to a website where unprecedented access to the most up-to-date environmental information is available. The leading-edge approach that SEWeb is undertaking will be of benefit to a range of audiences, including policymakers, the education sector, young people and the wider public.
 
David Pirie, Director of Science and Strategy at SEPA, says: “The environment is all interconnected. What happens in one area has a consequence for another and therefore the science and research we do needs to be multidisciplinary and joined up. We needed a way for partners to interact and bring data together and then to make that data accessible and communicate it. SEWeb achieves all of that.”
 
Ed Mackey, Unit Manager for Knowledge and Information Management at SEWeb partner, Scottish Natural Heritage (SNH), agrees and says the partnership aspect of SEWeb is fundamental: “More than ever before we need to work together to use our expertise, knowledge and other resources to best effect. SEWeb adds value through a shared understanding of environmental opportunities and constraints founded on common access to a comprehensive and trusted evidence base.”
 
Developing technology to aid the collection, analysis and interpretation of environmental data is at the forefront of SEWeb. The web is a more dynamic place than a few years ago. By providing information in a vibrant, interactive way the information becomes more engaging and more likely to change behaviour as a result. There are a number of tools which SEWeb and its partners are developing in their mission to make environmental data more accessible.
 
These include ‘Spotfire’ and a new interactive map search tool. It is a visual data analysis tool which can read data from any source, and multiple sources, all at once. It is web based and can be used without the need to download additional tools. It will allow users to interactively interrogate data using various forms of presentation including maps, tables and graphs. All data can be exported for use in reports and presentations.
 
ScotlandsEnvironmentFigure1.png

 

Through ‘Spotfire’, SEWeb provides users with a platform which will allow them to analyse and view multiple levels of data on the environment, allowing them to filter the information to their area or level of interest. Each application will vary in its complexity and therefore the audiences it is most suitable for. The first to be released will be a water quality application which is designed for users with varying levels of knowledge, from members of the general public, to the education sector, local authorities, planners and professionals. The second application on climate change projections, however, will use complex data which will be aimed towards industry professionals.
 
ScotlandsEnvironmentFigure2.png
 
Similarly, a new interactive map search tool called the Land Information Search (LIS) will be hosted on SEWeb. This tool will give information about any land designations or features in any chosen area, for example, special protection areas or woodland grant schemes. The LIS will be an updated version of the current tool which was created by the Forestry Commission and its development will be led by SNH. Ed Mackey from SNH says: “LIS recognises that information is a dynamic property and that the constituency of interests in it is diverse. By creating a ‘level playing field’ of readily accessible environmental information it saves time and money, for example, in development planning or environmental assessment.” He adds: “It makes efficient use of information, for differing and interrelated purposes, in ways that will benefit the operation of government, business, education and research, and the interested citizen.” The application can be used to identify any designations or sensitivities which may affect a potential project.
 
So how does this advanced approach benefit its key audiences? For the education sector, SEWeb is a useful tool in aiding teaching and research methods for environmental subjects. John Isaacs, lecturer at Abertay University, has been involved in the project and can see the benefits from an educational perspective. “Using this type of technology will allow teachers and students to find the information they are looking for much more easily. “Teachers will be able to find appropriate course material without having to trawl through in-depth reports and similarly students will be able to find the information they require for reports and assignments which is relevant and unbiased.” This highlights one of the key benefits of the way the information is presented on SEWeb; it is neutral so that the user can fully interpret the meaning for themselves.
 
Young people have been identified as being the most receptive to web-based information and an important audience. The project has been working with pupils from Stirling High School to gain insight and ideas as to how the website could help and support young people to get more involved in Scotland’s environment. The young people went on to produce prototype resources including a game, mobile phone app, youth pages on the website and social networking.
 
Gregor MacMillan, one of the students who took part, says: “We decided that a game would be the best way to get other young people interested because we all play games online and, once they started playing it, protecting the environment for fun, they’d have that same stuff in their head for real too. I’m really proud of what we’ve done, I never thought we’d get the chance to actually make the prototypes for real, so that’s been fantastic.” SEWeb intends to launch some of these prototypes in 2014.
 
Helping the public to better understand, protect and improve Scotland’s environment is key to SEWeb. As well as ‘Spotfire’ and the LIS, SEWeb is also developing tools and resources to help people measure and monitor their own environment. Citizen science, or public monitoring, is about public involvement in data collection and analysis. The significant increase in personal access to smart phones and tablet computers extends the scope and uptake of citizen science data gathering.
 
SEWeb will promote public monitoring in Scotland and encourage members of the public and the education sector to get involved in this type of data gathering. It will provide a toolkit of resources for new and developing projects, a searchable register of opportunities, and demonstrate how the data which is collected can be used to develop an understanding of the environment. A newly developed public engagement section will be launched on SEWeb in summer 2013.
 
SEWeb provides an authoritative and up to date account of the state of Scotland’s environment. It will continue to broaden the range of data, the way it is presented and increase accessibility. The web is becoming more graphics based, more intuitive and is increasingly being built around applications that allow people to draw on raw data and display it in the way they find most beneficial and this recent trend is likely to continue.
 
SEWeb will put Scotland at the European forefront of sharing environmental information, prioritising problems and involving citizens in assessing and improving their own environment, with a number of exciting developments and changes coming to the website over the coming year. “I think it’s a fantastic public resource,” says David Pirie. “It’s a tremendous opportunity to make a real difference to the environment. It’s an opportunity to allow us to engage with people, get the knowledge and understanding out of all our raw data and find out about the issues that matter the most.” 

Principles and Practices for a Federal Statistical Agency Fifth Edition

Principles and Practices for a Federal Statistical Agency, Fifth Edition, was publicly released on May 8, 2013.  The Committee on National Statistics first issued “P&P” in 1992. Beginning in early 2001, with the second edition, it committed to updating P&P every four years to coincide with a new presidential administration or second term, releasing updated editions in 2001, 2005, 2009, and now 2013.   This slim document underscores for the executive and legislative branches of government the important public good provided by strong federal statistical agencies. The full text is available in print and as a PDF; a 4-page leaflet of “Highlights” is also available.

http://download.nap.edu/cart/downloa...ecord_id=18318

Principles and Practices for a Federal Statistical Agency:
Fifth Edition (2013)

Authors
Constance F. Citro and Miron L. Straf, Editors; Committee on National Statistics; Division on Behavioral and Social Sciences and Education; National Research Council

Description
Publicly available statistics from government agencies that are credible, relevant, accurate, and timely are essential for policy makers, individuals, households, businesses, academic institutions, and other organizations to make informed decisions. Even more, the effective operation of a democratic system of government depends on the unhindered flow of statistical information to its citizens.

In the United States, federal statistical agencies in cabinet departments and independent agencies are the governmental units whose principal function is to compile, analyze, and disseminate information for such statistical purposes as describing population characteristics and trends, planning and monitoring programs, and conducting research and evaluation. The work of these agencies is coordinated by the U.S. Office of Management and Budget. Statistical agencies may acquire information not only from surveys or censuses of people and organizations, but also from such sources as government administrative records, private-sector datasets, and Internet sources that are judged of suitable quality and relevance for statistical use. They may conduct analyses, but they do not advocate policies or take partisan positions. Statistical purposes for which they provide information relate to descriptions of groups and exclude any interest in or identification of an individual person, institution, or economic unit.

Four principles are fundamental for a federal statistical agency: relevance to policy issues, credibility among data users, trust among data providers, and independence from political and other undue external influence. Principles and Practices for a Federal Statistical Agency: Fifth Edition explains these four principles in detail.

Highlights

Source: http://download.nap.edu/cart/downloa...ecord_id=18318 (PDF)

Publicly available statistics from government agencies that are credible, relevant, accurate, and timely are essential for policy makers, individuals, households, businesses, academic institutions, and other organizations to make informed decisions. Even more, the effective operation of a democratic system of government depends on the unhindered flow of statistical information to its citizens.

 
In the United States, federal statistical agencies in cabinet departments and independent agencies are the governmental units whose principal function is to compile, analyze, and disseminate information for such statistical purposes as describing population characteristics and trends, planning and monitoring programs, and conducting research and evaluation. The work of these agencies is coordinated by the U.S. Office of Management and Budget.
 
Statistical agencies acquire information from surveys or censuses of people and organizations, but may also derive data from such sources as government administrative records, private–sector datasets, and Internet sources that are judged of suitable quality and relevance for statistical use. They may conduct analyses but do not advocate policies or take partisan positions. Statistical purposes for which they provide information relate to descriptions of groups and exclude any interest in or identification of an individual person, institution, or economic unit.
 
Four principles are fundamental for a federal statistical agency: relevance to policy issues, credibility among data users, trust among data providers, and independence from political and other undue external influence.

Principle 1: Relevance to Policy Issues

A statistical agency must provide information that is relevant to issues of public policy and useful to a broad range of public and private-sector users as well as the general public. To establish priorities for its programs, a statistical agency must not only work closely with the executive branch, Congress, and interested nongovernmental groups, but also engage a broad spectrum of users in the business sector, academia, state and local governments, and elsewhere. Interaction with stakeholders is essential to enable a statistical agency to continually reassess the needs of its users for information.

Principle 2: Credibility Among Data Users

Because few data users are in a position to verify the completeness and accuracy of statistical information, they must rely on an agency’s reputation as a source of accurate and useful statistics that are free from political and other undue external interference. An agency builds credibility with users when it widely disseminates its data on an equal basis to all. Also essential to building credibility is that an agency exhibit openness about its data sources and their limitations, be willing to understand and meet user needs, and provide full documentation of the processes used to produce and disseminate statistical products.

Principle 3: Trust Among Data Providers

Data providers, such as survey respondents and custodians of administrative records, must be able to trust that the information they provide to a statistical agency will be used only for the purposes that the agency has described. Thus, they must be able to trust that information collected for statistical purposes will not be used for another purpose, such as law enforcement or regulation, directed at specific individuals or organizations. Data providers must also be able to trust that a statistical agency will honor its pledge to protect the confidentiality of their responses. A statistical agency further builds trust among data providers by respecting their privacy and autonomy through minimizing the intrusiveness of questions and the time and effort to respond, consistent with the agency’s requirements for information, and through informing them of the expected time required to participate in the data collection, whether the collection is mandatory or voluntary, and the important uses of the information.

Principle 4: Independence from Political and Other Undue External Influence

To be credible and unhindered in its mission, a statistical agency must maintain a widely acknowledged position of independence from undue external influences. It must avoid even the appearance that its collection, analysis, or reporting processes might be manipulated for political purposes or that individually identifiable data collected under a pledge of confidentiality might be turned over for administrative, regulatory, or law enforcement uses. Protection from undue outside influences requires that a statistical agency have authority for professional decisions  in its programs, including authority over the selection and promotion of staff, the processing, secure storage, and maintenance of data, and the timing and content of data releases, accompanying press releases, and documentation. The credibility that comes from independence is essential for users to maintain confidence in the accuracy and objectivity of a statistical agency’s data and for data providers to be willing to cooperate with agency requests.
 
By adhering to the principles and following the practices enumerated in the fifth edition (see next page), a federal statistical agency will be well positioned to provide the relevant, accurate, timely, and credible statistical information that policy makers and the public require.
 
Thirteen practices are critical for the effective, principled operation of a statistical agency:
(1) a clearly defined and well-accepted mission;
(2) necessary authority to protect independence;
(3) continual development of more useful data;
(4) openness about sources and limitations of the data provided;
(5) wide dissemination of data;
(6) cooperation with data users;
(7) respect for the privacy and autonomy of data providers;
(8) protection of the confidentiality of data providers’ information;
(9) commitment to quality and professional standards of practice; (10) an active research program;
(11) professional advancement of staff;
(12) a strong internal and external evaluation program; and
(13) coordination and collaboration with other statistical agencies.

Definition of a Statistical Agency

A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes. Statistical purposes include description, evaluation, analysis, inference, and research; they do not include administration, regulation, or enforcement.

Why We Need Statistical Agencies

Statistics that are publicly available from government agencies—estimates of business activity and productivity, consumer and producer prices, labor force and employment, high school and college completion, birth and death rates, prices of fuels, health and disability status, healthcare expenditures, health insurance coverage, transportation patterns, crime victimization, investment in research and development, air and water quality, crop forecasts, poverty, inequality, and welfare dependency, assets and wealth—are essential for a nation to advance the economic well–being and quality of life of its people and to operate a democratic system of government. Federal statistical agencies are established to be a credible source of relevant, accurate, and timely statistics in one or more subject areas that are available to policy makers in all levels and branches of government and for use by the broader public, including individuals, families, corporations, academic institutions, and other organizations.
 
• • • • •
 
In addition to definitions and commentary on the principles and practices, the fifth edition of Principles and Practices for a Federal Statistical Agency includes appendixes on the laws and regulations that cover the operations of the federal statistical system, the organization of the system, a reproduction of the Fundamental Principles of Official Statistics of the Statistical Commission of the United Nations, a reproduction of the European Statistics Code of Practice for the National and Community Statistical Authorities, and selected federal statistical websites.

About the Committee on National Statistics

CNSLogo.png

 
The Committee on National Statistics (CNSTAT) is a standing committee of the National Academy of Sciences. It was established in 1972 to improve the statistical methods and information on which public policy decisions are based. The Committee carries out studies, workshops, and other activities to foster better measure and fuller understanding of the economy, the environment, public health, crime, education, immigration, poverty, welfare, and other public policy issues. It also evaluates ongoing statistical programs and tracks the statistical policy and coordinating activities of the federal government, serving a unique role at the intersection of statistics and public policy.
 
The Committee is chaired by Lawrence Brown, Professor of Statistics at the University of Pennsylvania, and includes 14 distinguished members with expertise in survey research, statistics, computer science, biostatistics, economics, sociology, demography, and the operation of statistical agencies. They are assisted by a small professional staff. The Committee’s work is supported by a consortium of federal agencies through a National Science Foundation grant.
 
Principles and Practices for a Federal Statistical Agency, Fifth Edition, is available online at http://www.nationalacademies.org/Pri..._and_Practices or from The National Academies Press online at http://www.nap.edu.
 
COMMITTEE ON NATIONAL STATISTICS
500 Fifth Street NW, Washington, DC 20001
TEL (202) 334-3096 • FAX (202) 334-3751

Full Report

Source: http://www.nap.edu/nap-cgi/report.cg...8&type=pdfxsum (PDF)

Cover Page

5thReportCoverPage.png

The National Academies Press

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
 
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
 
This study was supported by Grant No. SES-1024012 between the National Academy of Sciences and the National Science Foundation, which provides funding from a consortium of federal agencies. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
 
Library of Congress Cataloging-in-Publication data are available from the Library of Congress.
 
Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
 
Copyright 2013 by the National Academy of Sciences. All rights reserved. Printed in the United States of America
 
Suggested citation: National Research Council. (2013). Principles and Practices for a Federal Statistical Agency, Fifth Edition. Committee on National Statistics. Constance F. Citro and Miron L. Straf, Editors. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

The National Academy of Sciences

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
 
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
 
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
 
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. 
 

Committee on National Statistics 2012-2013

The Committee on National Statistics (CNSTAT) was established in 1972 at the National Academies to improve the statistical methods and information on which public policy decisions are based. The committee carries out studies, workshops, and other activities to foster better measures and fuller understanding of the economy, the environment, public health, crime education, immigration, poverty, welfare, and other public policy issues. It also evaluates ongoing statistical programs and tracks the statistical policy and coordinating activities of the federal government, serving a unique role at the intersection of statistics and public policy. The committee’s work is supported by a consortium of federal agencies through a National Science Foundation grant.

LAWRENCE D. BROWN (Chair), Department of Statistics, The Wharton School, University of Pennsylvania

JOHN M. ABOWD, School of Industrial and Labor Relations, Cornell University
DAVID CARD, Department of Economics, University of California, Berkeley
ALICIA CARRIQUIRY, Department of Statistics, Iowa State University
CONSTANTINE GATSONIS, Center for Statistical Sciences, Brown University
JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan
MICHAEL HOUT, Survey Research Center, University of California, Berkeley
SALLIE ANN KELLER, Department of Statistics, University of Waterloo, Ontario, Canada
LISA M. LYNCH, The Heller School for Social Policy and Management, Brandeis University
SALLY C. MORTON, Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh
RUTH D. PETERSON, Criminal Justice Research Center, Ohio State University
EDWARD H. SHORTLIFFE, Columbia University and Arizona State University
HAL STERN, Donald Bren School of Information and Computer Sciences, University of California, Irvine
JOHN H. THOMPSON, NORC at the University of Chicago
ROGER TOURANGEAU, Westat, Rockville, MD
 
CONSTANCE F. CITRO, Director

Dedication

This fifth edition of Principles and Practices for a Federal Statistical Agency is dedicated to Margaret E. Martin (1912–2012), the founding director (1972–1978) of the Committee on National Statistics (CNSTAT) and one of the editors of the first edition of Principles and Practices for a Federal Statistical Agency, in 1992. After retiring as CNSTAT director, she continued as a senior research associate and then as a consultant to the committee, contributing to committee projects on the Survey of Income and Program Participation and sharing research data and subsequent editions of Principles and Practices. Her work and her leadership of CNSTAT set high standards that have guided all of those who followed her.
 
Before coming to CNSTAT, Dr. Martin had a distinguished career in the service of the federal statistical system, retiring in 1972 after 30 years with the U.S. Office of Management and Budget (formerly the U.S. Bureau of the Budget). She became well known for evaluating agency proposals for collecting relevant data and coordinating their efforts toward obtaining the best available information while maintaining reasonable limit burdens on respondents and containing expenditures.
 
Dr. Martin was deservedly recognized in many ways by the statistical profession. She was a fellow of the American Statistical Association (ASA), president of the Washington chapter, and the association’s 75th president. She received the first-ever ASA Founders Award. She was also an elected member of the International Statistical Institute, an honorary lifetime member of the board of the Council of Professional Associations on Federal Statistics, and chair of Section U (statistics) of the American Association for the Advancement of Science in 1986.
 
It is with fond memories, deep gratitude, and appreciation for her outstanding accomplishments that the Committee on National Statistics dedicates this volume to the memory of Margaret E. Martin.

Acknowledgments

The Committee on National Statistics (CNSTAT) thanks the many people who contributed their time and expertise to the preparation of this report. We are most appreciative of their cooperation and assistance.
 
In expressing our gratitude to the staff, a special measure of recognition is due to Miron Straf, who originated the concept for this report and who, with the late Margaret E. Martin, to whom this volume is dedicated, coedited the first (1992) edition. Straf has continued as an editor of the subsequent editions, joined by CNSTAT director Constance Citro. This edition, like its predecessors, benefited from the editing of Eugenia Grohman of the Division of Behavioral and Social Sciences and Education. We also thank Daniel Cork and Jacqui Sovde of the committee staff for their editing and graphic assistance. We are indebted to many others who offered valuable comments and suggestions, too numerous to mention.
 
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the National Research Council. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
 
We thank the following individuals for their review of this report: John Gardenier, National Center for Health Statistics, U.S. Department of Health and Human Services (retired); Turkan K. Gardenier, (formerly) U.S. Environmental Protection Agency and U.S. Energy Information Administration; Daniel Kasprzyk, Center for Excellence in Survey Research, NORC at the University of Chicago; David McMillen, Office of Strategy and Communications, U.S. National Archives and Records Administration; Andrew Reamer, Institute of Public Policy, George Washington University; and Katherine R. Smith, Executive Director’s Office, Council of Professional Associations on Federal Statistics.
 
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Robert Bell, Statistics Research Department, AT&T Labs Research. Appointed by the Report Review Committee, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
 
Finally, we recognize the many federal agencies that support the Committee on National Statistics directly and through a grant from the National Science Foundation. Without their support and their commitment to improving the national statistical system, the committee work that is the basis of this report would not have been possible.
 
Lawrence D. Brown, Chair
Committee on National Statistics

Executive Summary

Publicly available statistics from government agencies that are credible, relevant, accurate, and timely are essential for policy makers, individuals, households, businesses, academic institutions, and other organizations to make informed decisions. Even more, the effective operation of a democratic system of government depends on the unhindered flow of statistical information to its citizens.
 
In the United States, federal statistical agencies in cabinet departments and independent agencies are the governmental units whose principal function is to compile, analyze, and disseminate information for such statistical purposes as describing population characteristics and trends, planning and monitoring programs, and conducting research and evaluation. The work of these agencies is coordinated by the U.S. Office of Management and Budget.
 
Statistical agencies may acquire information not only from surveys or censuses of people and organizations, but also from such sources as government administrative records, private-sector datasets, and Internet sources that are judged of suitable quality and relevance for statistical use. They may conduct analyses, but they do not advocate policies or take partisan positions. Statistical purposes for which they provide information relate to descriptions of groups and exclude any interest in or identification of an individual person, institution, or economic unit.
 
Four principles are fundamental for a federal statistical agency: relevance to policy issues, credibility among data users, trust among data providers, and independence from political and other undue external influence.
 
Principle 1: Relevance to Policy Issues—A statistical agency must provide information that is relevant to issues of public policy and useful to a broad range of public- and private-sector users as well as the general public. To establish priorities for its programs, a statistical agency must not only work closely with the executive branch, Congress, and interested nongovernmental groups, but also engage a broad spectrum of users in the business sector, academia, state and local governments, and elsewhere. Interaction with stakeholders is essential to enable a statistical agency to continually reassess the needs of its users for information.
 
Principle 2: Credibility Among Data Users—Because few data users are in a position to verify the completeness and accuracy of statistical information, they must rely on an agency’s reputation as a source of accurate and useful statistics that are free from political and other undue external influence. An agency builds credibility with users when it widely disseminates its data on an equal basis to all. Also essential to building credibility is that an agency exhibit openness about its data sources and their limitations, be willing to understand and meet user needs, and provide full documentation of the processes used to produce and disseminate statistical products.
 
Principle 3: Trust Among Data Providers—Data providers, such as survey respondents and custodians of administrative records, must be able to trust that the information they provide to a statistical agency will be used only for the purposes that the agency has described. Thus, they must be able to trust that information collected for statistical purposes will not be used for another purpose, such as law enforcement or regulation, directed at specific individuals or organizations. Data providers must also be able to trust that a statistical agency will honor its pledge to protect the confidentiality of their responses. A statistical agency further builds trust among data providers by respecting their privacy and autonomy through minimizing the intrusiveness of questions and the time and effort to respond, consistent with the agency’s requirements for information, and through informing them of the expected time required to participate in the data collection, whether the collection is mandatory or voluntary, and the important uses of the information.
 
Principle 4: Independence from Political and Other Undue External Influence—To be credible and unhindered in its mission, a statistical agency must maintain a widely acknowledged position of independence from undue external influences. It must avoid even the appearance that its collection, analysis, or reporting processes might be manipulated for political purposes or that individually identifiable data collected under a pledge of confidentiality might be turned over for administrative, regulatory, or law enforcement uses. Protection from undue outside influences requires that a statistical agency have authority for professional decisions on its programs, including authority over the selection and promotion of staff, the processing, secure storage, and maintenance of data, and the timing and content of data releases, accompanying press releases, and documentation. The credibility that comes from independence is essential for users to maintain confidence in the accuracy and objectivity of a statistical agency’s data and for data providers to be willing to cooperate with agency requests.
 
Thirteen practices are critical for the effective, principled operation of a statistical agency:
 
1. a clearly defined and well-accepted mission,
2. necessary authority to protect independence,
3. continual development of more useful data,
4. openness about sources and limitations of the data provided,
5. wide dissemination of data,
6. cooperation with data users,
7. respect for the privacy and autonomy of data providers,
8. protection of the confidentiality of data providers’ information,
9. commitment to quality and professional standards of practice,
10. an active research program,
11. professional advancement of staff,
12. a strong internal and external evaluation program, and
13. coordination and collaboration with other statistical agencies.
 
By adhering to the principles and following these practices, a federal statistical agency will be well positioned to provide the relevant, accurate, timely, and credible statistical information that policy makers and the public require.

Introduction

The Committee on National Statistics (CNSTAT) has, since 1992, published a report on principles and practices for a federal statistical agency, which draws from CNSTAT’s many reports on specific agencies, programs, and topics. This report has been widely cited and used by Congress and federal agencies; it has helped shape legislation and executive actions to establish and evaluate statistical agencies; the U.S. Office of Management and Budget has cited it in guidance; and the U.S. Government Accountability Office has used it as a benchmark in reports to Congress. Statistical agencies have used it to inform new appointees, advisory committees, members of Congress and their staff, their own staff, and others about what constitutes an effective and credible statistical organization. Beginning with the second edition in 2001, CNSTAT committed to updating the document every 4 years to provide a current edition to newly appointed cabinet secretaries and other personnel at the beginning of each presidential administration (or second term).
 
The committee had the following statement of work for this volume:
 
In response to recurring requests for advice on what constitutes an effective federal statistical agency, CNSTAT issued the first edition of Principles and Practices for a Federal Statistical Agency (P&P) in 1992. In early 2001, 2005, and 2009, CNSTAT issued the second, third, and fourth editions, respectively, which reiterated the basic principles for federal statistical agencies, revised and expanded the discussion of some of the practices for an effective statistical agency, and updated the discussion with references to recent reports by CNSTAT and others. Changes in laws, regulations, and other aspects of the environment of federal statistical agencies over the past 4 years warrant preparation of a fifth edition, which the committee will prepare for release in early 2013.
 
All of the committee’s work is based on the key concepts of principles and practices. “Principles” are fundamental and intrinsic to the concept of a federal statistical agency; “practices” are ways and means of making the basic principles operational and facilitating an agency’s adherence to them.
 
This fifth edition presents and comments on four basic principles that statistical agencies must embody in order to carry out their mission fully: (1) they must produce data that are relevant to policy issues; (2) they must achieve and maintain credibility among data users; (3) they must achieve and maintain trust among data providers; and (4) they must achieve and maintain independence from the appearance and reality of political or other undue external influence in developing, producing, and disseminating statistics.
 
This edition also discusses 13 important practices that are the means for statistical agencies to implement the four principles. Some of these practices are continual development of more useful data, openness about sources and limitations of the data provided, a commitment to quality and professional standards of practice, a strong internal and external evaluation program, and coordination and collaboration with other statistical agencies. A new practice for this edition concerns the authority that is necessary for an agency to protect its independence. This edition also divides the practice of fair treatment of data providers into two practices: respect for privacy and autonomy of data providers and protection of the confidentiality of providers’ information.
 
In addition to this Introduction, this edition adds an Executive Summary (which is available separately), largely in response to users’ requests. The two main parts are those used in previous editions: Part I is a brief statement of the principles and practices for an effective statistical agency; Part II further explains, defines, and illustrates those principles and practices.
 
The fifth edition emphasizes the need for statistical agencies not only to actively seek out new ways of using multiple data sources (such as administrative records, private-sector datasets, and selected Internet sources in addition to surveys), but also to find new ways to integrate their activities with those of other agencies. The goal is to enable agencies to maintain and improve the relevance, accuracy, timeliness, and cost-effectiveness of their data in the face of the significant challenges they now face, including the increasing costs of obtaining responses from people, households, and organizations to traditional survey inquiries and increasingly constrained budgets for statistical activities.
 
The appendixes to this edition update material included in previous editions and add new information to help orient readers. Appendix A summarizes key legislation and regulations that affect federal statistical agencies, such as the Confidential Information Protection and Statistical Efficiency Act of 2002; statistical policy directives of the U.S. Office of Management and Budget that govern the release of statistical products and provide guidelines for statistical surveys; and the December 2010 Memorandum on Scientific Integrity of the Office of Science and Technology Policy. Appendix B provides an overview of the organization of the U.S. federal statistical system, which is one of the most decentralized in the developed world, and compares the size of the system to the size of the federal government as a whole. Appendix C reproduces the Fundamental Principles of Official Statistics of the United Nations Statistical Commission (for which the Preamble is currently being updated by the international community). Appendix D reproduces the European Statistics Code of Practice for the National and Community Statistical Authorities. Appendix E provides addresses of Internet sites for major federal agencies that provide statistical data.
 
Although focused on federal statistical agencies, many of the principles and practices articulated here likely also apply to statistical activities elsewhere, such as in federal policy, evaluation, research, and program agencies, in state and local government agencies, and in other countries. The principles and practices in this report remain guidelines, not prescriptions. We intend them to assist statistical agencies and to inform policy makers, data users, and others about the characteristics of statistical agencies that enable them to serve the public good. 

Part I: Principles and Practices for a Federal Statistical Agency

Definition of a Federal Statistical Agency
 
Establishment of a Federal Statistical Agency
 
Principles for a Federal Statistical Agency
• Relevance to Policy Issues
• Credibility Among Data Users
• Trust Among Data Providers
• Independence from Political and Other Undue External Influence
 
Practices for a Federal Statistical Agency
• A Clearly Defined and Well-Accepted Mission
• Necessary Authority to Protect Independence
• Continual Development of More Useful Data
• Openness About Sources and Limitations of the Data Provided
• Wide Dissemination of Data
• Cooperation with Data Users
• Respect for the Privacy and Autonomy of Data Providers
• Protection of the Confidentiality of Data Providers’ Information
• Commitment to Quality and Professional Standards of Practice
• An Active Research Program
• Professional Advancement of Staff
• A Strong Internal and External Evaluation Program
• Coordination and Collaboration with Other Statistical Agencies

Definition of a Federal Statistical Agency

A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes. 1
 
• The unit is generally recognized as a distinct entity. It may be located within a cabinet-level department or an independent agency, or it may be an independent agency.
• Compilation may include direct collection of data from individuals, organizations, or establishments through surveys or the acquisition of information from other sources, such as administrative records maintained by government agencies to operate a program, datasets available from the private sector, or data gleaned from selected Internet websites.
• Analysis may take various forms. It includes methodological research to improve the quality and usefulness of data. It also includes substantive analysis—for example, developing indicators from one or more data series, developing models to produce estimates, making projections, interpreting data, and explaining differences among statistics obtained by different methods, such as surveys and administrative records. An analysis by a statistical agency does not advocate policies or take partisan positions.
• Dissemination means making information available to the public, to the executive branch, and to Congress.
• Statistical purposes include description, evaluation, analysis, inference, and research. Statistical purposes relate to descriptions of groups and exclude any interest in or identification of an individual person or economic unit. For these purposes, a statistical agency may collect data directly from providers, or it may obtain data from other sources, but it does not use these data for administrative, regulatory, or law enforcement purposes. The data are used solely to describe and analyze statistical patterns, trends, and relationships involving groups of individuals or other units.

Establishment of a Federal Statistical Agency

Statistics that are publicly available from government agencies are essential for a nation to advance the economic well-being and quality of life of its people. Public policy makers are best served by statistics that are relevant for policy decisions, accurate, timely, and credible. Individuals, households, businesses, academic institutions, and other organizations rely on highquality, publicly available data as the basis for informed decisions on a wide variety of issues. Moreover, the effective operation of a democratic system of government depends on the unhindered flow of statistical information that citizens can use to assess government actions and for other purposes. Federal statistical agencies are established to be a credible source of relevant, accurate, and timely statistics in one or more subject areas that are available to the public and policy makers.
 
“Relevant statistics” are statistics that measure things that matter to policy making, program implementation, monitoring, and evaluation, and public understanding. Relevance requires concern for providing data that help users meet their current needs for decision making and analysis, as well as anticipating future needs. “Accurate statistics” are statistics that match the phenomena being measured and do so in repeated measurements. Accuracy requires proper concern for consistency across geographic areas and across time, as well as for statistical measures of errors in the data. “Timely statistics” are those that are known close in time to the phenomena they measure. Timeliness requires concern for issuing data as frequently as is needed to reflect important changes in what is being studied, as well as disseminating data as soon as practicable after they are collected. “Credibility” requires concern for both the reality and appearance of impartiality and independence from political and other undue external influence. Credibility also requires that agencies follow such practices as making their data and the information that users need to work with the data readily available to all. It is the primary mission of agencies in the federal statistical system to strive to ensure the relevance, accuracy, timeliness, and credibility of statistical information.
 
Reasons to establish a statistical agency include:
 
• the opportunity to achieve higher data quality and greater efficiency of statistical production through a consolidated and professional activity,
• the need for ongoing, up-to-date information in a subject area that extends beyond the scope of individual operating units, possibly involving other departments or agencies,
• the need to protect the confidentiality of responses of data providers, both individuals and organizations, and
• the need for data series that are independent—not subject to control by policy makers or regulatory or enforcement agencies and readily available on an equal basis to all users.
 
The principles and practices for a federal statistical agency that are reviewed in this report pertain to individual agencies as separate organizational entities in the context of a decentralized system for providing federal statistics. Historically, the response of the U.S. government to needs for information to support new federal responsibilities in such areas as agriculture, education, labor, health, science, energy, criminal justice, and transportation has been to create a separate statistical unit in the relevant cabinet department or independent agency. As a consequence, the United States now has one of the most decentralized statistical systems of any modern nation. This report does not comment on the advantages or disadvantages of the U.S. system nor compare it with other models for organizing government statistics. It discusses the critical importance of ensuring that federal statistical agencies coordinate and collaborate with each other and with other agencies on a range of activities, describes the coordinating role of the U.S. Office of Management and Budget (OMB), and reviews some mechanisms for interagency collaboration.

Principles for a Federal Statistical Agency

Principle 1: Relevance to Policy Issues
A federal statistical agency must be in a position to provide objective, accurate, and timely information that is relevant to issues of public policy.
 
A statistical agency must be knowledgeable about the issues and requirements of public policy and federal programs and able to provide information that is relevant to policy and program needs. In establishing priorities for statistical programs for this purpose, a statistical agency must work closely with the users of such information in the executive branch, Congress, and elsewhere.
 
Statistical agencies must also provide objective, accurate, and timely information on the subject area(s) in their purview that is useful to a broad range of private- and public-sector users as well as the public. To establish priorities for such information, a statistical agency must engage with a broad spectrum of users in state and local governments, businesses, academia, and other sectors.
Principle 2: Credibility Among Data Users
A federal statistical agency must have credibility with those who use its data and information.
 
It is essential that a statistical agency strive to maintain credibility for itself and for its data. Few data users are in a position to verify the completeness and accuracy of statistical information; they must rely on an agency’s reputation as a credible source of accurate and useful statistics.
 
Credibility derives from the respect and trust of users in the statistical agency and its data. Such respect results not only from an agency’s production of data that merit acceptance as relevant, accurate, timely, and free from political and other undue external influence, but also from many aspects of an agency’s policies and practices. Key among these are wide dissemination of data on an equal basis to all users; openness about the sources and processes used to produce data and the limitations of the data; commitment to quality and professional practice; a strong internal and external evaluation program to assess and improve an agency’s data systems; a willingness to understand and strive to meet user needs, even though users may not clearly articulate their needs; and a posture of respect and trust in the users of an agency’s data.
Principle 3: Trust Among Data Providers
A federal statistical agency must have the trust of those whose information it obtains.
 
Data providers, such as respondents to surveys and custodians of administrative records, must be able to rely on the word of a statistical agency that the information they provide about themselves or others needs to be collected and will be used only for the purposes that the agency has described. Importantly, data providers must be able to trust that a statistical agency will honor its pledges to protect the confidentiality of their responses. Such protection, in particular, precludes the use of individually identifiable information collected and maintained by a statistical agency under a pledge of confidentiality—whatever its source—for any administrative, regulatory, or law enforcement purpose.
 
The trust of data providers is also achieved by respecting their privacy and autonomy. 2 Such respect requires that an agency minimize the intrusiveness of questions and the time and effort to respond to them to the extent that is compatible with the agency’s requirements for information. An agency’s data collection staff should take care to treat respondents with courtesy and appreciation for their time.
 
Respect also requires that an agency provide sufficient information for the provider to make an informed decision about whether to supply the requested data, including the intended uses of the data being collected, their relevance for important public purposes, and the extent of confidentiality protection that will be provided.
Principle 4: Independence from Political and Other Undue External Influence
A federal statistical agency must be independent from political and other undue external influence in developing, producing, and disseminating statistics.
 
To fulfill its mission to provide objective, useful, accurate, and timely information, a statistical agency must not only be distinct from those parts of a department that carry out administrative, regulatory, law enforcement, or policy-making activities, but it also must have a widely acknowledged position of independence from political and other undue external influences and the necessary authority to protect independence. 3 It must be able to execute its mission without being subject to pressures to advance a political agenda. It must be impartial and avoid even the appearance that its collection, analysis, and reporting processes might be manipulated for political purposes or that individually identifiable data collected under a pledge of confidentiality might be turned over for administrative, regulatory, or law enforcement purposes. Independence from any undue outside influence is an essential element of credibility with data users and the public so that they maintain confidence in the accuracy and objectivity of a statistical agency’s data. It is also essential for trust among data providers so that they continue to be willing to cooperate with agency requests.

Practices for a Federal Statistical Agency

The effective operation of a federal statistical agency must begin with a clearly defined and well-accepted mission. With this prerequisite, effective operation involves a wide range of practices: necessary authority to protect independence, continual development of more useful data, openness about sources and limitations of the data provided, wide dissemination with ample documentation of data, cooperation with data users, respect for privacy and autonomy of data providers, protection of confidentiality of providers’ information, commitment to quality and professional standards of practice, an active research program, professional advancement of staff, a strong internal and external evaluation program, and coordination and collaboration with other statistical agencies.
Practice 1: A Clearly Defined and Well-Accepted Mission
An agency’s mission should include responsibility for all elements of its programs for providing statistical information—determining sources of data, measurement methods, efficient methods of data collection and processing, and appropriate methods of analysis—and ensuring the public availability not only of the data, but also of documentation and explanation of the methods used to obtain and process the data and their quality. The mission should include the responsibility for continually assessing information needs and priorities through proactive engagement with policy makers and other users of its data. The mission should also include the responsibility for identifying, evaluating, implementing, documenting, and explaining new ways to meet user needs, such as by the establishment, modification, or discontinuance of a survey or census or by the implementation of another method of data collection, such as extracting information from administrative records, private-sector data, or selected relevant Internet sources that meet quality standards.
Practice 2: Necessary Authority to Protect Independence
Protection from political or other undue outside influence requires that a statistical agency have the necessary authority for professional decisions on the scope, content, and frequency of data compiled, analyzed, and disseminated within the limits of budgetary resources, departmental requirements, review by OMB, and congressional mandates. It should also have authority over selection and promotion of professional, technical, and operational staff; processing, storage, and maintenance of the data that it collects; and the timing and content of data releases, including accompanying public announcements and documentation, without prior external clearance. An agency’s leaders and qualified technical staff should have authority to speak about the agency’s statistics before Congress, with congressional staff, and before public bodies. Such authority may come from legislation, OMB directives (which carry over from one administration to another), or policies and practices that are communicated by agency leadership to political appointees. 4
 
An agency’s independence is enhanced by adhering to fixed schedules that are announced in advance for the public release of important statistical indicators to prevent even the appearance of manipulation of release dates for political purposes. 5 Independence is also fostered by an agency’s maintaining a clear distinction between statistical information and policy interpretations of such information by executive branch officials and having dissemination policies that foster regular, frequent release of statistical findings and any data limitations to the public through the traditional media, the Internet, and other appropriate means. To bolster public credibility with regard to an agency’s independence, an agency’s website should include a clear description of the procedures it follows to protect against undue external influence in such matters as data dissemination.
Practice 3: Continual Development of More Useful Data
Statistical agencies should continually look to improve their data systems to provide information that is accurate, timely, and relevant for changing public policy and data user needs. They should also continually seek to improve the cost-effectiveness of their programs for collecting, analyzing, and disseminating statistical information.
 
There are many ways for an agency to achieve these goals:
 
• Establish a multifaceted program of data collection from individuals, households, businesses, and other organizations to provide relevant information to meet different data needs. Such a program could include one-time surveys on special topics; repeated surveys of cross-sections of the population that provide regularly updated statistics; and longitudinal surveys that track people, firms, and institutions over time in order to analyze the antecedents and consequences of changes in their circumstances.
• Seek opportunities to combine data from multiple surveys or to integrate survey data with other kinds of data, with appropriate safeguards for protection of confidentiality and the maintenance of quality standards. When separate datasets are collected and analyzed in such a manner that they may be used together, the value of the resulting information and the efficiency of obtaining it may be greatly enhanced.
• Use administrative records that are maintained by government agencies for program operations to improve the quality and cost-effectiveness of some kinds of statistics. Such uses could include producing data series derived from one or more administrative datasets, using administrative records to improve the quality of imputations for missing data in surveys or to adjust survey responses for misreporting or population undercoverage, and combining administrative and survey data in models to produce estimates with improved accuracy for small geographic areas or small population groups.
• Explore the use of other sources (e.g., data from selected Internet sources or private-sector transactions) to improve the relevance and timeliness of some information. For example, an Internet source might provide the basis for timely information, which is later revised on the basis of data from a survey or administrative records, or it might provide the basis for an additional indicator that is not otherwise readily available. Care must always be taken to evaluate a source of data before deciding to use it, perhaps initially on an experimental basis, and to fully explain the source and its limitations.
• Share technical information and ideas with other statistical agencies. Such sharing can stimulate the development of innovative data collection, analysis, and dissemination methods that improve the accuracy and timeliness of information and the efficiency of data operations.
Practice 4: Openness About Sources and Limitations of the Data Provided
A statistical agency should be open about the strengths and limitations of its data, taking as much care to understand and explain how its statistics may fall short of accuracy as it does to produce accurate data. Data releases from a statistical program should be accompanied by a full description of the purpose of the program; the methods and assumptions used for data collection, processing, and reporting; what is known and not known about the quality and relevance of the data; sufficient information for estimating variability in the data; appropriate methods for analysis that take account of variability and other sources of error; and the results of research on the methods and data.
 
When problems are found in a previously released statistic that could affect its use, an agency should issue a correction promptly and publicly. It should also consider maintaining an online list of previous corrections to assist both new and long-standing users. Generally, an agency should be proactive in seeking ways to alert known and likely users of the data about the nature of a problem and the appropriate corrective action that it is taking or that users should take.
Practice 5: Wide Dissemination of Data
A statistical agency should strive continually for the widest possible dissemination of the data it compiles in formats that are widely accessible. Data dissemination should be timely, and information should be made readily available on an equal basis to all users. Also, measures should be taken to ensure that data are preserved and accessible for use in future years.
 
There are many elements of an effective dissemination program:
 
• An established publications policy should describe, for a data collection program, the types of reports and other data releases to be made available, the formats to be used, the audience to be served, and the frequency of release.
• A variety of avenues for data dissemination should be chosen to reach as broad a public as reasonably possible—including, but not limited to, an agency’s Internet website, government depository libraries, conference exhibits and programs, newsletters and journals, email address lists, social media and blogs, and the traditional media for regular communication of major findings.
• Data should be released in a variety of forms, including printed reports, easily accessible website displays and databases, public-use microdata, and other publicly available computer-readable files, so that the information can be accessed by users with varying skills and needs for data retrieval and analysis. All data releases should be suitably processed to protect confidentiality and accompanied by careful and complete documentation, including explanatory material to assist users in appropriate interpretation. Particularly for complex databases, user training should be provided through such forums as webinars, online tutorials, and sessions at appropriate conferences.
• For research and other statistical purposes, an agency should provide access to relevant information that is not publicly available through restricted access modes that protect confidentiality. Such modes include protected research data centers, remote monitored online access systems, and licensing of datasets to individual researchers.
• Policies should be in place for the preservation of data that guide what data to retain and how they are to be archived and made accessible for future secondary analysis.
Practice 6: Cooperation with Data Users
A statistical agency shows cooperation with data users by facilitating their access to and ability to use data through well-designed websites and other dissemination vehicles, careful and complete documentation, and user training adapted to varying skills and needs. In addition, a statistical agency should seek input from users on every aspect of its programs. The goal is to make its data as relevant, accurate, timely, and accessible as possible to a broad range of users. It should:
 
• seek advice on data concepts, content, processing, estimation, products, and documentation from a wide spectrum of data users, as well as from professional and technical subject-matter and methodological experts, using a variety of formal and informal means of communication that are appropriate to the types of input sought;
• seek advice on its statistical programs and priorities from external groups, including those with relevant subject-matter and technical expertise; and
• widely disseminate its responses to those who have provided input.
 
In developing and implementing new methods or data sources to produce statistical information, it is particularly important to reach out to policy makers and other key data users so that they understand an agency’s criteria and decision process for the new methods or data. Statistics that are based on models (for example, for small geographic areas) or that use nontraditional data sources will likely require an explanation of their benefits and limitations that is more extensive than is usually provided. Reaching out to policy makers and other key data users when new data sources or methods are in a developmental stage can help in identifying and responding to users’ concerns and earning their acceptance of the resulting data products.
Practice 7: Respect for the Privacy and Autonomy of Data Providers
To maintain a relationship of respect and trust with survey participants and other data providers, a statistical agency should respect their privacy and minimize the burden imposed on them. Two key data collection practices demonstrate an agency’s respect for and fair treatment of data providers:
 
• Prior to collection of information, data providers should be informed of the purposes of data collection and the anticipated uses of the information, the expected burden of participation, whether their participation is mandatory or voluntary, and, if voluntary, using appropriate informed consent procedures to obtain their participation.
• The data collection method should minimize—to the extent possible and consistent with the need for the data—the intrusiveness of questions and the time and effort needed to respond. For items that may be perceived as burdensome, an agency should provide an explanation of their purpose.
 
In addition, agencies can recognize the value of respondents’ participation in data collection programs by accurately representing the statistical information they provide and by making it widely available on an equal basis to all.
Practice 8: Protection of the Confidentiality of Data Providers’ Information
To earn the respect and trust of data subjects and other data providers, it is essential for a statistical agency to protect the confidentiality of the information it collects for statistical purposes. An agency should have policies and procedures to maintain the confidentiality of data—whether collected directly in surveys or obtained from administrative records or other sources—so that individual data collected under a pledge of confidentiality cannot be used for administrative, regulatory, law enforcement, or any other nonstatistical purpose. As part of confidentiality protection, an agency should have the authority to manage the storage of confidential microdata on secure servers that are controlled by the agency and not by a departmentwide information technology system. A statistical agency should also have policies and procedures to inform data providers of the manner and level of confidentiality protection and the kinds of research and analysis that will be allowed with the data.
Practice 9: Commitment to Quality and Professional Standards of Practice
A statistical agency should:
 
• keep abreast of and use modern statistical theory and sound statistical practice in all technical work;
• document concepts, definitions, data collection methodologies, and measures of uncertainty and discuss possible sources of error in reports and other data releases to the public;
• develop strong staff expertise in the disciplines relevant to its mission, in the theory and practice of statistics, and in data collection, processing, analysis, and dissemination techniques;
• develop an understanding of the validity and accuracy of its data and convey the resulting measures of quality to users in ways that are comprehensible to nonexperts;
• maintain quality assurance programs to improve data quality and to improve the processes of compiling, editing, and analyzing data; and
• develop a strong and continuous relationship with appropriate professional organizations in the fields of statistics and relevant subject-matter areas.
Practice 10: An Active Research Program
A statistical agency should have a research program that is relevant to its activities. Because a small agency may not be able to afford an appropriate research program, agencies should collaborate and share research results and methods. Agencies can also augment their staff resources for research by using outside experts through consulting or other arrangements as appropriate.
 
Several elements should be part of a statistical agency’s research program:
 
• Research should be conducted on the substantive issues for which the agency’s data are compiled. Such research should be conducted not only to provide useful objective analytical results, but also as a means to identify potential improvements to the content of the data, suggest improvements in the design and operation of the data collection, and provide fuller understanding of the limitations of the data.
• An agency’s program should include research to evaluate and improve statistical methods, including the identification and creation of new statistical measures; improved methods for analyzing reporting and other errors in the data; ways to reduce the time and effort requested of respondents; and means to improve the timeliness, accuracy, and efficiency of data collection, analysis, and dissemination procedures.
• Research should be conducted to understand and estimate new sources of risk to confidentiality protection and to enhance mechanisms for access to data in ways that guard against disclosure.
• Research should be conducted to understand how the agency’s information is used, in order to make the data more relevant to policy concerns and more useful for policy research and decision making.
Practice 11: Professional Advancement of Staff
A statistical agency should recruit, develop, and support professional staff who are committed to the highest standards of quality work, professional practice, and professional ethics. To develop and maintain a high-caliber staff, a statistical agency must recruit qualified people with relevant skills for efficient and effective operations, including analysts in fields relevant to its mission (e.g., demographers, economists), statistical methodologists who specialize in data collection and analysis, and other skilled staff (e.g., computer specialists). To retain and make the most effective use of its staff, an agency should provide opportunities for work on challenging projects in addition to more routine, production-oriented assignments. An agency’s personnel policies, supported with significant resources, should enable staff to extend their technical capabilities through appropriate professional and developmental activities, such as attendance and participation in professional meetings, participation in relevant training programs, rotation of assignments, and involvement in collaborative activities with other statistical agencies.
 
An agency should also seek opportunities to reinforce the commitment of its staff to ethical standards of practice. Such standards are the foundation of an agency’s credibility as a source of relevant, accurate, and timely information obtained through fair treatment of data providers and data users.
Practice 12: A Strong Internal and External Evaluation Program
Statistical agencies should have regular, ongoing programs of evaluation for major statistical programs and program components and for their overall portfolio of programs. Regular formal reviews of major data collection programs and their components should consider, among other topics, how to produce relevant, accurate, and timely data in the most costeffective manner possible and evaluate whether there are ways to improve cost-effectiveness by combining data from multiple sources. Regular formal reviews of an agency’s portfolio should consider ways to reduce duplication, fill gaps, and adjust priorities so that the suite of programs remains as relevant as possible to the information needs of policy makers and the public given the available resources. Such evaluations should include internal reviews by staff and external reviews by independent groups.
Practice 13: Coordination and Collaboration with Other Statistical Agencies
A statistical agency should actively seek opportunities to collaborate with other statistical agencies to enhance the value of its own information and that of other agencies in the federal statistical system. Although agencies differ in their subject-matter focus, there is overlap in their missions and a common interest in serving the public need for credible, relevant, accurate, and timely statistics gathered as efficiently and fairly as possible.
 
When possible and appropriate, federal statistical agencies should collaborate not only with each other, but also with state and local statistical agencies in providing data for subnational areas. Federal statistical agencies should also collaborate with foreign and international statistical agencies to exchange information on both data and methods and to develop appropriate common classifications and procedures to promote international comparability of information.
 
Such collaborative activities as integrating data compiled by different statistical agencies, standardizing concepts and measures, sharing data among agencies, and identifying ways to reduce unneeded duplication invariably require effort to overcome differences in agency missions and operations. Yet with constrained budgets and increasing demand for more relevant, accurate, and timely statistical information, the importance of proactive collaboration and coordination among statistical agencies cannot be overstated. To achieve the most effective integration of their work for the public good, agencies must be willing to take a long view and to strive to accommodate other agencies.
 
The rewards can be data that are more efficiently obtained and more relevant to policy concerns. Another reward can be a stronger, more effective statistical system as a whole. To achieve these rewards, statistical agencies need to act as partners, not only in the development of statistical information for public use, but also for the entire panoply of statistical activities, including the definition and updating of concepts and classifications and the continual improvement of measurement methods, analytical tools, means for confidentiality protection, and modes of data dissemination. Statistical agencies, working with OMB, also need to be continually vigilant to refine, disseminate, and inculcate the highest standards of professional practice and policies in such areas as privacy and confidentiality protection, data release schedules, and scientific integrity—standards that are critical for credibility with the providers and users of their information.

Footnotes

1
The U.S. Office of Management and Budget (2007:33364) provides a similar definition of a statistical agency: “An agency or organizational unit of the executive branch whose activities are predominantly the collection, compilation, processing, or analysis of information for statistical purposes.”
2
We take the term “autonomy” from the 1978 report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, known as the Belmont Report: “To respect autonomy is to give weight to autonomous persons’ considered
opinions and choices.” Available: http://archive.org/details/belmontreporteth00unit [February 2013].
3
A statistical agency actively works to obtain a broad range of external input to develop its programs; “undue external influences” are those that seek to undermine an agency’s impartiality and professional judgment.
4
Thirteen agencies have adopted and made accessible a “Statement of Commitment to Scientific Integrity by Principal Statistical Agencies” (see Appendix A and http://bls.gov/bls/integrity.htm [February 2013]), which affirms their commitment to the principles and practices enunciated in National Research Council (2009c), OMB statistical policy directives, and guidelines required by the Information Quality Act. This statement and these supporting documents uphold the necessity for a statistical agency to have authority to protect its independence from undue outside influence.
5
Such adherence is required in OMB Statistical Policy Directive 3 for the principal economic indicators and in Directive 4 for all regular or recurring statistical products; see Appendix A.

Part II: Commentary

Definition of a Federal Statistical Agency

A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes.
 
A statistical agency may be labeled a bureau, center, division, or office or similar title, so long as it is recognized as a distinct entity. Statistical agencies have been established to serve several purposes, including:
 
• to develop new information for an area of public concern (e.g., the Bureau of Labor Statistics [BLS] and the National Center for Health Statistics);
• to conduct large statistical collection and dissemination operations specified by law (e.g., the U.S. Census Bureau);
• to compile and analyze statistics from sets of administrative records for policy purposes and public use (e.g., the Statistics of Income Division in the Internal Revenue Service [IRS]); and
• to develop broad and consistent estimates from a variety of statistical and administrative sources under a specified conceptual framework (e.g., the Bureau of Economic Analysis [BEA] in the U.S. Department of Commerce, which produces the U.S. National Income and Product Accounts). Once established, many statistical agencies engage in all of these functions to varying degrees.
 
This definition of a federal statistical agency does not include many statistical activities of the federal government because they are not performed by distinct units or because they do not result in the dissemination of statistics to others. Such activities include statistics compiled by the U.S. Postal Service to set rates or statistics developed by the U.S. Department of Defense in the testing of weapon systems (see National Research Council, 1998b, 2003b, and 2012d). Nor does it include agencies whose primary functions are the conduct or support of problem-oriented research, although their research may be based on information gathered by statistical means, and they may also sponsor important surveys: examples include the National Institutes of Health, the Agency for Healthcare Research and Quality, and other agencies in the U.S. Department of Health and Human Services.
 
This definition of a statistical agency also does not usually include agencies whose primary function is policy analysis and planning (e.g., the Office of Tax Analysis in the U.S. Department of the Treasury, the Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of Health and Human Services). Such agencies may collect and analyze statistical information, and statistical agencies, in turn, may perform some policy-related analysis (e.g., produce reports on trends in after-tax income or child care arrangements of families). However, to maintain credibility as an objective source of accurate, useful information, statistical agencies must be separate from units that are involved in developing policy and assessing policy alternatives.
 
Statistical agencies have as their primary purpose the dissemination of information that can be used for a wide range of statistical purposes but not for administrative, enforcement, or regulatory purposes that could affect an individual (person or business) data provider. Such data are usually collected under a pledge of confidentiality. Statistical agencies may collect information from government agencies in which individual reporting units are identified because the data are already public information—as, for example, in the Census Bureau’s program to collect financial and employment information for state and local governments (see National Research Council, 2007a) and the program of the National Center for Science and Engineering Statistics to collect information on research and development spending from federal agencies (see National Research Council, 2010c).
 
Occasionally, statistical agencies are charged to collect information that is made available for both statistical and nonstatistical purposes. For example, the Bureau of Transportation Statistics (BTS) maintains the Airline On-Time Statistics Program (originated by the former Civil Aeronautics Board), which identifies individual airlines.1 However, BTS does not itself use the data for administrative or regulatory purposes—those functions are carried out by the Federal Aviation Administration—and the data are not collected under a pledge of confidentiality. As another example, higher education institutions that participate in federal student aid programs are required by law (20 USC 1094(a)(17)) to respond to surveys conducted by the National Center for Education Statistics (NCES). The data collected on enrollments, graduation rates, faculty and staff, finances, institutional prices, and student financial aid feed into the Integrated Postsecondary Education Data System (IPEDS). The data are not collected under a pledge of confidentiality, and NCES makes information on individual institutions available to parents and students to help them in choosing a college, as well as to researchers and others.2 NCES also collaborates with institutions through the National Postsecondary Education Cooperative, which works to improve the quality of reporting and dissemination of information to the public, identify modifications to definitions that are necessary to keep abreast of changes in the field, and address other aspects of the IPEDS program.
 
Statistical agencies should carefully consider the advantages and disadvantages of undertaking a program with both statistical and nonstatistical purposes. One potential advantage is that there may be improved consistency and quality when a statistical agency collects information for its own use and that of other parts of its department. One potential disadvantage is that the program may compromise the public perception of the agency as objective and separate from government administrative, regulatory, and enforcement functions.
 
When an agency decides to carry out a program that has both statistical and nonstatistical uses, it must take care to clearly describe that program on such dimensions as the extent of confidentiality protection, if any (for example, some but not all of the data may be collected under a pledge of confidentiality); the statutory basis for the program and the public purposes it serves, including benefits to respondents from having comparative information available of uniform quality; and the role of the agency (for example, providing information to the public, working with respondents to improve reporting). Should an agency decide that the nature of a program is such that no amount of description or explanation is likely to make it possible for the agency to maintain its credibility as a statistical agency, it should decline to carry out the activity.
 
The work of federal statistical agencies is coordinated through the U.S. Office of Management and Budget (OMB) Statistical and Science Policy Office (SSP) and the Interagency Council on Statistical Policy (ICSP), which was created by OMB in the 1980s and authorized in statute in the 1995 reauthorization of the Paperwork Reduction Act (44 USC 3504(e)(8)). The ICSP is chaired by the chief statistician in OMB and currently includes representation from 14 agencies and units, which are housed in 9 cabinet departments and 3 independent agencies (see Appendix B):
 
• Bureau of Economic Analysis (U.S. Department of Commerce)
• Bureau of Justice Statistics (U.S. Department of Justice)
• Bureau of Labor Statistics (U.S. Department of Labor)
• Bureau of Transportation Statistics (U.S. Department of Transportation)
• Census Bureau (U.S. Department of Commerce)
• Economic Research Service (U.S. Department of Agriculture)
• Energy Information Administration (U.S. Department of Energy)
• National Agricultural Statistics Service (U.S. Department of Agriculture)
• National Center for Education Statistics (U.S. Department of Education)
• National Center for Health Statistics (U.S. Department of Health and Human Services)
• National Center for Science and Engineering Statistics (U.S. National Science Foundation [NSF])
• Office of Environmental Information (U.S. Environmental Protection Agency)
• Office of Research, Evaluation, and Statistics (Social Security Administration, U.S. Department of Health and Human Services)
• Statistics of Income Division (U.S. Department of the Treasury)
 
In addition to these 14 agencies, OMB currently recognizes 110 other units and agencies that are not statistical agencies but that have annual budgets of $500,000 or more for statistical activities (U.S. Office of Management and Budget, 2012b:Table 1). The principles for federal statistical agencies presented here are relevant to these other agencies that carry out statistical activities, and many of the detailed practices are also pertinent. Similarly, the principles and practices may be relevant to statistical units in state and local government agencies, as well as for international statistical agencies.

Establishment of a Federal Statistical Agency

One of the most important reasons for establishing a statistical agency is to provide information that will allow for an informed citizenry. A democracy depends on an informed electorate. A citizen has a right to information that comes from a trustworthy, credible source and that is relevant, accurate, and timely. Timely information of high quality is also critical to policy analysts and decision makers in both the public and private sectors. (For more information on the purposes of official statistics, see the Fundamental Principles of Official Statistics of the U.N. Statistical Commission in Appendix C; see also U.N. Economic Commission for Europe, 2003; U.N. Statistical Commission, 2003.) Federal statistical agencies serve the key functions of providing a broad array of information to the public and to policy makers and of ensuring the necessary quality and credibility of the data.
 
Commercial, nonprofit, and academic organizations in the private sector also provide useful statistical information, including data they collect themselves and data they acquire from government agencies and other data collectors to which they add useful other information or analysis. However, because the benefits of statistical information are shared widely throughout society and because it is often difficult to garner payments for these benefits, private entities are not likely to collect all of the data that are needed for public and private decision making or to make data as  widely available as needed for important public purposes. Nor are they likely to have the capacity or interest to continually work to make data comparable across geographic areas, population groups, and over time, or, in general, to continually work to provide the needed scope, scale, and quality of statistical information. Government statistical agencies are established to ensure that a broad range of relevant, accurate, timely, and credible information is publicly available. (See National Research Council, 1999b, 2005b, for a discussion of the governmental role in providing public goods, or near public goods, such as research and data.)
 
The U.S. government collected and published statistics long before any distinct federal statistical agency was formed (see Duncan and Shelton, 1978; Norwood, 1995). The U.S. Constitution mandated the conduct of a decennial census of population; the first such censuses (beginning in 1790) were conducted by U.S. marshals as one of their many duties. Legislation providing for the compilation of statistics on agriculture, education, and income was enacted by Congress in the 1860s. The Bureau of Labor (forerunner of the Bureau of Labor Statistics) was established by law in 1884 as a separate agency with a general mandate to respond to widespread public demand for information on the conditions of industrial workers. The Census Bureau was established as a permanent agency in 1902 to conduct the decennial census and related statistical activities.
 
Many federal statistical agencies that can trace their roots back to the 19th or early 20th century, such as the National Center for Education Statistics and the National Center for Health Statistics, were organized in their current form following World War II. Several relatively new agencies have since been established, including the Energy Information Administration, the Bureau of Justice Statistics, and the Bureau of Transportation Statistics. 3
 
In every case, the agency itself, in consultation with users of its information, has major responsibility for determining its specific statistical programs and for setting priorities. Initially, many of these agencies also had responsibilities for certain policy analysis functions for their department heads. More recently, policy analysis has generally been located in separate units that are not themselves considered to be statistical agencies, a separation that helps establish and maintain the credibility of statistical agencies as providers of data and analyses that are not designed for particular policy alternatives. Nevertheless, an effective statistical agency has a role as a creative, not just reactive, actor in the development of data needed for policy analysis. Statistical agencies may also play additional roles, such as reviewer and consultant on statistical matters for other units in the same department (see, e.g., National Research Council, 1985a) and collector of data on a reimbursable basis for other agencies.
 
There is no set rule or guideline for when it is appropriate to establish a separate federal statistical agency, carry on statistical activities within the operating units of departments and independent agencies, or contract for statistical services from existing federal statistical agencies or other organizations. Establishment of a federal statistical agency should be considered when one or more of the following conditions prevail: 4
 
• There is a need for information on an ongoing basis beyond the capacity of existing operating units, possibly involving other departments and agencies. Such needs may require coordinating data from various sources, initiating new data collection programs to fill gaps, or developing regularly updated time series of estimates.
• There is a need, as a matter of credibility, to ensure that major data series are independent of policy makers’ control.
• There is a need to establish the functional separation of data on individuals and organizations that are collected for statistical purposes from data on individuals and organizations that may be used for administrative, regulatory, or law enforcement uses. Such separation, recommended by the Privacy Protection Study Commission (1977), bolsters a culture and practice of respect for privacy and protection of confidentiality. Functional separation is easier to maintain when the data to be used for statistical purposes are compiled and controlled by a unit that is separate from operating units or department-wide data centers. The Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA) extended legal confidentiality protection to statistical data collections that may be carried out by any federal agency, whether a statistical agency or other type of agency (see Appendix A). Nonetheless, functional separation of statistical data from other kinds of data is important because it makes promises of confidentiality protection more credible. 5
 
• There is a need to emphasize the principles and practices of an effective statistical agency, for example, professional practice, openness about the data provided, and wide dissemination of data.
• There is a need to encourage research and development of a broad range of statistics in a particular area of public interest or of government activity or responsibility.
• There is a need to consolidate compilation, analysis, and dissemination of statistics in one unit to encourage high-quality performance, eliminate duplication, and streamline operations.

Principles for a Federal Statistical Agency

Principle 1: A federal statistical agency must be in a position to provide objective, accurate, and timely information that is relevant to issues of public policy
A statistical agency supplies information not only for the use of managers and policy makers in the executive branch and for legislative designers and overseers in Congress, but also for everyone who requires objective statistical information on public issues, whether the information is needed for purposes of production, trade, consumption, or participation in civic affairs. Just as a free enterprise economic system depends on the availability of economic information to all participants, a democratic political system depends on—and has a fundamental duty to provide—wide access to information on education, health, transportation, the economy, the environment, criminal justice, and other social issues.
 
Federal statistical agencies are responsible for providing statistics on conditions in a variety of areas. The resulting information is used both inside and outside the government not only to delineate problems and to guide courses of action, but also to evaluate the results of government activity or lack of activity. The statistics provide much of the basis on which the government itself is judged. This role places a heavy responsibility on federal statistical agencies for impartiality and objectivity.
 
In order to provide information that is relevant to public issues, statistical agencies need to reach out to users of the data. While they are usually in touch with the primary users in their own departments, considerable energy and initiative are required to open avenues of communication more broadly to other current and potential users, including analysts and policy makers in other federal departments, state and local government agencies, academic researchers, private-sector businesses and other organizations, organized constituent groups, associations that represent data users, the media, and members of Congress and their staffs. 6
 
One way to obtain the views of users outside an agency, as well as people with relevant technical expertise, is through advisory committees (see National Research Council, 1993a, 2007c). Many agencies obtain advice from committees that are chartered under the Federal Advisory Committee Act: examples include the Advisory Committee on Agriculture Statistics for the National Agricultural Statistics Service; the Board of Scientific Counselors for the National Center for Health Statistics; the Data Users Advisory Committee and the Technical Advisory Committee for the Bureau of Labor Statistics; and the Scientific Advisory Committee and the National Advisory Committee on Race, Ethnic, and Other Populations for the Census Bureau. The Federal Economic Statistics Advisory Committee (FESAC), chartered in November 1999, provides substantive and technical advice to three agencies—the Bureau of Economic Analysis, the Bureau of Labor Statistics, and the Census Bureau—thereby providing an important cross-cutting perspective on major economic statistics programs. 7 Some agencies obtain advice from committees and working groups that are organized by an independent association, such as the American Statistical Association’s Committee on Energy Statistics for the Energy Information Administration.
 
Other means to gather information about user priorities for federal statistics include workshops and conferences, which are valuable for facilitating interchange among users and agency staff (see National Research Council, 2012a). Online mechanisms, such as blogs and web surveys, may also assist a statistical agency obtain input from users.
 
It is also important for an agency’s own staff to engage in analysis of its data to improve them and make them more relevant to users (Martin, 1981; Norwood, 1995; Triplett, 1991). For example, relevant analysis may use the agency’s data to examine correlates of key social or economic phenomena or to study the statistical error properties of the data. Such in-house analysis can lead to improvements in the quality of the statistics, to identification of new needs, to a reordering of priorities, and to closer cooperation and mutual understanding with policy analysis units. In working with a policy analysis unit, a statistical agency may describe conditions and possibly measure progress toward some previously identified goal, but it refrains from making policy recommendations. The distinction between statistical analysis and policy analysis is not always clear, and a statistical agency will need to consider carefully the extent of policy-related activities that are appropriate for it to undertake.
Principle 2: A federal statistical agency must have credibility with those who use its data and information
Users of a statistical agency’s data must be able to trust that the data were collected and analyzed in an objective, impartial manner and that they are as relevant, accurate, and timely as the agency can make them. Without the reality and appearance of credibility, policy debates may deteriorate into attacks on the data, instead of using the information to inform policy choices. Credibility is enhanced when an agency fully informs users of the strengths and weaknesses of the data, makes data available widely, and actively engages with users about priorities for data collection and analysis. When it does so, an agency is perceived to be working in the national interest, not the interest of a particular administration (Ryten, 1990).
 
Credibility is also enhanced when a statistical agency’s website has readily accessible information about its policies on such topics as confidentiality and privacy protection, scientific integrity, standards for data quality and for documenting sources of error in data collections and the limitations of datasets and statistical models, procedures and schedules for the release of new and continuing data series, and procedures for timely notice of errors and corrections to previously released data. Links to policies of an agency’s parent cabinet department or independent agency that clearly specify the authority that is delegated to the statistical agency also enhance credibility and build trust with users.
Principle 3: A federal statistical agency must have the trust of those whose information it obtains
The statistical programs of the federal government rely on information supplied by many data providers, including not only other agencies of the federal government, but also individuals and organizations outside the federal government, such as state and local governments, businesses, and  other organizations. Some of this information is a by-product of data collections that are required by law or regulation for use in the administration of government tax and transfer programs, such as employers’ wage reports to state employment security agencies or records of payments to program beneficiaries. But much of it is obtained through the voluntary cooperation of respondents in statistical surveys. Even when response is mandatory, as in the case of statistical programs that are critical to the nation, such as the population and economic censuses, the cooperation of respondents reduces costs and likely promotes accuracy (see National Research Council, 1995b, 2004d).
 
Important elements in encouraging such cooperation are that respondents believe that the data requested are important and legitimate for the government to collect, that they are being collected in an impartial, competent manner, and that the confidentiality of their responses will be protected. With regard to confidentiality, trust depends on providing respondents with realistic promises of confidentiality that the agency can reasonably expect to honor and then scrupulously honoring those promises. Respondent trust also depends on adopting practices that respect personal privacy, such as taking steps to minimize the intrusiveness of questions and the time and effort required to participate in a survey to the maximum extent possible that is consistent with the needs for information.
 
When data are obtained from the administrative records of other federal, state, or local government agencies, the same principle of trust applies in order to secure the fullest possible cooperation of the agencies with a statistical agency’s needs for the records and their documentation. Provider agencies need to believe that their records are important and legitimate for a statistical agency to obtain, that their restrictions on data access will be honored, and that the statistical agency will make every effort to minimize their burden in responding to the agency’s requests.
Principle 4: A federal statistical agency must be independent from political and other undue external influence in developing, producing, and disseminating statistics
A statistical agency must be able to provide credible information that may be used to evaluate the program and policies of its own department or the government as a whole. More broadly, a statistical agency must be a trustworthy source of objective, relevant, accurate, and timely information for decision makers, analysts, and others—inside and outside the government—who want to use statistics to understand present conditions, draw comparisons with the past, and help guide plans for the future. 8 For these purposes, it is essential that a statistical agency has a strong position of independence that protects it from political and other undue external influence in developing, producing, and disseminating statistics.
 
Statistical agency independence is exercised in a broad framework. Legislative authority usually gives ultimate responsibility to the secretary of the department rather than to the head of the statistical agency. In addition, a statistical agency is subject to the normal budgetary processes and to various coordinating and review functions of OMB, as well as the legislative mandates and oversight of Congress.
 
Within this broad framework, a statistical agency has to work to maintain its credibility as an impartial purveyor of information. In the long run, the effectiveness of an agency depends on its maintaining a reputation for impartiality; thus, an agency must be continually alert to possible infringements on its credibility and be prepared to argue strenuously against such infringements.
 
For an agency head, independence and protection from undue political influence can be strengthened by the method of the appointment. Appointment by the President with confirmation by the Senate for a fixed term, as is the case for the BLS and the U.S. Census Bureau, and departmental appointment of a career civil servant, as is the case for many statistical agencies, are both methods that can bolster the professional independence of an agency head. 9 For a fixed term, it is desirable that it not coincide with the presidential term so that professional considerations, rather than political ones, are more likely to be paramount in the appointment process. Appointment by the President with Senate confirmation for a term that is at the pleasure of the President, as is the case for the head of the Energy Information Administration, is not ideal for agency independence. However, this agency does have strong legislative protection for the authority of its administrator (see Practice 2).
 
The Presidential Appointment Efficiency and Streamlining Act of 2011 (P.L. 112-166), which took effect on August 10, 2012, kept the commissioner of the National Center for Education Statistics and the director of the Bureau of Justice Statistics as presidential appointees but dropped the requirement for Senate confirmation. 10 It also provided that the director of the Census Bureau remain a presidential appointee with Senate confirmation but have a fixed 5-year term (with one renewal permitted) for terms beginning on January 1 of years ending in 2 and 7. Previously, the Census Bureau director served at the pleasure of the President.
 
It is valuable for the head of a statistical agency to have direct access to the secretary of the department or the head of the independent agency in which the statistical agency is located. Such access allows the head to inform new secretaries about the role of a statistical agency and be able to directly present the case for new statistical initiatives. Such direct access currently is provided by legislation only for the Bureau of Labor Statistics and the Energy Information Administration.
 
It is also desirable for a statistical agency to have its own funding appropriation from Congress and not be dependent on allocations from the budget of its parent department or agency, which may be subject to reallocation. Agencies that have been assisted by departmental allocations include the Bureau of Justice Statistics and the National Center for Health Statistics. Such organizational aspects as direct access to the secretary of the agency’s department and separate budgetary authority are neither necessary nor sufficient for a strong position of independence that protects a statistical agency from undue political influence, but they facilitate such independence. In contrast, some agencies are under several layers of supervision within their departments (see Appendix B).

Practices for a Federal Statistical Agency

Practice 1: A Clearly Defined and Well-Accepted Mission
A clear understanding of the mission of an agency, the scope of its statistical programs, and its authority and responsibilities are basic to planning and evaluating its programs and to maintaining credibility and independence from political control (National Research Council, 1986, 1997b). Some agency missions are clearly spelled out in legislation; other agencies have only very general legislative authority. On occasion, very specific requirements may be set by legislation or regulation.
 
A statistical agency’s mission should focus on the compilation, evaluation, analysis, and dissemination of relevant, accurate, and timely information for statistical purposes. When an agency is charged to carry out an activity that could be perceived as undermining its credibility as an objective source of information (e.g., collecting data not only for statistical, but also for administrative purposes), the agency should carefully describe and structure the activity (e.g., perhaps locating it within a clearly demarcated office) so as to preserve credibility. Should it not be possible to develop a satisfactory arrangement that is both responsive to the charge and credible, then a statistical agency should request that the activity be assigned elsewhere.
 
Agencies should clearly communicate their mission to others. The use of the Internet is one means to publicize an agency’s mission to a broad audience and to provide related information, including enabling legislation, the scope of the agency’s statistical program, confidentiality provisions, operating procedures, and data quality guidelines.
 
An agency also needs to pay considerable and formal attention to setting statistical priorities (National Research Council, 1976). Advice from outside groups should be sought on the agency’s statistical program, on setting statistical priorities, on the statistical methods used, and on data products and services. Such advice may be sought in a variety of formal and informal ways, but it should be obtained from both data users and providers and from professional or technical experts in the subject-matter area and in statistical methods and procedures. A strong research program in the agency’s subject-matter field can assist in setting priorities and identifying ways to improve an agency’s statistical programs (Triplett, 1991).
Practice 2: Necessary Authority to Protect Independence
Protection from political and other undue external influence over a statistical agency’s data collection, production, dissemination, and other operations necessitates that the agency have the necessary authority for professional decisions in key aspects of its work, including the following:
 
• authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, or published within the framework set by its authorizing legislation; 11 
• authority for selection and promotion of professional, technical, and operational staff;
• authority—and recognition by policy officials outside the statistical agency of that authority—to release statistical information, including accompanying press releases and documentation, without prior clearance regarding the statistical content of the release;
• authority to control information technology systems in order to securely maintain the integrity and confidentiality of data and reliably support timely and accurate production of key statistics; and
• authority for the statistical agency head and qualified staff to speak about the agency’s statistics before Congress, with congressional staff, and before public bodies.
 
In order to guard against even the perception of political and other undue external influence, it is important that a statistical agency strictly observe several basic operating procedures:
 
• adhere to fixed schedules in public release of important statistical indicators to prevent even the appearance of manipulation of release dates for political purposes;
• maintain a clear distinction between statistical information and policy interpretations of such information by the president, the secretary of the department, or others in the executive branch; and
• have dissemination policies that foster regular, frequent release of major findings from the agency’s statistical programs to the public through the traditional media, the Internet, and other means.
 
Another important aspect of independence is control over personnel actions, especially the selection and appointment of qualified professional staff, including senior executive career staff. The agency staff who report directly to the agency head should have formal education and deep experience in the substantive, methodological, operational, or management issues facing the agency as appropriate for their positions. For the head of a statistical agency, professional qualifications are of the utmost importance, whether the profession is that of statistician or the subject-matter field of the statistical agency (National Research Council, 1997b). Relevant professional associations can be a source of valuable input on suitable candidates.
 
The authority to ensure that information technology systems fulfill the specialized needs of the statistical agency is another important aspect of independence. A statistical agency must be able to vouch for the integrity, confidentiality, and impartiality of the information collected and maintained under its authority so that it retains the trust of its data providers and data users. Such trust is fostered when a statistical agency has control over its information technology resources, and there is no opportunity or perception that policy, program, or regulatory agencies could gain access to records of individual respondents. A statistical agency also needs control over its information technology resources to support timely and accurate release of official statistics, which are often produced under stringent deadlines.
 
Authority to decide the scope and specific content of the data collected or compiled and to make decisions about technical aspects of data collection programs is yet another important element of independence, although such authority can never be without limits. Congress frequently specifies particular data that it wishes to be collected (e.g., data on job openings and labor turnover by the BLS, data on family farms by the Economic Research Service and National Agricultural Statistics Service) and, in the case of the decennial census, requires an opportunity to review the proposed questions.
 
The OMB Office of Information and Regulatory Affairs, under the Paperwork Reduction Act (and under the preceding Federal Reports Act), has the responsibility for designating a single data collection instrument for information wanted by two or more agencies. It also has the responsibility
under the same act for reviewing all questionnaires and other instruments for the collection of data from 10 or more respondents (see Appendix A). In addition, the courts sometimes become involved in interpreting laws and regulations that affect statistical agencies, as in a number of issues concerning data confidentiality and Freedom of Information Act requests and in the use of sampling in the population census.
 
The budgetary constraints on statistical agencies and OMB’s review of data collections are ongoing. Other pressures depend, in part at least, on the relations between a statistical agency and those who have supervisory or oversight functions. Agencies need to develop skills in communicating to oversight groups the need for statistical series and credibility in assessing the costs of statistical work. In turn, although it is standard practice for the secretary of a department or the head of an independent agency to have ultimate responsibility for all matters in the department or agency, the head of a statistical agency, for credibility, should be allowed full authority in professional and technical matters. For example, decisions to revise the methodology for calculating the consumer price index (CPI), the gross domestic product (GDP), and the new Supplemental Poverty Measure (SPM) 12 have been and are properly made by the relevant statistical agency heads or their designees.
 
Other aspects of independence underscore a statistical agency’s credibility and work to protect it from political and other undue external influence. Authority to release statistical information and accompanying materials (including press releases) without prior clearance for the statistical content by department policy officials is essential so that there is no opportunity for or perception of political manipulation of any of the information. 13 Authority for the statistical agency head and qualified staff to speak about the agency’s statistics before Congress, with congressional staff, and before public bodies is also important to maintain an agency’s standing.
 
When a statistical agency releases information publicly, a clear distinction should be made between the statistical information and any policy interpretations of it. Not even the appearance of manipulation for political purposes should be allowed. This essential requirement is one reason that statistical agencies are required by Statistical Policy Directive Number 3 (U.S. Office of Management and Budget, 1985) to adhere to predetermined schedules for the public release of key economic indicators and take steps to ensure that no person outside the agency has access to such indicators before the official release time. Statistical Policy Directive Number 4 (U.S. Office of Management and Budget, 2008) requires agencies to develop and publish schedules for release of other important social and economic indicators as well (see Appendix A). When an agency modifies a customary release schedule for statistical purposes, it should announce and explain the change as far in advance as possible.
 
Regarding press releases, Statistical Policy Directive Number 4 encourages statistical agencies to use them as a way to publicize and thereby expand the dissemination of data to the public. The directive explicitly states that “statistical press releases must be produced and issued by the statistical agency and must provide a policy-neutral description of the data.” Policy pronouncements must be issued separately by executive branch policy officials and not by the statistical agency, and “policy officials of the issuing department may review the draft statistical press release to ensure that it does not include policy pronouncements” but for no other reason.
Practice 3: Continual Development of More Useful Data
Introduction
Federal statistical agencies cannot be static. To provide information of continued relevance for public and policy use, they must continually anticipate data needs for future policy considerations and look for ways to develop data systems that can serve broad purposes. To improve the quality and timeliness of their information, they must keep abreast of methodological and technological advances and be prepared to implement new procedures in a timely manner. They must also continually seek ways to make their operations more efficient and cost-effective. Preparing for the future requires that agencies periodically assess the justification, scope, and frequency of existing data series, plan new data series as required, and be innovative and open in their consideration of ways to improve their programs. Because of the decentralized nature of the federal statistical system, innovation often requires cross-agency collaboration. Innovation also implies a willingness to reach out to a broad range of users to identify emerging needs and to implement different kinds of data collection efforts to answer different needs.
 
In essence, statistical agencies need to define their primary business as that of providing relevant, accurate, and timely statistical information rather than continuing long-standing data collection and estimation programs for their own sakes. Over time, this goal will likely require the use of new methods and data sources and new ways of combining information from multiple sources, including cross-sectional and longitudinal surveys, administrative records, and other sources. In considering data collection, estimation, and dissemination strategies for the future, statistical agencies must be mindful of tradeoffs among relevance, accuracy, timeliness, costs, and transparency. It will not usually be possible to maximize all five criteria at the same time, and, indeed, a major challenge for federal statistical agencies is that of continuing to produce high-quality statistics in the face of constrained budgets and growing user demands for relevance and timeliness.
Multiple Data Sources
Statistical agencies need to continuously think creatively about using multiple data sources in their statistical programs, including such strategies as the use of small-area estimation with auxiliary data to expand the value of surveys without the need for increased sample sizes, 14 or the use of administrative records or possibly Internet sources to supplement, calibrate, and even replace data that would otherwise be collected in a survey.15 Statistical agencies are already using these and other strategies to maximize the relevance and cost-effectiveness of one or more statistical programs. However, there is much more that can be done, including the further development of infrastructure and policies for the federal statistical system as a whole to facilitate cost-effective approaches to the design of statistical programs. Examples include standard protocols for such essential operations as acquiring administrative records from federal and state agencies, evaluating the completeness and quality of data sources that were not originally designed for statistical use, and synchronizing individually identifiable information among statistical agencies for statistical purposes.
The Role of Surveys
Many current statistical programs rely on well-established probability sampling methods that draw representative samples of a population, such as household members or business establishments, interview the sample units, and produce estimates that account for known errors in population coverage and missing data and have a quantifiable level of uncertainty from sampling variability. The probability sampling paradigm represented a quantum leap forward in providing cost-effective information on a variety of subjects when it was first introduced for federal surveys beginning in the late 1930s (see Duncan and Shelton, 1978). For example, no longer did everyone in the United States have to answer a long battery of questions in the decennial census, when the use of a separate “long-form” questionnaire administered to a sample of the population could produce reliable estimates for the nation and many subnational geographies (see National Research Council, 2010d).16 Surveys are now widely used throughout the federal government to collect information on a wide array of characteristics of individuals, households, business establishments, government agencies, and other organizations.
 
However, declining rates of survey response over the past few decades in the United States and other countries pose increasingly difficult challenges to containing the costs of data collection with traditional surveys in ways that do not risk compromising the quality of the data (see, e.g., Brick and Williams, 2013; de Leeuw and de Heer, 2002). 17 Survey researchers are actively seeking techniques to maintain and improve both the quality and the cost-effectiveness of surveys (see National Research Council, 2013b), which are and will remain important components of federal statistical programs. Yet the challenges to the survey paradigm make it critically important to consider how other data sources can be used to bolster the completeness, quality, and utility of estimates from statistical agency programs while containing costs.
Roles for Administrative Records
Administrative records include records of federal, state, and local government agencies that are developed and maintained to administer a government program. Examples include the various records maintained by the U.S. Social Security Administration of taxes collected from workers and benefits paid out to retirees and other beneficiaries; records maintained by state agencies of information provided by applicants for various assistance programs and payments to applicants deemed eligible; and property tax records of local governments.
 
Administrative records do not fit the sample survey paradigm, but they are not dissimilar from household or business censuses in that they are designed to capture information for all instances of a specified population (e.g., program beneficiaries) according to a set of rules that typically have a statutory basis. The challenge for a statistical agency is to determine not only the extent to which a particular administrative records system covers a relevant population and uses similar enough concepts for the agency’s purposes, but also the extent to which the information recorded is accurate according to the system’s rules. Like a population census, administrative records may have errors of omission and duplication or other types of erroneous enumeration, and the variables in the system may be of inconsistent accuracy (for example, payments to beneficiaries may be more accurate than information provided at the time of application regarding the beneficiary’s characteristics). The records may also be stored in formats that are not readily usable by a statistical agency, they may not be well documented, and they may not be provided on a timely basis. Acquiring the records may involve challenging negotiations with the custodial agency, and because the records are designed for administrative and not statistical purposes, their contents and formats are subject to change that is not under the control of the statistical agency.
 
Yet given that government program administrative records are rule based, it is possible with sufficient effort to understand their concepts and error properties. It is also possible to develop productive relationships with the custodial agency to make them easier to access and use. That the payoffs can be great is illustrated by many examples of current well-established uses of administrative records for statistical programs. One such example is the Census Bureau’s quinquennial economic censuses, which rely heavily on IRS tax records to obtain basic information for the nation’s millions of sole proprietor businesses. The availability of the IRS records for statistical purposes by the Census Bureau obviates the need for costly and burdensome surveys of these businesses.18 The IRS information is not without error, but survey responses would also have errors, and the cost and burden savings are enormous. Another example is the regular use of population estimates based on a prior census updated with administrative records of births and deaths and estimates of net immigration to control household survey responses for population undercoverage relative to the census. This use reduces the nonsampling error in the survey estimates for key population groups defined by age, gender, race, and ethnicity.19
 
There are many ways in which administrative records from federal, state, and even local government agencies could be used more extensively in federal statistical programs. As an example, a study by the Committee on National Statistics (CNSTAT) on retirement income statistics concluded that some of the information that is essential for analysis of savings and retirement decisions and the effect of medical care use and expenditures on retirement income security is most efficiently and accurately obtained from existing administrative records (National Research Council, 1997a). To be useful for estimation, this information (e.g., Social Security earnings histories, Medicare and Medicaid benefits) must be linked to individual data from panel surveys, which has been done to some extent in the Health and Retirement Study sponsored by the National Institute on Aging, the National Longitudinal Surveys sponsored by the Bureau of Labor Statistics, 20 and the Census Bureau’s Survey of Income and Program Participation. Similarly, linkage of employer and employment survey data with administrative records can provide enhanced analysis and modeling capability: a good example is the Census Bureau’s Longitudinal Employer-Household Dynamics Program (LEHD). 21
 
In another example, a CNSTAT study on reengineering the Census Bureau’s Survey of Income and Program Participation (SIPP) outlined a strategy for using administrative records to improve the survey’s income information, which, like other household surveys, suffers from substantial underreporting of many income sources (National Research Council, 2009e). The recommended strategy includes such elements as conducting regular, routinized comparisons of SIPP data with appropriate control totals from administrative records to identify problematic income sources and monitor improvement (or deterioration) in completeness of the data; incorporating administrative records in model-based imputations to replace older methods; and adjusting SIPP income data to match administrative records control totals. This strategy is more practical for federal records to which the Census Bureau already has access (e.g., Social Security payments) than for state records (e.g., Temporary Assistance to Needy Families benefits), but the development of standard protocols for data sharing could facilitate readier access to relevant state records. In the longer term, it could be cost-effective to use administrative records to replace selected questions in surveys such as SIPP. 22
 
Another method for using survey and administrative records data to improve data quality and relevance is in models that produce improved estimates for specific quantities of interest, such as small-area estimates of school-age children in poverty or of people with health insurance coverage, which use data from the ACS and administrative sources (see National Research Council, 2000c, 2000d). An important step forward would be if methods could be developed to use models to improve all or large parts of entire datasets.
 
In most uses of administrative data, consideration needs to be given to at least three factors: (1) upfront investments to facilitate the most effective approach to their acquisition and use, accompanied by estimates of the likely longer term cost savings, data quality improvements, or both that can be used to determine priorities for moving forward; (2) the means by which the confidentiality of linked or augmented data files can be protected while allowing access for research purposes (see National Research Council, 2005b); and (3) the protocols and criteria that can be followed to ensure full understanding by the statistical agency of the properties of a specific administrative records system (e.g., population coverage, coding error rates, frequency of updating). In addition, it is important for statistical agencies to develop protocols and procedures for cooperative working relationships with the custodians of administrative records to facilitate joint understanding of the data needed for statistical use. 23 Finally, it is vital for statistical agencies to maintain transparency by developing full documentation for users of the sources of the data provided, including the role played by administrative records, and the limitations (as well as the benefits) of the sources.
Roles for Nontraditional Data Sources
Statistical agencies are currently exploring the use of data sources in addition to surveys and administrative records that hold promise to improve the relevance, accuracy, and timeliness of federal statistics (see National Research Council, 2013a). These nontraditional data sources include, among others, data gleaned from relevant Internet websites and data obtained from the private sector (e.g., scanner data on consumer purchases). Often, these sources generate large volumes of data that require data mining and other computationally intensive techniques for extracting information (see National Research Council, 2008a, esp. App. H).
 
There are already instances of effective use of nontraditional data sources by statistical agencies, some of long standing. For example, the Economic Research Service in USDA obtains data from private vendors of expenditure data scanned by households from store receipts and has evaluated the quality of the data. 24 The National Center for Health Statistics, in its surveys of hospitals and other health care providers, obtains data from questionnaires, abstracts of samples of patient records, and providers’ medical care claim records. 25
 
Most nontraditional data sources present significant challenges to statistical agencies to evaluate the accuracy and error properties of the information. For example, harvesting website data to develop up-to-the-minute consumer price indexes 26 may offer significant timeliness and cost savings compared with traditional methods, but it is not clear how to adjust these data for consumer expenditures that occur off-line so that they accurately represent the universe of purchases. More generally, information that is taken from the Internet cannot usually be described or evaluated according to either a probability survey paradigm or a rules-based administrative records paradigm—for example, people who post items to sell on an auction website do not comprise any specified population. Another challenge is that statistical agencies lack control over the consistency of nontraditional data over time or among vendors or sites, so that deciding to rely heavily on such data sources carries high risks of compromising key time series if a vendor or site then ceases operation or there are marked changes in data content or population coverage.
 
Yet in an era when data users expect timeliness and when budgets are constrained, it is important that statistical agencies actively explore means by which nontraditional data sources can contribute to their programs. Such means could include augmenting information obtained from traditional sources, replacing information elements that have been obtained from traditional sources, and providing earlier estimates that are later benchmarked using traditional sources. Just as more and more surveys use multiple data collection modes (including the Internet, by mail, by telephone, and in person), so more statistical programs will likely benefit from using multiple data sources.
 
To garner acceptance for the use of multiple data sources, particularly newer, nontraditional sources with which users are less familiar, statistical agencies should take care to invest resources in documentation and user training and education. Agencies will likely need to “wall off ” data series that are derived from nontraditional sources by labeling them as experimental or for research use until their properties can be fully understood. If it is not possible to evaluate a nontraditional source sufficiently to establish its quality and suitability for inclusion in a statistical program, then a statistical agency should not use the data, although it may assist users by informing them of the problems with the source.
Integration and Synchronization of Data Across Agencies
Another way to improve data quality, develop new kinds of information, and increase cost-effectiveness is for statistical agencies that collect similar information to integrate their microdata records for specified statistical uses. One such cost-effective use is for a large survey to provide the sampling frame and additional content for a smaller, more specialized survey. Currently, the National Health Interview Survey of the National Center for Health Statistics serves this function for the Medical Expenditure Panel Survey of the Agency for Healthcare Quality and Research, as does the Census Bureau’s American Community Survey for the National Survey of College Graduates that the Bureau conducts for the National Center for Science and Engineering Statistics (see National Research Council, 2008d).
 
Another cost-effective use is to synchronize or harmonize similar data held by different agencies. An example is the business establishment lists maintained by the Bureau of Labor Statistics and the Census Bureau. The lists derive from different sources (state employment security records in the case of the BLS and a variety of sources, including federal income tax records, in the case of the Census Bureau), and research has demonstrated that synchronization of the lists would not only improve the accuracy of the information, but also increase the coverage of business establishments in the United States. Such synchronization would make it possible to develop more useful and accurate statistics on the nation’s economy while decreasing the reporting burden on business data providers (National Research Council, 2006b, 2007b).
 
Enactment of Subtitle B of the 2002 Confidential Information Protection and Statistical Efficiency Act (CIPSEA) was a major achievement, authorizing the synchronization of business data among the three principal statistical agencies that produce the nation’s key economic statistics BEA, BLS, and the Census Bureau. The first formal proposal for data synchronization under CIPSEA involved matching data from BEA’s international investment surveys with data from the Survey of Industrial Research and Development, which is now the Business Research, Development, and Innovation Survey (conducted by the Census Bureau for the NSF). The results helped BEA improve its survey sample frames and enabled the Census Bureau to identify companies that were not previously known to engage in research and development activities (U.S. Office of Management and Budget, 2004b:44–45). However, synchronization of business establishment lists between BLS and the Census Bureau cannot be accomplished at present because of the requirement in Title 26 of the U.S. Code that prohibits the Census Bureau from sharing with BLS (or BEA) any tax information of businesses or individuals that it has permission to acquire from the IRS, even for statistical purposes. 27
Longitudinal Data
The need to understand temporal changes in important social or economic events may call for the development of longitudinal surveys that track people, institutions, or firms over time. Developing longitudinal data (and general-purpose, repeated cross-sectional data, as well) usually requires much coordination with policy research agencies, other statistical agencies, and academic researchers. Longitudinal data may require more sophisticated methods for collection and analysis than data from repeated or one-time cross-sectional surveys. In addition, considerable time may be needed to produce useful data products for analyzing transitions and other dynamic characteristics of longitudinal samples (although production of cross-sectional products from longitudinal surveys need not take long). Yet data from longitudinal surveys are potentially very useful—sometimes, they are the only means to answer important policy questions: see, for example, National Research Council (1997a) on data needs to inform retirement income policy and National Research Council (2001b) on data needs to evaluate the effects of the 1996 welfare reform legislation.
 
Historically, because statistical agencies are oriented toward the mission of their particular department, the longitudinal surveys they developed (and cross-sectional data activities as well) typically focused on subject matter and population groups (or other entities) that the department serves. For example, separate datasets are available on health characteristics of infants and children, educational characteristics for children and teenagers, and work force characteristics for adults. Increasingly, however, agencies have considered surveys that follow individuals across such key transitions as from early childhood to school and from school to the labor force (National Research Council, 1998a; National Research Council and Institute of Medicine, 2004, 2008).
 
Examples of statistical agency surveys that are designed for analysis of some kinds of transitions include the Early Childhood Longitudinal Study (ECLS), sponsored by the National Center for Education Statistics in collaboration with other agencies, and the National Longitudinal Surveys of Youth (NLSY79, NLSY97), sponsored by BLS. The ECLS includes two cohorts of children, one of kindergartners in 1998 who were followed through 8th grade and another of babies born in 2001 who were followed through kindergarten. A new cohort of kindergartners was sampled in fall 2010 and will be followed through 5th grade. 28 The NLSY includes two cohorts, one of people aged 14–22 in 1979 and the other of people aged 12–17 in 1997, both of which are currently being interviewed every other year. 29
 
Other important longitudinal surveys are sponsored by research agencies—for example, the National Institute on Aging sponsors the Health and Retirement Study (HRS), and the National Institute of Child Health and Human Development sponsors the new National Children’s Study (NCS) (see National Research Council and Institute of Medicine, 2008). The HRS, which began in 1992, includes people aged 50 and older, who are interviewed every 2 years, with a new cohort introduced every 6 years. 30 The NCS plans to follow 100,000 children and their families from before birth through age 21; it is currently in a testing stage. 31 Finally, administrative records can be turned into longitudinal datasets that are useful for research, policy analysis, and program evaluation. For example, the National Center for Education Statistics is assisting states through the Statewide Longitudinal Data Systems (SLDS) program to develop datasets from administrative records that follow school children through primary and secondary education and even into higher education and the workforce. 32
Operational Methods
It is important for statistical agencies to be innovative in the methods used for data collection, processing, estimation, analysis, and dissemination. Agencies need to investigate new or modified methods that have the potential to improve the accuracy and timeliness of their data and the efficiency of their operations. Careful evaluation of new methods is required to assess their benefits and costs in comparison with current methods and to determine effective implementation strategies, including the development of methods for bridging time series before and after a change in procedures.
 
For example, experience with the use of computer-assisted interviewing techniques, which are widely used for survey data collection, has identified their benefits. It has also identified challenges for the timely provision of data and documentation that require continued research to develop solutions that maximize the gains from these techniques (see National Research Council, 2003e). Currently, statistical agencies are exploring how best to use “paradata,” that is, data about the process by which the information from a survey was generated, such as the times of day that interviews were conducted, how long they took, how many contacts were attempted with sample cases, and similar data. Using paradata together with cost information for various survey processes offers a statistical agency the potential for optimizing the costs and timeliness of data collection and estimation and the accuracy of the survey results—for example, by varying contact modes depending on respondent characteristics (see National Research Council, 2013b).
 
Statistical agencies have turned to the Internet as a standard vehicle for data dissemination and are increasingly using it as a means of data collection. Internet dissemination facilitates the timely availability of data to a broad audience and provides a valuable tool for users to learn of related datasets from other agencies. However, it poses challenges in such areas as the best ways to provide information on the various dimensions of data quality, such as measurement error and sampling error, and appropriate use of the data to an audience that spans a wide range of analytical skills and understanding (see National Research Council, 2012b).
 
Internet data collection offers opportunities to reduce costs in comparison with other survey modes and to reduce errors by incorporating automatic edits, prompts, and other features. However, it also poses new challenges in such areas as sample design, questionnaire design, and protecting data confidentiality, and it requires careful evaluation of the effects on the quality of responses in comparison with traditional data collection modes (telephone, mail, personal interview). Yet even as work is ongoing on meeting these challenges, population censuses around the world, federal business surveys, and other surveys are using the Internet as one data collection mode to reduce costs and facilitate response (see National Research Council, 2010d, esp. Part II, App. B, on Internet use in population censuses).
Practice 4: Openness About Sources and Limitations of the Data Provided
A critically important means to instill credibility and trust among data users and data providers is for an agency to operate in an open and fully transparent manner with regard to the sources and the limitations of its data. Openness requires that an agency provide a detailed description of its data with acknowledgment of any uncertainty and a description of the methods used and assumptions made. Agencies should provide to users reliable indications of the kinds and amounts of statistical error to which the data are subject (see Brackstone, 1999; Federal Committee on Statistical Methodology, 2001a; see also President’s Commission on Federal Statistics, 1971).
 
Some statistical agencies developed detailed quality profiles for some of their major series, such as those produced for the American Housing Survey (Chakrabarty, 1996), the Residential Energy Consumption Survey (Energy Information Administration, 1996), the Schools and Staffing Survey (Kalton et al., 2000), and the Survey of Income and Program Participation (U.S. Census Bureau, 1998). Earlier, the Federal Committee on Statistical Methodology (1978c) developed a quality profile for employment as measured in the Current Population Survey. These profiles have proved helpful to experienced users and agency personnel responsible for the design and operation of major surveys and data series (see National Research Council, 1993a, 2007c). They were staff-intensive to produce, however, and so have not often been updated. Agencies should pursue creative use of the Internet as a means for easier maintenance and updating of quality profile-type information (e.g., separate web pages for major types of error). 33
 
Openness about data limitations requires much more than providing estimates of sampling error. In addition to a discussion of aspects that statisticians characterize as nonsampling errors—such as coverage errors, nonresponse, measurement errors, and processing errors—it is valuable to have a description of the concepts used and how they relate to the major uses of the data. Descriptions of the shortcomings of and problems with the data should be provided in sufficient detail to permit a user to take them into account in analysis and interpretation. Descriptions of how the data relate to similar data collected by other agencies should also be provided, particularly when the estimates from two or more series differ significantly in ways that may have policy implications.
 
Openness means that a statistical agency should describe how decisions on methods and procedures were made for a data collection program. It is important to be open about research conducted on methods and data and other factors that were weighed in such decisions.
 
Openness also means that, when mistakes are discovered after statistics are released, the agency has an obligation to issue corrections publicly and in a timely manner. The agency should use not only the same dissemination vehicles to announce corrections that it used to release the original statistics, but should also use additional vehicles, as appropriate, to alert the widest possible audience of current and future users of the corrections in the information.
 
In summary, agencies should make an effort to provide information on the quality, limitations, and appropriate use of their data that is as frank and complete as possible. Such information, which is sometimes termed “metadata,” should be made available in ways that are easy for users to access and understand, recognizing that users differ in their level of understanding of statistical data (see National Research Council, 1993a, 1997b, 2007c). Agencies need to work to educate users that all data contain some uncertainty and error, which does not mean the data are wrong, but that they should be used with understanding of the possible limitations.
 
The Information Quality Act of 2000 required all federal agencies to develop written guidelines for maintaining and documenting the quality of their information programs and activities. Using a framework developed collaboratively by the members of the Interagency Council on Statistical Policy (U.S. Departments of Agriculture et al., 2002), individual statistical agencies have developed quality guidelines for their own data collection programs, which are available on the Internet (see Practice 9 and Appendix A).
Practice 5: Wide Dissemination of Data
A statistical agency must have vigorous and well-planned dissemination programs to get information into the hands of users who need it on a timely basis. Planning should be undertaken from the viewpoint that the public has contributed the data elements, has paid for the data collection and processing, and should in return have the information accessible in ways that make it as useful as possible to the largest number of users.
 
A good dissemination program provides data to users in forms that are suited to their needs. Data release of aggregate statistics may take the form of regularly updated time series, cross-tabulations of aggregated characteristics of respondents, analytical reports, and brief reports of key findings. Such products should be made accessible through an agency’s Internet website, which should also make available more detailed tabulations in formats that are downloadable from the website. Agencies should take care in designing their websites to make it as easy as possible for users to locate and access information. They should also explore ways to provide data to developers of applications for smartphones and similar media (see National Research Council, 2012b).
 
Yet another form of dissemination involves access to microdata files, which make it possible to conduct in-depth research in ways that are not possible with aggregate data. Public-use microdata files can be developed for general release. Such files contain data for individual respondents that have been processed to protect confidentiality by deleting, aggregating, or modifying any information that might permit individual identification. Alternatively, an agency can provide or arrange for a facility on the Internet to allow researchers to analyze restricted microdata (that is, data that have not been processed for general release) to suit their purposes, with safeguards so that the researcher is not seeing the actual records and cannot obtain any output, such as too-detailed tabulations, that could identify individual respondents. 34 Another alternative is to grant a license to individual researchers to analyze restricted microdata at their own sites by agreeing to follow strict procedures for protecting confidentiality and accepting liability for penalties if confidentiality is breached. A fourth alternative is to allow researchers to analyze restricted microdata at secure sites maintained by a statistical agency, such as one of the Census Bureau’s Research Data Centers located at several universities and research organizations around the country or the National Center for Health Statistics’ Research Data Center at its headquarters. Agencies should consider all forms of dissemination in order to gain the most use of their data consistent with protecting the confidentiality of responses (see Doyle et al., 2001; National Research Council, 2005b).
 
The stunning improvements over the past three decades in computing speed, power, and storage capacity, the growing availability of information from a wide range of public and private sources on the Internet, and the increasing richness of statistical agency data collections have increased the risk that individually identifiable information can be obtained (see National Research Council, 2003d:Ch. 5, 2005b). Statistical agencies must be vigilant in their efforts to protect against the increased threats to disclosure from their summary data and microdata products while honoring their obligation to be proactive in seeking ways to provide data to users. When statistical data are not disseminated in useful forms, there is a loss to the public, not only of wasted taxpayer dollars, but also of research findings that could have informed public policy and served other important societal purposes.
 
A good dissemination program for statistical data uses a variety of channels to inform the broadest possible audience of potential users about available data products and how to obtain them. Such channels may include providing direct access to aggregated data on the Internet, depositing data products in libraries, establishing a network of centers to work with users (such as the Census Bureau’s state data centers), holding exhibits and making presentations at conferences, and maintaining lists of individuals and organizations to notify of new data. Agencies should also arrange for archiving of data with the National Archives and Records Administration (NARA) and other data archives, as appropriate, so that data are available for historical research in future years with suitable protections for confidentiality.
 
An effective dissemination program provides not only the data, but also information about the strengths and weaknesses of the data in ways that can be comprehended by diverse audiences. Information about the limitations of the data should be included in every form of data release, whether in a printed report, on a computer-readable data file, or on the Internet (for useful measures of data quality to report, see Federal Committee on Statistical Methodology [2001a]; National Research Council [2007b]).
 
On occasion, the objective of presenting the most accurate data possible may conflict with the needs of users for the information. The tension between frequency and promptness of release and accuracy should be explicitly considered. When concerns for timeliness prompt the release of preliminary estimates (as is done for some economic indicators), consideration should be given to the frequency of revisions and the mode of presentation of revised figures from the point of view of the users as well as the issuers of the data. Agencies that release preliminary estimates must educate the public about differences among preliminary, revised, and final estimates.
Practice 6: Cooperation with Data Users
Users of federal statistical data span a broad spectrum of interests and needs. They include policy makers, planners, government program administrators, members of Congress and their staffs, and researchers in federal agencies, state and local governments, the business sector, and academia. They also include activists, citizens, students, and media representatives. An effective statistical agency endeavors to learn about its data users and to obtain input from them on the agency’s statistical programs, including learning what data they use or want, how they use data, and for what purposes.
 
The needs of users can be explored by forming advisory committees, holding focus groups, analyzing requests and Internet activity, or undertaking formal surveys of users. The task requires continual alertness to the changing composition and needs of users and of potential users. An agency should cooperate with professional associations, institutes, universities, and scholars in the relevant fields to determine the needs of the research community and obtain their insight on potential uses. An agency should work with relevant associations and other organizations to determine the needs of business and industry for its data, as well as with user groups that are formed around federal statistics generally or particular statistical programs.
 
Within the limitations of its confidentiality procedures, as noted above, an agency should seek to provide maximum access to its data, including making the data available to external researchers for secondary analysis (National Research Council, 1985c, 2005b). Having data accessible for a wide range of analyses increases the return on the investment in data collection and provides support for an agency’s program. Once statistical data are made public, they may be used in numerous ways not originally envisaged. An agency should attempt to monitor the major uses of its data as part of its efforts to keep abreast of user needs.
 
Researchers and other users of data frequently request data from statistical agencies for specific purposes. The agency should have procedures in place for referring users to the appropriate professional staff who can understand the user’s purposes and needs and who have a thorough knowledge of the agency’s data. Statistical agencies should view these services as a part of their dissemination activities.
 
Ensuring equal access requires avoiding release of data to selected individuals or organizations in advance of other users. Agencies that prepare special tabulations of their data on request for external groups must be alert to the proposed uses. If the data are to be used in court cases, administrative proceedings, or collective bargaining negotiations, an agency should have an explicit and publicly known policy for ensuring that all sides may receive the special tabulations, regardless of which side requested them or paid the cost of the tabulation.
Practice 7: Respect for the Privacy and Autonomy of Data Providers
Introduction
Clear policies and effective procedures for respecting the privacy of respondents and, more broadly, protecting the rights and respecting the autonomy of human research participants are critical to maintaining the quality and comprehensiveness of the data that federal statistical agencies provide to policy makers and the public. Part of the challenge for statistical agencies is to develop effective means of communicating not only the agency’s protection procedures and policies, but also the importance of the data being collected for the public good.
Respecting Privacy
To promote trust and encourage accurate response from data providers, it is important that statistical agencies respect their privacy. When data providers are asked to participate in a survey, they should be told whether the survey is mandatory or voluntary, how the data will be used, and who will have access to the data. In the case of voluntary surveys, information on these matters is necessary in order for data providers to give their informed consent to participate (see National Research Council, 2003d, on regulations and procedures for informed consent).
 
Respondents invest time and effort in replying to surveys. The amount of effort or burden varies considerably from survey to survey, depending on such factors as the complexity of the requested information. Statistical agencies should attempt to minimize such effort, to the extent possible, by using concepts and definitions that fit respondents’ common understanding; by simplifying questionnaires; by minimizing the intrusiveness of questions and explaining why questions that may be perceived as intrusive are needed for important purposes; by allowing alternative modes of response when appropriate (e.g., via the Internet); and by using administrative records or other data sources, if they are sufficiently complete and accurate to provide some or all of the needed information. In surveys of businesses or other institutions, agencies should seek innovative ways to obtain information from the institution’s records and minimize the need for respondents to reprocess and reclassify information. It is also the responsibility of agencies to use qualified, well-trained interviewers. Respondents should be informed of the likely duration of a survey interview and, if the survey involves more than one interview, how many times they will be contacted over the life of the survey. This information is particularly important when respondents are asked to cooperate in extensive interviews, search for records, or participate in longitudinal surveys.
 
Ways in which participation in surveys can be made easier for respondents and result in more accurate data can be explored by such means as focus group discussions or surveys. Many agencies apply the principles of cognitive psychology to questionnaire design, not only to make the resulting data more accurate, but also to make the time and effort of respondents more efficient (National Research Council, 1984). Some agencies thank respondents for their cooperation by providing them with summaries of the information after the survey is compiled.
 
It is possible that increasing concerns about privacy are contributing to observed declines in survey response rates. In a time when individuals are inundated with requests for information from public and private sources, when there are documented instances of identity theft and other abuses of confidential information on the Internet, when individual information is being used for terrorism-related investigatory or law enforcement purposes, it is not surprising that some people object to responding to censuses and surveys, even when the questions appear noninvasive and the data are collected for statistical purposes under a pledge of confidentiality. 35 The E-Government Act of 2002 requires agencies to develop privacy impact assessments (PIAs) whenever “initiating a new collection of information [that] includes any information in an identifiable form.” The purpose of a privacy impact assessment is to ensure there is no collection, storage, access, use, or dissemination of identifiable information that  is not both needed and permitted. In response, statistical agencies have begun conducting and releasing PIAs for statistical programs and, in the process, rethinking how to respect individual privacy in order to maintain trust with data providers (see Appendix A).
 
Statistical agencies should devote resources to understanding the privacy and confidentiality concerns of individuals (and organizations). They should also devote resources to devising effective strategies for communicating privacy and confidentiality policies and practices to respondents. Such strategies appear to be more necessary—and more challenging—than ever before.
 
Finally, a reason that respondents reply to statistical surveys is that they believe that their answers will be useful to the government or to society generally. Statistical agencies should respect this contribution by compiling the data and making them accessible to users in convenient forms. A statistical agency has an obligation to publish statistical information from the data it has collected unless it finds the results invalid.
Protecting and Respecting the Autonomy of Human Research Participants
Collecting data from individuals for research purposes with federal funds falls under a series of regulations, principles, and best practices that the federal government has developed over a period of 50 years for research involving human participants (see National Research Council, 2003d). The pertinent regulations, which have been adopted by 10 departments and 7 agencies, are known as the “Common Rule” (45 CFR 46). The Common Rule regulations require that researchers adequately protect the privacy of human participants and maintain the confidentiality of data collected from them, minimize the risks to participants from the data collection and analysis, select participants equitably with regard to the benefits and risks of the research, and seek the informed consent of individuals to participate (or not) in the research. Under the regulations, most federally funded research involving human participants must be reviewed by an independent institutional review board (IRB) to determine that the design meets the ethical requirements for protection. 36
 
Some federal statistical agencies consider certain of their information collections to be subject to IRB review. Whether or not a given information collection is subject to formal IRB review, a statistical agency should strive to incorporate the spirit of the Common Rule regulations in the design and operation of its programs involving data collection from individuals. When an agency is required to obtain IRB approval for data collection, it should work proactively with the IRB to determine how best to apply the regulations in ways that do not unnecessarily inhibit participant responses. For example, implied consent is typically used for mail and telephone surveys of the general population; in these situations, written documentation tends not to provide any added protection to the respondent, and could reduce participation. An effective statistical agency will seek ways—such as sending an advance letter—to furnish information to potential respondents that will help them make an informed decision about whether to participate. Such information should include the planned uses of the data and their benefits to individuals and the public.
Practice 8: Protection of the Confidentiality of Data Providers’ Information
Data providers must believe that the data they give to a statistical agency will not be used by the agency to harm them. For statistical data collection programs, protecting the confidentiality of individual responses is considered essential to encourage high response rates and the accuracy of the responses. (For reviews of research on the relationship of concerns about confidentiality protection to response rates, see Hillygus et al., 2006; National Research Council, 2004d:Ch. 4.) Furthermore, if participants have been assured of confidentiality, then under federal policy for the protection of human subjects, disclosure of identifiable information about them would violate the principle of respect for persons even if the information is not sensitive and would not result in any social, economic, legal, or other harm (National Research Council, 2003d:Ch. 5).
 
Historically, some agencies have had provisions for promising respondent confidentiality written into their authorizing or enabling legislation (e.g., for the U.S. Census Bureau, Title 13 of the U.S. Code, first enacted in 1929, and for the National Agricultural Statistics Service, various provisions in Title 7 of the U.S. Code). However, other agencies (e.g., the Bureau of Labor Statistics) relied on strong statements of policy, legal precedents in court cases, or customary practices (see Gates, 2012; Norwood, 1995). These latter agencies risked having their policies overturned by judicial interpretations of legislation or executive decisions that might have required the agency to disclose identifiable data collected under a pledge of confidentiality (for an example involving the Energy Information Administration, see National Research Council, 1993b:185–186).
 
To give additional weight and stature to policies that statistical agencies had pursued for decades, OMB issued a Federal Statistical Confidentiality Order on June 27, 1997. This order assured respondents who provided statistical information to specified agencies that their responses would be held in confidence and would not be used against them in any government action “unless otherwise compelled by law” (U.S. Office of Management and Budget, 1997; see also Appendix A).
 
The Confidential Information Protection and Statistical Efficiency Act (CIPSEA) became law in 2002, as Title V of the E-Government Act of 2002. Subtitle A of CIPSEA provides a statutory basis for protecting the confidentiality of all federal data collected for statistical purposes under a confidentiality pledge, including but not limited to data collected by statistical agencies. Subtitle A places strict limits on the disclosure of individually identified information collected with a pledge of confidentiality; such disclosure to persons other than the employees or agents of the agency collecting the data can occur only with the informed consent of the respondent and the authorization of the agency head and only when the disclosure is not prohibited by any other law (e.g., Title 13 of the U.S. Code). It also provides penalties for employees or agents who knowingly or willfully disclose statistical information (up to 5 years in prison, up to $250,000 in fines, or both). OMB issued guidance in 2007 to assist agencies in implementing Subtitle A of CIPSEA (U.S. Office of Management and Budget, 2007; see also Appendix A).
 
Although confidentiality protection for statistical data is now on a much firmer legal footing across the federal government than prior to CIPSEA, there is an exception for some data from the National Center for Education Statistics. The USA PATRIOT Act of 2001, Section 508, amended the National Center for Education Statistics Act of 1994 to allow the U.S. attorney general (or an assistant attorney general) to apply to a court to obtain any “reports, records, and information (including individually identifiable information) in the possession” of NCES that are considered relevant to an authorized investigation or prosecution of domestic or international terrorism. Section 508 also removed the penalties for NCES employees who furnish individual records under this section. This section has not been invoked, and its possible effect on survey response rates has not been tested, but it is a type of language that is not helpful for the mission of statistical agencies and their need to be independent of undue political interference.
 
Statistical agencies continually strive to avoid inadvertent disclosure of confidential information in disseminating data. The widespread dissemination of statistical data through the Internet has heightened attention by agencies to ensuring that effective safeguards to protect confidential information are in place. Risks are increased when data for small groups are tabulated, when the same data are tabulated in a variety of ways, or when public-use microdata files (samples of records for unidentified individuals or units) are released with highly detailed content. Longitudinal surveys, for example, particularly newer ones, typically have richly detailed content for multiple domains (e.g., health, education, labor force participation) or multiple respondents (e.g., parents, students, teachers) or both. Risks may also be increased when surveys include linked administrative data or collect biomarkers from blood samples or other physiological measures (National Research Council, 2001a, 2010b).
 
Because of the disclosure risks associated with detailed tabulations and rich public-use microdata files, there is always a tension between the desire to safeguard confidentiality and the desire to provide public access to data. This dilemma is an important one to federal statistical agencies, and it has stimulated ongoing efforts to develop new statistical and administrative procedures to safeguard confidentiality while permitting more extensive access. An effective federal statistical agency will exercise judgment in determining which of these procedures are best suited to its requirements to serve data users while protecting confidentiality. 37
 
Finally, there is a tension between safeguarding confidentiality and departmental initiatives to consolidate data processing, storage, and maintenance as a way to satisfy requirements of the Federal Information Security Management Act (FISMA), which is Title III of the E-Government Act of 2002 (see Appendix A). FISMA is intended to bolster computer and network security in the federal government. An effective statistical agency will work with its department on approaches to computer security that recognize the need for the statistical agency to control the processing, storage, and maintenance of data that it collects under a pledge of confidentiality and for which it provides an assurance that such data will be not accessible for other departmental purposes, such as regulation or enforcement.
Practice 9: Commitment to Quality and Professional Standards of Practice
The best guarantee of high-quality data is a strong professional staff that includes experts in the subject-matter fields covered by the agency’s program, experts in statistical methods and techniques, and experts in data collection, processing, and other operations. A major function of an agency’s leadership is to strike a balance among these groups and promote working relationships that make the agency’s program as productive  as possible, with each group of experts contributing to the work of the others.
 
An effective statistical agency devotes resources to developing, implementing, and inculcating standards for data quality and professional practice. Although a long-standing culture of data quality contributes to professional practice, an agency should also seek to develop and document standards through an explicit process. The existence of explicit standards and guidelines, regularly reviewed and updated, facilitates training of new in-house staff and contractors’ staffs. The OMB document Standards and Guidelines for Statistical Surveys (U.S. Office of Management and Budget, 2006b) is helpful in that it covers every aspect of a survey from planning through data release (see also U.S. Office of Management and Budget, 2006a, and Appendix A). 38 It recommends that agencies develop additional, more detailed standards that focus on their specific statistical activities .39
 
An effective statistical agency keeps up to date on developments in theory and practice that may be relevant to its program, such as new techniques for imputing missing data (see, e.g., National Research Council, 2004d:App. F, 2010g). An effective agency is also alert to changes in the economy or in society that may call for changes in the concepts or methods used in particular datasets. 40 Yet the need for change often conflicts with the need for comparability with past data series, and this issue can easily dominate consideration of proposals for change. Agencies have the responsibility to manage this conflict by initiating more relevant data series or revising existing series to improve quality while providing information to compare old and new series, such as was done when the BLS revised the treatment of owner-occupied housing in the CPI. 41
 
To ensure the quality of its data collection programs and reports, an effective statistical agency has mechanisms and processes for obtaining both inside and outside review of such aspects as the soundness of the data collection and estimation methods and the completeness of the documentation of the methods used and the error properties of the data. For individual publications and reports, formal processes are needed that incorporate review by agency technical experts and, as appropriate, by technical experts in other agencies and outside the government. (See Appendix A for a description of recent OMB guidelines for peer review of scientific information; reviews at a program or agency-wide level are considered under Practice 12.)
Practice 10: An Active Research Program
Substantive Research and Analysis
A statistical agency should include staff with responsibility for conducting objective substantive analyses of the data that the agency compiles, such as analyses that assess trends over time or compare population groups:
 
• Agency analysts are in a position to understand the need for and purposes of the data from a survey or other data collection program and know how the statistics will be used. Such information has to be available to the agency and understood thoroughly if the survey design is to produce the data required.
• Those involved in analysis can best articulate the concepts that should form the basic framework of a statistical series. Agency analysts are well situated to understand and transmit the views of external users and researchers; at the same time, close working relationships between analysts and data producers are needed for the translation of the conceptual framework into the design and operation of the survey or other data collection program.
• Agency analysts have access to the complete microdata and so are in a better position than analysts outside the agency to understand and describe the limitations of the data for analytic purposes and to identify errors or shortcomings in the data that can lead to subsequent improvements.
• Substantive research by analysts on an agency’s staff will have credibility because of the agency’s commitment to openness about the data provided and maintaining independence from political influence.
• Substantive research by analysts on an agency’s staff can assist in formulating the agency’s data program, suggesting changes in priorities, concepts, and needs for new data or discontinuance of outmoded or littleused series.
 
As with descriptive analyses provided by an agency, substantive analyses should be designed to be relevant to policy by addressing topics of public interest and concern. However, such analyses should not include positions on policy options or be designed to reflect any particular policy  agenda. These issues are discussed in Martin (1981), Norwood (1975), and Triplett (1991).
Research on Methodology and Operations
For statistical agencies to be innovative in methods for data collection, analysis, and dissemination, research on methodology and operational procedures must be ongoing. Methodological research may be directed toward improving survey design, measuring error, and, when possible, reducing error from such sources as nonresponse and reporting errors. Other important topics for research include reducing the time and effort asked of respondents, evaluating the best mix of interview modes (e.g., Internet, mail, telephone, personal interview) to cope with increasing nonresponse rates due to such phenomena as cell-phone-only households, developing new and improved summary measures and estimation techniques, and developing innovative statistical methods for confidentiality protection. Research on operational procedures may be directed toward facilitating data collection in the field, improving the efficiency and reproducibility of data capture and processing, and enhancing the usability of Internet-based data dissemination systems.
 
Many of the current practices in statistical agencies were developed through research they conducted or obtained from other agencies. Federal statistical agencies, frequently in partnership with academic researchers, pioneered the applications of statistical probability sampling, the national economic accounts, input-output models, and other analytic methods. The U.S. Census Bureau pioneered the use of computers for processing the census, and research on data collection, processing, and dissemination operations continues to lead to creative uses of automated procedures and equipment in these areas. Several federal statistical agencies sponsor research using academic principles of cognitive psychology to improve the design of questionnaires (see National Research Council, 1984), the clarity of data presentation, and the ease of use of electronic data collection and dissemination tools such as the Internet. The history of the statistical agencies has shown repeatedly that methodological and operations research can lead to large productivity gains in statistical activities at relatively low cost.
 
An effective statistical agency actively partners with the academic community for methodological research. It also seeks out academic and industry expertise for improving data collection, processing, and dissemination operations. For example, a statistical agency can learn techniques and best practices for improving software development processes from computer scientists (see National Research Council, 2003e, 2004c).
Research on Policy Uses
Much more needs to be known about how statistics are actually used in the policy-making process, both inside and outside the government. Research about how the information produced by a statistical agency is used in practice should contribute to future improvements in the design, concepts, and format of data products. For example, public-use files of statistical microdata were developed in response to the growing analytic needs of government and academic researchers.
 
Gaining an understanding of the variety of uses and users of an agency’s data is only a first step. More in-depth research on the policy uses of an agency’s information might, for example, explore the use of data in microimulation or other economic models, or go further to examine how the information from such models and other sources is used in decision making (see National Research Council, 1991a, 1991b, 1997a, 2000b, 2001b, 2003a, 2010e).
Practice 11: Professional Advancement of Staff
An effective federal statistical agency has personnel policies that encourage the development and retention of a strong professional staff who are committed to the highest standards of quality work for their agency and in collaboration with other agencies. There are several key elements of such a policy:
 
• The required levels of technical and professional qualifications for positions in the agency are identified, and the agency adheres to these requirements in recruitment and professional development of staff. Position requirements take account of the different kinds of technical and other skills, such as supervisory skills, that are necessary for an agency to have a full range of qualified staff, including not only statisticians, but also experts in relevant subject-matter areas, data collection, processing, and dissemination processes, and management of complex, technical operations.
• Continuing technical education and training, appropriate to the needs of their positions, is provided to staff through in-house training programs and opportunities for external education and training.
• Position responsibilities are structured to ensure that staff have the opportunity to participate, in ways appropriate to their experience and expertise, in research and development activities to improve data quality and cost-effectiveness of agency operations.
• Professional activities, such as publishing in refereed journals and presentations at conferences, are encouraged and recognized, including presentations of technical work in progress with appropriate disclaimers. Participation in relevant statistical and other scientific associations, including leadership positions, is encouraged to promote interactions with researchers and methodologists in other organizations that advance the state of the art. Such participation is also a mechanism for disseminating information about an agency’s programs, including the sources and limitations of the data provided. Guidance from the Office of Science and Technology Policy issued in 2010 stresses the importance of participation in professional activities as a means of ensuring a culture of scientific integrity in federal agencies (see Appendix A).
• Interaction with other professionals inside and outside the agency is fostered through opportunities to participate in technical advisory committee meetings, establish and be active in listservs, blogs, and wikis that take advantage of Internet technology to foster informal exchanges on technical matters, supervise contract research and research consultants on substantive matters, interact with visiting fellows and staff detailed from other agencies, take assignments with other relevant statistical, policy, or research organizations, and regularly receive new assignments within the agency.
• Participation in cross-agency collaboration efforts, such as the Federal Committee on Statistical Methodology and its subcommittees, is supported. Such participation not only benefits the professional staff of an agency, but also contributes to improving the work of the statistical system as a whole.
• Accomplishment is rewarded by appropriate recognition and by affording opportunities for further professional development. The prestige and credibility of a statistical agency is enhanced by the professional visibility of its staff, which may include establishing high-level nonmanagement positions for highly qualified technical experts.
 
An effective statistical agency considers carefully the costs and benefits—both monetary and nonmonetary—of using contractor organizations, not only for data collection, as most agencies do, but also to supplement in-house staff in other areas.42 Outsourcing can have benefits, such as: providing experts in areas in which the agency is unlikely to be able to attract highly qualified in-house staff (e.g., some information technology functions), enabling an agency to handle an increase in its workload that is expected to be temporary or that requires specialized skills, and allowing an agency to learn from best industry practices. However, outsourcing can also have costs, including that agency staff become primarily contract managers and less qualified as technical experts and leaders in their fields. An effective statistical agency maintains and develops a sufficiently large number of in-house staff, including mathematical statisticians, survey researchers, and subject-matter specialists, who are qualified to analyze the agency’s data and to plan, design, carry out, and evaluate its core operations so that the agency maintains the integrity of its data and its credibility in planning and fulfilling its mission. Statistical agencies should also maintain and develop staff with the expertise necessary for effective technical and administrative management of contractor resources.
 
An effective statistical agency has policies and practices to instill the highest possible commitment to professional ethics among its staff, as well as procedures for monitoring contractor compliance with ethical standards. When an agency comes under pressure to act against its principles—for example, if it is asked to disclose confidential information for an enforcement purpose or to support an inaccurate interpretation of its data—it must be able to rely on its staff to resist such actions as contrary to the ethical principles of their profession. An effective agency ensures that its staff are aware of and have access to such statements of professional practice as the guidelines published by the American Statistical Association (1999) and the International Statistical Institute (1985), as well as to the agency’s own statements about protection of confidentiality, respect for privacy, standards for data quality, and similar matters. It endeavors in other ways to ensure that its staff are fully cognizant of the ethics that must guide their actions in order for the agency to maintain its credibility as a source of objective, reliable information for use by all.
Practice 12: A Strong Internal and External Evaluation Program
Introduction
Statistical agencies that fully follow such practices as continual development of more useful data, openness about sources and limitations of the data provided, wide dissemination of data, commitment to quality and professional standards of practice, and an active research program will likely be in a good position to make continuous assessments of and improvements in the relevance and quality of their data collection systems. Yet even the best functioning agencies will benefit from an explicit program of internal and independent external evaluations, which frequently offer fresh perspectives. Such evaluations need to address not only specific agency programs, but also the agency’s portfolio of programs considered as a whole.
Evaluating Quality
Evaluation of data quality for a continuing survey or other kind of data collection program begins with regular monitoring of quality indicators that are readily available to users. For surveys, such monitoring includes unit and item response rates, population coverage rates, and information on sampling error, such as coefficients of variation. In addition, in-depth assessment of quality on a wide range of dimensions—including sampling and nonsampling errors across time and among population groups and geographic areas—needs to be undertaken on a periodic basis and the results made public (National Research Council, 2007c).
 
Research on methods to improve data quality may cover such areas as alternative methods for imputing values for missing data and alternative question wordings, using cognitive methods, to reduce respondent report ing errors. Methods for such research may include the use of “methods panels” (small samples of respondents with whom experiments are conducted by using alternative procedures and questionnaires), matching with administrative records, simulations of sensitivity to alternative procedures, and the like. The goal of the research is the development of feasible, costeffective improved procedures for implementation.
 
In ongoing programs for which it is disruptive to implement improvements on a continuing basis, a common practice is to undertake major research and development activities at intervals of 5 or 10 years or longer Agencies should ensure, however, that the intervals between major research and development activities do not become so long that data collection programs deteriorate in relevance, quality, and efficiency.
 
Regular, well-designed program evaluations, with adequate budget support, are key to ensuring that data collection programs do not deteriorate. Having a set schedule for research and development efforts will enable data collection managers to ensure that the quality and usefulness of their data are maintained and help prevent the locking into place of increasingly less optimal procedures over time.
Evaluating Relevance
In addition to quality, it is important to assess the relevance of an agency’s data collection programs. The question in this instance is whether the agency is “doing the right thing” in contrast to whether the agency is “doing things right.” Relevance should be assessed not only for particular programs or closely related sets of programs, but also for an agency’s complete portfolio in order to assist it in making the best choices among program priorities given the available resources.
 
Keeping in close touch with stakeholders and important user constituencies—through such means as regular meetings, workshops, conferences, and other activities—is important to ensuring relevance. Customer surveys can be helpful on some aspects of relevance, although they typically provide only gross indicators of customer satisfaction, usually with regard to timeliness and ease of use of data products. As discussed in the next section, including other federal statistical colleagues in this communication, both as users and as collaborators, can also be valuable.
 
Statistical agencies commonly find that it is difficult to discontinue or scale back a particular data series, even when it has largely outlived its usefulness relative to other series, because of objections by users who have become accustomed to it. In the face of limited resources, however, discontinuing a series is preferable to across-the-board cuts in all programs, which would reduce the accuracy and usefulness of both the more relevant and less relevant data series. Regular internal and external reviews can help an agency not only reassess its priorities, but also develop the justification and support for changes to its portfolio.
Types of Reviews
Regular program reviews should include a mixture of internal and external evaluation. Agency staff should set goals and timetables for internal evaluations, which should involve staff who do not regularly work on the program under review. Independent external evaluations should also be conducted on a regular basis, the frequency of which should depend on the importance of the data and on how quickly changes in such factors as respondent behavior and data collection technology may adversely affect a program. In a world in which people and organizations appear increasingly less willing to respond to surveys, it becomes increasingly urgent to continually monitor response and have more frequent evaluations than in a more stable environment. In addition to program evaluations, agencies should seek outside reviews to examine priorities and quality practices across the entire agency.
 
External reviews can take many forms. They may include recommendations from advisory committees that meet at regular intervals (typically every 6 months). However, advisory committees should never be the sole source of outside review because the members of such committees rarely have the opportunity to become deeply familiar with agency programs. External reviews can also take the form of a “visiting committee” using the NSF model; 43 an academic-type visiting committee; a special committee established by a relevant professional association (see, e.g., American Statistical Association, 1984); or a study by a panel of experts. 44
Practice 13: Coordination and Collaboration with Other Statistical Agencies
Introduction
The U.S. federal statistical system consists of many agencies in different departments, each with its own mission. Nonetheless, statistical agencies do not and should not conduct their activities in isolation. An effective statistical agency actively explores ways to work with other agencies to meet current information needs, through such means as seeking ways to integrate the designs of existing data systems to provide new or more useful data than a single system can provide. An effective agency is also alert for occasions when it can provide technical assistance to other agencies—including not only other statistical agencies, but also program agencies in its department—as well as occasions when it can benefit from such assistance in turn.
 
Efforts to standardize concepts and definitions, such as those for industries, occupations, and race and ethnicity, can contribute to effective coordination of statistical agency endeavors (see, e.g., National Research Council, 2004a, 2004b; also see Appendix A), as does the development of broad macro models, such as the system of national accounts. Efforts to standardize categories on survey questionnaires among agencies can enhance data comparability, while efforts by agencies to adopt common standards for data documentation and other metadata can contribute to the ease of the use of statistical products. Initiatives for interrelating and synchronizing data among statistical agencies (including individual data and address lists when permitted by law) can be helpful for such purposes as achieving greater efficiency in drawing samples, evaluating completeness of population coverage, and reducing duplication among statistical programs, as well as reducing respondent burden.
Role of OMB
The responsibility for coordinating statistical work in the federal government is specifically assigned to the Office of Information and Regulatory Affairs (OIRA) in OMB by the Paperwork Reduction Act (previously, by the Federal Reports Act and the Budget and Accounting Procedures Act see Appendix A). The Statistical and Science Policy Office in OIRA, often working with the assistance of interagency committees, reviews concepts of interest to more than one agency; issues standard classification systems (of industries, metropolitan areas, etc.) and oversees their periodic revision; consults with other parts of OMB on statistical budgets; and, by reviewing statistical information collections as well as the statistical programs of the government as a whole, identifies gaps in statistical data, programs that may be duplicative, and areas in which interagency cooperation might lead to greater efficiency and added utility of data. The Statistical and Science Policy Office also is responsible for coordinating U.S. participation in international statistical activities. 45
 
The Statistical and Science Policy Office encourages the use of administrative data for statistical purposes, when feasible, and works to establish common goals and norms on major statistical issues, such as confidentiality. It sponsors and heads the interagency FCSM, which issues guidelines and recommendations on statistical issues common to a number of agencies, typically by working through subcommittees, and also hosts conferences that facilitate professional interaction and development (see Federal Committee on Statistical Methodology, 1978a–2005). 46 It encourages CNSTAT at the National Research Council to serve as an independent adviser and reviewer of federal statistical activities. The 1995 reauthorization of the Paperwork Reduction Act created a statutory basis for the existing Interagency Council on Statistical Policy, formalizing an arrangement whereby statistical agency heads participate with OMB in activities to coordinate federal statistical programs (see Appendixes A and B).
Forms of Interagency Collaboration
There are many forms of interagency collaboration and coordination. Some efforts are multilateral, some bilateral. Many result from common interests in specific subject areas, such as economic statistics, statistics on people with disabilities, or statistics on children or the elderly. The U.S. Office of Management and Budget (2011:Ch. 3) describes several interagency collaborative efforts, such as joint support for research that fosters new and innovative approaches to surveys; the development of a statistical community of practice for agencies to share, standardize, and improve statistical protocols and tools; a systemwide initiative to facilitate the statistical uses of administrative records under the leadership of an FCSM subcommittee; and implementation of comparable measures of disability on major household surveys.
 
A common type of bilateral arrangement is the agreement of a program agency to provide administrative data to a statistical agency to be used as a sampling frame, a source of classification information, or a summary compilation to check (and possibly revise) preliminary sample results. The Bureau of Labor Statistics, for example, benchmarks its monthly establishment employment reports to data supplied by state employment security agencies. Such practices improve statistical estimates, reduce costs, and eliminate duplicate requests for information from the same respondents. In other cases, federal statistical agencies engage in cooperative data collection with state counterparts to let one collection system satisfy the needs of both. A number of such joint systems have been developed, notably by the Bureau of Labor Statistics, the National Agricultural Statistics Service, the National Center for Education Statistics, and the National Center for Health Statistics.
 
Another example of a joint arrangement is one in which a statistical agency contracts with another to conduct a survey, compile special tabulations, or develop models. Such arrangements make use of the special skills of the supplying agency and facilitate use of common concepts and methods. The Census Bureau conducts many surveys for other agencies, both the National Center for Health Statistics and the National Agricultural Statistics Service receive funding from other agencies in their departments to support their survey work, and the National Center for Science and Engineering Statistics receives funding from agencies in other departments to support several of its surveys (see U.S. Office of Management and Budget, 2012b:Table 2).
 
The major federal statistics agencies are also concerned with international comparability of statistics. Under the leadership of OMB’s  Statistical and Science Policy Office, they contribute to the deliberations of the United Nations Statistical Commission, the OECD, and other international organizations; participate in the development of international standard classifications and systems; and support educational activities that promote improved statistics in developing countries. Statistical agencies also learn from and contribute to the work of established statistical agencies in other countries in such areas as survey methodology, record linkage, confidentiality protection techniques, and data quality standards. Several statistical agencies run educational programs for government statisticians in developing countries. Some statistical agencies have long-term cooperative relationships with international groups: examples include the Bureau of Labor Statistics with the International Labor Organization, the National Agricultural Statistics Service with the Food and Agriculture Organization, the National Center for Education Statistics with the International Indicators of Education Systems project of the OECD, and the National Center for Health Statistics with the World Health Organization.
 
To be of most value, the efforts of statistical agencies to collaborate as partners with one another need to involve the full range of their activities, including definitions, concepts, measurement methods, analytical tools, dissemination modes, and disclosure limitation techniques. Such efforts should also extend to policies and professional practices, so that agencies can respond effectively and with a coordinated voice to such government-wide initiatives as data quality guidelines, privacy impact assessments, institutional review board requirements, and others.
 
Collaboration efforts should also encompass the development of data, especially for emerging policy issues (see, e.g., National Research Council, 1999a, 2007b). In some cases, it may be not only more efficient, but also productive of needed new data for agencies to fully integrate the designs of existing data systems, such as when one survey provides the sampling frame for a related survey. In other instances, collaborative efforts may identify ways for agencies to improve their individual data systems so that they are more useful for a wide range of purposes.
 
Collaboration on ways and means of using alternative data sources, such as administrative records, should be pursued so that the entire statistical system can move forward to improve the relevance, accuracy, timeliness, and cost-effectiveness of their data programs. Toward this goal, in 2008 the FCSM established a Subcommittee on Administrative Records, which is working to develop standards and provide guidance to statistical agencies that will facilitate not only use of administrative records, but also evaluation of their quality and fitness to be part of an agency’s data collection, estimation, and evaluation programs. This subcommittee has released two products from its work: one is a compilation of case studies of successful statistical uses of administrative data (Federal Committee on Statistical Methodology, 2009); the other is a checklist tool for assessing the quality of administrative data (Federal Committee on Statistical Methodology, 2013).
 
Two continuing collaborative efforts for providing statistical information to the public in a broad area of interest are the Federal Interagency Forum on Aging-Related Statistics and the Federal Interagency Forum on Child and Family Statistics. The former was established in the mid-1980s by the National Institute on Aging, in cooperation with the National Center for Health Statistics and the Census Bureau. The forum’s goals include coordinating the development and use of statistical databases among federal agencies, identifying information gaps and data inconsistencies, and encouraging cross-national research and data collection for the aging population. The forum was reorganized in 1998 to include 6 new agencies and has grown since then to include 15 agencies. The forum develops a periodic indicators chart book, which was first published in 2000 and was most recently issued in 2012 (Federal Interagency Forum on Aging-Related Statistics, 2012).
 
The Federal Interagency Forum on Child and Family Statistics was formalized in a 1994 executive order to foster collaboration in the collection and reporting of federal data on children and families. Its membership currently includes 22 statistical and program agencies. The forum’s reports (e.g., Federal Interagency Forum on Child and Family Statistics, 2012) describe the condition of America’s children, including changing population and family characteristics, the environment in which children are living, and indicators of well-being in the areas of economic security, health, behavior, social environment, and education.
 
No single agency, whether a statistical agency or program agency, could have produced the forum reports alone. Working together in this way, federal statistical agencies contribute to presenting data in a form that is more relevant to policy concerns and to a stronger statistical system overall. Similar collaborative efforts aimed at integrating not only data dissemination, but also data collection and estimation, using traditional and nontraditional data sources, are critically important to improving the relevance, accuracy, timeliness, and cost-effectiveness of the output from the nation’s federal statistical system.

Footnotes

2
See http://nces.ed.gov/ipeds/about [February 2013].
3
Within the past decade, the Division of Science Resources Studies in the National Science Foundation became the Division of Science Resources Statistics in 2002 and, as provided by section 505 of the America COMPETES Reauthorization Act of 2010, became the National Center for Science and Engineering Statistics in 2011.
4
The National Research Council (2001b:Ch. 6) cited a number of these reasons in recommending to the U.S. Department of Health and Human Services that it establish or identify a statistical unit to be assigned responsibility and authority for carrying out statistical functions and data collection for social welfare programs and the populations they serve (although the recommendation was not adopted); see also National Research Council and Institute of Medicine (2004).
5
Under the guidance issued for CIPSEA in 2007, OMB has recognized 4 new statistical units in addition to the 12 statistical agencies originally recognized (see Appendix A). These agencies and units are authorized to assign agent status to researchers and contractors, which permits sharing individually identifiable information with them for statistical purposes and holding them legally liable for protecting the confidentiality of the information.
6
For example, there are more policy uses of statistical data from the American Community Survey (ACS) and ways to make the ACS data more relevant and accurate for these purposes than have yet been tapped: see National Research Council (2008d, 2011a, 2012g).
7
8
See the Fundamental Principles of Official Statistics of the U.N. Statistical Commission in Appendix C and the European Statistics Code of Practice for the National and Community Statistical Authorities in Appendix D.
9
Agencies headed by career civil servants, many of whom hold their positions for long periods of time, include the Bureau of Economic Analysis; the Bureau of Transportation Statistics; the Economic Research Service and the National Agricultural Statistics Service in the U.S. Department of Agriculture (USDA); the National Center for Health Statistics in the Department of Health and Human Services; the National Center for Science and Engineering Statistics in the National Science Foundation; the Office of Research, Evaluation, and Statistics in the Social Security Administration; and the Statistics of Income Division in the Internal Revenue Service.
10
These two positions were among a large number of subcabinet-level posts that were changed from presidential appointments requiring Senate confirmation to presidential appointments not requiring confirmation, reflecting a desire by the U.S. Senate to lessen its confirmation workload and increase the prospects for speedier confirmations for the remaining positions.
11
Most statistical agencies have such broad authority, limited by budgetary constraints, departmental requirements, OMB review, and congressional mandates.
12
See Observations from the Interagency Technical Working Group on Developing a Supplemental Poverty Measure (March 2010:1,3). Available: http://www.census.gov/hhes/www/pover...servations.pdf [February 2013].
13
One statistical agency, the Energy Information Agency, has its independence authorized in the statute that established the Department of Energy: “The Administrator [of EIA] shall not be required to obtain the approval of any other officer or employee of the Department in connection with the collection or analysis of any information; nor shall the Administrator be required, prior to publication, to obtain the approval of any other officer or employee of the United States with respect to the substance of any statistical or forecasting technical reports which he has prepared in accordance with law” (Section 205 of Department of Energy Organization Act of 1977; 42 USC 7135(d)).
14
For example, the Census Bureau’s programs of Small-Area Income and Poverty Estimates (SAIPE) and Small-Area Health Insurance Estimates (SAHIE) use various administrative datasets together with the American Community Survey (ACS) in models to produce small-area estimates with less error than the ACS estimates by themselves; see http://www.census.gov/saipe [February 2013]; http://www.census.gov/sahie [February 2013].
15
For example, the Current Employment Statistics (CES) program of the Bureau of Labor Statistics (BLS) collects monthly data on employment, hours, and earnings from surveys of business establishments; each year BLS uses a census of such information from state unemployment insurance records to benchmark or correct the monthly estimates; see http://bls.gov/sae/790faq2.htm#Ques8 [February 2013].
16
The ACS replaced the long-form sample in the 2010 census, after it entered full-scale data collection in 2005.
17
Lower response rates reduce the effective sample size and increase the sampling error of estimates from surveys; lower rates also increase response bias in survey estimates to the extent that nonrespondents differ from respondents in ways that affect analysis and are not addressed by weighting and imputation procedures.
19
See, for example, National Research Council (2007b), which describes the use of population controls in the ACS.
20
This group includes five surveys, the earliest of which began in 1966 and the most recent in 1997; see http://www.bls.gov/nls [February 2013].
21
See http://lehd.did.census.gov/led [February 2013]; see also National Research Council (2007b).
22
This approach is used by Statistics Canada, which asks respondents to its surveys that collect income data, such as its Survey of Labour and Income Dynamics (SLID), to permit the agency to use the respondent’s income tax information (which is available to the agency for statistical and research purposes) instead of asking a series of questions. See, for example, http://www23.statcan.gc.ca/imdb/p3In...db&adm=8&dis=2, first question in the SLID questionnaire IN module [February 2013].
23
A decades-old report from the Federal Committee on Statistical Methodology (FCSM) (1980b) still has much to offer on the issues and problems in using administrative records for statistical purposes, and an FCSM Subcommittee on Administrative Records is making progress toward the development of protocols for accessing, using, and evaluating administrative records (see discussion in Practice 13).
25
See, for example, http://www.cdc.gov/nchs/nhcs/faq.htm, “What Does Participation in the NHCS [National Hospital Care Survey] Entail,” and related frequently asked questions [February 2013]. Much of the data collected could be characterized as administrative records of private-sector organizations and evaluated under the framework outlined above for government administrative records.
26
This is currently being done by the Billion Prices Project at the Massachusetts Institute of Technology; see http://bpp.mit.edu [February 2013].
27
Efforts have been under way since CIPSEA was enacted to introduce legislation that would permit business data synchronization involving IRS records, but, to date, this has not occurred.
28
For information about the ECLS, see http://www.nces.ed.gov/ecls [February 2013].
29
For information about the NLSY, see http://www.bls.gov/nls/home.htm [February 2013].
30
For information about HRS, see http://hrsonline.isr.umich.edu [February 2013].
31
For information about NCS, see http://www.nationalchildrensstudy.gov/Pages/default.
aspx [February 2013].
32
For information about SLDS, see http://nces.ed.gov/programs/slds/about_SLDS.asp [February 2013].
33
The Census Bureau posts basic quality indicators for the ACS, such as sample size, population coverage, household response rates, and item response rates for the nation and states on the ACS website; see http://www.census.gov/acs/www/method...d_data_quality [February 2013].
34
Such a utility is provided by the Data Enclave of NORC at the University of Chicago, which provides secure access by researchers to selected microdata sets of the USDA Economic Research Service, the National Center for Science and Engineering Statistics, and several other federal agencies and private foundations (see http://www.dataenclave.org/index.php/home [February 2013]).
35
For a literature review of public opinion on privacy in the wake of the September 11, 2001, terrorist attacks, see National Research Council (2008a:App. M).
36
For information about the Common Rule and procedures for the certification of IRBs by the Office for Human Research Protections in the U.S. Department of Health and Human Services, see http://www.hhs.gov/ohrp [February 2013]. For information about proposed changes to the Common Rule, see also http://www.hhs.gov/ohrp/humansubject...m2011page.html [February 2013].
37
Several CNSTAT study panels have discussed these issues and alternative procedures for providing data access while maintaining confidentiality protection; see National Research Council (1993b, 2000a, 2003d, 2005b, 2009d, 2010f ).
38
The data quality guidelines of statistical agencies in other countries are also helpful; for example, see Statistics Canada (2009); United Kingdom Office for National Statistics (2007).
39
For examples, see the Statistical Standards of the National Center for Education Statistics, available: http://nces.ed.gov/statprog/2002/stdtoc.asp [February 2013], and the Energy Information Administration’s Standards Manual, available: http://www.eia.doe.gov/smg/Standards.html [February 2013].
40
Reviews of concepts underlying important statistical data series include National Research Council (1995a and 2005c) on concepts of poverty; National Research Council (2002a) on cost-of-living concepts; National Research Council (2005a) on “satellite” accounts for nonmarket activities, such as home production, volunteerism, and human capital investment; National Research Council (2006a) on concepts of food insecurity and hunger; National Research Council (2006c) on concepts of residence for the U.S. census and the ACS; National Research Council (2009b) on concepts of disability; National Research Council (2010a) on satellite accounts for health; and National Research Council and Institute of Medicine (2012) on concepts of medical care economic risk and burden.
41
See, e.g., Gillingham and Lane (1982).
42
Only BLS and the Census Bureau maintain their own interviewing staffs.
43
For examples of evaluations of NSF programs, see http://www.nsf.gov/od/oia/activities/cov/covs.jsp [February 2013].
44
See, e.g., National Research Council (1985a—study of the statistical programs of the Immigration and Naturalization Service); (1986—study of NCES); (1997b—study of BTS); (2004b—study of NCSES statistics on research and development expenditures); (2009a—study of BJS); and other National Research Council reports in the references.
45
The Statistical and Science Policy Office, formerly the Statistical Policy Office, was renamed to reflect added responsibilities with respect to the 2001 Information Quality Act standards and guidelines, OMB’s guidance on peer review planning and implementation, and evaluations of science underlying proposed regulatory actions.
46
The papers from the most recent FCSM research conference are available at: http://www.fcsm.gov/events/papers2012 [February 2013].

Appendixes

A Legislation and Regulations That Govern Federal Statistics

This appendix summarizes the major legislation and agency regulations and guidance that govern the operations of the federal statistical system as a whole, highlighting ten topics in three categories (see Appendix B for descriptions of the overall structure of the system and the principal statistical agencies):
 
1. Regulatory authority of the U.S. Office of Management and Budget (OMB)
a. The 1980 Paperwork Reduction Act, as reauthorized and amended in 1995, and associated guidance (covering also the 1942 Federal Reports Act and the 1950 Budget and Accounting Procedures Act)
b. OMB Statistical Policy Directives
2. Confidentiality and privacy protection
a. The 1997 Order Providing for the Confidentiality of Statistical Information
b. The 2002 Confidential Information Protection and Statistical Efficiency Act (CIPSEA—Title V of the E-Government Act) and associated guidance
c. The 2002 E-Government Act, Section 208, which requires privacy impact assessments for federal data collections and associated guidance (covering also the 1974 Privacy Act)
d. The 2002 Federal Information Security Management Act (FISMA—Title III of the E-Government Act)
3. Information quality, peer review, performance evaluation, and scientific integrity
a. The 2000 Information Quality Act and associated guidelines
b. 2004 OMB peer review guidance
c. The 2010 Government Results and Performance Modernization Act
d. 2009–2010 guidance on scientific integrity
 
Most of the legislation, regulations, and guidance pertains to the authority of OMB, which plays a critical role in oversight of the federal government’s widely dispersed statistical operations. The oversight dates to 1939, when the functions of a Central Statistical Board, created in 1933, were transferred to the then-named Bureau of the Budget (see Anderson, 1988; Duncan and Shelton, 1978; Norwood, 1995). Recent legislation and guidance address such systemwide issues as confidentiality protection and privacy of respondents, data quality (including peer review prior to dissemination), efficiency of operations, and scientific integrity.
The 1980 Paperwork Reduction Act and Associated Guidance
The Paperwork Reduction Act (PRA) of 1980 (44 USC 3501, amended in 1986 and reauthorized and amended in 1995 by P.L. 104-13) is the foundation for the modern statistical coordination and management mission of OMB. It establishes OMB’s review power over federal statistical agencies and myriad other agencies throughout the federal government that collect information from individuals and organizations. This review power covers both data collection budgets and methods and practices for data collection and dissemination.
Background
The PRA’s origins trace back to Executive Order 6226, signed by Franklin D. Roosevelt in July 1933, which established a Central Statistical Board to “appraise and advise upon all schedules of all Government agencies engaged in the primary collection of statistics required in carrying out the purposes of the National Industrial Recovery Act, to review plans for tabulation and classification of such statistics, and to promote the coordination and improvement of the statistical services involved.” Members of the board were appointed by the relevant cabinet secretaries. The board was established in law for a 5-year period in 1935. Its functions were transferred to the Bureau of the Budget (itself established in 1921) in 1939, when the Budget Bureau was transferred to the Executive Office of the President.
 
The 1942 Federal Reports Act represented another milestone by codifying the authority for the Budget Bureau to coordinate and oversee the work of federal statistical agencies. Most famously, it provided that no federal agency could collect data from 10 or more respondents without approval of the budget director. (Data collections by contractors on behalf of federal agencies are covered by this provision, although data collections by government grantees are generally not covered.) The 1950 Budget and Accounting Procedures Act (31 USC 1104(d)) further strengthened the statistical coordinating and improvement role of OMB, giving OMB authorization to promulgate regulations and orders governing statistical programs throughout the federal government.
 
The statistical policy function continued in the budget office in the Executive Office of the President when the Budget Bureau became the Office of Management and Budget in 1970. However, in 1977, the statistical policy staff was split into two groups: one group remained in the OMB to handle the paperwork clearance and review function for statistical agencies; the other group was moved to the Department of Commerce to address statistical policy and standards issues.
Paperwork Reduction
The overarching goal of the 1980 Paperwork Reduction Act was to reduce the burden of filling out federal forms by businesses and individuals. It moved the statistical policy office back to OMB (from the Department of Commerce) under a new Office of Information and Regulatory Affairs (OIRA), which was charged to reduce the combined burden imposed by regulatory agencies and administrative and statistical program agencies. The PRA required OMB, through the chief statistician, to engage in long-range planning to improve federal statistical programs; review statistical budgets; coordinate government statistical functions; establish standards, classifications, and other guidelines for statistical data collection and dissemination; and evaluate statistical program performance. In the 1995 reauthorization and extensive revision of the PRA, two of the most significant provisions added for statistical policy were (1) to codify the Interagency Council on Statistical Policy (ICSP), chaired by the chief statistician, and (2) to update the legislation to reflect the chief statistician’s role in coordinating U.S. participation in international statistical activities.
Survey Clearance Process
In January 2006 the OMB Statistical and Science Policy Office released Guidance on Agency Survey and Statistical Information Collections—Questions and Answers When Designing Surveys for Information Collections, 1 a set of 81 questions and answers that attempts to demystify the OMB clearance process (required by the PRA) for surveys and other statistical information collections. Its purpose is to explain OMB’s review process, assist agencies in strengthening their supporting statements for information collection requests, and provide advice for improving information collection designs.
 
The Guidance covers such topics as the purpose of the guidance; submission of information collection requests (often called clearance packages) to OMB; scope of the information collection (e.g., calculation of burden hours on respondents); choice of methods; sampling; modes of data collection; questionnaire design and development; statistical standards; informing respondents about their participation and the confidentiality of their data; response rates and incentives; analysis and reporting; and studies using stated preference methods (which ask respondents about the use or non-use value of a good in order to obtain willingness-to-pay estimates relevant to benefit or cost estimation). The Guidance includes a glossary of terms and information collection request supporting statement instructions.
 
The document outlines the statutory timing and process requirements for all statistical information collection requests in order to obtain OMB approval (which is indicated by an OMB control number on approved survey questionnaires). After an agency has developed a draft information collection plan and instrument, the agency must publish a 60-day notice in the Federal Register inviting public comment on the proposed collection. At this stage, the agency must have the draft survey instrument available for the public to review. Following this initial comment period, the agency may submit its clearance package to OMB, at which time it must place a second notice in the Federal Register, allowing a 30-day public comment period and notifying the public that OMB approval is being sought and that comments may be submitted to OMB. This notice runs concurrent with the first 30 days of OMB review, half of OMB’s total of 60 days after receipt of the clearance package to make its decision to approve or disapprove or to instruct the agency to make a substantive change to its proposed collection. Generally, agencies need to allow 6 months to complete the entire process, including survey development, public comment, and agency, departmental, and OMB review.
 
In recent years, OMB has issued several memoranda to clarify particular interpretations and applications of the PRA to agency activities. 2 Specifically, OMB issued a memorandum that provides an overview of PRA requirements, discusses PRA implications of social media and web-based interactive technologies, clarifies the use of generic clearances, provides options for streamlining the PRA process for scientific research, establishes a fast-track process for qualitative customer service delivery feedback, and provides answers to PRA questions related to challenges and prizes.
OMB Statistical Policy Directives
The OMB Statistical and Science Policy Office issues and periodically updates a number of directives that pertain to federal agency data collection and dissemination. The process involves expert review, agency consultation, and public comment. The oldest two directives, on standards for statistical surveys and publication of statistics, were first issued in the 1950s, updated in the 1970s, and then combined and revised in 2006. A directive on release of federal statistical products, issued in 2008, complements a directive issued in the 1970s and updated in 1985 on release of federal economic indicators. Three other directives are updated at least every 10 years: standards on metropolitan statistical areas (MSAs) (last revised in 2010), on industry (last revised in 2012), and on occupational classification systems (last revised in 2010). A 1969 directive on the official poverty measure was updated in a minor way in 1978.
 
Among the statistical policy directives are the following, which are briefly summarized below: 3
 
Standards and Guidelines for Statistical Surveys (replaces and combines Statistical Policy Directives Nos. 1 and 2)
Statistical Policy Directive No. 3—Compilation, Release, and Evaluation of Principal Federal Economic Indicators (and Schedule of Release Dates for Principal Federal Economic Indicators)
Statistical Policy Directive No. 4—Release and Dissemination of Statistical Products Produced by Federal Statistical Agencies
• Standards for the definition of metropolitan statistical areas (MSAs) and other statistical areas
• North American Industry Classification System (NAICS)/Standard Industrial Classification (SIC)
• Standard Occupational Classification (SOC)
Statistical Policy Directive No. 14—Definition of Poverty for Statistical Purposes
Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity
Standards and Guidelines for Statistical Surveys
After extensive consultation with agencies and the public, OMB issued Standards and Guidelines for Statistical Surveys in September 2006 as an update and revision of Statistical Policy Directive No. 1, Standards for Statistical Surveys, and Statistical Policy Directive No. 2, Publication of Statistics. 4 The new document includes 20 standards and one or more associated guidelines for every aspect of survey methodology from planning through data release:
 
1. survey planning,
2. survey design,
3. survey response rates,
4. pretesting survey systems,
5. developing sampling frames,
6. required notification to potential survey respondents,
7. data collection methodology,
8. data editing,
9. nonresponse analysis and response rate calculation,
10. coding,
11. data protection,
12. evaluation,
13. developing estimates and projections,
14. analysis and report planning,
15. inference and comparisons,
16. review of information products,
17. releasing information,
18. data protection and disclosure avoidance for dissemination,
19. survey documentation, and
20. documentation and release of public-use microdata.
Compilation, Release, and Evaluation of Principal Economic Indicators
OMB issued Statistical Policy Directive No. 3—Compilation, Release, and Evaluation of Principal Federal Economic Indicators in the 1970s and strengthened it in 1985. 5 Its purpose is clearly stated:
 
[This directive] designates statistical series that provide timely measures of economic activity as Principal Economic Indicators and requires prompt release of these indicators by statistical agencies in a politically-neutral manner. The intent of the directive is to preserve the time value of such information, strike a balance between timeliness and accuracy, prevent early access to information that may affect financial and commodity markets, and preserve the distinction between the policy-neutral release of data by statistical agencies and their
interpretation by policy officials.
 
Each September OMB issues the Schedule of Release Dates for Principal Federal Economic Indicators for the subsequent calendar year.6 At present, the following agencies issue one or more of the 38 principal economic indicators:
 
• Bureau of Economic Analysis (5 indicators, including gross domestic product [GDP], personal income and outlays, corporate profits)
• Bureau of Labor Statistics (7 indicators, including the employment situation, Consumer Price Index [CPI])
• Census Bureau (13 indicators, including new residential construction, monthly retail trade)
• Energy Information Administration (weekly natural gas storage report)
• Federal Reserve Board (4 indicators, including money stock measures, consumer installment credit)
• Foreign Agricultural Service (world agricultural production).
• National Agricultural Statistics Service (6 indicators, including agricultural prices and grain production)
• World Agricultural Outlook Board (world agricultural supply and demand estimates)
Release and Dissemination of Statistical Products
OMB issued Statistical Policy Directive No. 4—Release and Dissemination of Statistical Products Produced by Federal Statistical Agencies in 2008, which essentially covers all statistical releases other than those specified in Directive 3. 7 It includes not only statistical information released in printed reports or on the Internet, but also statistical press releases, which describe or announce a statistical data product. Statistical press releases are the sole responsibility of the relevant statistical agency. Each fall statistical agencies must issue a schedule of when they expect each regular or recurring product to be released and give timely notification of any change to the published schedule.
North American Industry Classification System/Standard Industrial Classification
The NAICS, most recently updated in 2012, substantially revised its predecessor, the SIC.8 NAICS was developed by the United States, Canada, and Mexico to provide a common, contemporary classification system for economic production activity following the enactment of the North American Free Trade Agreement (NAFTA). NAICS was first issued in 1997 as a conceptual and operational replacement for the SIC, which had been revised on an irregular basis since it was initially issued in various sections in 1938–1940. Interagency and country working groups (under the aegis of OMB in the United States) have the opportunity to update NAICS every 5 years for years ending in 2 and 7 so that it keeps up reasonably well with changes in the structure of industrial activity in the three countries.
Standard Occupational Classification
The SOC is used by federal statistical agencies to classify workers into occupational categories for collecting, tabulating, and disseminating data. 9 The first SOC was published in 1977 in an effort to standardize the collection of occupational data by multiple agencies. It was revised in 1980 but not universally adopted until an interagency process under the aegis of OMB further revised it in 1998 for use in the 2000 decennial census and surveys conducted in the following decade. Work to revise the 2000 SOC was completed in time for its use in 2010 for the American Community Survey (which provides occupational data in place of the decennial census “long form” sample) and other surveys. The next planned revision of the SOC is scheduled for 2018 and every 10 years thereafter.
Metropolitan Area Classification
For more than 60 years, the OMB Metropolitan Area Classification Program has provided standard statistical area definitions for use throughout the federal government. The usefulness of standardizing these classifications became clear in the 1940s, and the Bureau of the Budget (the predecessor to OMB) led an effort to develop what were then called “standard metropolitan areas” for use in 1950 census publications. Since then, OMB has updated as appropriate the definitional criteria for metropolitan areas before each census; based on those criteria, after each census, OMB has issued a list of recognized areas.
 
The definitional criteria issued before the 2000 census marked a major revision to the coverage of the program. Standards for Defining Metropolitan and Micropolitan Statistical Areas defined not only metropolitan statistical areas, but also, for the first time, micropolitan areas. 10 Metropolitan areas are those with a central urbanized core of 50,000 or more people in one or more counties; micropolitan areas are those with a central urbanized core of 10,000 or more people in one or more counties. The list of metropolitan and micropolitan areas using the 2000 criteria was initially issued in 2003 and was updated annually through 2008 by OMB on the basis of the Census Bureau’s population estimates.11 Two years later, OMB issued 2010 Standards for Delineating Metropolitan and Micropolitan Statistical Areaswhich largely continued the criteria adopted for the 2000 standards.12 Areas based on these standards, using data from the 2010 census and the American Community Survey, are scheduled to be delineated and announced in 2013.
Definition of Poverty
OMB first issued Statistical Policy Directive No. 14—Definition of Poverty for Statistical Purposes in 1969. The directive adopted the existing poverty thresholds (first defined by Mollie Orshansky of the Social Security Administration in 1963) for different categories of families defined by size, number of children, gender of the family head, and farm-nonfarm residences. For most family types, the thresholds represented the costs of a minimally adequate diet multiplied by three to allow for all other expenses.
 
The 1969 directive specified that the thresholds would be updated each year for the change in the CPI and compared with families’ total money income as measured in the Current Population Survey. The directive was reissued in 1978; additional minor modifications were made to the poverty thresholds beginning in 1982.13 With input from the observations in the 2010 report of an Interagency Technical Working Group, 14 the U.S. Census Bureau released a new supplemental poverty measure in fall 2011 (referencing poverty in calendar 2010), using thresholds developed by the U.S. Bureau of Labor Statistics. 15 The thresholds and definition of countable resources as money and near-money disposable income for the supplemental poverty measure were derived in large part from the recommendations in Measuring Poverty: A New Approach (National Research Council, 1995a).
Data on Race and Ethnicity
OMB issued Statistical Policy Directive No. 15—Race and Ethnic Standards for Federal Statistics and Administrative Reporting in 1977. It specified a minimum set of racial and ethnic categories for reporting of race and ethnicity on federal surveys and in administrative records systems. It recommended either two separate questions, one on ethnicity (Hispanic or non-Hispanic) and one on race (white, black, Asian or Pacific Islander, American Indian or Alaska Native), or a combined question that included Hispanic as a category. The U.S. census has historically included additional categories under the two-question format.
 
Following an intensive research, testing, and consultation process, OMB issued revised Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity in 1997.16 The updated directive retains a two-question format, includes separate categories for Asians and for Native Hawaiian and other Pacific Islanders, emphasizes self-identification, and allows respondents to select more than one racial category. The 2010 census included several experimental panels to test different strategies that incorporate alternative wording and format for the questions on race and ethnicity, including a combined race and ethnicity question. The results have been analyzed, 17 and additional research is planned during the next few years that could ultimately inform additional OMB guidance on categories for race and ethnicity.
The 1997 Order Providing For The Confidentialy of Statistical Information
An Order Providing for the Confidentiality of Statistical Information was issued by OMB in 1997.18 The order was designed to bolster the confidentiality protections afforded by statistical agencies or units (as listed in the order), some of which lacked legal authority to back up their confidentiality protection. CIPSEA (see below) placed confidentiality protection for statistical information on a strong legal footing across the entire federal government.
The 2002 Confidential Information Protection and Statistical Efficiency Act and Associated Guidance
The Confidential Information Protection and Statistical Efficiency Act (CIPSEA) (Title V of the E-Government Act of 2002, P.L. 107-347) was landmark legislation to strengthen the statistical system with regard to confidentiality protection and data sharing. Enactment of CIPSEA was the culmination of more than 30 years of efforts to standardize and bolster legal protections for data collected solely for statistical purposes by federal agencies while permitting limited sharing of individually identifiable business information among three statistical agencies for efficiency and quality improvement. CIPSEA has two subtitles, A and B, covering confidentiality and sharing data.
Subtitle A—Protecting Confidentiality
Subtitle A of CIPSEA, Confidential Information Protection, strengthens and extends statutory confidentiality protection for all statistical data collections of the U.S. government. Prior to CIPSEA, such protection was governed by a patchwork of laws applicable to specific agencies, judicial opinions, and agencies’ practices. For all data furnished by individuals or organizations to an agency under a pledge of confidentiality for exclusively statistical purposes, Subtitle A provides that the data will be used only for statistical purposes and will not be disclosed in identifiable form to anyone not authorized by the title. It makes knowing and willful disclosure of confidential statistical data a class E felony with fines up to $250,000 and imprisonment for up to 5 years.
 
Subtitle A pertains not only to surveys, but also to collections by a federal agency for statistical purposes from administrative records (e.g., state government agency records). Data covered under Subtitle A are not subject to release under a Freedom of Information Act request.
Subtitle B—Sharing Data
Subtitle B of CIPSEA, Statistical Efficiency, permits the Bureau of Economic Analysis (BEA), the Bureau of Labor Statistics (BLS), and the Census Bureau to share individually identifiable business data for statistical purposes. The intent of the subtitle is to reduce respondent burden on businesses; improve the comparability and accuracy of federal economic statistics by permitting these three agencies to reconcile differences among sampling frames, business classifications, and business reporting; and increase understanding of the U.S. economy and improve the accuracy of key national indicators, such as the National Income and Product Accounts. Several data-sharing projects have been initiated under Subtitle B.
 
The subtitle does not permit sharing among BEA, BLS, and the Census Bureau of any individually identifiable tax return data that originate from the Internal Revenue Service (IRS). This limitation currently blocks some important kinds of business data sharing, such as those for sole proprietorships, for improving the efficiency and quality of business data collection by statistical agencies. For tax return information, data sharing is limited to a small number of items for specified uses by a small number of specific agencies (under Title 26, Section 6103 of the U.S. Code and associated Treasury Department regulations, as modified in the 1976 Tax Reform Act). The law provides access to specific tax return items by the Census Bureau for use in its population estimates program and economic census and survey programs, by the National Agricultural Statistics Service for conducting the Census of Agriculture, by the Congressional Budget Office for long-term models of the Social Security and Medicare programs, and by BEA for producing the National Income and Product Accounts. (Prior to the 1976 act, the President could issue an executive order authorizing access to tax records.) The governing statute would have to be modified to extend sharing of tax return items to agencies not specified in the 1976 legislation.
 
A proposal for legislation to expand access to IRS information for limited statistical purposes has been developed through interagency discussions: it would authorize the Bureau of Labor Statistics to receive limited business data from the Census Bureau (comingled with business tax information) for the purpose of synchronizing the two agencies’ business lists. It would also authorize BEA to receive business tax information for noncorporate businesses with receipts of more than $750,000, allowing BEA to improve the measurement of income and international transactions in the national accounts. This proposal has yet to be introduced as a bill in Congress.
OMB CIPSEA Guidance
OMB is charged to oversee and coordinate the implementation of CIPSEA; after a thorough interagency development and coordination process, OMB released final guidance in 2007.19 The guidance, which pertains to both Subtitles A and B, covers such topics as the steps that agencies must take to protect confidential information; wording of confidentiality pledges in materials that are provided to respondents; steps that agencies must take to distinguish any data or information they collect for nonstatistical purposes and to provide proper notice to the public of such data; and ways in which agents (e.g., contractors, researchers) may be designated to use individually identifiable information for analysis and other statistical purposes and be held legally responsible for protecting the confidentiality of that information.
 
A key provision of the CIPSEA guidance defines statistical agencies or units, which are the only federal agencies that may assign agent status for confidentiality protection purposes to contractors, researchers, or others. The guidance defines a statistical agency or unit as “an agency or organizational unit of the executive branch whose activities are predominantly the collection, compilation, processing, or analysis of information for statistical purposes.” A total of 16 agencies are currently so recognized:
 
• Of the 16, 12 are those enumerated in OMB’s 1997 confidentiality order and named in the CIPSEA implementation guidance: the members of the Interagency Council on Statistical Policy (ICSP) (see Appendix B), excluding the Office of Environmental Information in the Environmental Protection Agency and the Office of Research, Evaluation, and Statistics in the Social Security Administration.
• In 2007, OMB recognized two more units that applied for designation under the procedures outlined in the guidance: the Center for
Behavioral Health Statistics and Quality in the Office of Applied Studies in the Substance Abuse and Mental Health Services Administration in the Department of Health and Human Services and the Microeconomic Surveys Statistical Unit of the Board of Governors of the Federal Reserve.
• Since 2007, two additional units have been recognized: the Office of Research, Evaluation, and Statistics in the Social Security Administration (a member of the ICSP), and the National Animal Health Monitoring System Program Unit in the Animal and Plant Health Inspection Service in the Department of Agriculture.
The 2002 E-Government Act, Section 208
Section 208 of the E-Government Act of 2002 requires federal agencies to conduct a privacy impact assessment whenever an agency develops or obtains information technology that handles individually identifiable information or whenever the agency initiates a new collection of individually identifiable information. 20 The assessment is to be made publicly available and cover such topics as what information is being collected and why, with whom the information will be shared, what provisions will be made for informed consent regarding data sharing, and how the information will be secured. Typically, privacy impact assessments cover not only privacy issues, but also confidentiality, integrity, and availability issues (see, e.g., U.S. Census Bureau, 2003). OMB is required to issue guidance for development of the assessments, which was done in a memorandum from the OMB director to the heads of executive agencies and departments on September 26, 2003. 21
 
Section 208, Title III (see below), and Title V (see above) are the latest in a series of laws dating back to 1974 that govern access to individual records maintained by the federal government. The Privacy Act of 1974 states in part: 22
 
No agency shall disclose any record which is contained in a system of records by any means of communication to any person, or to another agency, except pursuant to a written request by, or with the prior written consent of, the individual to whom the record pertains.
 
There are specific exceptions allowing the use of personal records without prior consent for statistical purposes by the Census Bureau, for statistical research or reporting when the records are to be transferred in a form that is not individually identifiable, for routine uses within a U.S. government agency, for archival purposes “as a record which has sufficient historical or other value to warrant its continued preservation by the United States Government,” for law enforcement purposes, for congressional investigations, and for other administrative purposes.
 
The Privacy Act mandates that every federal agency have in place an administrative and physical security system to prevent the unauthorized release of personal records; it also mandates that every agency publish in the Federal Register one or more system of records notices (SORNs) for newly created and revised systems of records that contain personally identifiable information as directed by OMB. 23 SORNs are to describe not only the records and their uses by the agency, but also describe procedures for storing, retrieving, accessing, retaining, and disposing of records in the system. 24
The 2002 Federal Information Security Management Act
The Federal Information Security Management Act (FISMA) was enacted in 2002 as Title III of the E-Government Act of 2002 (P.L. 107-347). The act was meant to bolster computer and network security in the federal government and affiliated parties (such as government contractors) by mandating yearly audits.
 
FISMA imposes a mandatory set of processes that must be followed for all information systems used or operated by a federal agency or by a contractor or other organization on behalf of a federal agency. These processes must follow a combination of Federal Information Processing Standards documents, the special publications issued by the National Institute of Standards and Technology (SP-800 series), and other legislation pertinent to federal information systems, such as the Privacy Act of 1974 and the Health Insurance Portability and Accountability Act of 1996.
 
The first step is to determine what constitutes the “information system” in question. There is no direct mapping of computers to an information system; rather, an information system can be a collection of individual computers put to a common purpose and managed by the same system owner. The next step is to determine the types of information in the system and categorize each according to the magnitude of harm that would result if the system suffered a compromise of confidentiality, integrity, or availability. Succeeding steps are to develop complete system documentation, conduct a risk assessment, put appropriate controls in place to minimize risk, and arrange for an assessment and certification of the adequacy of the controls.
 
FISMA affects federal statistical agencies directly in that each of them must follow the FISMA procedures for its own information system. In addition, some departments are taking the position that all information systems in a department constitute a single information system for purposes of FISMA: those departments are taking steps to require that statistical agencies’ information systems and personnel be incorporated into a centralized departmentwide system.
The 2000 Information Quality Act and Associated Guidelines
The Information Quality Act of 2000 (P.L. 106-554) directed OMB to issue government-wide guidelines that “provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies.” It also required federal agencies to develop their own implementing procedures, including “administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency.” After a public comment period, OMB issued government-wide guidelines on February 22, 2002. 25 A few months later, 13 principal statistical agencies issued a notice outlining a common approach to the development and provision of guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of disseminated information. 26 The notice directed people to the websites of each agency for more information and to learn how to comment on draft guidelines. Each agency then finalized its own guidelines. 27 The information quality framework developed by the agencies was followed in the 2006 revision of OMB’s standards and guidelines for statistical surveys (summarized in the section on
“OMB Statistical Policy Directives,” on page 83).
2004 OMB Peer Review Guidance
Consistent with the 2000 Information Quality Act (P.L. 106-554), OMB developed guidance for federal agencies with regard to seeking peer review of the policy-relevant scientific information an agency disseminates. After two rounds of public comment, OMB issued the Final Information Quality Bulletin for Peer Review on December 16, 2004 (hereafter referred to as the Bulletin): it requires federal agencies to conduct a peer review of “influential scientific information” before the information is released to the public. 28 “Influential scientific information” is defined as “scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions” (U.S. Office of Management and Budget, 2004a:10). The Bulletin allows agencies discretion to select the type of peer review process most appropriate for a given scientific information product. The Bulletin excludes from the guidelines “routine statistical information released by federal statistical agencies (e.g., periodic demographic and economic statistics) and the analysis of these data to compute standard indicators and trends (e.g., unemployment and poverty rates)” (U.S. Office of Management and Budget, 2004a:33); however, it does include research reports and nonroutine collections by statistical agencies that can be considered “influential scientific information.”
2010 Government Performance and Results Modernization Act
On January 4, 2011, President Obama signed the Government Performance and Results Modernization Act of 2010. 29 This act, which superseded the Performance Assessment Rating Tool and the Government Performance and Results Act of 1993 (GPRA), requires performance assessment of government programs for purposes of evaluating agency performance and improvement. In carrying out the provisions of the act, the director of OMB coordinates with agencies to develop the federal government performance plan. The act requires, with few exceptions, all federal agencies to establish performance indicators to be used in measuring or assessing progress toward their identified performance goals and an objective, quantifiable, and measurable means by which to compare actual program results with these established performance goals. Additionally, each agency must describe how it will ensure the accuracy and reliability of the data used, including validation of measures, data sources, required level of accuracy, data limitations, and management of those limitations.
 
The broad scope of agencies affected by this act, and the use of the act in making budgetary decisions based on measured achievement toward program goals, has fostered added focus among many agencies on how to collect high quality data and produce sound government statistics with which to conduct rigorous program evaluation.
2009–2010 Guidance on Scientific Integrity
In a memorandum on scientific integrity in March 2009, President Obama stated: 30
 
The public must be able to trust the science and scientific process informing public policy decisions. Political officials should not suppress or alter scientific or technological findings and conclusions. If scientific and technological information is developed and used by the Federal Government, it should ordinarily be made available to the public. To the extent permitted by law, there should be transparency in the preparation, identification, and use of scientific and technological information in policymaking. The selection of scientists and technology professionals for positions in the executive branch should be based on their scientific and technological knowledge, credentials, experience, and integrity.
 
The President directed the Office of Science and Technology Policy (OSTP) to develop a strategy to ensure scientific integrity in government decision making.
 
In response, the OSTP director issued a memorandum on December 17, 2010, that called for executive departments and agencies to develop policies to “ensure a culture of scientific integrity,” “strengthen the actual and perceived credibility of Government research,” “facilitate the free flow of scientific and technological information, consistent with privacy and classification standards,” and “establish principles for conveying scientific and technological information to the public.” 31 It included guidance on the selection of candidates for scientific positions, independent peer review, whistleblower protections, promoting access to scientific and technological information in online open formats, and agency communications. It also provided guidance on public communications, use of federal advisory committees, professional development of government scientists and engineers, and implementation.
 
Each statistical agency is covered by its department’s scientific integrity policies. In addition, the principal statistical agencies developed a Statement of Commitment to Scientific Integrity that documents in a single place their response to the OSTP memorandum. The statement articulates how the Principles and Practices for a Federal Statistical Agency, Fourth Edition (National Research Council, 2009c), various OMB statistical policy directives and standards, and each agency’s information quality guidelines together form “the foundation for achieving and maintaining scientific integrity within and among the principal statistical agencies.” 32
Footnotes
1
The document is available at http://www.whitehouse.gov/omb/infore...dance_2006.pdf [February 2013].
2
All the memoranda can be found at http://www.whitehouse.gov/omb/inforeg_infocoll [February 2013].
3
All but one of the directives are available at or can be linked to from http://www.whitehouse.gov/omb/inforeg/statpolicy.html [February 2013]. The exception is Statistical Policy Directive No. 14, which is available at http://www.census.gov/hhes/www/povmeas/ombdir14.html [February 2013].
4
The original directive is available at http://www.whitehouse.gov/sites/defa...at_surveys.pdf [February 2013]; the revision is available at http://www.whitehouse.gov/sites/defa...at_surveys.pdf [February 2013].
5
See September 25, 1985, Federal Register 50(186). Available: http://www.whitehouse.gov/omb/infore...r_09251985.pdf [February 2013].
7
See March 7, 2008, Federal Register 73(46):12622–12626. Available: http://www.whitehouse.gov/sites/defa...irective-4.pdf [February 2013].
8
Available: http://www.census.gov/eos/www/naics [February 2013].
9
Available: http://www.bls.gov/soc [February 2013].
10
See December 27, 2000, Federal Register 65(249):82228–82238. Available: http://www.whitehouse.gov/omb/fedreg...reas122700.pdf [February 2013].
11
See, for example, Update of Statistical Area Definitions and Guidance on Their Uses.Available: http://www.whitehouse.gov/omb/bullet...2009/09-01.pdf [February 2013].
13
Available: http://www.census.gov/hhes/www/pover...ns/p60-133.pdf (page 9) [February 2013].
15
The supplemental poverty measure statistics for 2011 were published in fall 2012. Available: http://www.census.gov/hhes/povmeas/m...rchSPM2011.pdf [February 2013].
18
See June 27, 1997, Federal Register 62(124):35044–35050. Available: http://www.whitehouse.gov/sites/defa...onf-order1.pdf [February 2013].
19
See June 15, 2007, Federal Register 72(115):33362–33377. Available: http://www.whitehouse.gov/sites/defa...ipsea_guidance
20
Section 208 also mandates that OMB lead interagency efforts to improve federal information technology and use of the Internet for government services.
21
22
Available: http://www.justice.gov/opcl/privstat.htm [February 2013].pdf [February 2013].
23
See OMB Circular A-130, Management of Government Information Resources, Appendix I, revised 1996. Available: http://www.whitehouse.gov/omb/circulars_a130 [February 2013].
24
For an example of SORNs for a statistical agency, see http://www.census.gov/privacy/sorn/l...le_sorns.html# [February 2013].
25
See February 22, 2002, Federal Register 67(36):8452–8460. Available: http://www.whitehouse.gov/sites/default/files/omb/assets/omb/fedreg/reproducible2.pdf [February 2013].
26
See June 4, 2002, Federal Register 67(107):38467–38470. Available: http://www.gpo.gov/fdsys/pkg/FR-2002...f/02-13892.pdf [February 2013].
30

B Organization of the Federal Statistical System

This appendix begins with an overview of the U.S. statistical system as a whole. It then briefly summarizes the statistical functions of the U.S. Office of Management and Budget (OMB), the principal statistical agencies, and a selection of major statistical programs housed or sponsored by other agencies.
Overview
Budget
For fiscal 2012 OMB estimated $6.2 billion in direct funding for government statistical programs in 129 agencies with directly funded statistical activities of $500,000 or more (U.S. Office of Management and Budget, 2012b:Table 1). “Statistical activities” are defined by OMB (2012b:3–4) to include not only survey and census design and data collection, but also data analysis, forecasting, and modeling. The 2012 total amount covers programs carried out by 14 designated statistical agencies and 115 policy, research, and program operation agencies, excluding the 2010 decennial census, which had a separate $0.4 billion in direct funding. 1
 
“Direct funding” is directly appropriated to an agency. Some agencies (e.g., the Census Bureau) carry out statistical activities for other agencies on a cost-reimbursable basis. The funding for these activities is allocated to the sponsoring agency and not to the data collection agency. OMB’s annual compilation of statistical programs generally includes the entire budget for each of the 14 agencies represented on the Interagency Council on Statistical Policy (ICSP); 2 other agencies determine which parts of their budgets should be included according to the OMB definition of statistical activities.
 
In fiscal 2012, the 14 ICSP agencies accounted for 34 percent of the total budget authority for statistical activities, excluding the 2010 census; with the 2010 census authority included, they accounted for 36 percent. Not all of the work of ICSP agencies is carried out in house. For fiscal 2012, OMB estimated that 37 percent of the total budget authority of ICSP agencies was used to purchase statistical services, such as data collection and analysis, from other organizations (U.S. Office of Management and Budget, 2011:App. A). Of total ICSP budget authority, 8 percent went to reimburse state and local governments for administrative records (e.g., birth and death records provided to the National Center for Health Statistics and unemployment insurance wage records provided to the Bureau of Labor Statistics); 21 percent was paid to private organizations for data collection and analysis services; and 8 percent was paid to other federal agencies, principally the Census Bureau.
 
In dollar terms, the National Center for Education Statistics, the Census Bureau, the Bureau of Labor Statistics, and the National Center for Health Statistics dedicated the largest amounts of their budgets to purchasing statistical services; by percentage of budget authority, the National Center for Education Statistics, the Bureau of Justice Statistics, the National Center for Health Statistics, and the National Center for Science and Engineering Statistics were the largest users of purchased services. These patterns have remained roughly constant over the past decade (see U.S. Office of Management and Budget, 2001:Table 3).
Value
Spending on statistical programs is a tiny fraction of overall federal spending: in fiscal 2012, the $6.2 billion in budget authority for all statistical programs identified by OMB amounted to less than 0.2 percent of the budget authority of about $3.7 trillion for the federal government. On a per capita basis, the $6.2 billion is equal to about $20 annually for every U.S. resident (315.1 million as of January 1, 2013; see http://www.census.gov).
 
A basic public policy question is the value of the statistical system for the federal government and the public. It is difficult to assign an overall valuation to the system or even to a specific agency or program (see National Research Council, 1985b:Ch. 3, App. 3A). A sense of value can be obtained in some instances by comparing the dollars spent on providing key statistics to the dollars that such statistics drive in the economy and society. For example, the prices and cost-of-living programs of the Bureau of Labor Statistics—including the Consumer Price Index (CPI), the Producer Price Index, the Consumer Expenditure Survey, and related activities—had an estimated budget authority of $206 million in fiscal 2012. 3 Output from the CPI component of the program is used for annual cost-of-living adjustments to payments for retirees and other beneficiaries under the Social Security Program, which provided $65.4 billion in benefits to 56.8 million people in December 2011: 4 a difference of 1 percentage point in the CPI amounts to almost $8 billion in additional (or reduced) Social Security benefits in the subsequent year. Annual changes in the CPI also affect changes in commercial and residential rents, public- and private-sector wages, and components of the federal income tax code. Reports of monthly changes in the CPI are a major input for Federal Reserve Board decisions in setting short-term interest rates and to financial decisions throughout the public and private sectors. There are other such examples of consequential statistics throughout government and the economy.
 
Some statistical programs may lack clear-cut links to public- and private-sector financial outlays, but they nonetheless serve other important purposes:
 
• Some programs provide information to inform policy makers and the public about the social and economic health of the nation, states, and localities. For example, the Bureau of Economic Analysis provides estimates of gross domestic product not only for the nation each quarter, but also for states and metropolitan areas each year, while the Census Bureau’s American Community Survey provides estimates of educational attainment, median income, immigration, poverty, and many other characteristics for large and small geographic areas annually (see National Research Council, 2007c).
• Some programs provide empirical evidence for developing and evaluating federal, state, local, and private-sector programs. For example, the American Housing Survey, sponsored by the Office of Policy Development and Research in the Department of Housing and Urban Development and conducted by the Census Bureau, provides valuable data on housing condition and housing finance with which to inform housing policy (see National Research Council, 2008b). The Commercial Buildings and Residential Energy Consumption Surveys, sponsored by the Energy Information Administration, provide valuable data for public- and private-sector policy making on end uses of various types of energy for heating, cooling, information technology, and other uses (see National Research Council, 2012c).
• Some programs provide input to important social science research that, in turn, informs the public and policy making. For example, the National Long-Term Care Survey, funded by the National Institute on Aging, produced unexpected findings of declining disability rates for older Americans over time (see National Research Council, 2009b), which has implications for understanding work-to-retirement transitions and the need for medical care by the elderly.
Structure
The United States has a highly decentralized statistical system in comparison with other developed countries (see Norwood, 1995). Essentially, the system grew by adding separate agencies whenever the need for objective empirical information on a particular aspect of the economy, society, or environment came to the fore (see Part II). Periodic recommendations from presidential commissions and other initiatives to consolidate one or more of the principal statistical agencies have never been adopted.
 
The statistical coordinating, clearance, review, and planning functions of the Statistical and Science Policy Office of OMB, which began in the 1930s (see Appendix A), provide an important integrative force for the U.S. statistical system. However, because statistics on agriculture, education, health, justice, labor, and other topics are housed in agencies in different cabinet departments with different statutory provisions and are reviewed by different congressional committees, the system has limited capability to respond to changing priorities by such means as reallocating budgets across subject areas or to streamline agency operations by such means as sharing data (with some important exceptions in recently enacted legislation—see Appendix A).
 
Figure B-1 shows the major statistical programs in the executive branch of government by cabinet department. At the center of the system (not shown in the figure) is the OMB Office of Information and Regulatory Affairs (OIRA), which includes the Statistical and Science Policy Office headed by the chief statistician of the United States, a senior executive civil service position. OIRA also includes the clearance officers who review individual survey and other information requests from most agencies: staff of the Statistical and Science Policy Office clear information requests from many of the principal statistical agencies and consult with the OIRA desk officers for the other agencies. Other parts of OMB recommend budgets for statistical agencies and programs in consultation with the Statistical and Science Policy Office.
Figure B-1 Organization of principal federal statistical agencies and programs, by department, 2013.

See text for discussion. SOURCE: Based on U.S. Office of Management and Budget (2012b:Table 1).

5thReportFigureB-1.png
The chief statistician chairs the Interagency Council on Statistical Policy (ICSP), whose 14 member agencies are in 9 cabinet departments and 3 independent agencies. In Figure B-1, the ICSP member agencies are designated by shaded circles; the open circles designate other agencies that expected to spend at least $500,000 on statistical services in fiscal 2012. Some of these agencies report directly to the secretary or other high-level official of their cabinet department; others are one, two, or even more layers further down the hierarchy (see Figure B-2). Several of these agencies have federal-state cooperative statistical programs that produce some of the nation’s most important statistics, such as birth and  eath rates from vital records maintained by state registrars and estimates of employment from wage records maintained by state employment security offices.
 
The 14 agencies and OMB have their yearly budget requests reviewed and approved by seven different subcommittees of the House and Senate Appropriations Committees (see Figure B-2). The fact that different statistical agencies fall into different components of the federal budget for purposes of annual congressional appropriations complicates the possibility of coordination of statistical programs across the government.
 
Figure B-2 Members of the Interagency Council on Statistical Policy: Organizational location and relevant congressional appropriations subcommittee based on subcommittee jurisdictions in the 113th Congress.
 
5thReportFigureB-2.png
Finally, there are some important federal agencies that have statistical activities that are not included in the OMB annual compilation because they are not part of the executive branch. These agencies include the Congressional Budget Office, which develops and applies projection models for the budgetary impact of current and proposed federal programs; the Federal Reserve Board, which compiles the widely used Flow of Funds report and other statistical series and periodically conducts the Survey of Consumer Finances; and the U.S. Government Accountability Office, which uses statistical data in evaluations of government programs.
Office of Management and Budget
The 1995 reauthorization of the Paperwork Reduction Act of 1980 and other legislation give the OIRA the authority to approve all agency information collection requests, including all survey and other statistical information requests. OIRA also reviews all proposed federal regulations.  The chief statistician’s office in OIRA (the Statistical and Science Policy Office) establishes statistical policies and standards, identifies priorities for improving programs, evaluates statistical programs for compliance with OMB guidance, reviews statistical agency budgets, approves information collections for many of the principal statistical agencies, provides guidance to OIRA desk officers who review statistical information requests from other federal agencies, and coordinates U.S. participation in international statistical activities. The chief statistician’s office currently has a staff of six professionals, some of whom focus largely on science policy.
 
As required by the Paperwork Reduction Act, the chief statistician’s office annually issues Statistical Programs of the United States Government  the “blue book”; see U.S. Office of Management and Budget, 2012b). It also prepares a chapter each year in the Analytical Perspectives volume of the President’s budget, which provides a cross-cutting analysis of the budget requests for the principal statistical agencies (see U.S. Office of Management and Budget, 2012a). The chief statistician chairs the ICSP, which began operating informally in the late 1980s and was authorized by statute in 1995. The chief statistician’s office also sponsors the Federal Committee on Statistical Methodology and other interagency collaborations, such as the Interagency Forum on Aging-Related Statistics and the Interagency Forum on Child and Family Statistics (see discussion in Part II under Practice 13). Appendix A provides background information on the Paperwork Reduction Act, statistical policy directives issued by the chief statistician’s office, and other legislation that the office oversees for the U.S. statistical system.
Principal Statistical Agencies
This section provides information—primarily from agency websites (see Appendix E) and OMB publications—on 13 of the 14 members of the ICSP, excluding only the Office of Environmental Information in the Environmental Protection Agency, which is not a self-contained statistical unit. The information provided for the 13 agencies includes origins, authorizing legislation or other authority, status of head (presidential appointee, career senior executive service official), budget and full-time permanent staffing levels in 2012 (see U.S. Office of Management and Budget, 2012b:Table 1 and App. B), and principal programs. The agencies are discussed in alphabetical order.
Bureau of Economic Analysis
The Bureau of Economic Analysis (BEA) is part of the Economics and Statistics Administration in the U.S. Department of Commerce (as is the Census Bureau), which is headed by the Under Secretary for Economic Affairs. The BEA director is a career senior executive service appointee, and the agency has a full-time staff of about 485 people and direct funding in fiscal 2012 of $92.2 million.
 
BEA’s history traces back to 1820 when the Secretary of the Treasury was directed by Congress to compile and publish statistics on U.S. foreign commerce. Three 20th-century predecessors of BEA were all located in the Department of Commerce: the Bureau of Statistics (1903–1912); the Bureau of Foreign and Domestic Commerce (1912–1945); and the Office of Business Economics (1945–1972).
 
BEA produces statistics on the performance of the nation’s economy. Although it collects some source data, it primarily compiles data from the Census Bureau, the Bureau of Labor Statistics, and other agencies as input to estimating the National Income and Product Accounts (NIPAs), which include estimates of the nation’s gross domestic product (GDP) and related measures. The GDP, which was recognized by the Department of Commerce as its greatest achievement of the 20th century in a December 2009 ceremony, has major influence on U.S. financial markets.
 
Since the NIPAs were first developed in the aftermath of the Great Depression, BEA has extended its estimates to cover a wide range of economic activities for the nation, regions, and industries and also for the nation’s position in the world economy. BEA also conducts research and development on “satellite accounts” in such areas as health care, transportation, and research and development investments. Satellite accounts enable experimentation with new accounts before they are ready to incorporate into the main accounts and with nonmarket sectors that are not part of the market-based NIPAs.
Bureau of Justice Statistics
The Bureau of Justice Statistics (BJS), in the U.S. Department of Justice (DOJ), was formally established by the Justice Systems Improvement Act of 1979 (P.L. 96-157), inheriting statistical functions that had previously been vested in an office of the Law Enforcement Assistance Administration, established in 1968. BJS is housed in the DOJ’s Office of Justice Programs (OJP), which also contains the National Institute of Justice (a research agency) and other agencies that are primarily focused on providing grant and technical assistance to state and local governments and law enforcement agencies. BJS’s director is a presidential appointee (without Senate confirmation as of August 2012) and reports to an assistant attorney general for OJP. The bureau has a full-time staff of about 45 and direct funding in fiscal 2012 of $53 million.
 
The centerpiece of BJS’s data collections is the National Crime Victimization Survey (originally the National Crime Survey), which has served as one of the nation’s principal measures of crime (particularly crime not reported to police) since its full-scale implementation in 1972. Data collection for most BJS surveys is conducted by the Census Bureau or private contractors, and OMB estimates that 86 percent of BJS’s fiscal 2012 budget authority was spent on purchased services.
 
BJS publishes annual statistics on criminal victimization, populations under correctional supervision, law enforcement management and administration, and case processing in the state and federal courts. Its periodic data series cover the administration of law enforcement agencies and correctional facilities, prosecutorial practices and policies, state court case processing, felony convictions, criminal justice expenditure and employment, civil case processing in state courts, and special studies on other criminal justice topics.
Bureau of Labor Statistics
The Bureau of Labor Statistics (BLS) is an agency of the Department of Labor. It is responsible for the production of some of the nation’s most sensitive and important economic data, including unemployment statistics and consumer and producer price indexes, which are closely watched by the public, Congress, other federal agencies, state and local governments, business, and labor. The BLS commissioner is a presidential appointee, subject to Senate confirmation, for a fixed term of 4 years; the agency has a full-time staff of about 2,050 people and direct funding in fiscal 2012 of $609 million.
 
The history of the BLS dates back to 1884, when the Bureau of Labor was established in the Interior Department to collect information about employment and labor. It was made an independent (subcabinet) agency by the Department of Labor Act in 1888; it was made part of the Department of Commerce and Labor (as the Bureau of Labor) in 1903 and transferred to the newly created Department of Labor in 1913.
 
BLS programs use a variety of data collection methods and sources. Certain wage, benefit, employment, and price data are collected by BLS staff located throughout the country, who contact employers, households, and businesses directly. BLS has contractual arrangements with various state agencies to collect much of its employment and workplace safety and health data. Contractual arrangements with the Census Bureau support collection of several programs, including the Current Population Survey (the source of monthly unemployment statistics) and the Consumer Expenditure Survey (the source of the market baskets for the CPI). Some BLS data, such as data for the various National Longitudinal Surveys, are collected by private contractors. Finally, certain BLS data, such as information on work stoppages, rely on information from secondary sources.
 
BLS’s surveys, indexes, and statistics fall into four main categories:
 
1. consumer expenditures and prices, including the CPI, producer price index, and U.S. import and export prices indexes;
2. the labor force, including monthly data on employment from households and business establishments, monthly and periodic data on
unemployment, time use, job openings and labor turnover, occupational employment and projections of trends, mass layoffs, and longitudinal data on the work experience of cohorts of the population;
3. compensation and working conditions, including the employment cost index, and workplace injuries and fatalities; and
4. productivity.
Bureau of Transportation Statistics
The Bureau of Transportation Statistics (BTS) is an agency in the Research and Innovative Technology Administration (RITA) of the Department of Transportation. RITA also includes the Intelligent Transportation Systems Joint Program Office; the Office of Research, Development, and Technology; the Transportation Safety Institute; and the Volpe National Transportation Systems Center.
 
BTS’s director is a career senior executive service appointee who reports to the head of RITA. Prior to 2004, the director was a presidential appointee with a fixed term of 4 years who reported directly to the Secretary of Transportation. BTS has a full-time staff of about 70 and direct funding of $25 million in fiscal 2012.
 
BTS was established by the 1991 Intermodal Surface Transportation Efficiency Act and began operations in late 1992. It was moved to the newly created RITA by the Norman Y. Mineta Research and Special Programs Improvement Act of 2004. Prior to the establishment of BTS, statistical programs of the Department of Transportation focused exclusively on specific modes of transportation (highways, airlines, railroads, etc.), except for the first 10 years of the department’s existence (1967–1977), when the Office of the Secretary funded intermodal surveys on commodity flows and long-distance personal transportation.
 
BTS is charged to produce an annual report on transportation statistics, develop intermodal data on commodity and passenger flows, administer the National Transportation Library, and carry out other functions to ensure that the department, the states, and other federal agencies have available comprehensive information on the nation’s transportation systems. BTS also operates the Office of Airline Information, which was transferred to it from the now-defunct Civil Aeronautics Board. BTS contracts with the Census Bureau for major surveys.
Census Bureau
The Census Bureau is part of the Economics and Statistics Administration in the Department of Commerce (as is BEA), headed by the Under Secretary for Economic Affairs. It conducts population and economic censuses and a wide array of surveys.
 
Population censuses are required by the U.S. Constitution to be conducted every 10 years for reapportioning among the states the 435 seats in the House of Representatives. The first censuses were conducted by U.S. marshals under the authority of the Secretary of State. Beginning in 1850, a separate census office was established each decade to supervise the census. In 1902 a permanent Census Bureau was established; it was made part of the new Department of Commerce and Labor in 1903 and moved to the newly created Department of Commerce in 1913. Title 13 of the U.S. Code includes the major legal provisions related to the Census Bureau, including strict provisions for protecting the confidentiality of population and business information.
 
The director of the Census Bureau is appointed by the President with Senate confirmation for a fixed 5-year term (that can be renewed once) to begin in years ending in 2 and 7. 5 The bureau has about 6,840 full-time staff, excluding staff hired for the decennial census, and direct funding in fiscal 2012 of $964.8 million including $446.1 million for the decennial census.
 
The major periodic activity of the Census Bureau is the decennial population census, which in 2010 consisted of basic questions on age, sex, race, Hispanic origin, relationship to household head, and housing tenure (own, rent). The Census Bureau also conducts the continuous American Community Survey (ACS), which replaced the content included for a sample of households in the census from 1940 through 2000 (the “long form”), a large number of household surveys (most under contract for other agencies), and a panoply of censuses and surveys of business establishments and governments. The Census Bureau produces annual population and housing estimates (in cooperation with state and local governments), estimates of poverty, median income, and health insurance coverage using statistical models for small areas, and geographic products based on its Topologically Integrated Geographic Encoding and Referencing (TIGER) system.
 
The Census Bureau has a substantial portfolio of reimbursable work for other agencies, which is primarily for the conduct of surveys, such as the Current Population Survey (for BLS), the American Housing Survey (for the Department of Housing and Urban Development), the Consumer Expenditure Survey (for BLS), and many others. OMB estimates that the Census Bureau will conduct $285 million of reimbursable work in fiscal 2013 (U.S. Office of Management and Budget, 2012b:Table 2).
Economic Research Service
The Economic Research Service (ERS), along with the National Agricultural Statistics Service and two other agencies, reports to the Under Secretary for Research, Education, and Economics in the U.S. Department of Agriculture (USDA). The administrator of ERS is a career senior executive service appointee; the agency has a full-time staff of about 355 and fiscal 2012 direct funding of $77.7 million.
 
The origins of ERS trace back to 1905, when USDA established the Office of Farm Management, renamed the Office of Farm Management and Farm Economics in 1919. The Office’s research areas included farm organization, cost of production, farm labor, farm finance, land economics, agricultural history, and rural life studies. Several reorganizations took place, and in 1961, USDA created the Economic Research Service with responsibility to conduct economic research and policy analysis that informs program and policy decisions throughout USDA. The agency’s mission is to anticipate food, agricultural, agri-environmental, and rural development issues that are on the horizon and conduct peer-reviewed economic research so that research findings are available when issues require decisions by policy makers. As a statistical agency, ERS does not make recommendations; instead, it designs its research to show the consequences of taking alternative policy or programmatic pathways.
 
ERS is also the primary source of statistical indicators that, among other things, gauge the health of the farm sector (including farm income estimates and projections), assess the current and expected performance of the agricultural sector (including trade and productivity), provide measures of food insecurity in the United States and abroad, and measure dimensions of food availability and access. ERS jointly funds with the National Agricultural Statistics Service a major survey on farm household and business income and crop practices, the Agricultural Resources Management Survey (ARMS).
Energy Information Administration
The Energy Information Administration (EIA) is an agency of the Department of Energy (DOE); its administrator is a presidential appointee with Senate confirmation. EIA has a full-time staff of about 365 and fiscal 2012 direct funding of $105 million.
 
EIA was created by Congress in 1977 as part of the newly established Department of Energy. Its mission is to provide policy-independent energy data, forecasts, and analyses in order to promote sound policy making, efficient markets, and public understanding regarding energy and its interaction with the economy and the environment. To assure EIA’s independence, the Department of Energy Organization Act specifies that EIA’s products are not subject to clearance by executive branch officials; in particular, the administrator does not need to obtain the approval of any other DOE official for data collection and analysis, and he or she does not need to “obtain the approval of any other officer or employee of the United States” before publishing energy data and analysis reports (42 USC 7135(d)).
 
Many EIA data products, such as weekly, monthly, and annual data on petroleum and natural gas supply, deal with specific industries; others contain data on all fuel types. EIA energy supply surveys, for which participation is mandatory, are conducted by private contractors, who survey energy producers, users, and transporters, and certain other businesses. Data on energy consumption are collected for households, commercial  buildings, manufacturing, and transportation. Analyses prepared by EIA staff cover energy economics, technology, production, prices, distribution, storage, consumption, and environmental effects.
 
EIA forecasts cover all energy types and include supply, consumption, prices, and other factors. Short-term forecasts cover 1–2 years; 20-year projections are also developed and often serve as the baseline for independent analyses of policy proposals that are prepared by EIA at the request of Congress or the administration. More than three-quarters of EIA’s resources are used for energy data collection and dissemination; the rest are used to support forward-looking forecasts, projections, and analyses.
National Agricultural Statistics Service
The National Agricultural Statistics Service (NASS) is under the Under Secretary for Research, Education, and Economics in USDA (as is ERS). The administrator of NASS is a career senior executive service appointee; NASS has about 1,200 full-time staff and direct funding of $158.6 million in fiscal 2012.
 
The foundation of NASS began with the establishment of USDA in 1862. Agricultural supply information was one of the purposes of the new department. The first official report on the condition of crops was issued in July 1863. This basic, mission-oriented program continues today in the USDA forecasts and estimates provided by the NASS Agricultural Statistics Board. NASS’s responsibilities are authorized under the Agricultural Marketing Act of 1946 (7 USC 1621–1627) and the Census of Agriculture Act of 1997, P.L. 105-113 (7 USC 2204g), which transferred responsibility for the census of agriculture and other special studies from the Department of Commerce to NASS. Conducted every 5 years, the census provides comprehensive information about the nation’s agriculture down to the county level.
 
NASS services the data needs of many agencies inside and outside the Department. NASS collaborates with state departments of agriculture and land-grant universities to meet state, local, and national needs for agricultural statistics. Through cooperative agreements going back as far as 1917 and memoranda of understanding, NASS also provides data collection and statistical services to other federal agencies and provides statistics to the public through trust fund agreements with private producer organizations when federal funding is inadequate. NASS also collaborates with universities throughout the United States, including the land-grant universities, on research to improve statistical methodologies  and practices. A recent partnership with the National Institute of Statistical Sciences brings together academic and NASS researchers to solve challenges facing the agricultural statistics program.
 
Slightly more than a third of the agency’s staff is located at its Washington, DC, headquarters offices, with the rest of the staff located at the National Operations Center (NOC) and in offices around the country serving all 50 states and Puerto Rico. NASS works with its state and regional field offices to carry out hundreds of surveys every year and prepares reports covering virtually every aspect of U.S. agriculture. Examples include production and supplies of food and fiber, prices paid and received by farmers, farm labor and wages, farm finances, chemical use, and changes in the demographics of U.S. producers. All field interviewing staff are obtained through contracting with the National Association of State Departments of Agriculture (NASDA), and a federal telephone data collection staff is supplemented with NASDA staff on an as-needed basis.
National Center for Education Statistics
The National Center for Education Statistics (NCES) is part of the Department of Education’s Institute of Education Sciences (IES, which also includes three research and evaluation centers). The NCES commissioner is a presidential appointee for a fixed term of 6 years (as of August 2012 without Senate confirmation); the agency has a full-time staff of about 85 people and direct funding of $302 million in fiscal 2012.
 
NCES’s origins date back to 1867 when Congress established a Department of Education and gave it a primary mission of “collecting such statistics and facts as shall show the condition and progress of education in the several States and Territories, and of diffusing such information respecting the organization and management of schools and school systems and methods of teaching” (14 Stat 434). The legislation also charged the Commissioner of Education to issue an annual report. However, only 2 years later the department was abolished and the renamed Office of Education was transferred to the Department of the Interior, where it remained through 1939. The Office of Education was part of the newly created Federal Security Agency from 1939 to 1953, when it was made part of the newly created Department of Health, Education, and Welfare. A separate Department of Education was re-established in 1980.
 
A major function of the Office of Education throughout its history was the collection and publication of education statistics. NCES was established in 1965 as a staff office reporting to the Commissioner of Education. NCES received statutory authority in 1974; in 1980 it was made part of the Office of Educational Research and Improvement, which in 2002 became the IES. Supporting the independence of NCES, the Education Sciences Reform Act of 2002 which created IES, stipulated that “each Commissioner [head of one of IES’s constituent centers], except the Commissioner for Education Statistics, shall carry out such Commissioner’s duties . . . under the supervision and subject to the approval of the Director” of IES (20 USC 9517(d)).
 
NCES has an extensive survey program, including longitudinal surveys that follow the educational experience of cohorts of the U.S. population from early childhood through adulthood, periodic surveys of adult literacy, and international studies of educational achievement. It also collects the “Common Core of Data” from administrative records of state and local K–12 educational agencies, and it collects data for the Integrated Postsecondary Education Data System. It regularly assesses the educational knowledge and achievement of primary and secondary school students in the National Assessment of Educational Progress (NAEP). It also administers the Statewide Longitudinal Data Systems (SLDS) program, which provides grants to the states to develop longitudinal databases of student records for analyzing student performance and for identifying methods to improve achievement.
 
NCES contracts for a substantial portion of its work, including not only data collection, but also data analysis and preparation of reports. In 2012, 97 percent of its estimated budget authority was spent on data and analysis from state agencies, the Census Bureau, and private contractors.
National Center for Health Statistics
The National Center for Health Statistics (NCHS) is part of the Centers for Disease Control and Prevention (CDC) in the Department of Health and Human Services (HHS). The NCHS director is a career senior executive service appointee; NCHS has a full-time staff of about 470 people and direct funding of $139 million in fiscal 2012.
 
NCHS’s roots lie in two formerly separate historical strands of the provision of national health statistics. The first strand is vital statistics on births, deaths, and other life events, which traces back to 1902, when Congress gave the newly created permanent Census Bureau the authority to establish registration areas to produce nationally comparable vital statistics by working with state agencies. This function was transferred in 1946 to the Federal Security Administration, which was folded into the new Department of Health, Education, and Welfare in 1953.
 
The second strand is general statistics on the nation’s health, which were authorized in the 1956 National Health Survey Act. NCHS was created in 1960 as the merger of the National Office of Vital Statistics and the National Health Survey Division; it was relocated every few years in HHS until its last relocation in 1987, when it was made part of CDC. In 2005 it became one of three centers reporting to the newly created Coordinating Center for Health Information and Service in CDC.
 
NCHS has four major programs:
 
1. The National Health Interview Survey (NHIS), in continuous operation since 1956, collects a wide range of information on self-reported health status and conditions and use of health care services by the population.
2. Several surveys collect information from health care providers, including nursing homes, hospitals, and outpatient facilities.
3. The National Health and Nutrition Examination Survey (NHANES) ascertains self-reported information on health and dietary intake and also, by use of mobile examining units, obtains extensive information from physical examinations and laboratory tests.
4. The nation’s basic vital statistics are collected and maintained. In 2012, according to OMB, 78 percent of NCHS’s estimated budget authority was used to purchase data collection and reporting services from state and local governments, the Census Bureau, and private contractors.
National Center for Science and Engineering Statistics
The National Center for Science and Engineering Statistics (NCSES) is part of the Social, Behavioral, and Economic Sciences Directorate of the National Science Foundation (NSF). Its director is a career senior executive service appointee; it has a full-time staff of about 45 people and direct funding in fiscal 2012 of $44 million.
 
NCSES was formerly the Division of Science Resources Statistics and before that the Division of Science Resources Studies. It became NCSES, with an expanded mandate to serve as a “central Federal clearinghouse for the collection, interpretation, analysis, and dissemination of objective data on science, engineering, technology, and research and development,” with passage of the America COMPETES Reauthorization Act of 2010 (Section 505; 42 USC 1862).
 
At its founding in 1950, NSF was charged to maintain a register of scientific and technical personnel so that the nation would be able to mobilize the scientific and technical work force in the event of a major war. Although no longer required to maintain a complete register, NSF has continued (by the terms of the National Science Foundation Act of 1950, as amended) to have responsibility “to provide a central clearinghouse for the collection, interpretation, and analysis of data on scientific and engineering resources and to provide a source of information for policy formulation by other agencies of the Federal Government” (42 USC 1862). NSF also has a congressional mandate from 1980 to provide information on women and minorities in science and engineering.
 
The NSF mandates provide the basis for two major statistical programs in NCSES: censuses and surveys of people trained in or working in science and engineering positions and of new bachelor’s graduates, new master’s graduates, and doctoral recipients in science and engineering fields; and surveys of research and development (R&D) expenditures by private industry and academic institutions and of R&D funding by the federal government. To support its programs, 75 percent of NCSES’s estimated budget authority in 2012 was used to purchase data collection and other services from the Census Bureau and private contractors. NCSES also serves as staff to the National Science Board in producing the biennial congressionally mandated Science and Engineering Indicators Report, which uses data from all 11 of the NCSES surveys.
Office of Research, Evaluation, and Statistics
The Office of Research, Evaluation, and Statistics (ORES) is located in the Social Security Administration (SSA), which became independent from HHS in 1995. ORES reports to the SSA Deputy Commissioner for Retirement and Disability Policy. ORES is headed by an associate commissioner, who is a career senior executive service appointee; it has a full-time staff of about 85 people and direct funding of $29 million in fiscal 2012.
 
SSA began as the Social Security Board in 1935; it became part of the Federal Security Agency in 1939, part of the Department of Health, Education, and Welfare in 1953, and part of HHS in 1980; it regained independent agency status in 1995. From the outset, SSA has had a research, statistics, and evaluation function.
 
ORES conducts research and evaluation on the effects of the Social Security and Supplemental Security Income Programs and proposed changes in those programs on individuals, the economy, and program solvency. It provides data on program benefits and covered workers and develops and operates microsimulation models that estimate the costs and distributional effects of proposed changes in the programs. Periodically, it has sponsored surveys of specific populations, such as new beneficiaries, linked with SSA administrative records.
Statistics of Income Division
The Statistics of Income Division (SOI) is housed in the Office of Research, Analysis, and Statistics of the Internal Revenue Service (IRS) in the Department of the Treasury. The director is a career senior executive service appointee and leads a full-time staff of approximately 150 employees with direct funding of $38.6 million in fiscal 2012.
 
SOI’s history traces back to the enactment of authority to levy individual income taxes in the 16th amendment to the U.S. Constitution, which was ratified in 1913. Section 21 of the Revenue Act of 1916 mandated the annual “publication of statistics reasonably available with respect to the operation of the Income tax law.” Identical language is found in the current Internal Revenue Code (see 26 USC 6108).
 
SOI provides income, financial, and tax information to the user community based largely on individual and corporate tax returns and on returns filed by most tax-exempt organizations. It also provides periodic data derived from other returns and schedules, such as estate and gift taxes, foreign income and taxes, and gains and losses from sales of capital assets. Upon written request, SOI data are available to staff in the Department of the Treasury and the Congressional Joint Committee on Taxation for policy analysis and revenue estimation. Likewise data are also available to the Congressional Budget Office for modeling Social Security and Medicare programs and no other purpose. Selected tax return data are also available, under strict confidentiality protection provisions, for use by the Census Bureau, the Bureau of Economic Analysis, and the National Agricultural Statistics Service in structuring censuses and national economic accounts and conducting related statistical activities authorized by law. (See discussion in Appendix A of the Confidential Information Protection and Statistical Efficiency Act.)
Other Statistical Programs
This section briefly describes seven statistical programs that are conducted or sponsored by agencies of the federal government other than the principal statistical agencies. The programs were selected purposively to illustrate the breadth and depth of the federal government’s statistical portfolio.They are in alphabetical order.
Health and Retirement Study
The Health and Retirement Study (HRS) is a longitudinal panel survey with over 20,000 respondents representing people ages 51 and older in the United States. It provides in-depth information on middle- and older-aged people’s transitions from the workforce to retirement, savings behavior and pension plans, physical and cognitive health, disability, family structure, health care expenditures, and many other aspects of financial, social, physical, and mental well-being. The HRS began in 1992 and currently introduces a new cohort of people ages 51–56 every 6 years. People in the sample are interviewed in-person or by telephone every 2 years. The HRS is conducted by the University of Michigan with support from the National Institute on Aging and SSA.
 
The HRS has provided data for a wide range of path-breaking research studies, has made innovations in data collection methods, and has inspired similar efforts in many countries around the world. Similar panel surveys in other countries include the English Longitudinal Study of Ageing (ELSA); the Survey of Health, Ageing, and Retirement in Europe (SHARE); the New Zealand Health, Work and Retirement Survey; the Korean Longitudinal Study of Aging; the Mexican Health and Aging Study (MHAS); the Longitudinal Aging Study in India (LASI); the Japanese Study of Aging and Retirement (JSTAR); and the China Health and Retirement Longitudinal Study (CHARLS).
Medical Expenditure Panel Survey
The Medical Expenditure Panel Survey (MEPS) is a statistical program of the Agency for Healthcare Research and Quality (AHRQ) in HHS. MEPS is the core health care expenditure survey in the United States, with a primary analytical focus directed to the topics of health care access, cost, and coverage. MEPS was designed to provide data for health care policy analysis and research; it was first conducted in 1977 and 1987 under different names, and became a continuous survey in 1996. The MEPS budget for 2012 was about $59 million.
 
MEPS consists of a family of three interrelated parts: the household component, the medical provider component, and the insurance component. The household survey collects information from household members and their health care providers and employers in order to construct a complete picture of medical care use, expenditures, and health insurance coverage and reimbursements. Households are in a MEPS panel for five rounds of interviewing that cover 2 years, so that patterns of medical care and expenditures can be observed over time; a new household panel begins every year. Data for the MEPS household and medical provider surveys are collected by private contractors; the household survey sample of about 14,000 households per year is drawn from the NCHS National Health Interview Survey. The MEPS insurance component collects data each year from a sample of about 30,000 private- and public-sector employers on the health insurance plans they offer their employees. The collected data include the number and types of private insurance plans offered (if any), premiums, contributions by employers and employees, eligibility requirements, benefits associated with these plans, and employer characteristics.Data for this component of MEPS are collected by the Census Bureau.
National Agricultural Workers Survey
The National Agricultural Workers Survey (NAWS) is an activity of DOL’s Employment and Training Administration. It provides data on wage and migration history, type of crops worked, unemployment benefits, housing, health care, use of public programs, and other characteristics of the U.S. crop labor force. The information, which is used by numerous federal agencies for occupational injury and health surveillance, estimating the need for services for workers, allocating program dollars to areas of greatest need, and program design and evaluation, is obtained directly from farm workers through personal interviews.
 
Since 1988, when the survey began, nearly 53,000 workers have been interviewed. The survey samples crop workers in three cycles each year to reflect the seasonality of agricultural production and employment. Workers are located at their farm job sites. During the initial contact, arrangements are made to interview the respondent at home or at another location convenient to the respondent. Depending on the information needs and resources of the various federal agencies that use NAWS data, between 1,500 and 4,000 workers are interviewed each year.
National Automotive Sampling System
The National Automotive Sampling System (NASS) is an administrative-records–based data collection system of the National Center for Statistics and Analysis (NCSA) of the National Highway Traffic Safety Administration in the Department of Transportation. NASS was created in 1979 as part of a nationwide effort to reduce motor vehicle crashes, injuries, and deaths on U.S. highways. NASS samples accident reports of police agencies in randomly selected areas of the country.
 
NASS has two components, one on crashworthiness and one on general estimates. For the crashworthiness component, NCSA field researchers collect detailed information from police accident reports for selected crashes on a wide range of factors, including exterior and interior vehicle damage, occupant injury, and environmental conditions at the time of the crash. For the general estimates component, which covers a larger sample of crashes, only basic information is recorded from the police accident reports. The NASS infrastructure is also used for special studies and surveys, such as the National Motor Vehicle Crash Causation Survey, conducted in 2005–2007, which sampled police accident reports in real time and obtained on-scene information in addition to the information reported by the police. NASS will undergo a major redesign over the next few years.
National Resources Inventory
The National Resources Inventory (NRI) is a statistical program of the Natural Resources Conservation Service (NRCS) in USDA. The current NRI is a longitudinal survey of soil, water, and related environmental resources designed to assess conditions and trends on nonfederal U.S. land parcels. NRCS has conducted the NRI in cooperation with the Iowa State University Center for Survey Statistics and Methodology since 1977.
 
The NRI was conducted on a 5-year cycle from 1982 to 1997; beginning in 2000 it is now conducted annually (with major data releases occurring at 5-year intervals). Before 2000, NRI data were collected every 5 years for 800,000 sample sites; annual NRI data collection covers slightly less than 200,000 sample sites. Year-by-year data on conditions for the same sites enable analysis of the effects of resource conservation programs and other applications.
National Survey on Drug Use and Health
The National Survey on Drug Use and Health (NSDUH) is a continuing survey of the Center for Behavioral Health Statistics and Quality (CBHSQ) in the Substance Abuse and Mental Health Services Administration (SAMHSA), which is part of the National Institutes of Health of HHS. It is the nation’s primary data system for collecting data on the incidence and prevalence of substance abuse and adverse health consequences associated with drug abuse from the civilian, noninstitutionalized population of the United States for people aged 12 and older.
 
NSDUH (formerly called the National Household Survey on Drug Abuse) was fielded periodically from 1972 to 1990 and annually beginning in 1991. It has been conducted for SAMHSA by a private firm, RTI International, since 1988. The sample size each year is about 70,000 people, with oversampling of teenagers and young adults.
 
The total budget for the statistical activities of SAMHSA is about $130 million. This covers not only NSDUH, but also the Behavioral Health Services Information System and its associated surveys (the primary data source for information on the nation’s substance abuse treatment system and outcomes), the Drug Abuse Warning Network (DAWN) (a public health surveillance system, which monitors drug-related visits to hospital emergency departments, as well as drug-related deaths investigated by medical examiners and coroners), and other programs.
Panel Study of Income Dynamics
The Panel Study of Income Dynamics (PSID) is a longitudinal survey that has followed several thousand families since it began in 1968. It is conducted by the Survey Research Center of the Institute for Social Research of the University of Michigan with funding from a consortium of agencies. Originally, PSID was funded by the Office of Economic Opportunity; currently, the study’s major funding source is the National Science Foundation, with substantial additional funding from the National Institute on Aging, the National Institute of Child Health and Human Development, the Office of the Assistant Secretary for Planning and Evaluation of HHS, the Economic Research Service of USDA, the Department of Housing and Urban Development, the Department of Labor, and the Center on Philanthropy at Indiana University–Purdue University, Indianapolis.
 
The PSID emphasizes the dynamic aspects of economic and demographic behavior, but its content is broad, including sociological and psychological measures. From 1968 to 1996, the PSID interviewed individuals in the original sample of about 4,800 families every year, whether or not they were living in the same dwelling or with the same people. In 1997 interviewing was changed to every other year, the original sample was reduced, and a sample of Hispanic families that had been added in 1990 was replaced by a sample of post-1968 immigrant families and their adult children of all ethnic groups. The current sample of families, including those formed by children leaving their parental homes, is about 8,700.
 
Since 1968, more than 3,000 journal articles, books and book chapters, government reports, working papers, and dissertations have been based on the PSID. The PSID was founded to study poverty and the effects of programs to combat poverty, and an important early finding was that family structure changes such as divorce are as important to family well-being as employment. As the survey has added content and extended its period of observation, the data have also contributed importantly to studies of intergenerational patterns of work, welfare receipt, and other behaviors; international comparisons with panel data from other countries; neighborhood effects on family well-being (using data files augmented with census-based characteristics of sample members’ communities); and long-term trends in marital and fertility histories and living arrangements.
Footnotes
1
The total number of policy, research, and program operation agencies treats each institute or center of the National Institutes of Health as a separate agency.
2
The 14 agencies are the Bureau of Economic Analysis; Bureau of Justice Statistics; Bureau of Labor Statistics; Bureau of Transportation Statistics; Census Bureau; Economic Research Service of the Department of Agriculture; Energy Information Administration; National Agricultural Statistics Service; National Center for Education Statistics; National Center for Health Statistics; National Center for Science and Engineering Statistics; Office of Environmental Information of the Environmental Protection Agency; Office of Research, Evaluation, and Statistics of the Social Security Administration; and the Statistics of Income Division of the Internal Revenue Service.
5
The fixed term was signed into law in August 2012; previously, the director served at the pleasure of the president.

C Fundamental Principles of Official Statistics of the Statistical Commission of the United Nations

The text below is excerpted from the report of the Statistical Commission on its Special Session, held in New York City, April 11–15, 1994. Official Records of the Economic and Social Council, 1994, Supplement No. 9. Available: http://unstats.un.org/unsd/dnss/gp/f...rinciples.aspx [January 2013].

Fundamental Principles of Official Statistics

The Statistical Commission,
 
• Bearing in mind that official statistical information is an essential basis for development in the economic, demographic, social and environmental fields and for mutual knowledge and trade among the States and peoples of the world,
• Bearing in mind that the essential trust of the public in official statistical information depends to a large extent on respect for the fundamental
values and principles which are the basis of any society which seeks to understand itself and to respect the rights of its members,
• Bearing in mind that the quality of official statistics, and thus the quality of the information available to the Government, the economy and the public depends largely on the cooperation of citizens, enterprises, and other respondents in providing appropriate and reliable data needed for necessary statistical compilations and on the cooperation between users and producers of statistics in order to meet users’ needs,
• Recalling the efforts of governmental and non-governmental organizations active in statistics to establish standards and concepts to allow comparisons among countries,
• Recalling also the International Statistical Institute Declaration of Professional Ethics,
• Having expressed the opinion that resolution C (47), adopted by the Economic Commission for Europe on 15 April 1992, is of universal significance,
• Noting that, at its eighth session, held at Bangkok in November 1993, the Working Group of Statistical Experts, assigned by the Committee
on Statistics of the Economic and Social Commission for Asia and the Pacific to examine the Fundamental Principles, had agreed in principle to
the ECE version and had emphasized that those principles were applicable to all nations,
• Noting also that, at its eighth session, held at Addis Ababa in March 1994, the Joint Conference of African Planners, Statisticians and Demographers, considered that the Fundamental Principles of Official Statistics are of universal significance,
 
Adopts the present principles of official statistics:
 
Principle 1. Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens’ entitlement to public information.
 
Principle 2. To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.
 
Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.
 
Principle 4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.
 
Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents.
 
Principle 6. Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.
 
Principle 7. The laws, regulations and measures under which the statistical systems operate are to be made public.
 
Principle 8. Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.
 
Principle 9. The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels.
 
Principle 10. Bilateral and multilateral cooperation in statistics contributes to the improvement of systems of official statistics in all countries.

D European Statistics Code of Practice for the National and Community Statistical Authorities

The text below reproduces the language adopted by the European Statistical System Committee on September 28, 2011.
Preamble
The vision of the European Statistical System 1
“The European Statistical System will be a world leader in statistical information services and the most important information provider for the European Union and its Member States. Based on scientific principles and methods, the European Statistical System will offer and continuously improve a programme of harmonised European statistics that constitutes an essential basis for democratic processes and progress in society.”
The mission of the European Statistical System
“We provide the European Union, the world and the public with independent high-quality information on the economy and society on European, national, and regional levels and make the information available to everyone for decisionmaking purposes, research and debate.” To realize this mission and vision, the members of the European Statistical System strive for joint cooperation and continuous interaction with users according to the Principles of the European Statistics Code of Practice and general quality management principles including commitment of leadership, partnership, staff satisfaction, and continuous improvement, in addition to integration and harmonization.
The European Statistics Code of Practice
The European Statistics Code of Practice is based on 15 Principles covering the institutional environment, the statistical production processes and the output of statistics. A set of indicators of good practice for each of the Principles provides a reference for reviewing the implementation of the Code. The quality criteria for European Statistics are defined in European Statistical Law. 2 Statistical authorities, 3 comprising the Commission (Eurostat), National Statistical Institutes, and other national authorities responsible for the development, production, and dissemination of European Statistics, 4 together with governments, ministries, and the European Council, commit themselves to adhere to the Code. The Principles of the Code of Practice together with the general quality management principles represent a common quality framework in the European Statistical System.
Institutional Environment
Institutional and organisational factors have a significant influence on the effectiveness and creditability of a statistical authority developing, producing, and disseminating European Statistics. The relevant issues are professional independence, mandate for data collection, adequacy of resources, quality commitment, statistical confidentiality, impartiality, and objectivity.
Principle 1: Professional Independence
Professional independence of statistical authorities from other policy, regulatory, or administrative departments and bodies, as well as from privatesector operators, ensures the credibility of European Statistics.
 
Indicators
1.1: The independence of the National Statistical Institutes and Eurostat from political and other external interference in developing, producing, and disseminating statistics is specified in law and assured for other statistical authorities.
1.2: The heads of the National Statistical Institutes and of Eurostat and, where appropriate, the heads of other statistical authorities have sufficiently high hierarchical standing to ensure senior level access to policy authorities and administrative public bodies. They are of the highest professional calibre.
1.3: The heads of the National Statistical Institutes and of Eurostat and, where appropriate, the heads of other statistical authorities have responsibility for ensuring that statistics are developed, produced and disseminated in an independent manner.
1.4: The heads of the National Statistical Institutes and of Eurostat and, where appropriate, the heads of other statistical authorities have the sole responsibility for deciding on statistical methods, standards and procedures, and on the content and timing of statistical releases.
1.5: The statistical work programmes are published and periodic reports describe progress made.
1.6: Statistical releases are clearly distinguished and issued separately from political/policy statements.
1.7: The National Statistical Institute and Eurostat and, where appropriate, other statistical authorities, comment publicly on statistical issues, including criticisms and misuses of statistics as far as considered suitable.
1.8: The appointment of the heads of the National Statistical Institutes and Eurostat and, where appropriate, of other statistical authorities, is based on professional competence only. The reasons on the basis of which the incumbency can be terminated are specified in the legal framework. These cannot include reasons compromising professional or scientific independence.
Principle 2: Mandate for Data Collection
Statistical authorities have a clear legal mandate to collect information for European statistical purposes. Administrations, enterprises and households, and the public at large may be compelled by law to allow access to or deliver data for European statistical purposes at the request of statistical authorities.
 
Indicators
2.1: The mandate of the statistical authorities to collect information for the development, production, and dissemination of European Statistics is specified in law.
2.2: The statistical authorities are allowed by law to use administrative data for statistical purposes.
2.3: On the basis of a legal act, the statistical authorities may compel response to statistical surveys.
Principle 3: Adequacy of Resources
The resources available to statistical authorities are sufficient to meet European Statistics requirements.
 
Indicators
3.1: Staff, financial, and computing resources, adequate both in magnitude and in quality, are available to meet current statistical needs.
3.2: The scope, detail, and cost of statistics are commensurate with needs.
3.3: Procedures exist to assess and justify demands for new statistics against their cost.
3.4: Procedures exist to assess the continuing need for all statistics, to see if any can be discontinued or curtailed to free up resources.
Principle 4: Commitment to Quality
Statistical authorities are committed to quality. They systematically and regularly identify strengths and weaknesses to continuously improve process and product quality.
 
Indicators
4.1: Quality policy is defined and made available to the public. An organizational structure and tools are in place to deal with quality management.
4.2: Procedures are in place to plan and monitor the quality of the statistical production process.
4.3: Product quality is regularly monitored, assessed with regard to possible trade-offs, and reported according to the quality criteria for European Statistics.
4.4: There is a regular and thorough review of the key statistical outputs using also external experts where appropriate.
Principle 5: Statistical Confidentiality
The privacy of data providers (households, enterprises, administrations, and other respondents), the confidentiality of the information they provide and its use only for statistical purposes are absolutely guaranteed.
 
Indicators
5.1: Statistical confidentiality is guaranteed in law.
5.2: Staff sign legal confidentiality commitments on appointment.
5.3: Penalties are prescribed for any willful breaches of statistical confidentiality.
5.4: Guidelines and instructions are provided to staff on the protection of statistical confidentiality in the production and dissemination processes.
The confidentiality policy is made known to the public.
5.5: Physical, technological, and organisational provisions are in place to protect the security and integrity of statistical databases.
5.6: Strict protocols apply to external users accessing statistical microdata for research purposes.
Principle 6: Impartiality and Objectivity
Statistical authorities develop, produce and disseminate European Statistics respecting scientific independence and in an objective, professional and transparent manner in which all users are treated equitably.
 
Indicators
6.1: Statistics are compiled on an objective basis determined by statistical considerations.
6.2: Choices of sources and statistical methods as well as decisions about the dissemination of statistics are informed by statistical considerations.
6.3: Errors discovered in published statistics are corrected at the earliest possible date and publicised.
6.4: Information on the methods and procedures used is publicly available.
6.5: Statistical release dates and times are pre-announced.
6.6: Advance notice is given on major revisions or changes in methodologies.
6.7: All users have equal access to statistical releases at the same time. Any privileged pre-release access to any outside user is limited, controlled, and publicised. In the event that leaks occur, pre-release arrangements are revised so as to ensure impartiality.
6.8: Statistical releases and statements made in press conferences are objective and non-partisan.
Statistical Processes
European and other international standards, guidelines, and good practices are fully observed in the processes used by the statistical authorities to organise, collect, process, and disseminate European Statistics. The credibility of the statistics is enhanced by a reputation for good management and efficiency. The relevant aspects are sound methodology, appropriate statistical procedures, nonexcessive burden on respondents, and cost effectiveness.
Principle 7: Sound Methodology
Sound methodology underpins quality statistics. This requires adequate tools, procedures, and expertise.
 
Indicators
7.1: The overall methodological framework used for European Statistics follows European and other international standards, guidelines, and good practices.
7.2: Procedures are in place to ensure that standard concepts, definitions, and classifications are consistently applied throughout the statistical authority.
7.3: The business register and the frame for population surveys are regularly evaluated and adjusted if necessary in order to ensure high quality.
7.4: Detailed concordance exists between national classifications systems and the corresponding European systems.
7.5: Graduates in the relevant academic disciplines are recruited.
7.6: Statistical authorities implement a policy of continuous vocational training for their staff.
7.7: Co-operation with the scientific community is organised to improve methodology, the effectiveness of the methods implemented, and to promote better tools when feasible.
Principle 8: Appropriate Statistical Procedures
Appropriate statistical procedures, implemented from data collection to data validation, underpin quality statistics.
 
Indicators
8.1: When European Statistics are based on administrative data, the definitions and concepts used for administrative purposes are a good approximation to those required for statistical purposes.
8.2: In the case of statistical surveys, questionnaires are systematically tested prior to the data collection.
8.3: Survey designs, sample selections, and estimation methods are well based and regularly reviewed and revised as required.
8.4: Data collection, data entry, and coding are routinely monitored and revised as required.
8.5: Appropriate editing and imputation methods are used and regularly reviewed, revised, or updated as required.
8.6: Revisions follow standard, well-established, and transparent procedures.
8.7: Statistical authorities are involved in the design of administrative data in order to make administrative data more suitable for statistical purposes.
8.8: Agreements are made with owners of administrative data which set out their shared commitment to the use of these data for statistical purposes.
8.9: Statistical authorities co-operate with owners of administrative data in assuring data quality.
Principle 9: Non-excessive Burden on Respondents
The reporting burden is proportionate to the needs of the users and is not excessive for respondents. The statistical authorities monitor the response burden and set targets for its reduction over time.
 
Indicators
9.1: The range and detail of European Statistics demands is limited to what is absolutely necessary.
9.2: The reporting burden is spread as widely as possible over survey populations.
9.3: The information sought from businesses is, as far as possible, readily available from their accounts and electronic means are used where possible to facilitate its return.
9.4: Administrative sources are used whenever possible to avoid duplicating requests for information.
9.5: Data sharing within statistical authorities is generalised in order to avoid multiplication of surveys.
9.6: Statistical authorities promote measures that enable the linking of data sources in order to reduce reporting burden.
Principle 10: Cost Effectiveness
Resources are used effectively.
 
Indicators
10.1: Internal and independent external measures monitor the statistical authority’s use of resources.
10.2: The productivity potential of information and communications technology is being optimised for data collection, processing, and dissemination.
10.3: Proactive efforts are made to improve the statistical potential of administrative data and to limit recourse to direct surveys.
10.4: Statistical authorities promote and implement standardized solutions that increase effectiveness and efficiency.
Statistical Output
Available statistics meet users’ needs. Statistics comply with the European quality standards and serve the needs of European institutions, governments, research institutions, business concerns and the public generally. The important issues concern the extent to which the statistics are relevant, accurate and reliable, timely, coherent, comparable across regions and countries, and readily accessible by users.
Principle 11: Relevance
European Statistics meet the needs of users.
 
Indicators
11.1: Processes are in place to consult users, monitor the relevance and utility of existing statistics in meeting their needs, and consider their emerging needs and priorities.
11.2: Priority needs are being met and reflected in the work programme.
11.3: User satisfaction is monitored on a regular basis and is systematically followed up.
Principle 12: Accuracy and Reliability
European Statistics accurately and reliably portray reality.
 
Indicators
12.1: Source data, intermediate results, and statistical outputs are regularly assessed and validated.
12.2: Sampling errors and non-sampling errors are measured and systematically documented according to the European standards.
12.3: Revisions are regularly analysed in order to improve statistical processes.
Principle 13: Timeliness and Punctuality
European Statistics are released in a timely and punctual manner.
 
Indicators
13.1: Timeliness meets European and other international release standards.
13.2: A standard daily time for the release of statistics is made public.
13.3: The periodicity of statistics takes into account user requirements as much as possible.
13.4: Divergence from the dissemination time schedule is publicised in advance, explained, and a new release date set.
13.5: Preliminary results of acceptable aggregate accuracy can be released when considered useful.
Principle 14: Coherence and Comparability
European Statistics are consistent internally, over time and comparable between regions and countries; it is possible to combine and make joint use of related data from different sources.
 
Indicators
14.1: Statistics are internally coherent and consistent (i.e., arithmetic and accounting identities observed).
14.2: Statistics are comparable over a reasonable period of time.
14.3: Statistics are compiled on the basis of common standards with respect to scope, definitions, units, and classifications in the different surveys and sources.
14.4: Statistics from the different sources and of different periodicity are compared and reconciled.
14.5: Cross-national comparability of the data is ensured within the European Statistical System through periodical exchanges between the European Statistical System and other statistical systems. Methodological studies are carried out in close cooperation between the Member States and Eurostat.
Principle 15: Accessibility and Clarity
European Statistics are presented in a clear and understandable form, released in a suitable and convenient manner, available, and accessible on an impartial basis with supporting metadata and guidance.
 
Indicators
15.1: Statistics and the corresponding metadata are presented, and archived, in a form that facilitates proper interpretation and meaningful comparisons.
15.2: Dissemination services use modern information and communication technology and, if appropriate, traditional hard copy.
15.3: Custom-designed analyses are provided when feasible and the public is informed.
15.4: Access to microdata is allowed for research purposes and is subject to specific rules or protocols.
15.5: Metadata are documented according to standardised metadata systems.
15.6: Users are kept informed about the methodology of statistical processes, including the use of administrative data.
15.7: Users are kept informed about the quality of statistical outputs with respect to the quality criteria for European Statistics.
Footnotes
1

Regulation (EC) 223/2009, Article 4.

2

Regulation (EC) 223/2009, Article 12.

3

Regulation (EC) 223/2009, Articles 4 and 5.

4

Regulation (EC) 223/2009, Article 1. In the Code of Practice, «other national authorities responsible for the development, production and dissemination of European Statistics» are referred to as «other statistical authorities».

E Selected Federal Statistical Websites, February 2013

The information in this appendix is adapted from a listing in U.S. Office of Management and Budget (2012b); it includes both federal statistical agencies and other federal agencies that produce statistical information. http://
Web Sites
Congressional Budget Office (CBO): http://www.cbo.gov
 
Consumer Financial Protection Bureau (CFPB): http://www.consumerfinance.gov
 
Consumer Product Safety Commission (CPSC): http://www.cpsc.gov
 
Department of Agriculture (USDA): http://www.usda.gov
Agricultural Research Service (ARS): http://www.ars.usda.gov
Economic Research Service (ERS): http://www.ers.usda.gov
Food and Nutrition Service (FNS): http://www.fns.usda.gov
Foreign Agricultural Service (FAS): http://www.fas.usda.gov
Forest Service (FS): http://www.fs.fed.us
National Agricultural Statistics Service (NASS): http://www.nass.usda.gov
Natural Resources Conservation Service (NRCS): http://www.nrcs.usda.gov
World Agricultural Outlook Board (WAOB): http://www.usda.gov/oce/commodity/wasde/index.htm
 
Department of Commerce: http://www.doc.gov
Bureau of Economic Analysis (BEA): http://www.bea.gov
Census Bureau: http://www.census.gov
Economics and Statistics Administration (ESA): http://www.esa.doc.gov
International Trade Administration (ITA): http://www.ita.doc.gov
National Environmental Satellite, Data, and Information Service (NESDIS): http://www.nesdis.noaa.gov
National Institute of Standards and Technology (NIST): http://www.atp.nist.gov
National Marine Fisheries Service (NMFS): http://www.nmfs.noaa.gov
National Oceanic and Atmospheric Administration (NOAA): http://www.noaa.gov
Patent and Trademark Office (PTO): http://www.uspto.govabout/stats/index.jsp
 
Department of Defense: http://www.dod.gov
Army Corps of Engineers (CORPS): http://www.usace.army.mil
Defense Manpower Data Center (DMDC), Statistical Information Analysis Division: http://www.siadapp.dmdc.osd.mil
 
Department of Education: http://www.ed.gov
Institute of Education Sciences (IES): http://www.ies.ed.gov
National Center for Education Statistics (NCES): http://www.nces.ed.gov
 
Department of Energy: http://www.doe.gov
Energy Information Administration (EIA): http://www.eia.doe.gov
Office of Health, Safety, and Security (HSS): http://www.hss.doe.gov
 
Department of Health and Human Services: http://www.hhs.gov
Administration for Children and Families (ACF): http://www.acf.hhs.gov
Administration on Aging: http://www.data.aoa.gov
Agency for Healthcare Research and Quality (AHRQ): http://www.ahrq.gov
Agency for Toxic Substances and Disease Registry (ATSDR): http://www.atsdr.cdc.gov
Centers for Disease Control and Prevention (CDC): http://www.cdc.gov
Centers for Medicare and Medicaid Services (CMS): http://www.cms.hhs.gov
Health Resources and Services Administration (HRSA): http://www.hrsa.gov
HHS Data Council: http://www.hhs-stat.net
Indian Health Service (IHS): http://www.ihs.gov
National Center for Health Statistics (NCHS): http://www.cdc.gov/nchs
National Institute of Child Health and Human Development (NICHD): http://www.nichd.nih.gov
National Institute on Aging (NIA): http://www.nia.nih.gov
National Institutes of Health (NIH): http://www.nih.gov
Office of the Assistant Secretary for Planning and Evaluation (ASPE):http://www.aspe.hhs.gov
Office of Population Affairs (OPA): http://www.hhs.gov/opa
Substance Abuse and Mental Health Services Administration, Office of Applied Studies (SAMHSA/OAS): http://www.oas.samhsa.gov
 
Department of Homeland Security: http://www.dhs.gov
Bureau of Customs and Border Protection: http://www.cbp.gov
Federal Emergency Management Agency (FEMA): http://www.fema.gov
Office of Immigration Statistics (OIS): http://www.dhs.gov/office-immigrationstatistics
U.S. Citizenship and Immigration Services: http://www.uscis.gov
 
Department of Housing and Urban Development: http://www.hud.gov
Office of the Assistant Secretary for Policy Development and Research (PD&R): http://www.huduser.org
Office of Federal Housing Enterprise Oversight (OFHEO): http://www.fhfa.gov
Office of Public and Indian Housing (PIH): http://www.hud.gov/offices/pih
 
Department of the Interior: http://www.doi.gov
Bureau of Land Management (BLM): http://www.blm.gov
Bureau of Ocean Energy Management, Regulation and Enforcement (BOEMRE): http://www.boem.gov
Bureau of Reclamation (BoR): http://www.usbr.gov
Bureau of Safety and Environmental Enforcement (BSEE): http://www.bsee.gov
National Park Service (NPS): http://www.nps.gov
U.S. Fish and Wildlife Service (FWS): http://www.fws.gov
U.S. Geological Survey (USGS): http://www.usgs.gov
 
Department of Justice: http://www.usdoj.gov
Bureau of Justice Statistics (BJS): http://www.ojp.usdoj.gov/bjs
Bureau of Prisons (BoP): http://www.bop.gov
Drug Enforcement Administration (DEA): http://www.usdoj.gov/dea
Federal Bureau of Investigation (FBI): http://www.fbi.gov
National Institute of Justice: http://www.ojp.usdog.gov/nij
 
Department of Labor: http://www.dol.gov
Bureau of Labor Statistics (BLS): http://www.bls.gov
Employment and Training Administration (ETA): http://www.doleta.gov
Mine Safety and Health Administration (MSHA): http://www.msha.gov
Occupational Safety and Health Administration (OSHA): http://www.osha.gov
Office of Federal Contract Compliance Programs (OFCCP): http://www.dol.gov/ofccp/ofcpinfo.htm
 
Department of Transportation: http://www.dot.gov
Bureau of Transportation Statistics (BTS): http://www.bts.gov
Federal Aviation Administration (FAA): http://www.faa.gov
Federal Highway Administration (FHWA): http://www.fhwa.dot.gov
Federal Motor Carrier Safety Administration (FMCSA): http://www.fmcsa.dot.gov
Federal Railroad Administration (FRA): http://www.fra.dot.gov
Federal Transit Administration (FTA): http://www.fta.dot.gov
Maritime Administration (MARAD): http://www.marad.dot.gov
National Highway Traffic Safety Administration (NHTSA): http://www.nhtsa.gov
Pipeline and Hazardous Materials Safety Administration (PHMSA): http://www.phmsa.dot.gov
 
Department of the Treasury: http://www.ustreas.gov
Internal Revenue Service (IRS): http://www.irs.treas.gov
Statistics of Income (SOI): http://www.irs.gov/taxstats/index.html
Department of Veterans Affairs (VA): http://www.va.gov
National Center for Veterans Analysis and Statistics (NCVAS): http://www.va.gov/vetdata
 
Environmental Protection Agency (EPA): http://www.epa.gov
Office of Environmental Information: http://www.epa.gov/oei
 
Equal Employment Opportunity Commission (EEOC): http://www.eeoc.gov
 
Executive Office of the President
Federal Committee on Statistical Methodology: http://www.fcsm.gov
Federal Interagency Council on Statistical Policy, Federal Statistics:http://www.fedstats.gov
Federal Interagency Forum on Aging-Related Statistics: http://www.agingstats.gov
Federal Interagency Forum on Child and Family Statistics: http://www.childstats.gov
Office of Management and Budget (OMB): http://www.whitehouse.gov/omb
Statistical and Science Policy Office, Statistical Programs and Standards: http://www.whitehouse.gov/omb/inforeg_statpolicy
 
Federal Reserve Board: http://www.federalreserve.gov
 
National Aeronautics and Space Administration (NASA): http://www.nasa.gov
 
National Science Foundation (NSF): http://www.nsf.gov
Directorate for Social, Behavioral, and Economic Sciences: http://www.nsf.gov/sbe
Methodology, Measurement, and Statistics Program (MMS): http://www.nsf.gov/sbe/ses/mms/start.htm
National Center for Science and Engineering Statistics (NCSES): http://www.nsf.gov/statistics
 
Small Business Administration (SBA): http://www.sba.gov
 
Social Security Administration (SSA): http://www.socialsecurity.gov
 
Office of Research, Evaluation, and Statistics (ORES): http://www.ssa.gov/policy/about/ORES.html
 
U.S. Agency for International Development (USAID): http://www.usaid.gov
 
U.S. Government Accountability Office (GAO): http://www.gao.gov
Index of Federal Statistical Sites Listed
Administration for Children and Families—see Department of Health and Human Services
Administration on Aging—see Department of Health and Human Services
Agency for Healthcare Research and Quality—see Department of Health and Human Services
Agency for International Development—see U.S. Agency for International Development
Agency for Toxic Substances and Disease Registry—see Department of Health and Human Services
Agricultural Research Service—see Department of Agriculture
Army Corps of Engineers—see Department of Defense
Bureau of Customs and Border Protection—see Department of Homeland Security
Bureau of Economic Analysis—see Department of Commerce
Bureau of Justice Statistics—see Department of Justice
Bureau of Labor Statistics—see Department of Labor
Bureau of Land Management—see Department of the Interior
Bureau of Ocean Energy Management, Regulation and Enforcement—see Department of the Interior
Bureau of Prisons—see Department of Justice
Bureau of Reclamation—see Department of the Interior
Bureau of Safety and Environmental Enforcement—see Department of the Interior
Bureau of Transportation Statistics—see Department of Transportation
Census Bureau—see Department of Commerce
Centers for Disease Control and Prevention—see Department of Health and Human Services
Centers for Medicare and Medicaid Services—see Department of Health and Human Services
Congressional Budget Office—see Congressional Budget Office
Consumer Product Safety Commission—see Consumer Product Safety Commission
Defense Manpower Data Center—see Department of Defense
Directorate for Social, Behavioral, and Economic Sciences—see National Science Foundation
Drug Enforcement Administration—see Department of Justice
Economic Research Service—see Department of Agriculture
Economics and Statistics Administration—see Department of Commerce
Employment and Training Administration—see Department of Labor
Energy Information Administration—see Department of Energy
Environmental Protection Agency—see Environmental Protection Agency
Equal Employment Opportunity Commission—see Equal Employment Opportunity Commission
Federal Aviation Administration—see Department of Transportation
Federal Bureau of Investigation—see Department of Justice
Federal Committee on Statistical Methodology—see Executive Office of the President
Federal Emergency Management Agency—see Department of Homeland Security
Federal Highway Administration—see Department of Transportation
Federal Interagency Council on Statistical Policy, Federal Statistics—see Executive Office of the President
Federal Interagency Forum on Aging-Related Statistics—see Executive Office of the President
Federal Interagency Forum on Child and Family Statistics—see Executive Office of the President
Federal Motor Carrier Safety Administration—see Department of Transportation
Federal Railroad Administration—see Department of Transportation
Federal Reserve Board—see Federal Reserve Board
Federal Transit Administration—see Department of Transportation
Food and Nutrition Service—see Department of Agriculture
Foreign Agricultural Service—see Department of Agriculture
Forest Service—see Department of Agriculture
Government Accountability Office—see U.S. Government Accountability Office
Health Resources and Services Administration—see Department of Health and Human Services
HHS Data Council—see Department of Health and Human Services
Indian Health Service—see Department of Health and Human Services
Institute of Education Sciences—see Department of Education
Internal Revenue Service—see Department of the Treasury
International Trade Administration—see Department of Commerce
Maritime Administration—see Department of Transportation
Methodology, Measurement, and Statistics Program—see National Science Foundation
Mine Safety and Health Administration—see Department of Labor
National Aeronautics and Space Administration—see National Aeronautics and Space Administration
National Agricultural Statistics Service—see Department of Agriculture
National Center for Education Statistics—see Department of Education
National Center for Health Statistics—see Department of Health and Human Services
National Center for Science and Engineering Statistics—see National Science Foundation
National Center for Veterans Analysis and Statistics—see Department of Veterans Affairs
National Environmental Satellite, Data, and Information Service—see Department of Commerce
National Highway Traffic Safety Administration—see Department of Transportation
National Institute of Child Health and Human Development—see Department of Health and Human Services
National Institute of Justice—see Department of Justice
National Institute of Standards and Technology—see Department of Commerce
National Institute on Aging—see Department of Health and Human Services
National Institutes of Health—see Department of Health and Human Services
National Marine Fisheries Service—see Department of Commerce
National Oceanic and Atmospheric Administration—see Department of Commerce
National Park Service—see Department of the Interior
Natural Resources Conservation Service—see Department of Agriculture
National Science Foundation—see National Science Foundation
Occupational Safety and Health Administration—see Department of Labor
Office of the Assistant Secretary for Planning and Evaluation—see Department of Health and Human Services
Office of the Assistant Secretary for Policy Development and Research—see Department of Housing and Urban Development
Office of Environmental Information—see Environmental Protection Agency
Office of Federal Contract Compliance Programs—see Department of Labor
Office of Federal Housing Enterprise Oversight—see Department of Housing and Urban Development
Office of Health, Safety, and Security—see Department of Energy
Office of Immigration Statistics—see Department of Homeland Security
Office of Management and Budget—see Executive Office of the President
Office of Population Affairs—see Department of Health and Human Services
Office of Public and Indian Housing—see Department of Housing and Urban Development
Office of Research, Evaluation, and Statistics—see Social Security Administration
Patent and Trademark Office—see Department of Commerce
Pipeline and Hazardous Materials Safety Administration—see Department of Transportation
Small Business Administration—see Small Business Administration
Social Security Administration—see Social Security Administration
Statistical and Science Policy Office—see Executive Office of the President
Statistics of Income Division—see Department of the Treasury
Substance Abuse and Mental Health Services Administration—see Department of Health and Human Services
U.S. Citizenship and Immigration Services—see Department of Homeland Security
U.S. Fish and Wildlife Service—see Department of the Interior
U.S. Geological Survey—see Department of the Interior
World Agricultural Outlook Board—see Department of Agriculture

References and Selected Bibliography

American Academy of Political and Social Science. (2010). The federal statistical system: Its vulnerability matters more than you think. The Annals of the American Academy of Political and Social Science 631(September).
 
American Statistical Association. (1984). Report of the ASA technical panel on the census undercount. American Statistician 38(4):252–256.
 
_______. (1999). Ethical Guidelines for Statistical Practice. Alexandria, VA. Available: http://www. amstat.org/profession/index.cfm?fuseaction=main [February 2013].
 
Anderson, M.J. (1988). The American Census: A Social History. New Haven, CT: Yale University Press.
 
Brackstone, G. (1999). Managing data quality in a statistical agency. Survey Methodology 25(2):139–149.
 
Brick, J.M., and D. Williams. (2013). Explaining rising nonresponse rates in cross-sectional surveys. Pp. 36–59 in The Nonresponse Challenge to Surveys and Statistics, D.S.
 
Massey and R. Tourangeau, eds. The Annals of the American Academy of Political and Social Science 654 (January).
 
Chakrabarty, R., assisted by G. Torres. (1996). The American Housing Survey—A Quality Profile. Current Housing Reports, H121/95-1. Office of Policy Development and Research, U.S. Department of Housing and Urban Development and U.S. Census Bureau. Washington, DC: U.S. Department of Commerce.
 
de Leeuw, E., and W. de Heer. (2002). Trends in household survey nonresponse: A longitudinal and international comparison. Pp. 41–54 in Survey Nonresponse, R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, eds. New York: John Wiley & Sons.
 
Doyle, P., J.I. Lane, J.J.M. Theeuwes, and L.V. Zayatz, eds. (2001). Confidentiality, Disclosure, and Data Access: Theory and Practical Applications for Statistical Agencies. Amsterdam: Elsevier North-Holland.
 
Duncan, J.W., and W.C. Shelton. (1978). Revolution in United States Government Statistics, 1926–1976. Office of Federal Statistical Policy and Standards. Washington, DC: U.S. Department of Commerce.
 
Energy Information Administration. (1996). Residential Energy Consumption Survey Quality Profile. Prepared by T.B. Jabine. Washington, DC: U.S. Department of Energy.
 
Federal Committee on Statistical Methodology. (1978a). Report on Statistics for Allocation of Funds. Statistical Policy Working Paper 1 (NTIS PB86211521/AS). Washington, DC: U.S. Department of Commerce.
 
_______. (1978b). Report on Statistical Disclosure and Disclosure-Avoidance Techniques. Statistical Policy Working Paper 2 (NTIS PB86211539/AS). Washington, DC: U.S. Department of Commerce.
 
_______. (1978c). An Error Profile: Employment as Measured by the Current Population Survey. Statistical Policy Working Paper 3 (NTIS PB86214269/AS). Washington, DC: U.S. Department of Commerce.
 
_______. (1978d). Glossary of Nonsampling Error Terms: An Illustration of a Semantic Problem in Statistics. Statistical Policy Working Paper 4 (NTIS PB86211547/AS). Washington, DC: U.S. Department of Commerce.
 
_______. (1980a). Report on Exact and Statistical Matching Techniques. Statistical Policy Working Paper 5 (NTIS PB86215829/AS). Washington, DC: U.S. Department of Commerce.
 
_______. (1980b). Report on Statistical Uses of Administrative Records. Statistical Policy Working Paper 6 (NTIS PB86214285/AS). Washington, DC: U.S. Department of Commerce.
 
_______. (1982a). An Interagency Review of Time-Series Revision Policies. Statistical Policy Working Paper 7 (NTIS PB86232451/AS). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1982b). Statistical Interagency Agreements. Statistical Policy Working Paper 8 (NTIS PB86230570/AS). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1983a). Contracting for Surveys. Statistical Policy Working Paper 9 (NTIS PB83233148). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1983b). Approaches to Developing Questionnaires. Statistical Policy Working Paper 10 (NTIS PB84105055). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1984a). A Review of Industry Coding Systems. Statistical Policy Working Paper 11 (NTIS PB84135276). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1984b). The Role of Telephone Data Collection in Federal Statistics. Statistical Policy Working Paper 12 (NTIS PB85105971). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1986). Federal Longitudinal Surveys. Statistical Policy Working Paper 13 (NTIS PB86139730). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1987). Workshop on Statistical Uses of Microcomputers in Federal Agencies. Statistical Policy Working Paper 14 (NTIS PB87166393). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1988). Quality in Establishment Surveys. Statistical Policy Working Paper 15 (NTIS PB88232921). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1990a). A Comparative Study of Reporting Units in Selected Employer Data Systems. Statistical Policy Working Paper 16 (NTIS PB90205238). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1990b). Survey Coverage. Statistical Policy Working Paper 17 (NTIS PB90205246). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1990c). Data Editing in Federal Statistical Agencies. Statistical Policy Working Paper 18 (NTIS PB90205253). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1990d). Computer Assisted Survey Information. Statistical Policy Working Paper 19 (NTIS PB90205261). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1991). Seminar on Quality of Federal Data. Statistical Policy Working Paper 20 (NTIS PB91142414). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1993). Indirect Estimators in Federal Programs. Statistical Policy Working Paper 21 (NTIS PB93209294). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1994). Report on Statistical Disclosure Limitation Methodology. Statistical Policy Working Paper 22 (NTIS PB94165305). Washington, DC: U.S. Office of Management and Budget. [revised in 2005]
 
_______. (1995a). Seminar on New Directions in Statistical Methodology. Statistical Policy Working Paper 23 (NTIS PB95182978). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1995b). Electronic Dissemination of Statistical Data. Statistical Policy Working Paper 24 (NTIS PB96121629). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1996). Data Editing Workshop and Exposition. Statistical Policy Working Paper 25 (NTIS PB97104624). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1997). Seminar on Statistical Methodology in the Public Service. Statistical Policy Working Paper 26 (NTIS PB97162580). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1998). Training for the Future: Addressing Tomorrow’s Survey Tasks. Statistical Policy Working Paper 27 (NTIS PB99102576). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1999a). Seminar on Interagency Coordination and Cooperation. Statistical Policy Working Paper 28 (NTIS PB99132029). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1999b). Federal Committee on Statistical Methodology Research Conference (Conference Papers). Statistical Policy Working Paper 29 (NTIS PB99166795). Washington, DC: U.S. Office of Management and Budget.
 
_______. (1999c). 1999 Federal Committee on Statistical Methodology Research Conference: Complete Proceedings. Statistical Policy Working Paper 30 (NTIS PB2000105886). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2001a). Measuring and Reporting Sources of Error in Surveys. Statistical Policy Working Paper 31 (NTIS PB2001104329). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2001b). Seminar on Integrating Federal Statistical Information and Processes. Statistical Policy Working Paper 32 (NTIS PB2001104626). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2001c). Seminar on the Funding Opportunity in Survey Research. Statistical Policy Working Paper 33 (NTIS PB2001108851). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2001d). Federal Committee on Statistical Methodology Research Conference. Statistical Policy Working Paper 34 (NTIS PB2002100103). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2003a). Seminar on the Funding Opportunity in Survey Research. Statistical Policy Working Paper 36. Washington, DC: U.S. Office of Management and Budget.
 
_______. (2003b). Federal Committee on Statistical Methodology Research Conference. Statistical Policy Working Paper 37. Washington, DC: U.S. Office of Management and Budget.
 
_______. (2004a). Seminar on Challenges to the Federal Statistical System in Fostering Access to Statistics. Statistical Policy Working Paper 35 (NTIS PB2006109291). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2004b). Summary Report of the FCSM-GSS Workshop on Web-Based Data Collection. Statistical Policy Working Paper 38 (NTIS PB2006109290). Washington, DC: U.S. Office of Management and Budget.
 
_______. (2005). Federal Committee on Statistical Methodology Research Conference, Statistical Policy Working Paper 39. Washington, DC: U.S. Office of Management and Budget.
 
_______. (2007, 2009, 2010, 2012). FCSM Conference Papers (Proceedings). Washington, DC: U.S. Office of Management and Budget. Available: http://www.fcsm.gov/events/prior.html [February 2013].
 
_______. (2009). Profiles in Success of Statistical Uses of Administrative Data. Subcommittee on Administrative Records. Available: http://bls.gov/osmr/fcsm_related.htm [February 2013].
 
_______. (2013). Data Quality Assessment Tool for Administrative Data. Subcommittee on Administrative Records. Available: http://bls.gov/osmr/fcsm_related.htm [February 2013].
 
Federal Interagency Forum on Aging-Related Statistics. (2012). Older Americans 2012: Key Indicators of Well-Being. Available: http://www.agingstats.gov [February 2013].
 
Federal Interagency Forum on Child and Family Statistics. (2012). America’s Children in Brief: Key National Indicators of Well-Being, 2012. Washington, DC: U.S. Government Printing Office. Available: http://www.childstats.gov [February 2013].
 
Gates, G.W. (2012). Confidentiality. Pp. 93–95 in M.J. Anderson, C.F. Citro, and J.J. Salvo, eds. Encyclopedia of the U.S. Census—From the Constitution to the American Community Survey. Second Edition. Washington, DC: CQ Press.
 
Gillingham, R., and W. Lane. (1982). Changing the treatment of shelter costs for homeowners in the CPI. Monthly Labor Review 105(6):9–14.
 
Hillygus, D.S., N.H. Nie, K. P.rewitt, and H. Pals. (2006). The Hard Count: The Political and Social Challenges of Census Mobilization. New York: Russell Sage Foundation.
 
International Statistical Institute. (1985). Declaration on Professional Ethics. Voorburg, The Netherlands. Available: http://www.isi.cbs.nl/ethics.htm [February 2013].
 
Kalton, G., M. Winglee, S. Krawchuk, and D. Levine. (2000). Quality Profile for SASS Rounds 1–3: 1987–1995, Aspects of the Quality of Data in the Schools and Staffing Surveys (SASS). National Center for Education Statistics. Washington, DC: U.S. Department of Education.
 
Martin, M.E. (1981). Statistical practice in bureaucracies. Journal of the American Statistical Association 76(373):1–8.
 
National Research Council. (1976). Setting Statistical Priorities. Panel on Methodology for Statistical Priorities. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1984). Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines. Report of the Advanced Research Seminar on Cognitive Aspects of Survey Methodology, T.B. Jabine, M.L. Straf, J.M. Tanur, and R. Tourangeau, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1985a). Immigration Statistics: A Story of Neglect. Panel on Immigration Statistics, D.B. Levine, K. Hill, and R. Warren, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1985b). Natural Gas Data Needs in a Changing Regulatory Environment. Panel on Natural Gas Statistics. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1985c). Sharing Research Data. S.E. Fienberg, M.E. Martin, and M.L. Straf, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1986). Creating a Center for Education Statistics: A Time for Action. Panel to Evaluate the National Center for Education Statistics, D.B. Levine, ed. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1991a). Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I: Review and Recommendations. Panel to Evaluate Microsimulation Models for Social Welfare Programs, C.F. Citro and E.A. Hanushek, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1991b). Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume II: Technical Papers. Panel to Evaluate Microsimulation Models for Social Welfare Programs, C.F. Citro and E.A. Hanushek, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1992). Principles and Practices for a Federal Statistical Agency. M.E. Martin and M.L. Straf, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1993a). The Future of the Survey of Income and Program Participation. Panel to Evaluate the Survey of Income and Program Participation, C.F. Citro and G. Kalton, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1993b). Private Lives and Public Policies: Confidentiality and Accessibility of Government Statistics. Panel on Confidentiality and Data Access, G.T. Duncan, T.B. Jabine, and V.A. de Wolf, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1995a). Measuring Poverty—A New Approach. Panel on Poverty and Family Assistance, C.F. Citro and R.T. Michael, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1995b). Modernizing the U.S. Census. Panel on Census Requirements in the Year 2000 and Beyond, B. Edmonston and C. Schultze, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1997a). Assessing Policies for Retirement Income: Needs for Data, Research, and Models. Panel on Retirement Income Modeling, C.F. Citro and E.A. Hanushek, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1997b). The Bureau of Transportation Statistics: Priorities for the Future. Panel on Statistical Programs and Practices of the Bureau of Transportation Statistics, C.F. Citro and J.L. Norwood, eds. Committee on National Statistics and Transportation Research Board. Washington, DC: National Academy Press.
 
_______. (1998a). Longitudinal Surveys of Children. Committee on National Statistics and Board on Children, Youth, and Families, K.K. West, R.M. Hauser, and T.M. Scanlan, eds. Washington, DC: National Academy Press.
 
_______. (1998b). Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Panel on Statistical Methods for Testing and Evaluating Defense Systems, M.L. Cohen, J.E. Rolph, and D.L. Steffey, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1999a). Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Panel on Performance Measures and Data for Public Health Performance Partnership Grants, E.B. Perrin, J.S. Durch, and S.M. Skillman,
eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (1999b). Sowing Seeds of Change: Informing Public Policy in the Economic Research Service of USDA. Panel to Study the Research Program of the Economic Research Service, J.F. Geweke, J.T. Bonnen, A.A. White, and J.J. Koshel, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2000a). Improving Access to and Confidentiality of Research Data: Report of a Workshop. C. Mackie and N. Bradburn, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2000b). Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resource Studies. Committee to Assess the Portfolio of the Division of Science Resources Studies of NSF. Office of Scientific and Engineering Personnel and Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2000c). Small-Area Estimates of School-Age Children in Poverty: Evaluation of Current Methodology. Panel on Estimates of Poverty for Small Geographic Areas, C.F. Citro and G. Kalton, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2000d). Small-Area Income and Poverty Estimates: Priorities for 2000 and Beyond. Panel on Estimates of Poverty for Small Geographic Areas, C.F. Citro and G. Kalton, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2001a). Cells and Surveys: Should Biological Measures Be Included in Social Science Research? C.E. Finch, J.W. Vaupel, and K. Kinsella, eds. Committee on Population. Washington, DC: National Academy Press.
 
_______. (2001b). Evaluating Welfare Reform in an Era of Transition. Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs, R.A. Moffitt and M. Ver Ploeg, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2001c). Principles and Practices for a Federal Statistical Agency, Second Edition. M.E. Martin, M.L. Straf, and C.F. Citro, eds. Committee on National Statistics. Washington, DC: National Academy Press.
 
_______. (2002). At What Price?: Conceptualizing and Measuring Cost-of-Living and Price Indexes. Panel on Conceptual, Measurement, and Other Statistical Issues in Developing Cost-of-Living Indexes, C. Schultze and C. Mackie, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2003a). Estimating Eligibility and Participation for the WIC Program: Final Report.Panel to Evaluate the USDA’s Methodology for Estimating Eligibility and Participation for the WIC Program, M. Ver Ploeg and D.M. Betson, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2003b). Innovations in Software Engineering for Defense Systems. Oversight Committee for the Workshop on Statistical Methods in Software Engineering for Defense Systems, S.R. Dalal, J.H. Poore, and M.L. Cohen, eds. Committee on National Statistics and Division on Engineering and Physical Sciences. Washington, DC: The National Academies Press.
 
_______. (2003c). Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics’ Surveys—Special Report 227. Committee to Review the Bureau of Transportation Statistics’ Survey Programs. Transportation Research Board and Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2003d). Protecting Participants and Facilitating Social and Behavioral Sciences Research. Panel on Institutional Review Boards, Surveys, and Social Science Research, C.F. Citro, D.R. Ilgen, and C.B. Marrett, eds. Committee on National Statistics and Board on Behavioral, Cognitive, and Sensory Sciences. Washington, DC: The National Academies Press.
 
_______. (2003e). Survey Automation: Report and Workshop Proceedings. Oversight Committee for the Workshop on Survey Automation, D.L. Cork, M.L. Cohen, R. Groves, and W. Kalsbeek, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2004a). Eliminating Health Disparities: Measurement and Data Needs. Panel on DHHS Collection of Race and Ethnic Data, M. Ver Ploeg and E. Perrin, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2004b). Measuring Research and Development Expenditures in the U.S. Economy. Panel on Research and Development Statistics at the National Science Foundation, L.D. Brown, T.J. Plewes, and M.A. Gerstein, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2004c). Reengineering the 2010 Census: Risks and Challenges. Panel on Research on Future Census Methods, D.L. Cork, M.L. Cohen, and B.F. King, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2004d). The 2000 Census: Counting Under Adversity. Panel to Review the 2000 Census, C.F. Citro, D.L. Cork, and J.L. Norwood, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2005a). Beyond the Market: Designing Nonmarket Accounts for the United States. Panel to Study the Design of Nonmarket Accounts, K.G. Abraham and C. Mackie, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2005b). Expanding Access to Research Data: Reconciling Risks and Opportunities. Panel on Data Access for Research Purposes. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2005c). Experimental Poverty Measures: Summary of a Workshop. J. Iceland, rapporteur. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2005d). Principles and Practices for a Federal Statistical Agency, Third Edition. M.E. Martin, M.L. Straf, and C.F. Citro, eds. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
 
_______. (2006a). Food Insecurity and Hunger in the United States: An Assessment of the Measure. Panel to Review USDA’s Measurement of Food Insecurity and Hunger, G.S. Wunderlich and J.L. Norwood, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2006b). Improving Business Statistics Through Interagency Data Sharing: Summary of a Workshop. C. Kuebler and C. Mackie, rapporteurs. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2006c). Once, Only Once, and in the Right Place: Residence Rules in the Decennial Census. Panel on Residence Rules in the Decennial Census, D.L. Cork and P.R. Voss, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2007a). State and Local Government Statistics at a Crossroads. Panel on Research and Development Priorities for the U.S. Census Bureau’s State and Local Government Statistics Program. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2007b). Understanding Business Dynamics: An Integrated Data System for America’s Future. Panel on Measuring Business Formation, Dynamics, and Performance, J. Haltiwanger, L.M. Lynch, and C. Mackie, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2007c). Using the American Community Survey: Benefits and Challenges. Panel on the Functionality and Usability of Estimates from the American Community Survey, C.F. Citro and G. Kalton, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2008a). Protecting Individual Privacy in the Struggle Against Terrorists—A Framework for Program Assessment. Committee on Technical and Privacy Dimensions of Information for Terrorism Prevention and Other National Goals. Committee on Law and Justice, Committee on National Statistics, and Computer Science and Telecommunications Board. Washington, DC: The National Academies Press.
 
_______. (2008b). Rebuilding the Research Capacity at HUD. Committee to Evaluate the Research Plan of the Department of Housing and Urban Development, Center for Economic, Governance, and International Studies. Washington, DC: The National Academies Press.
 
_______. (2008c). Surveying Victims: Options for Conducting the National Crime Victimization Survey. Committee on National Statistics and Committee on Law and Justice. Washington, DC: The National Academies Press.
 
_______.(2008d). Using the American Community Survey for the National Science Foundation’s Science and Engineering Workforce Statistics Program. Panel on Assessing the Benefits of the American Community Survey for the NSF Division of Science Resources Statistics. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2009a). Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Panel to Review the Programs of the Bureau of Justice Statistics, R.M. Groves and D.L. Cork, eds. Committee on National Statistics and Committee on Law and Justice. Washington, DC: The National Academies Press.
 
_______. (2009b). Improving the Measurement of Late-Life Disability in Population Surveys: Beyond ADLs and IADLs: Summary of a Workshop. G.S. Wunderlich, rapporteur. Committee on National Statistics and Committee on Population. Washington, DC: The National Academies Press.
 
_______. (2009c). Principles and Practices for a Federal Statistical Agency, Fourth Edition. Committee on National Statistics. C.F. Citro, M.E. Martin, and M.L. Straf, eds. Washington, DC: The National Academies Press.
 
_______. (2009d). Protecting Student Records and Facilitating Education Research: A Workshop Summary. M. Hilton, rapporteur. Committee on National Statistics and Center for Education. Washington, DC: The National Academies Press.
 
_______. (2009e). Reengineering the Survey of Income and Program Participation. Panel on the Census Bureau’s Reengineered Survey of Income and Program Participation, C.F. Citro and J.K. Scholz, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2010a). Accounting for Health and Health Care: Approaches to Measuring the Sources and Costs of Their Improvement. Panel to Advance a Research Program on the Design of National Health Accounts, Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2010b). Conducting Biosocial Surveys: Collecting, Storing, Accessing, and Protecting Biospecimens and Biodata. Panel on Collecting, Storing, Accessing, and Protecting Biological Specimens and Biodata in Social Surveys, R.M. Hauser, M. Weinstein, R. Pool, and B. Cohen, eds. Committee on National Statistics and Committee on Population. Washington, DC: The National Academies Press.
 
_______. (2010c). Data on Federal Research and Development Investments: A Pathway to Modernization. Panel on Modernizing the Infrastructure of the National Science Foundation Federal Funds Survey, Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2010d). Envisioning the 2020 Census. Panel on the Design of the 2010 Census Program of Evaluations and Experiments, L.D. Brown, M.L. Cohen, D.L. Cork, and C.F. Citro, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2010e). Improving Health Care Cost Projections for the Medicare Population: Summary of a Workshop. G.S. Wunderlich, rapporteur. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2010f). Protecting and Accessing Data from the Survey of Earned Doctorates: A Workshop Summary. T.J. Plewes, rapporteur. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2010g). The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2011a). Allocating Federal Funds for State Programs for English Language Learners. Panel to Review Alternative Data Sources for the Limited-English Proficiency Allocation Formula under Title III, Part A, Elementary and Secondary Education Act. Committee on National Statistics and Board on Testing and Assessment. Washington, DC: The National Academies Press.
 
_______. (2011b). Change and the 2020 Census: Not Whether But How. Panel to Review the 2010 Census, T.M. Cook, J.L. Norwood, and D.L. Cork, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2011c). Facilitating Innovation in the Federal Statistical System: Summary of a Workshop. H. Habermann, rapporteur. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2011d). The Future of Federal Household Surveys: A Workshop Summary. K. Marton and J.C. Karberg, rapporteurs. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2012a). Benefits, Burdens, and Prospects for the American Community Survey: Summary of a Workshop. D.L. Cork, rapporteur. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2012b). Communicating Science and Engineering Data in the Information Age. Panel on Communicating National Science Foundation Science and Engineering Information to Data Users. Committee on National Statistics and Computer Science and Telecommunications Board. Washington, DC: The National Academies Press.
 
_______. (2012c). Effective Tracking of Building Energy Use: Improving the Commercial Buildings and Residential Energy Consumption Surveys. Panel on Redesigning the Commercial Buildings and Residential Energy Consumption Surveys of the Energy Information Administration, W.F. Eddy and K. Marton, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2012d). Industrial Methods for the Effective Development and Testing of Defense Systems. Panel on Industrial Methods for the Effective Test and Development of Defense Systems. Committee on National Statistics and Board on Army Science and Technology. Washington, DC: The National Academies Press.
 
_______. (2012e). Measuring What We Spend: Toward a New Consumer Expenditure Survey. Panel on Redesigning the BLS Consumer Expenditure Surveys, D.A. Dillman and C.C. House, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2012f). Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Panel on Statistical Methods for Measuring the Group Quarters Population in the American Community Survey, P.R. Voss and K. Marton, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
_______. (2012g). Using American Community Survey Data to Expand Access to the School Meals Programs. Panel on Estimating Children Eligible for School Nutrition Programs Using the American Community Survey, A. Schirm and N. Kirkendall, eds. Committee on National Statistics. Washington, DC: The National Academies Press.
 
U.N. Statistical Commission. (1994). Fundamental Principles of Official Statistics. Official Records of the Economic and Social Council, 1994, Supplement No. 9. New York: U.N. Statistical Commission. Available: http://unstats.un.org/unsd/dnss/gp/fundprinciplesaspx [February 2013].
 
_______. (2003). Implementation of the Fundamental Principles of Official Statistics. Report of the Secretary-General. New York: U.N. Statistical Commission. Available: http://www.unstats.un.org/unsd/dnss/gp/globreview.aspx [February 2013].
 
U.N. Statistical Commission and Economic Commission for Europe. (1991). Resolution on fundamental principles of official statistics in the Economic Commission for Europe (draft). Report of the Thirty-Ninth Plenary Session, June 17–21.
 
United Kingdom Office for National Statistics. (2007). Guidelines for Measuring Statistical Quality. London (November). Available: http://www.ons.gov.uk/ons/guide-method/method-quality/quality/guidelines-for-measuring-statistical-quality/index.html [February 2013].
 
U.S. Census Bureau. (1998). Survey of Income and Program Participation Quality Profile 1998. Third Edition. Prepared by Westat, G. Kalton, project director. Washington, DC: U.S. Department of Commerce. Available: http://www.bls.census.gov/sipp/workpapr/230.pdf [February 2013].
 
_______. (2003). U.S. Census Bureau Data Stewardship/Privacy Impact Assessment—Demographic Surveys Program. Washington, DC: U.S. Department of Commerce. U.S. Departments of Agriculture, Commerce, Education, Energy, Health and Human Services, Justice, Labor, Transportation, and Treasury; National Science Foundation; and Social Security Administration. (2002). Federal statistical organizations’ guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of disseminated information. Federal Register 67(107):38467–38469.
 
U.S. General Accounting Office. (1995). Statistical Agencies: Adherence to Guidelines and Coordination of Budgets. Washington, DC: U.S. Government Printing Office.U.S. Government Accountability Office. (2012). Agencies Can Make Greater Use of Existing Data, but Continued Progress Is Needed on Access and Quality Issues. GAO-12-54. Washington, DC: U.S. Government Accountability Office.
 
U.S. Office of Management and Budget. (1985). Statistical policy directive on compilation, release, and evaluation of principal federal economic indicators. Federal Register 50(186):932–934.
 
_______. (1997). Order providing for the confidentiality of statistical information. Federal Register 62(124):35044–35050. Available: www.whitehouse.gov/omb/inforeg/conforder.pdf [February 2013].
 
_______. (2000). Statistical Programs of the United States Government, Fiscal Year 2001. Statistical and Science Policy Office, Office of Information and Regulatory Affairs. Washington, DC: U.S. Government Printing Office.
 
_______. (2001). Statistical Programs of the United States Government, Fiscal Year 2002. Statistical and Science Policy Office, Office of Information and Regulatory Affairs. Washington, DC: U.S. Government Printing Office.
 
_______. (2004a, December 16). Issuance of OMB’s “Final Information Quality Bulletin for Peer Review.” Memorandum from J.B. Bolten, director, for heads of departments and agencies. Washington, DC. Available: http://www.whitehouse.gov/omb/memoranda/fy2005/m05-03.pdf [February 2013].
 
_______. (2004b). Statistical Programs of the United States Government, Fiscal Year 2005. Statistical and Science Policy Office, Office of Information and Regulatory Affairs. Washington, DC: U.S. Government Printing Office.
 
_______. (2006a). Guidance on Agency Survey and Statistical Information Collections. Memorandum for the President’s Management Council, J.D. Graham, January 20. Available: http://www.whitehouse.gov/omb/infore...dance_2006.pdf [February 2013].
 
_______. (2006b). Standards and Guidelines for Statistical Surveys. Federal Register 71(184):55522–55523. Complete document available: http://www.whitehouse.gov/omb/inforeg/statpoliy/standards_stat_surveys.pdf [February 2013].
 
_______. (2007). Implementation guidance for Title V of the E-Government Act, Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA), Federal Register 72(115):33362–33377. Available: http://www.whitehouse.gov/omb/fedreg/2007/061507_cipsea_guidance.pdf [February 2013].
 
_______. (2008). Statistical policy directive no. 4—Release and dissemination of statistical products produced by federal statistical agencies, Federal Register 73(46):12622–12626. Available: http://www.whitehouse.gov/omb/fedreg...irective-4.pdf [February 2013].
 
_______. (2011). Statistical Programs of the United States Government, Fiscal Year 2012. Statistical and Science Policy Office, Office of Information and Regulatory Affairs. Washington, DC: U.S. Government Printing Office.
 
_______. (2012a). Budget of the United States Government, Fiscal Year 2013, Analytical Perspectives, Chapter 19: Strengthening Federal Statistics. Washington, DC: U.S. Government Printing Office.
 
_______. (2012b). Statistical Programs of the United States Government, Fiscal Year 2013. Statistical and Science Policy Office, Office of Information and Regulatory Affairs. Washington, DC: U.S. Government Printing Office.
Page statistics
17383 view(s) and 202 edit(s)
Social share
Share this page?

Tags

This page has no custom tags.
This page has no classifications.

Comments

You must to post a comment.

Attachments