Management Response to the 2011-12 PMPRB Program Evaluation

Table Of Contents

• Executive Summary
• 1. Background
• 2. Methodology
• 3. Evaluation Report
• 4. Management Response and Action plan
• 5. Communications and Reporting
• Glossary
• Annex A – Action Plan

Executive Summary

In accordance with a commitment made within the context of having additional funds approved in 2008-2009, the PMPRB conducted a program evaluation in 2011-12 to assess its relevance, efficiency and effectiveness and the extent to which the increased resources helped the PMPRB achieve its objectives. The evaluation featured multiple lines of evidence which provided for a more rounded assessment and greater confidence in the results. Overall the evaluation findings were positive.

The Program Evaluation found that both the Patented Medicine Prices Regulation Program and the Pharmaceutical Trends Program continue to be relevant. However, the relevance of the Reporting Program has been reduced as a result of the evolution of the pharmaceutical sector and a lack of timeliness in product development and dissemination. The findings also indicate that both PMPRB Programs are appropriate for delivery by a federal agency and are well aligned with government-wide priorities and the PMPRB's Strategic Outcome.

The Regulation Program has very successfully achieved its expected outcomes thus ensuring that prices of patented medicines sold in Canada are not excessive as determined by legislated criteria. The Reporting Program, while quite successful in some aspects of its operations, could improve utilization of its price trends and analytical information by improving the timeliness of the development and dissemination of these products.

Although no formal cost-benefit analyses were undertaken, anecdotal evidence suggests that the PMPRB's operations are cost-effective. Further, the PMPRB collects non-respendable revenue on behalf the Government of Canada as a result of payments made by patentees through its enforcement activities. While the salary and operational budget of the PMPRB for 2012-13 is $7.7 million, over its operations of the past 25 years, this non-respendable revenue has totaled $138.6 million. The Patent Act provides that the Minister of Health may enter into agreements with any province respecting the distribution to that province of amounts received by the Receiver General, less any costs incurred in relation to the collection and distribution of those amounts.

Building on this success, the Board has undertaken to address the four considerations included in the Evaluation Report.

1. Expedite all PMPRB processes

Board Staff is examining current administrative and regulatory processes for ways to improve their timeliness. Quicker processes could mean, among other things, more efficient use of resources, shorter periods of excessive pricing, quicker access to information on price, sales and utilization trends which could result in greater use of information produced by the PMPRB.

2. Further simplify the Guidelines

The Monitoring and Evaluation Plan for the Major Changes in the Guidelines (GMEP) is proving to be an effective tool as it allows Board Staff to monitor the implementation of the Guidelines and to report to the Board on a regular basis. Annual reporting will be done in December of each year. The Board is committed to having Guidelines that are responsive to the changing environment.

3. Expand plain language offerings throughout all PMPRB communications

Board Staff will continue to expand the use of plain language in all PMPRB communications, for example, by way of its “In Brief” series. Board Staff is currently assessing the usefulness of a plain language Guide to Price Regulation, which will explain the pricing framework as a whole and promote awareness and compliance.

4. Expand the target audience for outreach efforts

The PMPRB will continue implementing its non-stakeholder engagement policy through the development of a bi-lateral meeting series with non-industry stakeholders. In addition, Board Staff has developed a “PMPRB 101 workshop” and continues to take advantage of opportunities to engage with a broad range of stakeholders.

1. Background

In conjunction with the Treasury Board Secretariat of Canada's (TBS) provision of increased funding in 2008-09 and ongoing, the PMPRB committed to conduct an evaluation in 2011-12 to assess the extent to which the increased resources helped the PMPRB achieve its objectives.

In October 2011, consultants were contracted to conduct the evaluation. The evaluation assessed the relevance and performance of the Patented Medicine Prices Regulation Program (Regulatory Program)1 and the Pharmaceutical Trends Program (Reporting Program)2 during the period from 2008-09 through 2011-12. Although the additional TBS funding was not directed to the Reporting Program it was included in the evaluation in order to provide a more comprehensive assessment of how the PMPRB has performed in recent years. The evaluation was conducted in accordance with TBS´s Policy on Evaluation and the Directive on the Evaluation Function. The full Evaluation Report can be found at on the PMPRB website under Accountability. As required, the evaluation addressed the five key issues presented in the table below.

Source: Patented Medicine Prices Review Board Performance Measurement Strategy (January 2011)

In December 2011 the evaluation work plan was presented to the Board for acceptance. The Board also received a progress report and information on the preliminary findings of the survey and interviews that had been conducted as at February 14, 2012. Board Staff worked with the consultants to gather and provide the performance information necessary to complete the program evaluation. A draft Evaluation Report was presented to the Board for discussion in mid-May. The final Evaluation Report was accepted by the Chairperson and presented to the Board members for information on September 12, 2012. The Evaluation Report included a number of comments and observations. Based on the findings of the evaluation as a whole, the consultants proposed items for consideration.

The Management Team worked with the Executive Committee3 to develop and document this Management Response and Action Plan to address the considerations presented in the Evaluation Report.

1 A description of the Regulation Program can be found in the 2012-13 Report on Plans and Priorities (RPP), Section II, Analysis of Program Activities by Strategic Outcome. The RPPis available on the PMPRB website under Reports to Parliament.

2 A description of the Reporting Program can be found in the 2012-13 Report on Plans and Priorities (RPP), Section II, Analysis of Program Activities by Strategic Outcome. The RPP is available on the PMPRB website under Reports to Parliament.

3 The PMPRB Executive Committee consists of the Chairperson, Vice-chairperson and Executive Director. The Secretary of the Board is also a member and provides secretariat functions for the Committee. Other members of the PMPRB management may be requested to attend on an ad-hoc basis. The Chairperson may also request other Board members to attend on an ad-hoc basis.

2. Methodology

The evaluation featured multiple lines of evidence. The use of multiple sources and multiple methods of evidence provided a more rounded assessment and greater confidence in the results. The combination of qualitative and quantitative evidence established a strong foundation to support the considerations.

The evaluation included:

• Interviews with stakeholders who have had a significant role in or experience with the PMPRB, or who have a key stake in it. A separate interview guide was developed for each of the following categories of interviewees. In total, 46 interviews were conducted as follows:
  • 9 Board members (current and former);
  • 11 pharmaceutical patentees;
  • 5 representatives from consumer/advocacy groups;
  • 3 representatives from private health insurance firms; and
  • 14 provincial and territorial government representatives

The Report noted that interviewee categories were somewhat artificial since many individuals interviewed could fit into more than one of the above categories. In cases where interviewees self-selected into more than one category they were asked with which category they identified best.

• An Internet-based survey was developed to collect data from stakeholders. The survey questionnaire included a combination of open and close-ended questions to address issues identified for this project. Email invitations to participate in the survey were directed to approximately 700 individuals, seventy-four (approximately 10%) completed the survey, broken into categories as follows:
  • 39 pharmaceutical patentees or organizations representing pharmaceutical patentees;
  • 4 consumer and/or patient advocacy organizations;
  • 7 provincial, territorial or federal drug plan providers;
  • 8 researchers from think tanks, universities, etc.; and
  • 16 others who self-identified as consultants, lawyers, and pharmacists.
• A document/literature review

The document review was used by the evaluators to develop a thorough understanding of the PMPRB and address many of the evaluation issues. The documents reviewed were gathered from the PMPRB, interviewees as well as through a range of targeted Internet searches.

• Analysis of PMPRB Performance Data

The PMPRB's Performance Measurement Strategy (PMS) was finalized in January 2011. At the time of the evaluation the PMS had been, or was in the process of being implemented by most branches within the PMPRB. Given how recently the PMS had been finalized, it yielded only limited performance measurement data. However, the PMS was substantially bolstered by data provided in the PMPRB's Annual Reports and/or in a range of other PMPRB documents listed in the bibliography and by data maintained and provided by the various PMPRB activity areas.

2.1 Methodological Constraints/Limitations

The following sets out the constraints and limitations that are noted in the Report.

All parties to this evaluation recognized from the outset that the numbers of both interviewees and survey participants would be too small to yield real statistical validity – either for the overall group or categories within it. This does not obviate the value of the information gathered. One of the merits of multiple lines of evidence is that it allows a higher degree of confidence if/when different lines of evidence yield consistent and congruent information. For this study, it was anticipated that the combination of input from interviews and surveys combined would provide a reasonable degree of confidence in the input received.

This anticipation proved accurate for the Regulatory Program. However, the survey responses were dominated by patentees, which thus skewed results for the Reporting Program because it is targeted at government policy-makers, not patentees. Given that these two groups (government policy-makers and patentees) may have conflicting priorities, overall survey input about the Reporting Program was interpreted with caution.

3. Evaluation Report

In accordance with TBS's Policy on Evaluation and the Directive on the Evaluation Function the evaluation addressed the issues of relevance and performance as well as efficiency and economy.

Overall the evaluation findings were positive.

3.1 Conclusions

3.1.1 Relevance

Continued Relevance of the PMPRB Programs

The Regulatory Program continues to be relevant. Although the pharmaceutical environment has changed considerably, the Regulation Program continues to meet the needs it was established to address.

The Reporting Program is also still relevant, but less so than the Regulatory Program. Its reduced relevance was attributed by some stakeholders to the evolution of the pharmaceutical sector, making some program parameters, such as the definition of Research and Development, appear less relevant. Another significant factor in the program´s reduced relevance was a lack of timeliness in product development and dissemination.

Alignment with Government Priorities and Federal Roles and Responsibilities

Both PMPRB Programs are appropriate for delivery by a federal agency and are well-aligned with both government-wide priorities and with PMPRB's Strategic Outcome.

3.1.2 Performance

Achievement of Expected Outcomes for the Regulatory Program

The Regulatory Program performed very successfully. It achieved a high level of compliance and has ensured that prices are not excessive as determined by legislated criteria. In addition, the program succeeded at enhancing stakeholder knowledge and awareness, and at creating a regulatory environment perceived by stakeholders to be increasingly clear and transparent. The program's success regarding the predictability of the environment was more guarded, primarily because the Guidelines, which have been improved considerably, require further simplification.

Achievement of Expected Outcomes for Reporting Program

The Reporting Program was quite successful in some aspects of its operations. When considering how well the program met the needs of the policy-makers for whom it was designed, it performed well in terms of the accessibility, comprehensiveness and accuracy of the products over which it has control - NPDUIS and Price Trends outputs. However, the program has not produced these products in a manner which was sufficiently timely for them to be utilized in policy-making and decision-making. It is likely that utilization would improve significantly if and when the timeliness issue is addressed.

3.1.3 Efficiency and Economy

The evaluation supports the current level of funding for salaries and operating expenses at the PMPRB. The incremental funding received in 2008-09 was well-utilized, having achieved the expected results for which it was approved. Though formal cost-benefit analyses were not undertaken, anecdotal evidence suggests that PMPRB operations are cost-effective.

Some gains in efficiency and economy are likely to be achieved by modernizing or streamlining aspects of both the PMPRB's operational and methodological framework. The most important of these is simplifying the Guidelines.

3.2 Considerations

The Report concludes by setting out the following considerations:

1. Expedite all PMPRB processes to facilitate a better fit between those processes and the very rapid pace of change that is now the norm for the pharmaceutical sector.
2. Further simplify the Guidelines, including methodologies used and explained in the Guidelines.
3. Expand plain language offerings throughout all PMPRB communications to enhance the value of all information products.
4. Expand the target audience for outreach efforts, with a strong focus on establishing ongoing working relationships with third party payers and patient advocacy groups.

4. Management Response and Action plan

The PMPRB´s management team has worked with our Executive Committee4, to develop these management responses and an action plan.

The final Management Response and Action Plan was approved by the Chairperson and presented to the Board for information at its December meeting. Management responses are set out below. The related action plan is included at Annex A.

In addition to the Considerations addressed in this document, Board Staff has discussed all comments and observations contained in the Report. While these are not included specifically in the Management Response or Action Plan, Board Staff will consider them, based on resources and regulatory and statutory constraints, as the daily operations of the Board and its regulatory framework evolve over the next few years.

Considerations:

1. Expedite all PMPRB processes

Board Staff is examining current administrative and regulatory processes for ways to improve their timeliness. Quicker processes could mean, among other things, more efficient use of resources, shorter periods of excessive pricing, quicker access to information on price, sales and utilization trends which could result in greater use of information produced by the PMPRB.

Regulatory Program:

The plans to expedite PMPRB processes include:

• Implementation of the Rules of Practices and Procedures for Hearings

The Rules of Practice and Procedure is a published standard set of procedures for all participants to follow in hearings before the Board. The Rules set out the Board's procedures in accordance with the requirement under the Patent Act to resolve matters as informally and expeditiously as the circumstances and considerations of fairness permit. The Rules provide a fair opportunity for interested parties to participate in the Board's hearings. The Board will be implementing the Rules now that they have received Treasury Board approval and have been published in the Canada Gazette, Part II.

• Implementation of performance targets for selected processes/procedures.

The PMPRB will formally establish and publish performance targets for notification to patentees of their failure to file Form 3 and the issuance of “Review and onside letters”. The PMPRB will also formally establish and publish timeframes for patentee responses in the Voluntary Compliance Undertaking (VCU) process.

Reporting Program:

The plans to expedite PMPRB processes include:

• Improving the timeliness of analyses and reports produced under the National Prescription Drug Utilization Information System (NPDUIS) initiative

Board Staff will review the information needs of key stakeholders and the planned allocation of internal resources. In conjunction with the NPDUIS Steering Committee, Board Staff will also examine ways to make the topic selection, project proposal, and report approval and study publication processes more efficient and timely. Board Staff will also seek to update the NPDUIS Steering Committee Terms of Reference.

• Improving the efficiency of the consultation process

Board Staff will evaluate the efficiency of making greater use of informal processes such as working groups and Notice and Comments for consultations with key stakeholder groups. The PMPRB is committed to effective and meaningful stakeholder involvement which is essential to enabling the PMPRB to fulfil its mandate, deliver its programs, launch new initiatives, and build public trust.5

2. Further simplify the Guidelines

Regulatory Program:

• Continue to assess the application and impact of changes made to the Guidelines

Board Staff and patentees continue to gain experience in interpreting and applying the new Guidelines. One of the tools the PMPRB uses to monitor the Guidelines is the Monitoring and Evaluation Plan for the Major Changes in the Guidelines (GMEP). The GMEP is proving to be an effective tool which allows Board Staff to monitor the implementation of the Guidelines and to report to the Board on a regular basis. The first reporting was in December 2011. An annual reporting will be done in December of each year. The Board is committed to having Guidelines that are responsive to the environment, so the monitoring and evaluation will continue. Further precision, and, as appropriate, changes to the text of the Guidelines will be adopted over the coming reporting periods as experience is gained through their application.

• Reduce regulatory burden for patentees

Board Staff will examine a number of potential mechanisms for reducing the regulatory burden on pharmaceutical patentees, including the possibility of once-a-year filing, changes to the excess revenue threshold trigger for commencing an investigation as well as modifications to the CPI Adjustment Methodology.

• Perform analysis of the CPI Adjustment Methodology

Board Staff will conduct an analysis of the CPI Adjustment Methodology to determine if changes to the methodology could be adopted and what impact such changes may have on the pricing of patented medicines.

• Monitor results of hearing panel decisions and issues under judicial review

The Board and Board Staff will also continue to monitor the results of hearing panel decisions and judicial reviews of Hearing Panel decisions. The Hearing Panel decisions and court findings may provide clarification of the application of the regulatory framework.

Reporting Program:

The Reporting Program is unrelated to the Board's Guidelines.

3. Expand plain language offerings

The PMPRB has recently introduced the use of webinars and begun to develop communication materials better suited to a quick-paced, electronic work environment. These materials include the establishment of our twitter account and our series of Analysis Briefs6.

Regulatory Program

Staff will continue to use and expand these initiatives. In addition, consideration will be given to developing and publishing a plain language Guide to Price Regulation which will explain the rationale for the pricing framework and how it works.

The Regulatory Affairs and Outreach Branch will continue its annual workshops and is also developing a “PMPRB 101”7 training program intended for new contacts who have recently taken responsibilities related to the PMPRB´s Regulatory Program with, or for a patentee.

Reporting Program

In 2011-12, in addition to its Annual Report 2011 the PMPRB published an Annual Report 2011: In Brief. The document is intended to provide stakeholders/subscribers with all of the pertinent information contained within the Annual Report in a condensed form.

Board Staff will expand the use of the “Analysis Brief” series to communicate its trend analysis and findings, and related policy implications.

In addition, the PMPRB will continue to include an Executive Summary highlighting policy and/or program implications of the analysis in its studies and other information products produced under the NPDUIS initiative.

4. Expand the target audience for outreach efforts

Regulatory Program

The PMPRB has developed a stakeholder engagement strategy to re-engage and broaden its current stakeholder base. Essentially, the PMPRB's aim is to further develop strategic networks and alliances, to gather information on policy and program design issues which impact stakeholders and, as a result, further improve the PMRPB's transparency and accountability.

Reporting Program

The PMPRB has expanded its Communications tools to further facilitate exchanges with stakeholders and cultivate an understanding of the PMPRB's role and to extend the reach and influence of its activities. To this end the PMPRB will continue implementing its non-stakeholder engagement policy through the development of a bi-lateral meeting series with non-industry stakeholders.

The PMPRB has developed and approved a Communications Strategy for studies and other information products produced under the NPDUIS initiative. Targeted audiences will be identified for each publication to ensure that the language, distribution, depth of treatment and accessibility match the intended users.

In addition, Board Staff is examining ways to better adapt the PMPRB´s consultation process to facilitate participation from a broad stakeholder group.

4 The PMPRB Executive Committee consists of the Chairperson, Vice-chairperson and Executive Director. The Secretary of the Board is also a member and provides secretariat functions for the Committee. Other members of the PMPRB management may be requested to attend on an ad-hoc basis. The Chairperson may also request that other Board members to attend on an ad-hoc basis.

5 Pursuant to subsection 96(5) of the Patent Act, before the Board issues any guidelines, it shall consult with the Minister, the provincial ministers of the Crown responsible for health and such representatives of consumer groups and representatives of the pharmaceutical industry as the Minister may designate for the purpose.

6 Analysis Briefs are short summaries of recent research and analysis work produced by the PMPRB. They can include such topics as trends of sales and prices of pharmaceuticals in Canada; Canadian pharmaceutical markets compared with international markets; R&D spending; and research report summaries.

7 The “PMPRB 101” training program provides an introduction to the price review process, and its filing requirements and filing forms for new contacts who have recently taken on responsibilities related to the PMPRB's Regulatory Program with, or for a patentee. The first training session was held December 6, 2012.

5. Communications and Reporting

In order to keep stakeholders informed, the PMPRB included a brief status update of the Evaluation Report in the October 2012 issue of its NEWSletter.

The final Evaluation Report and this Response and Action Plan were presented to the Minister of Health, The Honourable Leona Aglukkaq and to the Deputy Minister of Health at the end of January 2013.

The final Evaluation Report and this Response and Action Plan were released publicly by means of posting on the PMPRB website on February 7, 2013 and an electronic notification was sent to stakeholders informing them that the documents were available.

The PMPRB Management Team will communicate the results of the Evaluation Report and this Management Response and Action Plan during outreach sessions and speaking engagements. Updates may also be communicated by way of our NEWSletter.

The Chairperson will monitor progress on the Action Plan over time during the regular Executive Committee meetings and Board members will be kept abreast of progress via regularly scheduled Board meetings.

Glossary

Existing Patented Medicines: For purposes of the PMPRB, existing patented medicines include all patented medicines sold in Canada for which the PMPRB has established a benchmark price in accordance with the Board’s Guidelines.

Patentee: As defined by subsection 79(1) of the Patent Act, “the person for the time being entitled to the benefit of the patent for that invention and includes, where any other person is entitled to exercise any rights in relation to that patent other than under a license continued by subsection 11(1) of the Patent Act Amendment Act, 1992, that other person in respect of those rights;”

Research and Development Expenditures: For purposes of the Patented Medicines Regulations, in particular Sections 5 and 6, research and development includes activities for which expenditures would have qualified for the investment tax credit for scientific research and experimental development under the Income Tax Act as read on December 1, 1987.

Review and Onside Letter: This letter provides the results of the price review of existing patented medicines including its compliance status.

Special Access Programme (SAP): A program operated by Health Canada to give practitioners access to drugs that are not approved or otherwise available for sale in Canada.

Voluntary Compliance Undertaking (VCU): A written undertaking by a patentee to adjust its price to comply with the Board’s Guidelines. The Chairperson may approve a VCU in lieu of issuing a Notice of Hearing if it is in the public interest. A VCU can also be submitted following the issuance of a Notice of Hearing. A VCU submitted at this point must be approved by the Board Hearing Panel struck to hear the matter. The Board reports publicly on all VCUs approved by the Chairperson or the Board.

Annex A – Action Plan

The Management Team worked with the Executive Committee to develop and document this Action Plan to address the considerations presented in the PMPRB Program Evaluation Report. This Action Plan was approved by the Chairperson and is posted on the PMPRB web site: www.pmprb-cepmb.gc.ca, under Accountability.

Progress against this Action Plan will be reported to the Chairperson at Executive Committee meetings.

⁸ The “Analysis Brief” series was first introduced in fiscal year 2011-12.