Approach to uncertainty in health technology assessment in a Central and Eastern European country: appraisal of cancer drugs by a Polish HTA agency in presence of high crossover rates in clinical trials

Background: Health technology assessment (HTA) plays an important role in reimbursement decision making in Poland and its principles are similar to those used in other countries. However, specific inter-country differences, such as substantial divergence in budgetary resources, may lead to variation in actual HTA practices, e.g. in the approach to uncertainty. Cancer drug reimbursement is a decision-taking area associated with substantial uncertainty. One of its important sources is the presence of crossover (treatment switching) in clinical trials. Objectives: To review the appraisal processes completed for cancer drugs by the Polish HTA agency (AOTMiT) and to compare AOTMiT to the British, Australian and Canadian HTA bodies with respect to strategies of addressing crossover-related uncertainty. Methods: Cancer drug assessment processes in AOTMiT, where a substantial crossover took place were reviewed and subsequently matched with the assessments conducted by NICE, PBAC and pCODR. Ways to approach the crossover-related uncertainty, the influence of uncertainty on the recommendation and uncertainty management strategies were examined. Results: 29 HTA processes related to 6 drugs were included. The crossover rate ranged from 51% to 85% and ITT analyses did not show statistically significant survival benefit. AOTMiT more often yielded negative recommendation, showed less consistent approach to crossover-related uncertainty and a narrower scope of adopted uncertainty management strategies. Conclusions: Crossover constitutes a vital source of uncertainty in the assessments of new cancer therapies. The lack of consistent standards decreases the transparency of assessment processes and can contribute to undertaking suboptimal reimbursement decisions.


Introduction
The change of the political system which took place in the 1990s in Central and Eastern European (CEE) countries resulted in many vital changes in the healthcare policies, including recognition of the need for rational management of healthcare budgets. This contributed to the support for the Health Technology Assessment (HTA) given by the authorities of particular countries. [1,3,4,5] In Poland, HTA Agency was established in 2005 (since 2015: the Agency for Health Technology Assessment and Tariff System, AOTMiT) and 2007 saw the publication of the first HTA guidelines [1] . Moreover, since 2012, key submission requirements have been legally regulated. [6] The submission of a systematic review of clinical studies, cost-effectiveness analysis and budget impact analysis is required and a cost-effectiveness threshold is set by a legal act. [1,7] The previous version of the Polish HTA guidelines (effective 2009-08.2016) [8] was methodologically similar to "Western" guidelines [1] , and the aim of the recent (08.2016) update was to decrease the existing differences even more by implementing the recommendations of the European Network for HTA (EUnetHTA). [9,10] Apparently HTA methodology in Poland does not diverge from those adopted in other countries with well-established HTA system. However, it's not clear whether the actual HTA practice can converge between countries that have different income and healthcare budgets. OECD data indicate that in Poland, Hungary and in the Czech Republic, 2015 healthcare expenditures amounted to 4.5-6.3% of GDP, and in Great Britain, France and Germany -7.7-9.4% of GDP.
[11] Moreover, significant differences in GDP between "old" and "new" European countries still exist.
[12] At the same time, it was demonstrated that the differences in decisions taken by individual HTA bodies often reflected not the differences in scientific evidence interpretation but different approaches to risk and uncertainty. [13,14] The reimbursement of innovative oncological products seems to be the area particularly burdened with a coincidence of numerous sources of uncertainty, risky decisions and unmet needs, where HTA agencies often struggle to evaluate expensive palliative therapies, not resulting -by nature -in either long-term life prolongation or a noticeable decrease in productivity loss. [15,16] Consequently, cancer drugs appraisals may constitute an adequate sample for examination of uncertainty management strategies in the HTA practice.
One of recognised sources of uncertainty in cancer treatments appraisal is the permitted switch (crossover) from a randomly assigned control therapy to the investigational treatment, usually at the point of disease progression. The switching opportunity is considered ethically justified and it speeds up clinical study enrolments. However, high cross-over rate causes problems in interpretation of the outcomes by leading to the dilution of relative treatment effect, and resulting increase in the incremental cost-effectiveness ratio (ICER). [17,18] Attempts are taken to demonstrate the close relationship between OS and progression free survival (PFS) or response rate, which would justify decisions on reimbursing a given drug despite no reliable OS estimation. [19][20][21] Often, however, such approach is considered inadequate within the HTA process, failing to meet the expectations of the substantial improvement in the "final endpoint" which in advanced cancer patients is considered to be the OS gain. [22,23] In response to such requirements, efforts are being made to adjust survival estimates with statistical methods. [24, 17, 18] The vast variety of switching adjustment methods that are applied in HTA submissions, resulted in the need to develop the principles of good practice for HTA institutions, in particular in those regions where appraisal outcomes depend on the ICER estimation. National Institute for Health and Care Excellence Decision Support Unit (NICE DSU) developed a review of switching adjustment methods and provided an analytic framework for identification the method suitable to the case under evaluation, indicating at the same time the lack of consistent approach to the analysed issue in the current practice of NICE. [ We examined the approach of a Polish HTA agency to uncertainties associated with the assessment of clinical efficacy and cost-effectiveness, using the crossover-related uncertainty in cancer drug studies as an example. Practice of HTA agencies in countries with higher GDPs -Great Britain, Australia and Canada served as the reference.

Methods
Data search and selection. At the first stage the AOTMiT website was searched for the documentation of processes covering the full HTA assessment between 2012 and 2016. The HTA assessment processes completed with the issue of a recommendation and concerning a cancer drug were subject of a detailed review. Then, the collected documents were examined for information on crossover in pivotal trials and where such information was found -original study publications were retrieved. Cases of crossover rate exceeding 50% of control group were eligible for inclusion in the review.
At the second stage, completed HTA processes were searched for, on the websites of The National Institute for Health and Care Excellence (NICE), The pan-Canadian Oncology Drug Review (pCODR) and the Pharmaceutical Benefits Advisory Committee (PBAC), with respect to the health technologies selected at the first search stage. In order to ensure comparability, the review covered only the assessments based on the evidence common with the AOTMiT processes (i.e. the processes based on new studies, not considered in the AOTMiT appraisals, were excluded). In the case of an process concerning drug use in multiple indications, assessment for each indication were considered a separate process. The data set is based solely on publicly available documents, which included non-concealed results of clinical efficacy or cost-effectiveness assessment. In Poland, a full HTA process, including costeffectiveness assessment, is currently required only in the cases of reimbursement applications initiated by the MAH, for the authorised products which does not contain the active substance already reimbursed for the given clinical condition. The processes in which the full HTA assessment is not required (i.e. reimbursement withdrawal, off-label indications) were not eligible.
The following document categories were retrieved: (1) recommendations issued at the completion of the HTA process-made available by each of the analysed HTA agencies; (2) report or summary of assessment report of the company submission -made available by AOTMiT, NICE and pCODR; (3) company submission, in parts concerning the clinical efficacy and cost-effectiveness -made available by AOTMiT and NICE; (4) clinical trials -publications and conference abstracts identified based on reference lists of the HTA documents. The basic sources of data were recommendations and assessment reports, and where discrepancies occurred or data were incomplete, the remaining sources were used.
Where multiple HTA processes for the same health technology were identified, each of them was separately considered for eligibility. The possibility to include multiple processes (resubmissions) in the review depended on the given HTA agency's practice regarding the leaving or removal of expired recommendations from website. It should be noted that for the scope of the review it was important that the HTA processes conducted for the same health technologies in particular agencies to be based, to a large extent, on the results of the same pivotal trial, in which a substantial crossover took place. As a result, the most up to date recommendations for some health technologies could have been excluded from the review.
Data extraction and analysis. For each included HTA process, the following data have been extracted: active substance and drug name, indication, comparator and comparison type (direct vs indirect), type of economic analysis, type of OS estimation in the economic model with regard to crossover adjustment (unadjusted, adjusted or both; if adjusted -adjustment methods were extracted), comments on the crossover included in the assessment reports, type of recommendation (positive, conditional/restricted, deferred or negative), key reasons for recommendation, conclusion of the recommendation on clinical efficacy and, separately, cost-effectiveness, comments on crossover included in the recommendation. Moreover, the crossover rate in the study control group and unadjusted (intention-to-treat, ITT) and crossover-adjusted hazard ratios (HR) of death were extracted from the trial publications and HTA documents (if available).
Based on the extracted data, own assessment was made of the relative influence of crossover in pivotal trial on the result of each of the HTA processes analysed, by classifying it to one of the following influence categories: considerable (+++), moderate (++), weak (+), uncertain or difficult to assess due to concealing of an important part of evaluation results (+/-) or lack of noticeable influence (-). Reasons were given for each assessment of the crossover influence, together with a list of strategies for handling the crossover problem identified in the given HTA process based on the documents included.
Finally, differences and similarities between the practice of the Polish HTA agency and that of the remaining agencies were analysed in terms of the approach to the crossover issue at the assessment and appraisal stages, the severity of the impact of this problem on the recommendation and types of strategies adopted in order to manage the crossover-generated uncertainty.

Results
By 22 June 2016, AOTMiT published on its website HTA assessment documentation of 251 processes concerning cancer drugs which were conducted from 2012 to 2016 (Supplementary Figure 1) , so a total number of 29 HTA processes were analysed ( Table 1).
A total of 36 comparisons were identified ( AOTMiT analysis seemed to be less thorough. In three processes (sunitinib, everolimus, crizotinib), AOTMiT comments were limited to an indication that crossover is a study limitation, whereas the analyses of the other agencies dealt also with other issues, such as the meeting of assumptions critical for the use of specific crossoveradjustment methods (PBAC -regorafenib, NICEcrizotinib, crizotinib -PBAC) and choice of the best estimate (PBAC2 -sunitinib). The analyses of reference agencies more often acknowledged the direction of crossover-related bias-i.e. the underestimation of the OS gain (pCODR -sunitinib, pCODRpazopanib), and used external data to assess the possible extent of OS overestimation in the control group (pCODR -pazopanib). Differences across the reference agencies are also noticeable, however, the NICE, PBAC and pCODR assessments seemed more comprehensive than the ones conducted by AOTMiT. In the case of regorafenib, AOTMiT apparently mirrored the comments presented in the previously published pCODR reports, whereas the pazopanib assessment directly referenced to the NICE recommendation (issued in 2011, currently unavailable), justifying the selection of crossover adjustment method recalling reasons given by NICE. The most thorough AOTMiT analysis of the crossover-related bias was found in a sorafenib assessment report, though a clear influence of the prior PBAC recommendation was noted in this process.  A total of 29 recommendations were included (Table 3, Supplementary Figure). AOTMiT and NICE had the highest percentage of negative recommendations (50%). pCODR recommended reimbursement of the assessed drug most often, however, in each case, the positive recommendation included additional conditions to be met. Crossover-related uncertainty was directly pointed out among the reasons for 7 nonpositive recommendations. However, the true influence of crossover on the final recommendation was not limited to those cases, since high crossover rate often contributed to a conclusion of "uncertain efficacy," "uncertain clinical superiority", "uncertain risk-to-benefit ratio" or "uncertain cost-effectiveness" -stated among direct recommendation reasons (Supplementary Table 2). Moreover, lack of evaluator's approval for the crossover handling approach initially proposed in the submission, in some cases resulting in an adjustment of economic model assumptions during the HTA assessment towards more conservative ones, contributed to the fact that submission estimates were considered uncertain or not sufficiently robust, the clinical effect estimatesoverestimated and the ICER value -high and/or uncertain, also forming grounds for non-positive recommendations.
Crossover-related uncertainty influence on the HTA outcome were the least frequently identified in the Polish HTA documentation -however, at least weak influence was judged to be present in 50% of the analysed processes. In the cases of NICE, pCODR and PBAC assessments, it amounted to 100%, 100% and 80% (67% -with multiple PBAC recommendations being accounted for as a single comprehensive process), respectively. Accordingly, the share of HTA processes, in which the influence of crossover was rated as moderate or considerable, was the lowest for AOTMiT (33%), whereas the remaining agencies' score was at least 50%. Detailed reasons for each of the influence judgements are given in the Supplementary appendix (fifth column in Supplementary Table 2). For positive recommendations, the influence of crossover was generally considered weak or uncertain, with the exception of cases where crossoverrelated uncertainty caused a prolongation of the whole HTA process (PBAC -sunitinib). In the cases in which crossover-related uncertainty contributed to AOTMiT issuing negative recommendations, in the other agencies corresponding processes more often accounted for the conditional reimbursement (pCODR -regorafenib, crizotinib; PBAC3 -crizotinib) or the decision being deferred (PBAC3 -sorafenib).
Various strategies implemented by individual HTA agencies were identified for the crossover-related uncertainty management in the analysed sample. Among the identified approaches, "calculation" strategiesmainly aimed at identifying the best estimates of HR of death and/or ICER -as well as approaches basically independent from OS estimation can be distinguished (Table 4; for further details see sixth column of Supplementary Table 2). Table 3. The direction of recommendation and the result of assessment of the influence of crossover on the HTA process.

Drug
Recommendationa (rating of the influence of crossover-related uncertainty  Within "calculation" approach category, the following strategies were mainly identified in the AOTMiT processes: the use of adjusted assessments provided in submissions, accounting for highly conservative assessments (mainly -based on the ITT analysis) and one case of referring to the recommendation of another HTA agency (NICE) in determination of the most reliable adjustment method. Documentation of the remaining HTA agencies indicated the use of a more varied scope of actions. NICE and PBAC examined validity of using particular methods of statistical adjustment, considering the direction and extent of a possible crossover-related bias, and the possibility of meeting the critical assumptions for the proper performance of the given adjustment technique. During the assessments conducted for NICE, PBAC and pCODR, evaluators made changes in the applicant model. Furthermore, in PBAC documents, cases of recommendations for use of crossover statistical adjustment were identified, and use of an external, published analysis which addressed the crossover problem.
The Polish HTA agency seemed to have been reluctant to use methods of drug value assessment other than the calculation ones. Only one AOTMiT recommendation (everolimus) directly indicated other benefits than OS gain among the arguments for drug reimbursement. Australian and Canadian agencies, apart from quite consistent examination of PFS predictive value for OS and the clinical analysis of PFS improvement significance as such, also resorted to such decision-making tools as: analysis of the magnitude of the observed hazard ratios, regardless of statistical significance (pCODR), reduction of the risk of financing a cost-ineffective therapy by the use of a risk-sharing instrument addressing the identified uncertainty or reduction of therapy cost (PBAC) and accounting for, in the comprehensive cost-effectiveness assessment, the importance of such decision-making factors as unmet clinical need, lack of alternative treatment or a small number of patients resulting in a minor budget impact (PBAC).
In order to accurately illustrate the AOTMiT approach to the crossover-generated uncertainty, the case of crizotinib was selected for a more detailed description.
Case study: crizotinib in advanced or metastatic non-small-cell lung cancer associated with an anaplastic lymphoma kinase fusion gene (ALK-positive NSCLC).
Crizotinib efficacy in previously treated patients with advanced or metastatic ALK-positive NSCLC was studied in the randomised phase 3 PROFILE 1007 study (n=347) versus chemotherapy (docetaxel or pemetrexed). The study protocol allowed for one-way crossover from the control group in the case of disease progression and additional OS analysis with an crossover adjustment with the RPSFT method was planned. At data cut-off, 64.4% patients in the control group crossed to crizotinib [101] . Unadjusted HR of death in the interim analysis did not show crizotinib effect (HR=1.02; 95% CI: 0.68-1.54). Upon the crossover adjustment, the point estimates indicated 17% death risk reduction, but the confidence interval was broad and still included unity (RPSFT HR=0.83; 95% CI: 0.36-1.35). [71,108] The ITT analysis showed significant superiority of crizotinib over chemotherapy in PFS (HR=0.49; 95% CI: 0.37-0.64; p<0.001), being the primary study endpoint. [101] AOTMiT's report [43] pointed out that the crossover allowed by the study protocol was the limitation which hampered the evaluation of the drug's impact on OS and safety assessment and indicated PFS as the most reliable efficacy endpoint. The cost-effectiveness assessment results were concealed to a considerable degree. However, available data indicate that some crossover adjustment method was applied in the submission model, which was not accepted by AOTMiT, due to use of unpublished HR estimate and objections to the model description. The ICER estimate, indicated as the most reliable in the assessment report, exceeded the cost-effectiveness threshold established in Poland. [43] When comparing the scope of strategies of managing the crossover-related uncertainty, used by AOTMiT in the case of crizotinib, with the course of assessments of the other drugs, significant differences in the evaluation course can be observed. The assessment report for crizotinib [ When comparing the strategies for managing the crossover-related uncertainty in the crizotinib assessment conducted by AOTMiT, with the approach of the other HTA agencies, it seems that, although the approaches of NICE, PBAC and pCODR varied, each of them dealt with the problem in a more comprehensive manner. Although the NICE recommendation, similarly to that of AOTMiT, was negative, it was preceded with a detailed analysis of adjusted HR estimates, taking into account the sensitivity of particular methods to such factors as sample size or the time-dependency of the treatment effect. [71] Finally, despite clearly stated limitations of OS gain estimation, NICE recognised the therapy as associated with the clinical benefit as based on the "noteworthy extension to PFS" and "very high response rate." It was also recognised that treatment with crizotinib would result in an overall survival gain compared with docetaxel, with the uncertainty related to the exact magnitude of the gain.
[70] In this case, negative recommendation was issued due to the high ICER and objections related to the indirect comparison with BSC, which AOTMiT did not consider at all. The two remaining agencies decided to apply compromise solutions by recommending crizotinib reimbursement under additional conditions. The PBAC's crizotinib assessment seems particularly interesting for analysing crossover-related uncertainty approaches. In total, it included three recommendations (first submission [75] and re-submission [74] resulting with a deferred recommendation and the second re-submission that yielded a conditional recommendation [73] ), and it seems that here the uncertainty related with OS estimation constituted the central problem. The OS modelling method was the subject of discussion between the company and the evaluator and after all, a change of assumptions in the base-case analysis was made by PBAC. The PBAC's scenario was more conservative than the manufacturer's submission base-case, but it still assumed the non-zero incremental OS gain as a result of crizotinib use. The resulting ICER indicated the lack of crizotinib cost-effectiveness, but PBAC proposed a complex Managed Entry Scheme in which future therapy financing conditions depended on the outcomes to be observed in the real clinical practice [73] . Thus, by recognising that the available scientific evidence does not provide sufficient data for an accurate clinical benefit assessment (mainly due to crossover-caused interference), it was made possible to gather data for the purpose of further drug evaluation with the uncertainty-related risk shared by the MAH and the public payer. The recommendation also indicated an unmet clinical need and a small number of patients.
[73] The pCODR process included into the review was the re-submission analysis, since at the time of the first submission [112] , the RCT PROFILE 1007 results were not yet available. A substantial uncertainty around the OS benefit and related difficulties in the interpretation of the result were noted [77] , whereas the economic evaluation report (partly confidential) focused on uncertainty sources other than crossover, such as the selection of utility weights. The changes made in the model resulted in an increase in the ICER beyond the acceptable level. , in Australia -within the Managed Entry Scheme [114] and in Canada -in insurance plans of all provinces. [115][116][117][118][119][120][121][122][123][124] Eventually, crizotinib was granted reimbursement in Poland in November 2016, which was over 3 years after the completion of the HTA process.

Discussion
The review demonstrated that the Polish HTA agency, similarly to more experienced agencies, identifies crossover as an issue in the assessment of the survival benefit. However, at the same time AOTMiT uses a narrower scope of strategies to manage the crossoverrelated uncertainty, which results in a less consistent influence of the crossover issue on final recommendations. Lack of regular approach to the crossover issue across assessments is also noticeable. More comprehensive analyses of the crossover-related uncertainty in the AOTMiT processes were conducted in cases where there was the possibility to use assessment of another HTA agency as reference. Such a trend in the AOTMiT practice was already noticed. [125] Making references to the recommendations of other HTA agencies, especially well-established ones, is not controversial as such. In some cases, however, such practice may lead to unequal treatment of particular health technologies -a technology the assessment of which cannot account for, e.g., NICE or PBAC recommendation, may be assessed in a less comprehensive manner and due to the lack of the basis to indicate the appropriate adjustment method in the final assessment, the possibility of such adjustment may be ignored.
The distinctive difference between the AOTMiT processes and these of pCODR and PBAC is the fact that the two latter agencies use a broad array of "non-calculative" methods to reduce uncertainty, showing clear willingness to work out, in cooperation with the applicant, the conditions under which the drug could be made available to patients and the risk of financing a costineffective therapy would be adequately reduced. NICE, PBAC and pCODR recommendations include content indicating that substantial attention is paid to identifying unmet clinical needs and relevant patients' values. These decision-making aspects seem to be rather scarcely present in the analysed AOTMiT documents what may lead to the recommendation process being more freely finalised as negative, without sufficient attempts at dealing with the uncertainty. This situation can partially explain the limited array of strategies used when compared with the remaining agencies.
There are obvious limitations to the method of comparing the assessment process with respect to the same HTA technologies by different agencies, as adopted in our study. The considered HTA bodies work under different formal and legal conditions. For example, at NICE, a health technology assessment starts with a scoping phase, during which the HTA agency position on the key submission elements such as comparators are presented to the applicant, before the final recommendation is preceded by the publication of a draft version. In Poland, there is no formal procedure for any initial arrangements between the evaluator and the MAH preceding the submission and the applicant has very limited options of making adjustments in the submission during the assessment process. Such cross-agency differences certainly have an impact on the final HTA outcome. Therefore, comparison of the recommendations' direction in our review was of supportive significance, whereas our focus was on the details of assessment processes. To ensure the true presence of the crossover issue, only cases affected with very high (>50%) crossover rate were eligible. However, it was acknowledged that the final HTA recommendation is the result of a multifaceted evaluation in which different weights are attached to specific factors . The simultaneous presence of another source of substantial uncertainty, such as the lack of a relevant head-to-head study or early study termination, overlapped with the crossoverrelated uncertainty -the extraction of a direct influence of crossover on a given HTA process in such cases was particularly challenging.
In our study, the crossover issue served as an example of the uncertainty source in the HTA process, whereas a detailed analysis of the justification to use certain strategies, e.g. specific statistical models, was beyond the scope. It is, however, an important issue, with far reaching consequences for reimbursement recommendations and decisions. [24, 30, 31] Another important limitation of our study was the need to rely solely on the data disclosed in the public domain. We are aware that disclosed content may not fully reflect the actual contribution of particular factors to the final recommendation. Transparency is, however, acknowledged as one of a key features of a properly conducted HTA process. Thus, it should be assumed that the publicly disclosed HTA content may enable development of an unbiased picture of the appraisal conduct, in spite of a confidential exclusions.

Conclusions
According to our knowledge, this review is the first study analysing the methods used by an HTA agency in a CEE country to manage the crossover-related uncertainty. Our data indicate that the Polish HTA agency, in comparison with its counterparts in Great Britain, Australia and Canada, uses a narrower array of strategies to manage uncertainty and its approach to the crossover issue demonstrates little consistency. Considering the dynamic development of the HTA methodology in Poland and a strong influence of HTA assessment results on the drug reimbursement decisions, it can be expected that in the years to come, the decision-taking practice of the Polish HTA agency will, also in terms of the issue analysed herein, be approaching the standards developed by more experienced institutions.