Objective There is poor evidence to determine the superiority of combination regimens versus monotherapy against infections due to carbapenem-resistant (CR) Gram-negative bacteria. In vivo models can simulate the pathophysiology of infections in humans and assess antibiotic efficacy. We aim to investigate in vivo effects of antibiotic combination on mortality and disease burden for infections due to CR Acinetobacter baumannii, Pseudomonas aeruginosa and Enterobacteriaceae and provide an unbiased overview of existing knowledge. The results of the study can help prioritising future research on the most promising therapies against CR bacteria.
Methods and analysis This protocol was formulated using the Systematic Review Protocol for Animal Intervention Studies (SYRCLE) Checklist. Publications will be collected from PubMed, Scopus, Embase and Web of Science. Quality checklists adapted by Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies and SYRCLE’s risk of bias tool will be used. If the meta-analysis seems feasible, the ES and the 95% CI will be analysed. The heterogeneity between studies will be assessed by I2 test. Subgroup meta-analysis will be performed when possible to assess the impact of the studies on efficacy of the treatments. Funnel plotting will be used to evaluate the risk of publication bias.
Dissemination This systematic review and meta-analysis is part of a wider research collaboration project, the COmbination tHErapy to treat sepsis due to carbapenem-Resistant bacteria in adult and paediatric population: EvideNCE and common practice (COHERENCE) study that includes also the analyses of in vitro and human studies. Data will be presented at international conferences and the results will be published in peer-reviewed journals.
PROSPERO registration number CRD42019128104(available at: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42019128104).
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study aiming to summarise the in vivo evidence of antibiotic combination versus monotherapy to treat highly resistant (eg, carbapenem-resistant) infections in Gram-negative bacteria.
The results of the study can help directing and prioritising future research on the most promising therapy against carbapenem-resistant bacteria.
The heterogeneity of the studies may be high; however, we expect to include over 50 studies with at least 10–15 studies eligible for each Gram-negative bacterial species.
Meta-analysis (and, if possible, network meta-analysis) and subgroup analysis will be performed to compare combination therapies and monotherapies.
Quality of the studies will be systematically assessed according with Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies and Systematic Review Protocol for Animal Intervention Studies’s risk of bias tools.
Dimension of the problem: limited options for carbapenem-resistant infections
Infections caused by multidrug-resistant (MDR) Gram-negative bacteria (GNB) are associated with increased mortality compared with those caused by their susceptible counterpart.1 The increase of MDR infections, coupled with a limited number of novel antibiotics, has recently generated a significant unmet global medical need.2 An increase in resistance to carbapenems, considered as agents of last resort for MDR GNB severe infections, has become an urgent global health threat to address.3 In 2017, WHO published a priority pathogens list aimed at guiding research and development of new antibiotics.4 Critical pathogens included carbapenem-resistant (CR) Acinetobacter baumannii, Pseudomonas aeruginosa and Enterobacteriaceae. Although new drugs showing in vitro activity against CR GNB have recently become available, their use in real-world studies remains limited, and the optimal treatment for CR infections has yet to be established.5 Combination therapy, defined as the association of two or more antibiotics, is frequently used in clinical practice to treat CR A. baumannii, P. aeruginosa and Klebsiella pneumoniae infections. Aims of combination therapy include: (1) prevention of selection of drug-resistant strains; (2) maximisation of in vivo bacterial killing using antibiotics that act synergistically; (3) expansion of the antimicrobial spectrum of antibiotics. To date, however, the superiority of combination therapy compared with monotherapy has not been clearly demonstrated, and the impact of drug synergy on the evolution of resistance remains unclear. Moreover, increased toxicity can derive from the combination of antibiotics, and their efficacy may vary according to the pathogen species.
Animal models to study combination therapy in CR infections
Provided that appropriate models are used according to the research objective6 and the associated limitations are clearly recognised,7 in vivo animal models represent valuable tools for testing activity, pharmacokinetics (PK) and toxicity of single antibiotics and their combinations.8 The capability of in vivo studies to investigate the host immune response, inoculum of bacteria at the infected site and antibiotic pharmacodynamics (PD) adds a dimension to in vitro testing and is essential to translate preclinical data into clinical trials. Animal models have served as a platform for preclinical assessments of novel antibiotics against CR bacteria.9 Furthermore, they help to investigate strategies to enhance antibiotic activities and overcome bacterial resistance.8 Various animal models (eg, murine, other rodents or vertebrates) are used to simulate the pathophysiology of MDR GNB infections and/or the antibiotic exposures observed in humans (table 1).
Septicaemia models are relevant to study MDR GNB since bloodstream infections have been used to compare antibiotic combination with monotherapy in humans.10 11 These models use easy-to-assess endpoints (eg, survival rate and bacterial load, measured as colony forming unit (CFU) in the blood) and are used in preclinical assessment of novel antimicrobials.12 13 Other models can test combinations against CR pathogens due to their capability to assess antibiotic efficacy against bacterial strains with minimum inhibitory concentrations (MICs) above the recommended breakpoint concentrations that designate drug susceptibility or to simulate human antibiotic exposure and immune status.14–17 Animal bacterial endocarditis is often used for antibacterial PK/PD studies.18 Due to the connection between bacterial persistence within the cardiac vegetation and development of resistance,19 it is also relevant for the study of infection relapse following treatment. The murine thigh infection model can also assess antibiotic PK/PD indices (eg, T>MIC, area under the curve (AUC)/MIC or Cmax/MIC).20–22 Urinary tract infection models have also been used to investigate novel antibiotics against MDR GNB.13 Finally, animal models of lung infection can simulate human pneumonia, which has the highest mortality in CR GNB infections.23 To study the efficacy of antibiotic combinations, synergy between different drug combinations is usually determined by statistically significant survival rate or bacterial load reduction in the combination therapy compared with the most active single-drug regimen. To differentiate synergistic activity from an additive effect, some authors24 suggested analysing in vivo synergy as a significant bactericidal effect of the drug combination in comparison with the sum of the bactericidal effect of each agent alone.
Due to ethical, technical and economic issues, invertebrate models of infections including Zebrafish, Galleria mellonella and other invertebrates currently represent an attractive option to study MDR infections.25 G. mellonella (greater wax moth) larvae, in particular, are used to investigate antimicrobial efficacy against CR GNB due to favourable turnaround times, easy procedures and defined endpoints.26 27 Invertebrate models still have important limitations since they may not be representative of human pathophysiology and PK/PD assessments are not feasible; thus, their results require validation in vertebrates.
Immunocompetent and neutropenic models
Animal models of infections often use immune suppression to reduce the potential impact of immune responses on the effect of antimicrobials. Immunosuppression may be also necessary to establish infections that cause disease in humans but not in animals that may be inherently resistant. Pathogens with limited virulence may establish viable infections only in neutropenic animals and require lower bacterial inoculum.28 29 In general, a reduction of the amount of drug necessary to achieve similar microbiological outcome (ie, 1-log kill) is necessary in non-neutropenic compared with neutropenic antibacterial models according to the type of antibiotic and bacteria. Immunocompetent models remain key to reproduce what can happen in clinical practice and can be used to study the impact of leucocytes on antimicrobial efficacy. Both in vivo immunocompetent and neutropenic models of infection have been used to study MDR bacteria; furthermore, the efficacy of monotherapy or antimicrobial combinations can be compared in the same study between immunocompetent and neutropenic animals.30 31
The development of large-scale, randomised controlled trials enrolling patients with MDR GNB infections is often not feasible due to the limited number of patients with MDR infections. In 2017, the Food and Drug Administration (FDA) approved the Limited Population Antibacterial Drug pathway to facilitate regulatory approval from smaller clinical studies targeting urgent unmet medical needs.32 This pathway, similar to regulatory submission guidelines, acknowledges the relevance of solid preclinical PK/PD data to support the efficacy and safety of antibiotics in patients.33 34 Specifically, in vitro PK/PD studies coupled with animal infection models are key tools for directing the antibiotic development process. Animal models of MDR GNB infections integrate the need for optimisation of antibiotic use (eg, optimum dosing strategy) with information that cannot be obtained in vitro, allowing for: (1) preclinical assessment for antibiotics; (2) selection of dose and dosing intervals; (3) support for establishing in vitro susceptibility breakpoint concentrations; (4) evaluation of resistance to antibiotics. Design of both animal and clinical studies rely on PK/PD indices of antibacterial activity and the related values that are set to achieve different magnitudes of bacterial killing.35 Preclinical PK/PD models are key in clarifying exposure–response relationship and design dosage regimens that can be applied in humans, according to the site of the infection.36 Limitations in the use of in vivo PK/PD models include the differences in the PK properties of antibiotics compared with those in humans. Furthermore, there is a limited feasibility of performing prolonged administrations, especially in small animals.35 For these reasons, multiple doses are usually administered, and transient renal impairment in animals through administration of uranyl nitrite is used to prolong the antibiotic half-life. Finally, animal ethical concerns are associated with high bacterial inoculum (eg, ≥108 CFU/mL) that may cause increased early mortality, although high bacterial loads could be useful to investigate development of in vivo drug-resistant mutants.37
The aim of this analysis is to evaluate the impact of antibiotic combinations on infections due to CR (including carbapenemase-producing) Gram-negative bacilli, namely A. baumannii, P. aeruginosa and Enterobacteriaceae (especially K. pneumoniae) analysing the available evidence from preclinical studies.
Systematic review questions
What is the effect of antibiotic combination therapy versus monotherapy on mortality in animal models for infections due to CR A. baumannii, P. aeruginosa and Enterobacteriaceae?
What is the effect of antibiotic combination therapy versus monotherapy on disease burden reduction in animal models for infections due to CR A. baumannii, P. aeruginosa and Enterobacteriaceae?
What is the effect of antibiotic combination therapy versus monotherapy on drug-resistance development in animal models for infections due to CR A. baumannii, P. aeruginosa and Enterobacteriaceae?
Methods and analysis
The structure of this protocol was formulated using the Systematic Review Protocol for Animal Intervention Studies (SYRCLE) Checklist.38
PubMed, Scopus, Embase and Web of Science databases will be selected. All search strings have been discussed with a qualified librarian. The choice of keywords is based on the combinations of terms for carbapenem resistance, Gram-negative bacilli/bacteria (specifically, Enterobacteriaceae, K. pneumoniae, P. aeruginosa, A. baumannii), and related terms (eg, carbapenem, animal, in vivo) and included, specifically:
(“Gram-Negative Bacteria”[Majr] OR “gram negative” OR klebsiella OR acinetobacter OR pseudomonas OR enterobacteriaceae) AND (carbapenem* OR “carbapenem resistant” OR “carbapenem resistance” OR “multi drug resistant” OR “multi drug resistance” OR “pan drug resistant” OR “pan drug resistance” OR “extensive drug resistance” OR “extensive drug resistant”) AND (“in vivo” OR animal* OR murine).
Additional articles will be identified by manually searching the reference lists of included studies and relevant reviews related to the study question. Potentially eligible publications will be also searched on relevant conference proceedings of professional societies (Infectious Diseases Society of America (IDSA); European Society of Clinical Microbiology and Infectious Diseases) published between January 2016 and December 2018. Additionally, we will contact experts in the field.
Studies corresponding to PECOs (review questions for experimental animal exposure review) format will be selected.39 Population: all preclinical (animal) studies including infection models; Exposure: combination therapy for treatment of infections due to CR Enterobacteriaceae, A. baumannii, P. aeruginosa; Comparator: at least one intervention group in the study has to receive the combination of two (or more) antibiotics; Outcomes: (1) primary outcomes: proportion of animal mortality and/or reduction of disease burden (eg, change in the biomass based on culture results); (2) secondary outcomes: resistance development.
Study selection procedure
The study selection procedure will include two screening phases with two observers per phase:
Phase 1: articles retrieved from the databases will be screened based on title and abstract outlined in the search strategy by two independent researchers. In the case of at least one researcher opting for potential eligibility of the study, the full text will be retrieved.
Phase 2: full text of the articles obtained after preliminary selection will be retrieved. Two researchers will independently assess the eligibility of full text for inclusion. Discrepancies during article screening will be resolved by consensus between the two reviewers. Disagreement between them over the eligibility of particular studies will be resolved through discussion with a third reviewer.
Study selection criteria
Inclusion criteria related to the type of study design, study population and disease model, the type of intervention and the outcome measures will be applied to the systematic review.
Type of study design, animals and disease model, intervention
All types of studies describing the effect of combination therapy against CR infections due to A. baumannii, P. aeruginosa and Enterobacteriaceae will be included. Reviews, editorials and protocol papers will be excluded.
All animal species, regardless of age and sex, will be included. All models of CR infections due to A. baumannii, P. aeruginosa and Enterobacteriaceae will be considered. Studies testing disseminated infections and/or measuring bacterial growth from blood will be included.
We will consider as type of intervention any antibiotic combination therapy, defined as the association of two or more antibiotics, including that of an antibiotic with a beta-lactam beta-lactamase (BLBLI) combination treatment and dual beta-lactam therapy (eg, dual carbapenem therapy). Studies with any route, dose and treatment schedule for antibiotic administration will be eligible for inclusion. Single antibiotic (monotherapy, including BLBLIs) will be considered as comparator. When applicable, comparison between combination therapies will be performed; network meta-analysis (NMA) will be considered if more than one study will have at least one treatment arm in common to infer.
A list of antibiotics that can be used in combination against CR bacteria in clinical practice is reported in table 2. In the majority of cases, we expect to analyse combination therapies versus monotherapy with a carbapenem, colistin, an aminoglycoside (amikacin, gentamicin or tobramycin) or tigecycline.
Exclusion criteria for type of intervention will be:
Studies that do not include at least one intervention group receiving combination therapy.
Treatment with antibiotics that are not approved for treatment by FDA and/or European Medicines Agency or antibiotics that are not in late stage of clinical development for use in clinical practice (eg, phase 3 studies).
Studies will be included if they report separately for each treatment regimen (combination or monotherapy or blank) group at least one among: (1) animal mortality rate and proportion measured at completion of study; (2) extent of bacterial burden reduction based on culture results (total bacterial counts measured as change in the biomass, eg, CFU/g or mL) timed at completion of study. Studies will be excluded if the relevant outcome measures cannot be obtained through extraction. Whenever possible, data extraction from graphs will be attempted if no raw data are provided.
We did not include any language restrictions in our search. If an English language version is not available, abstracts and full text that are relevant to the review will be translated into English. Studies published from 1 January 1985 until 31 December 2018 will be included. In case data or experimental groups were used repeatedly in more than one study, for example, to address various hypotheses, data will be included only once.
Criteria order of priority according to the screening phase is reported in figure 1.
Study characteristics to be extracted and data extraction
Data will be extracted from either text or tables in the Results section of the manuscripts of interest. We will pilot test and use a standardised Microsoft Excel form to extract data regarding PECOs questions, including study characteristics, animal model of infection, antibiotic treatment and outcomes of interest. Once the database for data extraction is piloted and validated, two reviewers will independently complete data extraction. In case the control group serves as comparison for multiple treatment groups, the number of animals reported in the control group will be divided by the number of treatment groups served in order to control for multiple comparisons when weighting of effect size. Categorical variables within study results will be entered as proportions (eg, proportion of animals surviving) using a defined coding system in an electronic database. Continuous variables will be expressed as mean (normally distributed data), or median (non-normally distributed data; eg, median survival in days). Back calculation of the necessary data will be allowed.
Parameters for data extraction
For full-text papers, digital object identifier, first author, journal, publication year, source of funding will be retrieved. For conference proceedings, first author, conference, publication year and source of funding will be extracted.
The following data will be extracted: animal type (eg, non-vertebrates, eg, Galleria; rodent or non-rodent), species, breed or strain; sex, age, weight; immunocompetent or immunocompromised status, uranyl nitrate administration.
Data to be extracted will include: site of inoculum (intraperitoneal, tail injection, inhalation, thigh injection, urethral injection, other), positivity of blood for bacteria, type of infection (eg, sepsis and/or peritonitis, lung, thigh, urinary tract infection, other), type of bacteria (origin, eg, biobank vs clinical sample, strain type, molecular mechanism of resistance, MICs); mortality among untreated animals (controls).
Type of antibiotic administered, time of delivery relative to time of infection, loading and maintenance dose, administration schedule, PK/PD indices (eg, T>MIC, AUC/MIC or Cmax/MIC), simulated human antibiotic exposure, duration of treatment and duration of follow-up will be extracted. In case of multiple time points, the latest one available (eg, end of study) will be included for consistency and because of its relevance for clinical implications.
Mortality rate will be expressed as percentage of dead animals at end of study. Reduction of infection burden will be quantified according to the type of infection model by subtracting bacterial load (expressed as CFU per tissue or mL) at end of treatment (EOT) from that of the untreated (control) mice at time 0 or EOT.
We expect to retrieve outcomes mainly from the first time point following completion of therapy administration, while a limited number of studies will include long-term bacterial infections (eg, models with untreated infections that can last for weeks) allowing to analyse outcomes at multiple timepoints (eg, short-term and long-term mortality).
Absence of bacterial growth at end of study (yes/no), reduction of infection burden with combination therapy versus monotherapy and emergence of resistant mutants at 24–48 hours (bacterial growth in the presence of drug, yes/no) will be also collected.
To assess the efficacy of combination treatment, mortality rate and/or bacterial load reduction in the combination therapy arm will be compared with the most active single-drug regimen. According to the number of studies available, different combination regimens will be compared. We will analyse and compare combination regimens that can be used in clinical practice according to the evidence provided by available reports or recommendations.
No clear consensus to assess the methodology and potential biases of animal studies, including those reporting toxicology results—that are essential to evaluate the damage from exposure to environmental chemicals or drug safety prior to human testing—is available.40 Various tools are used and, of these, only a minority has been tested for reliability or validity.41 42 We will use SYRCLE’s tool for assessing risk of bias based on 10 domains including selection, performance, detection, attrition and reporting biases as summarised in box 1.43 A specific part of the Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies Checklist will be used to detect additional risk of bias42 44 including four additional questions (box 1). Finally, to assess the absence of bias related to the use of antibiotics and bacterial strains in the infection model, an adapted version of the criteria for reporting and evaluating ecotoxicity data tool for animal studies will be used assessing seven elements,45 and four additional elements applying specifically to infection models will be included (box 1). Publications will receive a point for compliance of each item in the checklist from which group median scores will be calculated. Data not reported will score 0 in the checklist.
Questions included in the risk-of-bias assessment
1. Was the allocation sequence adequately generated and applied?
2. Were the groups similar at baseline or were they adjusted for confounders in the analysis?
3. Was the allocation adequately concealed?
4. Were the animals randomly housed during the experiment?
5. Were the caregivers and/or investigators blinded from knowledge which intervention each animal received during the experiment?
6. Were animals selected at random for outcome assessment?
7. Was the outcome assessor blinded?
8. Were incomplete outcome data adequately addressed?
9. Are reports of the study free of selective outcome reporting?
10. Was the study apparently free of other problems that could result in high risk of bias?
11. Was the publication reported in a peer-reviewed journal?
12. Was a sample size calculation and/or a power calculation method reported?
13. Was compliance with animal welfare regulations reported?
14. Were potential conflicts of interest reported?
15. Are the antibiotics used identified clearly with name?
16. Is the source of the antibiotic trustworthy?
17. Are the microorganisms well described (eg, name, growth, strain)?
18. Is the experimental system described for the microorganism (eg, choice of medium for bacterial growth)?
19. Are frequency and duration of exposure as well as timepoints of observations explained?
20. Are the study endpoints and their methods of determination clearly described?
21. Is the number of replicates (or complete repetitions of experiment) given?
22. Was the presence of bacteria assessed in blood in sepsis models?
23. Was mortality reported in untreated controls?
24. Were drug concentrations in blood assessed in at least one arm?
25. Was mortality tested as outcome in sepsis models (eg, vs clearance of infection from blood)?
Strategy for data synthesis and meta-analysis
We plan to conduct a quantitative synthesis through meta-analysis if conditions apply. Data will be combined in a systematic review, forest plot and subsequent meta-analysis. For PECO where quantitative synthesis will not be feasible, available data obtained from the included studies will be summarised in terms of PECO questions, overall study conclusions and risk of bias. If the comparability of experimental conditions will allow it, we will perform an NMA. Statistical analyses will be performed using Stata V.15 or higher. We expect to include over 50 studies with at least 10 studies eligible for each GNB species.
We expect to face significant differences in effect between non-vertebrates (eg, Galleria) and vertebrates (eg, rodents) and non-rodents (eg, large animals) for our analyses. Therefore, we will stratify these groups upfront and pool these data separately for additional meta-regression analyses. We expect a limited number of studies reporting monotherapy at different doses, since the review will include articles comparing treatments rather than studies on the efficacy of a specific antibiotic for registration purposes. If multiple doses are provided, pooled results will be analysed and, if applicable, data analyses according to the dose used (eg, low, standard, high dose) will be performed. For the other parameters, we expect outcome measures to be consistent and generally used in the same way, although differences may apply to different infection models (eg, sepsis, thigh infection, pneumonia, etc). Results will be analysed through a pooled outcomes meta-analysis.
Pooled effect size will be expressed as relative risks for mortality compared with monotherapy and as standardised mean difference for reduction in bacterial burden. All data will be calculated with the corresponding 95% CI. When necessary, medians and IQR will be used to estimate means and SDs (and SEs). Monotherapy will be considered as comparator for combination therapies, while comparison with no therapy (eg, saline) will not be included.
If conditions apply, results will be pooled via fixed (low or fair heterogeneity) or random (moderate) effect meta-analysis.
Statistical heterogeneity among studies will be measured by I2 test and will be considered as follows: 0–0.25 low, 0.25–0.5 fair, 0.5–0.75 moderate and >0.75 high. Any reduction in heterogeneity will be assessed through subgroup analysis and meta-regression analysis for categorical and continuous variables, respectively.
Analysis of subgroups or subsets
If a sufficient number of studies (eg, at least 2) are eligible for inclusion in the meta-analysis, a subgroup analysis will be performed to analyse the effect of combination versus monotherapy according to: (1) bacteria species (eg, A. baumannii, P. aeruginosa, K. pneumoniae or other Enterobacteriaceae); (2) animal model, including site of infection (eg, sepsis, thigh infection, pneumonia, etc) and immunocompromised or immunocompetent status; (3) type of inoculum (eg, low or high inoculum infections); (4) molecular mechanisms of carbapenem resistance. In these subgroup analyses, Bonferroni correction will be applied for p values and 95% CI.
Sensitivity and publication bias
Sensitivity analyses excluding one study each time and recalculating the combined results will be completed to investigate the influence of an individual dataset on the pooled data.
Funnel plots will be generated to explore the possibility of publication bias.
Conclusions and clinical implications
Since in vivo models can simulate the pathophysiology of infections in humans and assess antibiotic efficacy, the results of the study can help prioritising future research on the most promising therapies against CR bacteria and inform clinical trials in humans. Furthermore, the study of clinical practice implications of combination therapy against CR GNB is part of the wider project funded by Global Antibiotic Research and Development Partnership/WHO, the COmbination tHErapy to treat sepsis due to carbapenem-Resistant bacteria in adult and paediatric population: EvideNCE and common practice (COHERENCE) study. The COHERENCE project includes also the analysis of in vitro and human studies.
Funding This work was supported by Global Antibiotic Research and Development Partnership.
Competing interests None declared.
Ethics approval Not applicable
Provenance and peer review Not commissioned; externally peer reviewed.
Preregistration This systematic review and meta-analysis protocol reported in this article was formally preregistered on PROSPERO.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.