Figure 1
Design of Partners In Health/Rwanda’s Intermediate Operational Research Training in 2016 and 2017.
Table 1
Iterations of the PIH/Rwanda Intermediate Operational Research Course.
| Area | The first course, 2013 | The second course, 2015 | The third and fourth course, 2016/2017 |
|---|---|---|---|
| Training structure, duration, and competencies | A deliverable-driven training model implemented over 8–12 months, using learning by doing teaching approach and intensive hands-on mentorship. The course has in-person sessions (lectures, open sessions, and plenary sessions) and practicum periods. Competencies covered include developing research protocol, research ethics, study design, research team, data collection and management, data analysis and interpretation, manuscript writing, dissemination. Trainees complete all milestones to stay in the program and to receive certificate. | ||
| 7 2-day modules, every 4–6 weeks, with 7 milestones. The first 6 milestones completed prior to the next training session. The final manuscript submitted to a peer-reviewed journal but had no specific deadline. All course mentors were Rwanda-based. | 3 6-day modules, every 2–5 months, with 3 milestones. A longer gap between Modules 1 and 2 allowed for ethical review and data collection. Final manuscript submitted to a peer-reviewed journal by a set deadline. All course mentors were non-Rwanda-based. Introduced peer-review lecture and study site dissemination before paper publication. | ||
| Mentorship and follow-up | All trainees receive hands-on in-person mentorship during course delivery and practicum, supplemented by ongoing virtual mentorship through emails and skype meetings. | ||
| 2 course mentors and 1 junior mentor. The course mentors supported all 5 research teams for Modules 1–6 and paired with 2–3 teams for Module 7. The junior mentor supported all projects. | 2 course mentors, 6 project mentors‡ and 3 junior mentors. Course mentors supported all 7 projects together throughout all modules. Each junior mentor supported 2–3 projects. Project mentors mentored 1–2 projects each but did not attend training sessions. | 2 course mentors, 4 project mentors and 5 junior mentors. Course mentors supported 4 projects each. Project mentors and junior mentors supported 1–2 projects each. Project mentors did not attend training sessions. | |
| Selection of trainees and research projects | A selection committee including institutional leaders and training mentors who selected district-based program and clinical staff. Trainees implemented quantitative retrospective projects that could be completed within 12 months of course implementation. We hired and trained data collectors for trainee projects. Trainees completed data analysis using Stata. PIH/Rwanda provided research administrative support. | ||
| Trainees applied to the program with a research proposal. | Trainees selected a research topic from a list of research topics submitted by project mentors | ||
| Trainees supervised data collection with support from junior mentors. Trainees led data cleaning and analysis. | Junior mentors supervised data collection and initial data management. Trainees completed data management and led data analysis. Trainees included post-graduate students and university lecturers. | ||
| Training and research project costs | All trainees received full scholarship excluding per-diems discouraged by organizational policies. Research costs included publication, conferences, data collection, software, communication, and travels. | ||
| Each project received US$4000 and trainees managed the budget to cover research costs and trainee supplies and communication. | Each project received US$3000 and trainees managed the budget to cover research costs and trainee supplies and communication. | No project was assigned a specific budget. Research costs were centrally managed for all projects. | |
| Training evaluation | Participants completed milestones and submitted their papers to peer-reviewed journals. | ||
| Participants completed course evaluation. | |||
[i] ‡ Project mentors were principal investigators with active protocols featured in the second and third courses only. They donated research topics/data to the training; PIH/Rwanda: Partners In Health/Rwanda.
Table 2
Cost items for IORT cost estimation.
| Trainee costs | Research project costs | Training delivery and mentorship costs |
|---|---|---|
| •Travels to training venue | •Data collection | •Training venue |
| •Meals during training | •Training of data collectors | •Administrative support |
| •Room and board | •Ethics review costs | •Staff time
|
| • Research communications | • Conference attendance† | |
• Project supplies:
| •Research meetings | |
| •Publication fees† |
[i] † We relied on existing resources to meet these costs e.g. using trainees’ own laptop or borrowed laptop from PIH/Rwanda and requesting for scholarships for international conference attendance outside Rwanda and fee waivers for publication.
Table 3
Description of trainees and training outputs in the IORT course, n (%).
| 2013 | 2015 | 2016 | 2017 | Total | |
|---|---|---|---|---|---|
| Total applications | 24 | 28 | 36 | 44 | 132 |
| Male | 23 (95.8) | 20 (71.4) | 30 (83.3) | 28 (63.6) | 101 (76.5) |
| Female | 1 (4.2) | 8 (28.6) | 6 (16.7) | 16 (36.4) | 31 (23.5) |
| Selected applicants | 10 (41.7) | 13 (46.4) | 16 (44.4) | 16 (36.4) | 55 (41.7) |
| Applicants completing the training | 9 (90.0) | 13 (100.0) | 15 (93.8) | 16 (100.0) | 53 (96.4) |
| Male | 8 (88.9) | 11 (84.6) | 11 (73.3) | 12 (75.0) | 42 (79.2) |
| Female | 1 (11.1) | 2 (15.4) | 4 (26.7) | 4 (25.0) | 11 (20.8) |
| Occupation of trainees | |||||
| Clinical staff† | 3 (33.3) | 6 (46.2) | 5 (33.3) | 2 (20.0) | 14 (37.8) |
| Post-graduate students | 0 | 0 | 3 (20.0) | 2 (13.3) | 3 (8.1) |
| Program managers and coordinators | 6 (66.7) | 5 (38.5) | 3 (20.0) | 8 (53.3) | 14 (37.8) |
| Research staff | 0 | 2 (15.4) | 2 (13.3) | 3 (20.0) | 4 (10.8) |
| University lecturers/tutorial assistants | 0 | 0 | 2 (13.3) | 3 (20.0) | 2 (5.4) |
| Employer of trainees | |||||
| Partners In Health/Rwanda | 5 (55.6) | 7 (53.8) | 5 (33.3) | 6 (40.0) | 23 (44.2) |
| Rwanda Ministry of Health | 4 (44.4) | 6 (46.2) | 5 (33.3) | 3 (20.0) | 18 (34.6) |
| University of Rwanda | 0 | 0 | 5 (33.3) | 3 (20.0) | 8 (15.4) |
| Rwanda Biomedical Centre | 0 | 0 | 0 | 3 (20.0) | 3 (5.8) |
| Research topic selection | |||||
| First choice topic/trainee proposed topic | 9 (100.0) | 9 (69.2) | 13 (86.7) | 14 (93.3) | 45 (86.5) |
| Second choice topic | n/a | 2 (15.4) | 1 (6.7) | 1 (6.7) | 4 (7.7) |
| Topic assigned | n/a | 2 (15.4) | 1 (6.7) | 0 | 3 (5.8) |
| Research proposals | 5 | 7 | 8 | 8 | 28 |
| Manuscripts published | 5 (100.0) | 7 (100.0) | 8 (100.0) | 7 (87.5) | 27 (96.4) |
| Manuscripts under-review | 0 | 0 | 0 | 0 | 0 |
| Manuscripts under-development | 0 | 0 | 0 | 1 (12.5) | 1 (3.6) |
| Conference/meeting presentations†† | 3 | 3 | 18 | 6 | 30 |
| Study facility | 0 | 0 | 13 (72.2) | 0 | 13 (43.3) |
| Rwanda conference | 3 (100.0) | 2 (66.7) | 3 (16.7) | 4 (66.7) | 12 (40.0) |
| International conference | 0 | 1 (33.3) | 2 (11.1) | 2 (33.3) | 5 (16.7) |
| Research capacity outcomes | |||||
| Trainee become junior or project mentor | 0 | 1 | 1 | 1 | 3 |
| Junior mentor become facilitator/mentor | 1 | 1 | 0 | 0 | 2 |
| Trainee enroll in graduate research training | 2 | 3 | 2 | 1 | 8 |
[i] † Included general practitioners, nurses, physiotherapists, pharmacists; This is captured in the text part of results. †† Includes oral and poster presentations. Some projects were presented to >1 conference; In 2016, all projects were presented to the relevant facilities of study and we counted individuals project presentation; IORT: Intermediate Operational Research Course.
Table 4
Research projects completed in the four offerings of the intermediate operational research course in Rwanda.
| Theme | Title of a research project | Year of training | Journal, year of publication |
|---|---|---|---|
| Infectious disease | A novel combined mother-infant clinic to optimize post-partum maternal retention, service utilization, and linkage to services in HIV care in rural Rwanda | 2015 | International Journal of Maternal Child Health and AIDS, 2017 |
| Adherence to renal function monitoring guidelines for HIV-infected patients on tenofovir-based antiretroviral therapy in rural Rwanda | 2013 | Africa Journal of AIDS and HIV Research, 2016 | |
| Dental caries management at a rural district hospital in northern Rwanda: a neglected disease | 2013 | Public Health Action, 2015 | |
| Maternal child health | A retrospective review of the Pediatric Development Clinic implementation: a model to improve medical, nutritional and developmental outcomes of at-risk under-five children in rural Rwanda | 2016 | BMC Maternal Health, Neonatology and Perinatology, 2017 |
| Assessing retention in care after 12 months of the Pediatric Development Clinic implementation in rural Rwanda: a retrospective cohort study | 2016 | BMC Pediatrics, 2018 | |
| Bubble CPAP to support preterm infants in rural Rwanda: a retrospective cohort study | 2013 | BMC Pediatrics, 2015 | |
| Care seeking patterns of families that experienced under-five deaths in rural Rwanda | 2015 | PLoS One, 2018 | |
| A retrospective study of neonatal case management and outcomes in rural Rwanda post implementation of a national neonatal care package for sick and small infants | 2015 | BMC Pediatrics, 2018 | |
| Developmental outcomes of preterm/low birth weight toddlers and term peers in Rwanda | 2017 | Annals of Global Health, 2020 | |
| High burden of undernutrition among at-risk children in neonatal follow-up clinic in Rwanda | 2017 | Annals of Global Health, 2020 | |
| Non-communicable disease | Pregnancy-associated breast cancer in rural Rwanda: The experience of Butaro Cancer Center of Excellence | 2016 | BMC Cancer, 2018 |
| Integration of Chronic Oncology Services in Noncommunicable Disease Clinic in Rural Rwanda | 2016 | Annals of Global Health, 2020 | |
| Treating persistent asthma in rural Rwanda: Characteristics, management, and 24-month outcomes | 2015 | International Journal of Tuberculosis and Lung Disease, 2017 | |
| A clinical mentorship and quality improvement program to support health center nurses manage type 2 diabetes in rural Rwandan settings | 2016 | Journal of Diabetes Research, 2017 | |
| Factors associated with loss to follow-up among cervical cancer patients at Butaro Cancer Center of Excellence, 2012–2017 | 2017 | Annals of Global Health, 2020 | |
| Treatment details and outcomes of Kaposi Sarcoma patients treated with paclitaxel at Butaro Cancer Center of Excellence, Rwanda, 2012–2017 | 2017 | Under-development | |
| Surgery | Assessing the cost of laparotomy at a rural district hospital in Rwanda using time-driven activity-based costing | 2016 | BJS Open, 2018 |
| Longer travel time to district hospital worsens neonatal outcomes: a retrospective cross-sectional study of the effect of delays in receiving emergency cesarean section in Rwanda | 2016 | BMC Pregnancy and Childbirth, 2017 | |
| Maternal predictors of neonatal outcomes after emergency cesarean section: A retrospective study in three rural district hospitals in Rwanda | 2016 | BMC Maternal Health, Neonatology and Perinatology, 2017 | |
| Non-obstetric surgical care at three rural district hospitals in Rwanda: more human capacity and surgical equipment may increase operative care | 2015 | World Journal of Surgery, 2016 | |
| Emergency general surgery in Rwandan district hospitals: a cross-sectional study of spectrum, management, and patient outcomes | 2016 | BMC Surgery, 2017 | |
| Referral patterns and predictors of referral delays for patients with traumatic injuries in rural Rwanda | 2015 | Surgery, 2016 | |
| Presentation of Paediatric unintentional injuries at rural hospitals in Rwanda: A retrospective study | 2017 | Annals of Global Health, 2020 | |
| Perioperative management and outcomes following cesarean section – a cross-sectional study from rural Rwanda, 2017 | 2017 | Journals of Surgical Research, 2019 | |
| Postoperative rheumatic heart disease follow-up: creating a national registry and first results from Rwanda | 2017 | Annals of Global Health, 2020 | |
| Supply chain | Assessment of essential medicines stock-outs at health centers in Burera District in Northern Rwanda | 2013 | Rwanda Journal of Medicine and Health Sciences, 2015 |
| Assessing prescribing patterns of essential medicines in three rural district hospitals in Rwanda | 2013 | International Journal of Pharmacy, 2015 | |
| Caring for NCD patients in rural Africa: A descriptive study of the availability, costs and stock-outs for essential drugs in three rural districts in Rwanda | 2017 | Annals of Global Health, 2020 |
[i] HIV: Human immunodeficiency virus; CPAP: Continuous positive airway pressure; NCD: Non communicable diseases.
Table 5
Training evaluation completed by the third cohort of IORT trainees.
| n | % | Median LC† | |
|---|---|---|---|
| Pre-training evaluation completed by 2016 applicants N = 36 | |||
| Trainee background during course application | |||
| Participation in a research project | 25 | 69.4 | |
| Completion of research related short course | 17 | 47.2 | |
| Presentation at local meetings | 11 | 30.6 | |
| Data collection and management | 8 | 22.2 | |
| Participation in a research writing workshop | 7 | 19.4 | |
| Completion of Introduction to Research course by PIH/Rwanda | 5 | 13.9 | |
| Member of a Research committee or research journal | 4 | 11.1 | |
| Research ethics training | 3 | 8.3 | |
| Data analysis | 2 | 5.6 | |
| Mentoring students on research skills | 2 | 5.6 | |
| Trainee motivations for course application | |||
| Improve research skills | 30 | 83.3 | |
| Enhance the ability to identify and close practice gaps | 27 | 75.0 | |
| Critical consumption of scientific literature | 14 | 38.9 | |
| Become a research leader | 13 | 36.1 | |
| For professional development | 7 | 19.4 | |
| Improve ability to mentor or teach research skills | 7 | 19.4 | |
| Effective participation in research projects | 6 | 16.7 | |
| Improve the use of existing data | 5 | 13.9 | |
| Mid-course trainee reaction completed 2016 trainees N = 15 | |||
| Training content was interesting | 15 | 100.0 | 4 |
| Training content was relevant | 13 | 86.7 | 4 |
| Training did not cover all concepts | 6 | 40.0 | 2 |
| Training improved my research skills | 14 | 93.3 | 4 |
| Training improved my analytical skills | 13 | 86.7 | 4 |
| The facilitators were engaging | 14 | 93.3 | 4 |
| The facilitators were knowledgeable | 15 | 100.0 | 4 |
| The training team was knowledgeable | 13 | 86.7 | 4 |
| The training team was not available to help | 10 | 66.7 | 1 |
| Training pace was just right | 13 | 86.7 | 2 |
| I would recommend this training to my colleagues because it†† | 13 | 86.7 | |
| Builds research skills‡ | 14 | 93.3 | |
| Provides Stata to facilitate future research projects | 9 | 60.0 | |
| Improves critical research consumption | 7 | 46.7 | |
| Uses learner-centered learning by doing methodology | 4 | 26.7 | |
| Training should improve on | |||
| In-person and practicum time for data analysis | 8 | 53.3 | |
| More training time for data management | 5 | 33.3 | |
| Increasing in-person days per session to 7–8 days | 3 | 20.0 |
[i] IORT: Intermediate Operational Research Training; ††Two participants did not respond to this question. ‡ Skills included critical thinking, writing, analytical skills, research production, mentorship, and leadership. † Likert scores ranged from strongly agree (score = 4), agree (score = 3), disagree (score = 2) and strongly disagree (score = 1) or too fast (score = 3), just right (score = 2) and too slow (score = 1).
Table 6
Cost of designing and delivering an operational research course with research project implementation from research idea to publication in Rwanda.
| Median and (Range) for unit costs (US$) | Median and (Range) for total costs (US$)† | |
|---|---|---|
| Trainee costs | ||
| Local travel per participant* | 57 (25–100) | 1257 (550–2200) |
| Training meals per participant | 261 (193–358) | 5736 (4242–7865) |
| Room and board per participant | 236 (212–382) | 5188 (4656–8404) |
| Printing training materials per participant | 23 (21–40) | 505 (455–880) |
| Communication (airtime and data) per trainee | 103 (83–133) | 1648 (1321–2124) |
| Supplies per trainee | ||
| Data analysis software (Stata) | 200 (200–213) | 3200 (3200–3408) |
| Flash drives | 28 (6–28) | 448 (96–449) |
| Laptopsǂ | 500 (500–500) | 8000 (8000–8000) |
| Sub-total | 1408 (1239–1753) | 25982 (22520–33330) |
| Sub-total (excluding laptops)ǂ | 908 (739–1253) | 17982 (14520–25330) |
| Research project costs | ||
| Data collection per project | 892 (643–4528) | 7139 (5141–36224) |
| Training data collectors per course | 234 (194–274) | 234 (194–274) |
| Ethical review per protocol | 1158 (500–1450) | 9267 (4000–11600) |
| Local/regional conference travel per project | 300 (300–300) | 2400 (2400–2400) |
| Research meetings per project | 123 (111–189) | 986 (891–1515) |
| Publication fee per projectǂ | 1300 (775–2100) | 10400 (6200–16800) |
| International conference travel per projectǂ | 2645 (1500–3790) | 21160 (12000–30320) |
| Sub-total | 6653 (4023–12631) | 51585 (30826–99133) |
| Sub-total (excluding publication fee and international conference travel)ǂ | 2708 (1748–6741) | 20025 (12626–52013) |
| Training delivery and mentorship costs | ||
| Training venue and equipment per course | 1035 (1032–1111) | 1035 (1032–1111) |
| Research administrative support per course | 4438 (4438–4438) | 4438 (4438–4438) |
| Senior course mentor (20% time) | 21770 (17847–25693) | 43540 (35694–51386) |
| Training coordinator (50% time)§ | 14234 (5176–23291) | 14234 (5176–23291) |
| Junior mentor (20% time)ǂ | 5693 (2070–9316) | 17079 (6210–27948) |
| International travel per US-based mentorǂ | 6970 (6541–7273) | 13940 (13082–14546) |
| Sub-total | 54140 (37104–71122) | 94265 (65633–122720) |
| Sub-total (excluding international travels for mentors, and paid junior mentors)ǂ | 47170 (30563–63849) | 63247 (46340–80226) |
| Total costs | 171833 (118978–255183) | |
| Total cost excluding laptops, publication fees, international conference travel, international travel for mentors and paid junior mentorsǂ | 101254 (73486–157569) | |
[i] † This includes a total of 22 participants – 16 trainees paired to implement 8 research projects, with 2 senior course mentors and 4 junior mentors, one of whom also functions as the training coordinator. * The cost of travel per participant is sensitive to the design of the course – a seven 2-day sessions course has high travel cost compared to a course of three 1-week sessions; ǂ Laptops, publication fees, international conference travels, international travels for US-based mentors, and junior mentors were considered avoidable direct costs. Trainees used their own laptops or borrowed from PIH/Rwanda, we sought publication fee waivers and scholarships for international conference travels, we prioritized Rwanda-based mentors where possible and junior mentors time, except for training coordinator, was in-kind support provided through collaborating research teams; § Training coordinator also functioned as a junior mentor.