- Chris White — Malaria Technical Advisor, Population Services International Myanmar
- Tom Kanyok — Senior Program Officer, Field Studies, Bill & Melinda Gates Foundation
- Jenna Brereton — Consultant (Geneva Global), Bill & Melinda Gates Foundation
- Cari Tuna — Co-Founder, Good Ventures
- Elie Hassenfeld — Co-Executive Director, GiveWell
- Natalie Crispin — Research Analyst, GiveWell
Note: This set of notes gives an overview of the major points made by Chris White, Tom Kanyok and Jenna Brereton.
Good Ventures and GiveWell spoke with PSI Myanmar about its artemisinin monotherapy replacement project.
Monitoring and evaluation of the project
Population Services International (PSI) is planning to conduct monitoring and evaluation of project through a variety of methods. Primarily, PSI will utilize three globally recognized study methods:
- Annual Private Outlet Surveys
- Baseline and endpoint Household Surveys
- A baseline Supply Chain Assessment
For a published summary description of these three components see: http://www.malariajournal.com/content/10/1/325.
Private Outlet Surveys:
Outlet surveys allow PSI to examine the market share of different antimalarials passing through private outlets. The specific objective is to assess trends over time regarding:
- Availability, Price and Market Share of antimalarials, including newly introduced, highly subsidized, quality-assured Artemisinin Combination Therapy (ACT)
- Provider knowledge
- Availability of Rapid Diagnostic Tests (RDT)
These surveys are multi-round and cross-sectional, utilizing a multi-staged cluster sampling technique. Within clusters, a census of ALL outlets with the potential to sell or provide antimalarials is completed. Outlet types include:
- Private clinics and hospitals
- Private pharmacies
- Itinerant drug vendors
- General retailers
- Community-level providers.
These outlets are then screened to ascertain which meet study inclusion criteria. Data are weighted to ensure results are nationally representative. Two tools are used for these surveys in every outlet included in the study: A physical audit of ALL antimalarials in stock on the day of the visit and a provider interview to assess knowledge.
The first annual (and therefore baseline) outlet survey has already been completed. In this first survey, 3476 outlets were screened. Of these, 1274 met the inclusion criteria, with 1182 outlets being fully audited.
The next outlet survey will take place before the end of 2012 and will be paired with a Household Survey (and ongoing Supply Chain Assessment). Triangulation of results from these three survey methods in this initial phase is possible as they are all nationally representative, collected over similar periods, and use a common sampling approach.
For a published description of the Outlet Survey methods, and PSI’s utilization of them across six malaria endemic countries, see: http://www.malariajournal.com/content/10/1/326.
Population-based Household Surveys
Nationally representative household surveys capture treatment-seeking behavior and use of antimalarial drugs, as well as respondent knowledge of antimalarials. The design mirrors that typically used in population-based surveys and follows standard Demographic Health Survey (DHS) sampling procedures. Eligibility for household inclusion in the Myanmar epidemiological context is determined by the presence of any family member with fever in the past two weeks. As a result of relatively low malaria transmission (and therefore lower fever incidence rates) compared to sub-Saharan Africa, the survey planned for August 2012 will include over 4,000 households.The survey will serve as the baseline, with a second end point survey occurring in year three at a similar point in time (i.e. towards the end of the monsoon season). If continuation funding is secured, the end point (year 3) survey will serve as a mid-term assessment.
A full description of the methods, and PSI’s utilization of them across six malaria endemic countries, can be found at: http://www.malariajournal.com/content/10/1/327.
Supply chain assessment
The supply chain assessment is a structured survey of wholesalers and a series of in-depth interviews with wholesalers and other key informants. The aim is to get a picture of the supply chain serving various types of outlet and measure the mark-ups at each supply chain level.
Goal-level impact assessment
PSI’s artemisinin monotherapy replacement project is but one component (albeit an important one) of a broader Myanmar Artemisinin Resistance Containment (MARC) framework. As such, the project contributes to, but wouldn’t be solely responsible for, any goal-level impact observed. Lastly, many aspects determining the nature of artemisinin resistance spread remain unclear, thus further complicating evaluation of impact at this level.
Goal-level impact will be tracked by the World Health Organization (WHO) and relevant academic institutions, through the ongoing assessment of parasite clearance rates (an indicator of the prevalence of drug resistance) across Myanmar.
PSI regularly attends technical meetings where information from relevant institutions is disseminated.
While the methods described above allow PSI to evaluate the project at various intervals during the project term, more qualitative and frequent monitoring activities are required to assess whether or not underlying assumptions in the project model are correct and allow PSI to respond promptly if not. The following are examples of activities that PSI will implement to ensure the project is on track:
- Review of monthly sales data (and replenishment orders) from the main distributor
- Annual quantitative Mystery Client Surveys that will enable PSI to monitor indicators of provider practices (esp. in relation to partial dose provision and price setting). Mystery Client Surveys use actors who pretend to seek treatment for fever at a variety of outlets. Qualitative spot checks will be conducted on a more ad hoc basis throughout the year. The audits will target all outlets, regardless of whether they are supplied by AA Medical Products Ltd.
- Routine monitoring of the supply chain to ensure consistent availability of stock at all levels
Overall risk level: In its pre-funding report on the project, DFID assessed the risks involved in the project as high. The Gates Foundation noted that this report was written before Myanmar's political situation "opened up," and that they believe the chances of success have improved since then. They also believe that while the project continues to be somewhat risky, the objective of the project is very important and potentially very high impact.
Risk of drugs being misappropriated by the military: Through its outlet and household surveys and routine monitoring, PSI expects that it will be able to detect if large quantities of drugs are missing. However, it does not expect to be able to detect whether individuals employed by the military benefit from accessing subsidized antimalarials.
Patients continuing to take partial treatment regimens: The full regimen of ACTs is three days long with two doses per day and multiple tablets at each dose (number dependent on age/weight category), compared with a 7 day full regimen for the most common current treatment, monotherapy. Currently, patients often buy only enough pills for a few days of treatment (with cost being the most likely barrier to full course purchases). PSI plans to try to set prices such that a full regimen of ACT costs about what patients currently pay for a partial course of monotherapy (around 500 Kyat). Treatment adherence is a "known risk." Tom Kanyok of the Gates Foundation noted that PSI's provider and consumer education efforts (and price setting strategy) will play a large role in changing behavior around completing drug regimens. The published literature suggests that once consumers are able to obtain full doses (usually determined by cost and geographic access) that compliance is not a major obstacle. For example, see Smithuis et al. 2006. Efficacy and effectiveness of dihydroartemisinin-piperaquine versus artesunate-mefloquine in falciparum malaria: an open-label randomized comparison. Lancet 2006; 367: 2075-85, which states, "The effectiveness ofthe unsupervised treatment, as in the usual context of use, equaled its supervised efficacy, indicating good adherence without supervision."
A national ban on importing monotherapy is currently coming into effect as import licenses expire and are not renewed, though stocks will remain in the country for some time. Deliveries of ACTs subsidized by PSI will begin in July/August 2012. It is a large country with porous borders, so PSI expects that country-wide complete elimination of AMT will take some time.