The USAID | DELIVER PROJECT uses monitoring and evaluation activities to improve program performance; the focus is on integrating routine monitoring into activities at the country and central level. The project’s monitoring and evaluation plans guide central- and country-level activities as they measure the project’s contribution to USAID’s strategic objective of increasing the availability of essential health supplies in the private and public sector facilities. The plans outline key objectives and specific activities needed to achieve those objectives, including performance measurement indicators and expected outcomes.
The project uses various qualitative and quantitative measurements or indicators to monitor project and commodity logistics systems performance. These indicators are analyzed to develop or modify project strategic plans, promote and advocate best practices in public health supply chain management, and report to various stakeholders.
Using these existing tools, the project’s performance will be measured by the monitoring logistics system performance and evaluate program outcomes. The Logistics System Assessment Tool (LSAT), a qualitative diagnostic instrument, provides an overview of the strengths and weaknesses of a logistics system. The Logistics Indicator Assessment Tool (LIAT), a standardized quantitative assessment instrument, measures product availability. In countries with robust logistics management information systems (LMISs), the project will use LMIS data to analyze and evaluate product availability and other essential logistics data for both country and project use; while at the same time, validating the quality of the LMIS data, as needed.
Project monitoring and system performance are integrated into all supply chain improvement activities. Supply chain managers utilize programmatic information to perform day-to-day activities to ensure commodity availability throughout the supply chain pipeline. The project will also monitor the country’s quarterly reports; they provide valuable information about country-specific objectives and key logistics, commodity security, procurement, and finance indicators. The information from the country reports will highlight issues and concerns that could have an impact on commodity availability. For other countries with existing baseline data, the project will document the data and periodically assess smaller sample sizes. At this time, for our country programs, the M&E team is investigating the use of Lot Quality Assessment Sampling (LQAS), personal data assistants (PDAs) for data collection, and Geographic Information System (GIS) technology.
The monthly Procurement Planning and Monitoring Report (PPMR) provides important information on the supply situation of country programs and any short-term supply issues. The report has two benefits: it monitors a country’s short-term supply, which measures the program’s overall health; and second, the report documents how well the program responds to supply problems. Additionally, annual and mid-year CPTs provide information for central-level procurement planning and information for measuring forecast accuracy.
The project also measures the effectiveness of its own procurement activities. We use data in the central database to track funds and orders (ORION) that will measure system performance, such as on-time shipments and inventory management. We also measure procurement performance; for example, conducting comparative price studies to ensure competitive pricing.
