This paper reports the first benchmarking of smallholder tilapia farming practices in rural northern Zambia, measuring fish growth, feed utilization and water quality parameters, using a participatory action research approach. The effects of three different feeding regimes that are currently used in smallholder farming systems in northern Zambia on the growth of the native tilapia, Oreochromis macrochir, were tested with 15 farmers (40% female), each having three ponds (one pond per feed treatment per farmer = 45 in total), over eight months in 2016–2017. The three feed treatments included: (1) formulated feed only comprising maize and soybean; (2) goat manure only; and (3) formulated feed and manure together. Three hundred (300) O. macrochir mixed-sex fingerlings, 0.5 ± 0.1 g average weight, were stocked in each pond. Water quality (pH, dissolved oxygen, and water temperature and transparency) was measured monthly, as was fish growth in each pond by randomly sampling thirty (30) fish and taking individual weights and lengths. Results showed that the water quality parameters were broadly similar in the different feed treatments over the eight-month experiment period, except for the water transparency values that were higher in the feed only treatment when compared to the other two treatments at months four through eight. Means for survival rate, weight gain, specific growth rate, total amount of fish harvested, and yield were all higher for the feed and manure together treatment than the other two feed treatments at the end of the experiment. A linear mixed-effects regression analysis that controlled for several fixed effects and included pond as a random effect found that the feed and manure together treatment increased fish growth by 10.8 ± 3.7 g at month eight compared to the manure only treatment. The analysis also found a negative effect on fish growth (−9.4 ± 3.8 g) in ponds that used the feed only treatment. The pond effect accounted for 10.1% of the total variance in the data at month eight. Future research and development efforts in Zambia should examine more closely the factors that influence fish growth and that make up the pond effect to determine which innovations and better management practices could help smallholder farmers increase the productivity of their ponds.