Previous clinical studies have reported adverse cognitive outcomes for people living with HIV (PLWH), but there are no population-based studies comparing cognitive function between older PLWH and comparators without HIV in sub-Saharan Africa. We analyzed baseline data of 40 + years-old participants in “Health and Aging in Africa: A Longitudinal Study of an INDEPTH Community in South Africa” (HAALSI) cohort. We measured cognition using a battery of conventional instruments assessing orientation, immediate- and delayed-recall, and numeracy (N = 4560), and the Oxford Cognitive Screen [OCS]-Plus, a novel instrument for low-literacy populations, assessing memory, language, visual-spatial ability, and executive functioning (N = 1997). Linear regression models comparing cognitive scores between participants with and without HIV were adjusted for sex, education, age, country of birth, father’s occupation, ever-consumed alcohol, and asset index. PLWH scored on average 0.06 (95% CI 0.01–0.12) standard deviation (SD) units higher on the conventional cognitive function measure and 0.02 (95% CI − 0.07 to 0.04) SD units lower on the OCS-Plus measure than HIV-negative participants. We found higher cognitive function scores for PLWH compared to people without HIV when using a conventional measure of cognitive function but not when using a novel instrument for low-literacy settings.