Background Population aging has intensified challenges related to polypharmacy, underscoring the need for pharmacist-led comprehensive medication review (CMR) services. PhAI (Pharmacist AI Assistant) is a generative AI-powered clinical decision support system that employs a document-grounded generation approach, incorporating medication review criteria tailored to the Korean clinical context into the model\'s prompt and deterministically fetching regulatory-approved drug labeling information to generate structured, evidence-based outputs. This study evaluated the clinical accuracy, reproducibility, and usability of PhAI in pharmacy practice settings. Methods To evaluate accuracy and reproducibility, six standardized clinical cases (three community-dwelling older adults and three long-term care facility residents) were developed. An expert panel established reference standards for medication-related problem (MRP) identification and intervention recommendations using a weighted scoring framework. PhAI underwent five repeated test runs per case. Reproducibility was assessed using the perfect agreement rate, Fleiss\' kappa, and Gwet\'s AC1. Usability was evaluated by 25 experienced pharmacists who used PhAI at least twice in practice settings, using the System Usability Scale (SUS) and five-point Likert scales. Results PhAI achieved an overall accuracy of 93.0%, exceeding the excellent threshold (≥90%), with 94.1% for MRP identification and 91.8% for intervention recommendations. Accuracy improved to 96.0% for clinically essential items, surpassing the mean expert pharmacist accuracy of 83.4%. Gwet\'s AC1 was 0.94 (95% CI: 0.90–0.97), indicating almost perfect reproducibility. The SUS score was 82.3, exceeding the industry benchmark of 68, with all usability domains scoring above 4.0/5. Following PhAI adoption, 80% of pharmacists reported reduced review time (p<0.001). Conclusions PhAI demonstrated high accuracy, reproducibility, and usability in supporting pharmacist-led medication reviews for older adults. These findings support the potential role of generative AI as a clinical decision support tool that augments rather than replaces pharmacist expertise in pharmaceutical care.