Background: The diagnosis of Alzheimer's disease (AD) is based on an ever-increasing body of data and knowledge making it a complex task. The PredictAD tool integrates heterogeneous patient data using an interactive user interface to provide decision support. The aim of this project was to investigate the performance of the tool in distinguishing AD from non-AD dementia using a realistic clinical dataset. Methods: We retrieved clinical data from a group of patients diagnosed with AD (n = 72), vascular dementia (VaD, n = 30), frontotemporal dementia (FTD, n = 25) or dementia with Lewy bodies (DLB, n = 14) at the Copenhagen Memory Clinic at Rigshospitalet. Three classification methods were applied to the data in order to differentiate between AD and a group of non-AD dementias. The methods were the PredictAD tool's Disease State Index (DSI), the naïve Bayesian classifier and the random forest. Results: The DSI performed best for this realistic dataset with an accuracy of 76.6% compared to the accuracies for the naïve Bayesian classifier and random forest of 67.4 and 66.7%, respectively. Furthermore, the DSI differentiated between the four diagnostic groups with a p value of <0.0001. Conclusion: In this dataset, the DSI method used by the PredictAD tool showed a superior performance for the differentiation between patients with AD and those with other dementias. However, the methods need to be refined further in order to optimize the differential diagnosis between AD, FTD, VaD and DLB.