Genz, L. R.; Nair, S.; Topf, M.

The recent breakthroughs in AI-driven protein structure prediction have revolutionized structural biology, unlocking new possibilities to model complex biomolecular interactions. We evaluated widely-used scoring metrics for assessing such models predicted by ColabFold with templates, ColabFold without templates, and AlphaFold3. We benchmarked the optimal cutoffs for these assessment scores using a set of 325 heterodimeric high-resolution protein structures and their predictions. Our results show that ColabFold with templates and AlphaFold3 perform similarly and both outperform ColabFold without templates. Furthermore, interface-specific scores are found to be more reliable for evaluating protein complex predictions compared to the corresponding global scores. Notably, ipTM and model confidence achieve the best performance in distinguishing correct from incorrect predictions. Based on our results, we developed a weighted combined score, C2Qscore, to improve model quality assessment, which was used to analyze dimers from large assemblies solved by cryoEM. This revealed potential limitations of the assessment scores when multiple configurations of heterodimers are possible. C2Qscore has been integrated as a tool into our ChimeraX plug-in PICKLUSTER v2.0, facilitating interactive access to the scoring metrics. The plug-in is freely available for download from the ChimeraX Toolshed and https://gitlab.com/topf-lab/pickluster-v2.0.git. This study provides insights into the strengths and weaknesses of current scores and offers guidance for improving protein complex model assessment.