The framework of $α$-potential games has recently been introduced as a tool to analyze finite-player dynamic games, reducing the challenging task of finding approximate Nash equilibria to a control problem of minimizing a single function called $α$-potential function. In this work, we investigate the limiting behavior of $α$-potential games as the number of players $N$ tends to infinity. We show that potential mean field games (MFGs) arise naturally as this limit. Specifically, both the optimal values and the minimizers of normalized $N$-player $α_N$-potential functions converge to those of a mean field control (MFC) problem with measure-valued controls. We establish the equivalence of $\lim_{N\to\infty}α_N= 0$ with the existing conditions for potential MFGs, and provide an unified approach to construct the potential function for MFGs using the techniques from differential geometry in Wasserstein space. We further demonstrate that the objective of the limiting MFC problem serves as a potential function for the corresponding MFGs, an extension of the analogous finite-player setting. This connection yields new constructions of potential MFGs from a finite-player game, through the asymptotic condition $\lim_{N\to \infty}α_N= 0$. As a by-product, we establish propagation of chaos for $N$-player games converging to MFGs for general controlled diffusions with common noise and non-separable control interactions.
