The continuous rise in heat dissipation of integrated circuits necessitates advanced thermal solutions to ensure system reliability and efficiency. Thermoelectric coolers are among the most promising techniques for dealing with localized on-chip hot spots. This study focuses on establishing a holistic optimization methodology for such thermoelectric coolers, in which a thermoelectric element's thickness and the electrical current are optimized to minimize source temperature with respect to ambient, when the thermal and electrical parasitic effects are considered. It is found that when element thickness and electrical current are optimized for a given system architecture, a "heat flux vs. temperature difference" Pareto frontier curve is obtained, indicating that there is an optimum thickness and a corresponding optimum current that maximize the achievable temperature reduction while removing a particular heat flux. This methodology also provides the possible system level Delta T's that can be achieved for a range of heat fluxes, defining the upper limits of thermoelectric cooling for that architecture. In this study, use was made of an extensive analytical model, which was verified using commercially available finite element analysis software. Through the optimization process, 3 pairs of master curves were generated, which were then used to compose the Pareto frontier for any given system architecture. Finally, a case study was performed to provide an in-depth demonstration of the optimization procedure for an example application.