In this thesis, we study some of the problems that arise in mathematical physics due to divergences. There are a number of situations in modern physics which force us to contend with infinities. In particular, this paper traces a technique known as dimensional regularization from its roots in fractional calculus to what is now termed split dimensional regularization. In the first part of our thesis, we discuss fractional calculus. Our emphasis is on the Riesz integral and how it can be used to eliminate infinities in certain situations. The first example shows how the Riesz integral may be used to solve the m-dimensional Cauchy problem, and the second shows how it can be used to treat the relativistic electron. The second part of the paper focuses on standard dimensional regularization. We approach quantum field theory by using path integrals instead of the canonical approach. We then focus on noncovariant gauges, because in these, ghost particles are known to decouple. One of the difficulties that arises from use of noncovariant gauges, occurs because spurious propagators appear in the theory. Thus, we consider the principal-value and Mandelstam-Leibbrandt prescriptions designed to alleviate this problem. We next discuss the methods of standard dimensional regularization and split dimensional regularization, and give an example of the latter by applying it to the calculation of the gluon self-energy in Yang-Mills Theory. We see how the use of split dimensional regularization, at the one-loop level in the pure axial gauge, yields results identical to those obtained when we use the principal-value prescription along with standard dimensional regularization. (Abstract shortened by UMI.)