Mathematical physics is best described as the intersection between mathematics and physics. This intersection is quite large but the compliment is non-empty. Here are two quotes, one of which talks exclusively about physics while the other talks exclusively about mathematics:

• We never experiment with just one electron or atom or (small) molecule. In thought-experiments we sometimes assume that we do; this invariably entails ridiculous consequences.

  Erwin Schrodinger
  1952

• [after proving Euler's formula in a lecture] ... we cannot understand it, and we don't know what it means. But we have proved it, and therefore we know it is the truth.

  Quoted in E Kasner and J Newman Mathematics and the Imagination (New York 1940)

Goals

The goal of a mathematician is to be correct with certainty. They do this using proof. A proof consists of three parts, the assumptions, the logical arrows, and the conclussion. A mathematician can always be certain that their conclussions are right given whatever assumptions were necessary to their proof.

• Logic is invincible because in order to combat logic it is necessary to use logic.

  Pierre Boutroux

The goal of a physicist is to understand the universe. Currently, their main method of accomplishing this is by using mathematics. Physicists are not always interested in saying things which are correct, especially if they do not believe it will help them gain a deeper understanding of the universe.

• It all works because Avogadro's number is closer to infinity than to 10.

  Ralph Baierlein

A mathematical physicist is someone whose goals happen to align with both mindsets: they want to understand the universe while being correct. Most physicists do not have the luxory to care about correctness and proof. Most historic innovations in physics came from a leap of faith, and these leaps typically led to new perspectives on physical phenomena that could not have been reached through rigour alone. The contributions of mathematical physicists may be incremental, but they play an important role in cementing a foundation for physicists to work off of.

On the Process

My research in mathematical physics is best described as "motivated by physics, performed using mathematics". To rigorously answer some question about the universe, one first has to formulate the question in a mathematical framework. Formulating the question mathematically is still considered in the realm of physics. After solving the problem we have to analyze the results and see what it says about the universe. This is also considered in the realm of physics. What defines the mathematical vs theoretical physicist is how they go about the middle step, solving the mathematical problem.

• In a world in which the price of calculation continues to decrease rapidly, but the price of theorem proving continues to hold steady or increase, elementary economics indicates that we ought to spend a larger and larger fraction of our time on calculation.

  John Tukey

Most american mathematical physicists are found in math departments. This is not because mathematical physicists do not participate in the first and third steps, they do indulge in pure physics. But the middle step is what occupies most of their time and so they are mathematicians. We often spend our time cleaning up arguments made by physicists.

• [Dirac] required a function which was zero everywhere, except at a single point, where it was discontinuous and behaved like an infinitely high, infinitely narrow spike of unit area. Mathematicians were quick to point out that, strictly speaking, there is no function which has these properties. But Dirac supposed there was, and proceeded to use it so successfully that a new branch of mathematics was developed in order to justify its use.

  B.R Krusse

Ontology

A statement is defined by its assumptions as much as its contents. The ontology of any physical theory is what determines the axioms that a mathematical physicist uses in their analysis. All statements about the universe should follow from these axioms. If the ontology is unclear then the physical theory will be too.

• Try to get your understanding of the physics to the same level of understanding that you have for the mathematics. If you can't do that, then it's probably a problem with the physics and not you.

  Sheldon Goldstein

Here are some examples of physical theories which have clear ontologies.

Newtonian mechanics is a classical theory of physics in which point particles interact via forces. We study Newtonian mechanics by translating this ontology into a purely mathematical framework. The positions of particles are vector quantities which evolve according to second order differential equations.

Maxwell-Lorentz Electrodynamics is a relativistic theory in which point chargess act as singularities in electromagnetic fields which are defined over Minkowski space-time. The motion of the point charges partly governs the motion of the fields, and the field are supposed to govern the motion of its singularities via the Lorentz force. Unfortunately this Lorentz force law is not well-defined due to a divergent self-force term. The ontology of this theory is clear, but it is not consistent.

General Relativity is a relativistic theory in which energy acts as a source for the curvature of space-time. The ontology of this theory is clear, and is consistent for energy of nice "regularity". As long as you don't evolve the system for too long.

Physics as a Foundation

None of these physical theories describe the universe we live in. Regardless of how well they happen to predict the outcomes of experiments, it is important not to forget this. Most physicists are not bothered by this. But some go too far, and claim that physical theories need not have a clear ontology. For them it is good enough to be able to predict the outcomes of their desired experiments.

• There will always be voices (even from otherwise reasonable people) insisting that it's simply not the role and purpose of physics to provide a coherent and objective description of nature.

  Detleff Durr and Dustin Lazarovici

This sentiment is common among those who study quantum mechanics. Luckily, the history of quantum mechanics also provides us with the perfect counterargument against this line of reasoning. The experiments which we so desperately want to predict the results of are a part of the universe they exist in. If we can't provide a coherent description of said universe, we cannot possibly provide a coherent description of the experiments which live in it. Confusion regarding the definition of an experiment would inevitably arise, as evidenced by the measurement problem of quantum mechanics. At some point we must admit that the numbers we are getting are not predictions of orthodox quantum mechanics about any physical reality, because the theory does not describe a physical reality.

• Are billions upon billions of acts of observer-participancy the foundation of everything? The very fact that we can ask such a strange question shows how uncertain we are about the deeper foundations of the quantum and its ultimate foundation.

  John Wheeler

What if the universe is fundamentally inconsistent?

Then I wouldn't want to study it. And yet, I would want to study it. Which statement is true? Both.
If you see a problem with this then you likely agree with my belief that an inconsistent universe isn't worth studying.

• Why go on? And you only go on if the game is worth the gamble. Now the universe has been going on for an incredible long time. And so really, a satisfactory theory of the universe has to be one that's worth betting on... If you make a theory of the universe which isn't worth betting on, why bother? Just [quit].

  Alan Watts