I have seen this question asked on a number of StackExchange sites, and the answer is invariably that it would impose too high a burden on the pages that don't use MathJax. I think this response may be based on a misunderstanding of how MathJax works, and stems from an early comment from Jeff Atwood where he incorrectly stated that MathJax would require downloads of more than 500KB, which is off by an order of magnitude, as I show below.
One thing that contributes to this misunderstanding is that MathJax loads components only when needed, and so there is a big difference between what is loaded on a page without mathematics from one that does include mathematics. The size of that difference depends on how MathJax is configured (and so on how many components are pre-loaded versus loaded on demand).
On a page with no mathematics where MathJax is loaded, MathJax requires two files: MathJax.js itself and a configuration file. If taken from the MathJax CDN, the file will be shipped in compressed form, and the actual data transfered for MathJax.js will be about 14.4KB. For comparison, the background image for this site (http://cdn.sstatic.net/betameta/img/bg-noise.png) is 12.3KB, and the sprite image (http://cdn.sstatic.net/betameta/img/sprites.png?v=3) that holds the various button imagery is 16.5KB. So MathJax.js can be included in a page with little more cost than the background image, and less then the other button images.
The configuration file can vary greatly in size depending on what is included. For example, the configuration used at math.stackexchange.com (which includes most of the components that MathJax will load for processing TeX commands) comes in at about 40.5KB, for a grand total of 55KB for every page on MSE. Note that that is about a tenth of Jeff's claim.
Now the configuration file used at MSE contains more of the MathJax code than you might want to preload on a site that doesn't have that much math. Another reasonable configuration would come in at 25KB (leaving out the code that actually processes TeX commands to be loaded on demand when math actually appears). This still includes much of MathJax's code that is needed common to all its input and output processors. Leaving that out and having a truly minimal configuration that just has enough to define the input and output processors and search for TeX code on the page, you could get a configuration file that is only 3.7KB.
So it is possible to include MathJax on every page with as little as 18KB, or just over the size of the button images on this page. That can range up to 55KB, with the decision about how much to include being based on how much math you expect to have on your site.
Just as a comparison, here are some other scripts that are included on this page already (these are the compressed numbers, so are actual network transfer amounts; the uncompressed files are larger):
- ga.js 13.8KB (google analytics)
- wmd.js 48.7KB (the markdown editor for creating answers)
- full.js 78.9KB (I presume this is the main code for running the site)
- stubs.js 19.0KB (not sure what this one is)
- all.css 109.3KB (the CSS controlling the look of the site)
In total, along with the two image files I already mentioned, these are over 330KB of data. It seems a little disingenuous to call MathJax the "great white whale", as Jeff did, when it is possible to bring it in at 18KB, while the CSS file for this site is over 100KB all by itself. Even at 55KB for a reasonably fully preloaded version of MathJax, that is comparable to the editor that is part of SE already, and much less than "full.js", whatever that is.
Of course, this is for pages without mathematics, but I'm making this distinction because the claim has always been that the MathJax dependency is too great for pages that don't include math. I don't think you can really say that about 18KB (or even 55KB).
For a page that does include mathematics, MathJax would need to load more code, so pages with math do have a heavier footprint. For a typical page in math.stackechange.com that does include mathematics, MathJax has to load about 48KB more (it depends on what the math actually is). That is about 100KB total (and had they used the 18KB configuration instead, that would have translated into about 80KB needed extra for a page with mathematics). For users who don't have the STIX fonts, the web fonts would be used, and that is additional download on pages that have mathematics (again, dependent on what the mathematics includes, but usually around 80KB). So it is true that pages with math are heavy, by even at 200KB total for MathJax and its fonts, it is still not the 500KB that were claimed.
For pages without math, 18KB to 55KB seems perfectly doable, especially when the page is already at over 330KB. This is one myth about MathJax that seems to be prevalent in the StackExchange community, and I hope that we are able to finally put it to rest. Since StackExchange is a MathJax partner, I'd be happy to help put together the proper configuration file (or files) to make MathJax more readily available on SE sites.