In fact, in Toronto proper, the FED with the largest middle class* is North Etobicoke, also known as Rexdale, despite being among Toronto's lower income FEDs (see bottom of post).
The middle class is largely concentrated in Brampton, adjacent parts of Toronto and other suburbs, and in eastern part of the GTA in Scarborough and Durham Region. It's smallest in Toronto's core, and in the northern and western suburbs.
The lowest income FED in Toronto - Humber River-Black Creek - which includes Jane-Finch, Emery, Humber Summit, Chalkfarm and York University has the 3rd largest middle class of Toronto's 25 FEDs. North Etobicoke and Humber River-Black Creek have low average incomes not because of a small middle class, but because of a lower/working class that greatly outnumbers the upper class.
Meanwhile, Willowdale, which is the most middle income of Toronto's FEDs, ranks only 20th out of 25 for the size of its middle class. The FED with the smallest middle class is Don Valley West, which is also Toronto's wealthiest. Don Valley West contains wealthy streetcar suburbs and very wealthy large lot suburbs, as well as the low income apartment community of Thorncliffe Park separated from the wealthy neighbourhoods by an industrial park and railway line.
In fact, within Toronto, the general trend appears to be one where the FEDs with the lowest average income have the largest middle class, and the size of the middle class shrinks as average incomes increase.
The trend is similar in the Greater Toronto Area suburbs, although these lack any low income FEDs.
However, combining the City of Toronto with the GTA suburbs causes the trend to break down, as the middle income suburban FEDs tend to have a larger middle class than the lower income City of Toronto FEDs.
The upper-middle class looks like it can still be priced out of certain neighbourhoods. While North-Central Toronto has a large upper class population, the upper-middle class population is still relatively small. Nonetheless, the upper middle class can still (and do) live away from the neighbourhoods with large low income populations in the east and northwest of the city proper. The upper-middle class largely lives in the suburbs, although there seem to still be some in the parts of the urban core where a house/rowhouse is still somewhat affordable of where there are a lot of condos (probably yuppies?).
While the middle class was relatively unsegregated, the upper class is the most segregated income group in the Toronto area. Not only do they avoid the lower-income eastern and northwestern parts of Toronto, they also largely avoid certain suburban areas such as middle class Brampton. The upper class mostly live in areas with good access to high paying jobs and with high end homes, both near downtown and in the suburbs.
While most of the middle class can probably still afford buy single family homes (or townhouses) in the less expensive parts of the GTA, most of the lower-middle class probably cannot. They are mostly concentrated in the less expensive parts of Toronto, usually neighbourhoods that have a lot of older rental apartments. Perhaps there are also some cases of large (extended?) families living in lower cost single family homes.
The low income population lives primarily in the city proper, with more moderate numbers in closer-in suburban areas. They mostly live in similar neighbourhoods to the lower-middle class, although perhaps in greater numbers in neighbourhoods with large amounts of public housing, especially in downtown areas. However, I'm not sure how to explain the large low income population in Willowdale and Rosedale-University.
And finally, a map of the average income decile of each FED.
*I'm defining upper class as people who fall within the wealthiest decile of Ontario's income distribution, upper-middle class as the 2nd and 3rd decile, middle class as the 4th-7th decile, lower-middle class as the 8th and 9th decile and low income as the lowest decile.