New View of Eisav's 400 Men

The Torah Temimah has the following comment:

The Medrash says that when Eisav was pursuing Yaakov with 400 men, he went and consulted with the customs collectors of Egypt… He took the tax responsibilities of Egypt and passed them on to Yaakov. This is difficult. How does it explain the 400 men? And why specifically from Egypt? Look at the insights of Rada"l… But the truth seems to be that the explanation for "customs collectors" is that Eisav sought permission to be a tax collector. And this hints to the idea that he sought permission specifically from Egypt because they were the ones to be enslaving Yaakov's family for 400 years – this is the reason for his having "400 men."
וע' במ"ר כאן על הפסוק וארבע מאות איש עמו, הלך ונטל [עשו] אגרומי ממצרים, ועמלו מאוד המפרשים בבאור זה, ופירשו שלקח מכס שנוטלין במצרים להטילן על יעקב, וכמה קשה באור זה, וגם מה זה הסבר על ד' מאות איש, ולמה דוקא ממצרים, ועיין בהגהות הרד"ל, אבל האמת נראה, כי הפי' אגרומי הוא רשות להיות מוכס, ורומז שלקח רשות ממצרים שעליהם הובטח לשעבד ביעקב ד' מאות שנה, וזהו הרמז מד' מאות איש:

And so I offer this new perspective. 

There are many possibilities as to what Eisav's intentions were when bringing 400 men. They boil down to greeting Yaakov with honors, coming to fight, and just that Eisav doesn't travel alone.

This Midrashic perspective, however, sheds an entirely different light. Perhaps Eisav, after having gone to Egypt and having learned about what was going on there, came to the realization that the exile promised to Avraham would be taking place there. And Eisav wanted no part of it.

So, when Eisav approaches Yaakov with 400 men, he is essentially saying, "I know you wanted the birthright and to be the spiritual heir of our father and grandfather. ENJOY IT! I don't want to be exiled to Egypt, I don't want 400 years away from my home. And I am bringing 400 men with me to indicate, without necessarily articulating this directly, that I am happy with the version of 400 I have (strong men)."

Therefore there never was a threat. The magnitude of the numbers accompanying Eisav was mostly for show - to indicate Eisav was happy to be a tax collector and to return to his land - not to be the ancestor of a slave people who would experience 400 years of exile, but as leader of a band of 400 men who would serve him honorably and symbolically as he pursued his own destiny, apart from his slavery-bound brother and his family.