The mystery that perplexes me the most...

[zenmaster]

What main factor makes "physical" incarnation the main preferred realm of existence over the inner planes aka the afterlife?

The physical body provides a particular configuration of mind. It is an orientation of awareness where that which is not yet accepted is placed into the unconscious. This unconscious is then funneled to consciousness based on that which has been accepted, in the form of interpreted experience. In order to feel and to be congruent with experience, the balancing work proceeds.

What prevents the inner planes from being as enticing as a place to live in?

Apparently, the inner-plane vehicles do not provide the range of experience and intensity and compelling enough illusion for optimal progression.

Especially for negative entities: What satisfaction of control do they get incarnating in physical bodies over staying in the inner planes?

They probably desire to progress. Physicality provides a relatively richer set of opportunities for that purpose and they recognize that. Pre-incarnative overview shows that which they desire as an attraction to some experience in the physical.

What removes most if not all catalyst from the inner planes?

The type of body dictates the mind you are working with. The mind provides the opportunity to process experience in a certain manner which calls upon "the core of your being". If the core of your being is recognized in that which you experience, things become less compelling. You are satisfied with this mere recognition (and the feeling it engenders) even without actual acceptance of it.

Maldek entities were actually provided with inner-plane bodies (of a particular nature) so that they could work with some type of illusion, and so some type of progression could be allowed, until physical bodies could be furnished.