2. Why does the anion derived from dimethyl malonate add to the β-position (1,4-position) rather than directly to the carbonyl group of mesityl oxide?
I think that the 1-4 addition allows for a p-orbitial of pi systems to form which will stabilize the reaction but if the addition was to the carbonyl, this will just form a highly reactive and unstable system.
I reason this along a slightly more practical line of thinking. Diethyl malonate has a pKa of 13. If its anion added to an aldehyde or ketone, you create a carbinol with a pKa of about 16. Deprotonation can easily catalyze a reverse reaction. This reaction can easily be in equilibrium.
If the anion adds to the beta carbon, even though it may be slower, it can form a new ketone or aldehyde. The pKa of a ketone will be about 18-20. In the presence of the malonate, poorer leaving group, and any stereoelectronic constraints that may apply, this reverse reaction is now much slower. I reason the conjugate addition is a thermodynamic product.
If a very strong base/nucleophile is used, then even though conjugate addition can occur, 1,2-additions predominate. This isn't a proof of how the reactions actually take place. It is a rationalization of why 1,2 and 1,4-additions might occur. Weak bases, amines, thiols, HCl, beta diketones, etc., favor 1,4-addition and Grignard reagents, organolithium reagents, acetylides, etc. favor 1,2-addition.