There are places you can go to get a starter set that are much less expensive (but still pricey!). For example:
http://www.elementsales.com/This site sells a set of 30 elements for $200 or at their largest you can get a set of 76 elements for $700 (plus the main elements you are missing from this set are the group I metals which aren't so hard to track down). They also sell wall-mounted display cases, and it is here where you will spend the large amount of the money. For instance, getting the 30 piece set is $200 for the elements and $500 (!) for the display case. As 408 said, in the end it is best to go the homemade route for the display case. To be honest, as someone that has this, the set of the elements isn't that exciting to look at either. Almost all of the metals look just like that, grey metals with only a few like copper, gold, cesium, and mercury that appear unique and interesting. The non-metals are even less interesting...they are about as fun to look at as an empty vial (which, in fact, is what you will see without applying instruments). Though the coin collection on the above site looks very cool I must admit
It won't be a lifelong gift for that element.
Nor for other unstable elements such as Astatine. For these collections what you usually do is include a rock that contains the element in question as part of the decay series for elements in that rock (so at any one time, maybe an atom of what you are looking at is astatine in the astatine vial). None of the very unstable elements can survive as pure sources for a periodic table (time is cruel to collectionists!). Getting any of the elements above Uranium is pointless too; what you are looking at a month from now will contain practically nothing of the element it purports to display.