I think you are severely overlooking the fact that beryllium, while it does create neutrons, is HORRIFCALLY slow at generating neutrons and is pretty damned inefficient. A clandestine lab would blow its budget in seconds in order to come up with the raw material needed to produce microscopic amounts of these materials.
When an "element dealer" sells elements, they are sold in very small amounts at a HUGE markup over the bulk price. This is how they make money. Terrorists and "bad people" aren't magical creatures that have unlimited resources. Like everyone else on earth, if they want something they need to come up with the money to procure it. The amount of money they would need to spend to get what they need would bankrupt them. Do you honestly think Osama Bin Laden would be able to do the s#*$ he does if he wasn't loaded with money? While terrorists are spineless cowards, they aren't stupid.
A great analogy is to say that you could ruin the economy by flooding it with gold and causing the price of gold to plummett to nothing. Gold exists in the ocean dissolved as a salt. You just need to extract that gold salt and then refine it. There's millions of pounds of gold dissolved in the waters of the Earth, and if you purify it you can cause the price of gold on the market to drop to nothing and cripple the economy. It may not be easy, but you can do it