Beilstein is a very old database - it started as Friedrich Beilstein's Handbook of Organic Chemistry in 1881 and has been adding reactions and organic compounds as they were reported in the chemical literature ever since. It is combined with Leopold Gmelin's Handbook of Inorganic Chemistry, which was started in 1887. The truth is there simply isn't anybody out there who has been keeping track of every new organic and inorganic chemical as it is discovered for nearly this long.
The Chemical Abstracts database started with a somewhat different purpose in the early 1900's, but the indexing system has evolved to the point where it forms a similar reaction and compound database. It is the database that underlies Scifinder.
Anybody that wanted to start a new database now would either have to use one of these databases or attempt to create one from scratch. It doesn't make any sense for a government agency to start from ground zero and build a database that has been operating for over 100 years on a solid commercial footing. And it certainly doesn't make sense for a company that has been building a database for this long to just give it away. Instead, what the NSF and other science funding agencies try to do is build enough money into their grants to pay for their researchers to access at least one of these two databases. Almost all research universities have student and researcher access to at least one of these two databases as well, because they are indispensible for chemical research. Yes, they are expensive - good tools usually are, especially considering the amount of literature they have to comb through every year to keep it up-to-date.