You start with the presumption that the product will only consist of the two elements that are reacting, i.e., calcium and nitrogen. So, your product will be something like, as you mentioned, CaxNy. Placing the x and y reminds us that Ca and N could be but are not necessarily in a 1:1 ratio (reminder that we usually do not write the numeral 1 in formulas). To determine what x and y are, you must use the fact molecules and formulas and reaction equations must be charge balanced. Since calcium metal and nitrogen gas are both charge neutral, the product must also be charge neutral. Following that, many elements that you'll be dealing with have a number of preferred oxidation states determined by the octet rule. You can use this information to make a great guess at what the formula of the product will be.
In this case, what oxidation state does calcium prefer? What about nitrogen? Look at their positions on the periodic table and think about how many electrons they need to gain or lose to satisfy a full octet and what their charges would be if they gained or lost that many electrons. Once you have that information, you can figure out how many calcium (ions) and nitrogen (ions) respectively would combine to form a charge neutral molecule.
Let's take a different example, say combining lithium and oxygen.
Li + O2 --> ??
So, we know this will form LixOy. Lithium likes to be +1 since it wants to lose one electron. Oxygen likes to be -2 since it wants to gain two electrons. So to balance, you need two lithiums for everyone one oxygen, so the correct formula for the product is Li2O and the equation is:
Li + O2 --> Li2O
But we have to balance the equation as well so that both side have the same number of each atom type, which leads us to:
4Li + O2 --> 2Li2O
Now, try to complete your problem.
(Do note that elements can take on several different formal oxidation states to reach a completed octet. For instance, nitrogen could also conceivably give up five electrons (5+) or gain three electrons (3-) to satisfy the ocetet rule. In these cases, you have to think about electronegativities to help you predict whether the element will go on way or another. Nitrogen would take electrons from lithium but would give them up to oxygen. In other cases like the transition metals, the octet rule is not really a great guide and instead you kind of just have to memorize what the preferred oxidation states are... and multiple different product formulas are viable solutions. Copper, for example, can be +1 or +2, and so can form two different oxide products. In those kinds of more advanced cases, the problem will usually give you more information to help you decide what the appropriate answer is.)