My question is, should the concentration of the electrolyte affect how long it takes for the voltage to reach zero(equilibrium)?
Definitely time required depends on the initial concentration of the electrolyte.
Think this way: imagine you have a 0.001M solution of Cu
2+ and it needs 10 minutes till the voltage measured is zero (these are completely random numbers, but they will do). Now imagine you start with a 0.01M solution. Concentration is higher than in the previous case, so it will go down for some time till the concentration is 0.001M - and at this point you need 10 minutes. So for 0.01M solution you need 10' + some time.