Adding a base to deprotonate the benzoic acid and make it soluble in water would allow you to separate naphthalene and benzoic acid in EtOAc. If you had a mixture of phenol, benzoic acid, and naphthalene, you could still separate the benzoic acid by taking advantage of the difference in pKa between benzoic acid and phenol. For example, sodium bicarbonate is a strong enough base to deprotonate benzoic acid, but it will not deprotonate your phenol.