I would probably classify most strong detergents (e.g. SDS and other ionic detergents) as chaotropic agents, but some of the weaker non-ionic detergents (e.g. tween-20, NP-40) don't denature protein so they probably would not be classified as chaotropic agents.
Although detergents are one way of denaturing proteins by solublizing the hydrophobic groups in the interior of proteins, other non-detergent substances are capable doing this such as various organic solvents (e.g. phenol). Other types of chaotropic agents can denature proteins in other ways too. For example, urea and guanidinium denature proteins by interfering with hydrogen bonding.