Toluene does not evaporate through the plastic walls. What you observe, the collapsing walls, can be the cooling of toluene after you have used the bottle and closed it.
When toluene is warmer, more of its vapour fill the bottle, pushing the air away. If toluene cools in the closed bottle, the vapour pressure drops, and the external atmospheric pressure lets the walls collapse.
You can observe this with a plastic water bottle. Pour a bit of hot water in it from the tap, let the hot water move in the bottle and wet the surfaces for some time, then close the bottle and wait, let optionally cold water flow on the bottle outside. Less water in the bottle makes it faster. The walls collapse.
Your toluene isn't supposedly as hot as tap water, and its vapour pressure resembles water. Maybe your bottle is less stiff than a water bottle, especially if it has no ripples and is designed to be squeezed by hand.