They degrade, albeit slowly.
Question is - which one is better, the one that degrades (and liberates chemicals) fast, or the one that degrades slowly, and is almost inert. I have not seen arguments that would make me think polyethylene is in general bad for environment. It is bad in some specific places - I recall reading or hearing about sea turtles dying of starvation after they got their oesophagus blocked. Or something close, I am not sure about details.