Hardware designed to play high quality music. (Think using Ubuntu vs Ubuntu Studio for music production)
Dedicated buttons instead of all touch screen.
More options for integration with other devices or systems
No distractions. Phones nowadays demand our attention for every little thing. Every app, no matter what it is, has notifications.
The Bluetooth is better.
You can literally hear the difference in the quality of the music if you use good quality headphones/ear buds. The same song, same file, will not sound the same if it’s a good quality FLAC.
I prefer an MP3 player over my phone. Here is the one I use. Why I like this one: