The early 2010s was when I first got into music in a big way. Early on, I just had some files in the first music player app I thought looked cool, but as my library grew my methods for listening to music got more and more convoluted, culminating in a DIY cloud streaming setup. I wanna write about it, for my own memory and amusement. Maybe you’ll get a kick out of it too!
When I first got into music, I didn’t have a smartphone yet. I did have an iPod Touch, but honestly I didn’t really find it all that compelling to try to use. iTunes was really confusing and frustrating to me, and I kinda just used it to play games and listen to podcasts. So I listened to all my music on my computer in quod libet, a GTK music app that’s still my favorite desktop music player to this day. It takes ages to index a large library from scratch, but at that time that wasn’t even a consideration for me, and once everything is indexed adding new stuff goes fast.
Once I got my first smart phone, the world of music on mobile opened up to me. I could just plug the thing in and drag music to it like a flash drive. But also, there were music apps that looked so cool. This was in the early 2010s and I was rocking Poweramp, an app with gapless playback and an incredible neon aesthetic that felt right at home with the Holo era of android. Gods I miss the Holo era. Just look at this.
At the time I used a file browser called “ES File Explorer”, which I refuse to link to since as far as I remember they eventually got bought out by an adware or spyware company or something like that, but it could act as an SFTP server. Whenever I got new music, I’d fire up the SFTP server and transfer it over from my desktop. I didn’t have too many albums so this worked ok. I had a lot of pony fan music, some furry artists, and a bunch of more mainstream EDM DJ sets.
At first it was mostly 128-192KB/s ish MP3s (for the podcasts), and some high quality MP3s for the other stuff, since I was mostly pirating music at the time. Later, I started getting paid to write game mods, and so I had some disposable income to buy music from artists directly. Most of the stuff I was into was on Bandcamp, so I started amassing a collection of FLAC files. This was great for my archivist tendencies, but terrible for my storage space on disk. What was I to do?
Before this ever truly became a problem, I recognized that my SFTP strategy was just not going to cut it on its own. So I took the obvious course of action and started transcoding. I’d used ffmpeg a couple times in the past, but this is when I really cut my teeth on it. Over a few days I put together a fish script that would incrementally transcode my audio library from my main archive to another folder for holding the phone transcodes. But I also wasn’t content to just transcode everything; why trancode my mp3s and m4as? They were already compressed enough as it was. Eventually I could just run a single command to transcode any new music, SFTP the results to my phone, and have my music library in sync in a matter of minutes.
This again worked well for awhile. Then I discovered Vaporwave.
The Collection Grows
The thing about Vaporwave is that there is a LOT of it. Not all of it is good, but even reddit’s recommendation list at the time was massive. There was an open file server that had a huge collection of nearly all the releases to date at that point, so I figured I’d just download the whole collection so I’d have everything on hand as I went through others’ recommendations. Around the same time, the Business Casual label was offering their entire discography at an absurdly low price, so I picked that up too.
Now I had decidedly Far Too Much Music. I may have been able to afford an upgrade to a 64GB microSD, but even with that I’d be pushing up against it with my mp3 transcode. That’s when I learned about opus. I fucking love the opus codec. I don’t know that many people can say they have a passion for codecs, but I sure do, and opus has gotta be my number one favorite. Opus was designed for low-latency interactive applications like Skype calls, but sort of on accident ended up outperforming AAC-LC, AAC-HE, and Vorbis too, all while being open and royalty free. It also degrades gracefully at lower bitrates; Even once artifacting is noticeable, it ends up just having a lo-fi vibe to it rather than the obnoxious arifacts you hear with something like low bitrate mp3.
All this made opus the perfect fit for my phone, which desperately needed something new to let me continue cramming my entire music library on it. I updated my previously-mentioned script to transcode my entire library to 96kbit/s Opus files. I experimented a bit with other bitrates, but this was my personal sweet spot. I could notice a little loss of detail in the high end in the percussion of a few particular tracks, but I really had to actively listen for that, and by and large the quality degradation wasn’t noticeable to me.
The other problem was that encoding my huge new influx of audio files was going to take ages. I now had 300 gigs of audio, and one core just wasn’t cutting. This was how I learned about GNU Parallel. GNU Parallel is like
xargs, but it runs commands in parallel (wow go figure). This way I could let it handle juggling ffmpeg invocations so I could fully saturate my 8-thread i7-2600k.
However, this move to opus necessitated a change in audio app too. Nowadays Poweramp supports opus, but at the time it did not. In fact, most apps at the time didnt, and of the ones that did, only one had both gapless playback and a UI that I found enjoyable: GoneMAD. Nowadays, the UI I used is available in “GoneMAD Classic”, but I’m on iOS now so I don’t know much about how it holds up. At the time, it did everything I needed, most of important of which was browsing the file tree directly. A lot of my music library didn’t have proper metadata tags, so I manually sorted things into folders and relied on being able to browse my own organization structure. Surprisingly, a lot of music players didn’t have this feature.
Time To Make Mistakes
The thing is, when you’re into archival, you only ever get more data. I collected large music dumps as certain artists and labels disappeared from various scenes, and my own tastes expanded. Eventually, even my opus solution was hitting the limits of my on-phone storage. I could’ve upgraded my storage again, but at the time I finally had consistent LTE coverage over my town, so I decided to be a bit more ambitious. I wanted to stream my music from home, and put my storage problems to rest once and for all.
So what did the stack look like for that?
- MPD ran on my computer and transcoded music in real time
- Icecast ran on a server in the cloud, taking in a stream from MPD and providing multi-client support, better buffering, and a web URL.
- ympd ran on the cloud server too and let me control mpd on the go (the MPD control apps didn’t work as well).
- nginx sat in front of icecast and ympd providing TLS and HTTP Basic Auth.
Here’s a screenshot of ympd, which I’ve inverted because otherwise it’s obnoxiously bright.
The link between MPD and Icecast/ympd also ran over a pair of SSH port forwards established with autossh, since otherwise I’d be sending unencrypted credentials over the network.
I didn’t really want to run this on my desktop though, since I sometimes left it booted into Windows for gaming purposes, so I intially tried to set this up on a first generation Raspberry Pi B. Now, I don’t know if you remember just how incredibly slow the first pi was, but it could not handle real-time transcoding to opus. I tried. These days, a RockPro64 can transcode opus at 32x real time speed; that’s how far we’ve come in the low-cost ARM SoC world. If memory serves, the Pi could barely do AAC but it was borderline enough that I didn’t trust it. So, for a bit at least, I was back to transcoding to mp3 or vorbis. But I bumped up the bitrate to compensate since I had the data for it in my data plan, and this worked fine.
Except for when it didn’t work out. Because the problem with real-time streaming to a phone is that sometimes a phone loses and regains connection when switching cell towers. Sometimes its IP address changes for no good reason. Sometimes you go through an area of spotty reception and get a bunch of packet loss. Neither Icecast nor MPD nor the apps I used for streaming music were prepared to deal with any of this properly. As a result, I had buttons on my home screen to launch a script to restart icecast and MPD as needed. I had to kill my streaming app from time to time. It was really REALLY cool when it worked, but it was so frustrating when it didn’t. But, I’m nothing if not comitted to the bit, so I used this setup long enough to get a Raspberry Pi 2 and switch over to opus encoding for a little bit.
I wouldn’t do it again. MPD is a pain to set up and configure, so is icecast, and making them work together is even worse. It took me a good week before I finally had everything working together reliably. I don’t remember the details, and my config files are lost to time; all I can say is, do not try this at home. Unless you’re into that kinda shit. Eventually, I got a phone with enough storage that the whole streaming thing was no longer relevant and went back to my old ways. Then I got an iPhone and tried Apple Music, at which point I was well into my synthwave phase, so all the music I wanted to listen to was streamable on there anyway.
Nowadays I’ve chilled out quite a bit. I bought a rockbox-compatible music player a few years ago, and I copy audio files over to it like it’s a flash drive once again. If I run out of space, I just delete some stuff I haven’t listened to in awhile. It’s not complicated. I even use CDs sometimes, when I’m feeling in the mood. I might bring back my opus transcoding script some day to bring my full collection over, but I’m very much over streaming music. It’s just nicer to press play and know it’ll always work.