MP3 Fidelity Results

Spent a few hours last night at Mike’s house listening to the MP3 fidelity test discs I burned the other day. The discs included these tracks:

Coltrane – Resolution (the Michael Cuscuna – produced version of A Love Supreme)
Alban Berg: Three Pieces for Orchestra, Ostinato (on RCA Living Presence)
Rob Wasserman – Angel Eyes (cello / vocal duet – superb recording)
Klip – Dyslexiana
Theresa Anderson – Summertime (my fave female vocals test and fave cover of this song)
Gerry Hemingway – Endpiece 1 (from Special Detail)
Tim Buckley – Hullucinations (from Live Dreamletter)

We did not get around to listening to all four versions of all seven pieces. The system we used was:

Audio Alchemy CD transport with DTI Pro jitter reduction
Audible Illusions Modulus 20 pre-amp (heavily modified)
Muse 100 amplifier
NHT 3.3 speakers

System value approximately $20,000

I’m still trying to figure out what we learned from all of this. First of all, I can unambiguously say that I have never heard MP3s sound this good in my life. Second, this is the first time I’ve ever been able to distinguish between 256kbps MP3 and the original source material. However, I also learned how much the mind plays tricks on you in tests like this, and hence why audio mags often avoid them. The only valid test is really to live with the material, doing what you do normally, whether that be sitting down and doing concentrated listening or playing it in the background while going about your business. In other words, if we listened to these tracks over and over again, and if we lived with them and subsumed their presence into the subconscious, we would be able to discern additional differences we didn’t notice yesterday.

I also have to say that Mike has a lifetime of intensive experience and training in audiophile listening environments, and in building hifi audio equipment. I care a lot about fidelity, but don’t have ears as well trained as his. He was able to discern more differences than I was.

Mike also pointed out some problems with the methodology of my experiment. Because each track can last 5-10 minutes, it’s extremely difficult if not impossible to make meaningful comparison notes without a standard of reference. IOW, I wanted for us to try and pick out the reference track blind, while he felt it would be more meaningful to know in advance which one was the uncompressed WAV, so we could compare the other bitrates to it. His point made more sense after we listened blind for a while – your mind really does play tricks on you. For example, you might hear more air in a cymbal splash in one track, but you could be unsure whether that “air” is in the reference (a good thing) or an artifact of the encoding process (a bad thing). So in the end we did most of our listening with knowledge of which track was the WAV reference.

Neither of us had any difficulty picking out the 160kbps tracks, which was disappointing knowledge for me, as I’ve encoded tons of music at 160kbps. From here on in, I’ll use 160 only for crappy old recordings, 192kbps for average stuff, 256kbps for stuff I really love or that’s really well recorded, and 320kbps for those amazing gems that are stellar both as music and as recordings.

Mike is going even higher. His argument is that disk space is just so cheap now, there’s no point in compressing at all. Why risk it? It’s interesting because this is the same argument I made in the intro to my MP3 book – that the only reason we even talk about compression is because storage is too expensive and the internet is slow. But think about it – 40GB drives are under $100. It’s possible to fit around 60-70 albums on a 40GB drive in uncompressed WAV format. That’s about $1.50 per album in storage space. If you care enough about your music to have a system like his but you still want the flexibility of soft storage for your collection, why compress at all? Then you just build an array of 10 or so IDE drives to house your collection, which would be like spending a grand on an important new stereo component – worth it.

I counter that by saying that the difference b/w 256kbps and WAV is so incredibly subtle, and 320kbps would be virtually indistinguishable in every important way. You still get around 4x compression at 320kbps, and you have the advantage of ID3 tags. Why not take advantage of it?

I could tell there was a difference between the 256 and the WAV in the Wasserman duet/vocal, but had trouble putting my finger on the difference. Mike nailed it down, and had me focus on the spatial relationship or distance between the vocalist near the middle of the room and the cello in the rear left of the room. At 256kbps, there is a sort of “veil” that enshrouds the vocals, and ties it spatially to the cellist in the rear left. It’s subtle, but he’s right – the spatial distinction between them is just not as pronounced at 256kbps as it is with the WAV. But the fact that it had to be pointed out to me, and that I had to concentrate to find the difference even then, tells me that this is a difference I can live with, even if he feels he can’t.

However, the same effect at 160kbps is atrocious. It sounds like there’s a blanket thrown over the whole production – like you’re listening through a thin layer of gauze that sort of erases or diminishes the space between musicians. However, it requires a system better than most people own or have even heard in their lives to reveal some of these differences. On a standard Japanese consumer stereo, 160 might be adequate for most people.

My problem is that I haven’t followed my own advice. I have professed that people’s MP3 collections would eventually migrate to their home stereos, and have warned that they shouldn’t trust their computer sound systems to tell the whole story. Nevertheless, I’ve encoded almost everything at 160kbps, and now have a collection that will translate to the home stereo with less than stellar results. Hrmmm…