In his July 1 editorial, “Rich Guys and Reviewers Running Amok in Hi-Fi,” SoundStage! Network founder and publisher Doug Schneider discussed the amount of exorbitantly priced hi-fi products available today, and the fact that their performance often does not justify their crazy price tags. And he cited academic studies showing that consumers often mistakenly equate price with quality.
As Doug observed, reviewers also fall prey to this placebo effect, and this influences the products they choose to evaluate. “The natural inclination for most reviewers is to seek out gear that’s more and more expensive,” Doug wrote. “That’s because when they see what they can get their hands on, they wind up like kids in a candy store and start asking for more. Before they know it, they’ve gone from reviewing components costing three and four figures to ones that run to five and sometimes six figures.”
Not this reviewer: I’m much more attracted by engineering elegance than opulence. That’s why this corner of the SoundStage! Network, which is all about “convenient, lifestyle-oriented hi-fi,” is such a great fit for me. In fact, I find the idea of spending half-a-million dollars on a turntable or a pair of speakers almost obscene.
Apparently, I’m an outlier. “It’s much easier to get people to write about expensive audio equipment rather than the inexpensive gear,” Doug observed. “In fact, when you ask some reviewers to review something that costs $400, rather than $4000 or $40,000, they’ll turn it down or they’ll make it seem like they’re doing you a favor if they do agree to do it.”
As an explanation for this behavior, I point to James Boswell’s The Life of Samuel Johnson, in which the 18th-century essayist and lexicographer is reported as saying “no man but a blockhead ever wrote, except for money.” Dr. Johnson’s witticism has been the guiding principle for generations of writers, including yours truly.
Who’re you calling a blockhead?
When you consider the hours spent unpacking and setting up equipment, listening, tweaking the setup, taking notes, troubleshooting problems, writing the piece, repacking the gear, and sending it back to the manufacturer, audio reviewing isn’t very lucrative. While many outlets (print and online) pay for reviews, quite often that payment amounts to an honorarium, a token of appreciation, rather than a proper fee for services rendered. So are audio reviewers blockheads?
No, they’re enthusiasts. Dr. Johnson’s quip notwithstanding, many writers are motivated by something other than money when they put pen to paper, or fingers to keyboard. It may be a need to create, to memorialize, to document. Or it may be a passion for the subject matter—as it surely is with most audio reviewers.
If money isn’t the motivator for these reviewers, what is? It’s likely the possibility of trying out a $4000 component they’re scheming to add to their system, or spending some quality time with a $40,000 component they dream about, but could never afford. Review a $400 component that they have no personal interest in buying or hearing? No thanks!
That’s not me. I consider myself an audiophile, but my relationship to reviewing is strictly professional. If I’m going to unbox a product, set it up in my living room, spend hours listening to it, write up a report, then ship the thing back, I want my pockets lined. With that understanding, I’m as happy to review powered speakers that cost $400/pair (PSB Alpha AM3, see above) as I am to review leading-edge active speakers that cost more than $17,000/pair (Kii Audio Three, see below). I love learning about how manufacturers are pushing the boundaries of audio, but I also love learning about how companies create affordable products that punch way above their weight.
This month marks my third anniversary on Simplifi, so I thought I’d use the occasion to share my approach to reviewing. When I started this gig, I’d been writing professionally about technology for 40 years—not just audio, but about video, photography, computing, and telecoms, too. Most of my clients had been technology magazines, but I’d also written for daily newspapers, lifestyle magazines, and business mags. And I’d been an audio enthusiast for over 50 years.
I had done some product reviews, but most of my writing had been feature articles, how-to pieces, show reports, company profiles, and the like. In my first year on Simplifi, I wrote more audio reviews than I had done in the previous 40 years. I’ve learned a lot since then, not just about the products, but about reviewing. The main thing I’ve learned is that reviewing is hard work!
The race against time
Of necessity, my mercenary approach influences the way I review products, and what readers can expect from my reviews. Most months, I crank out two full reviews plus a feature for Simplifi, because that’s what it takes to generate the income I want from this gig. On some months, I also mix in some work for other clients.
Typically, it takes me just over a week to turn a review around. That’s not a lot of time, and sometimes my work feels rushed—especially if I have to conduct some back-and-forth troubleshooting with the manufacturer. With software-driven audio products, that happens more often than it should. That’s why many of my reviews have a heavy focus on software and the user experience. With network audio products, usability is a vitally important consideration.
I’ve seen manufacturers promote features in their product literature and on their websites that don’t work well in real life. One example was a DAC with a USB port for connecting an external drive to play locally stored music. But the software wouldn’t let users sort music by the name of the artist, album, or song—it just displayed all the files in alphabetical order. Useless!
I’ve come across products with finicky setup procedures that required complex workarounds. I’ve reviewed software that would have some users pulling their hair out every time they sat down to listen. One bugaboo of mine is software that doesn’t support gapless playback—that will ruin the experience of listening to any album where one song is supposed to segue into the next.
But I’ve also come across software that makes setup a breeze and day-to-day use a joy. Whether the user experience is poor, great, or just OK, I make it a point to cover this stuff before I even talk about sound quality.
The need for speed
Coupled with occasional software issues that can take days to resolve, the compressed timelines I deal with mean that I often have less time to actually listen to a review product than I’d like. I’ve read opinion pieces by other reviewers who insist that the listening process takes time, and can’t be rushed.
They have a point. Focusing intently on some aspect of a component’s sound while working against a deadline is very different from listening for pleasure. Many’s the time that I’ve noticed some aspect of a component’s sound during relaxed listening (some of it herbally enhanced, I must confess) that I missed during focused listening sessions.
But there’s another side to this issue. If you’re thinking about buying a heavily hyped hot new product, you don’t want to wait months for a reviewer to put in the requisite hours of relaxed listening. There’s value in providing readers with reviews of new products as quickly as possible.
If I may be permitted to toot my own horn, I think Simplifi’s record on covering hot new products is pretty impressive. We were among the first sites in the world to review the Bowers & Wilkins Formation Duo active speakers; DALI’s Callisto 2 C, Rubicon 6 C, and Oberon 1 C active speakers; iFi Audio’s Aurora and Naim Audio’s Mu-so 2nd Generation music players; PSB’s Alpha AM3 and Alpha AM5 powered speakers; SVS’s 3000 Micro and Bluesound’s Pulse Sub+ subwoofers; McIntosh Laboratory’s MTI100 integrated turntable; and Bryston’s BR-20 streaming preamplifier-DAC.
In instances when we weren’t the first with a review, we were often the first to cover important enhancements and features.
For example, Simplifi wasn’t the first outlet to review NAD’s Masters M10 streaming integrated amplifier when it launched in 2019. But as far as I’m aware, we were the first to review the M10 after the company pushed out a firmware update enabling Dirac Live room correction. Dirac is a killer feature, but the outlets who reviewed the M10 before Simplifi weren’t able to cover it.
Other outlets reviewed Andover Audio’s Model-One turntable music system before Simplifi. But as far as I know, we were among the first to review the Model-One in conjunction with its matching subwoofer and record cabinet.
Given Simplifi’s compressed timelines, a review containing a factual error occasionally goes out. But that’s rare, because every review is vetted by a capable editing and proofing team (whose ministrations I deeply appreciate) and checked by the manufacturer for factual accuracy before publication. And as the SoundStage! Network publishes online rather than on dead trees, it’s easy to fix an error after publication, which I’ve done more than once.
A more serious problem is not being able to fully troubleshoot networking or software problems before my deadline. There have been a few reviews where I’ve mentioned buggy software or network hiccups. In some cases, given more time, I might have been able to resolve these problems. A related issue is the fact that many of the products reviewed on Simplifi have very rich feature sets, and there’s not enough time to test all those features.
Even so, when I look back on my reviews over the past three years, I can spot details I’d now like to change, but I’m still confident about the overall conclusions.
The space race
There’s another reason I want to push reviews out as quickly as possible: I don’t have a dedicated listening room. Most of the rooms where I review products—the main-floor living room and basement family room in our small rowhouse in Toronto, and the family room in our vacation condo in Southampton, Ontario—are multipurpose spaces. My wife and I use them for hanging out and entertaining, and cluttering up these rooms with review gear interferes with those activities. There have been several occasions when I’ve had to disconnect sample equipment during the course of a review and move it to the basement or my home office, because we needed the room for some other function. The only room where this isn’t an issue is my home office, where I review desktop audio products.
There’s an upside to this. Reviewing products in multipurpose spaces strikes me as being true to Simplifi’s mission to cover “convenient, lifestyle-oriented hi-fi,” because I’m experiencing the products in conditions similar to those in which they’ll be used in real life. But most of the time, I want to get review gear out of our living areas as quickly as I can, not because I don’t like the stuff, but because I don’t like clutter.
Room boom
There’s a downside, too. As I’ve written several times on Simplifi, the living room in our Toronto home has some nasty bass modes that cause steep low-frequency peaks and dips. These can make the bass sound muddy and indistinct. And the problem is exacerbated by the fact that my listening position is right against the wall opposite the speakers, where the effects of these room modes are most severe. This is unavoidable, given the size and layout of the room. For day-to-day listening, I deal with room problems by enabling Dirac Live full-bandwidth room correction on my NAD C 658 streaming DAC-preamp.
But what about when I’m reviewing an active or powered loudspeaker? If the bass sounds muddy and indistinct, is the loudspeaker causing the problem, or the room? To address this question, I’ll sometimes run Dirac Live room correction for the product under review, but apply correction only for the low frequencies. Sounds above the “curtain” frequency I set are ignored by Dirac. Used this way, even with Dirac enabled, I’m still hearing the speaker’s innate midrange and high-frequency response, and can draw appropriate conclusions.
But what about the lows? Dirac may be smoothing out irregularities caused by my room, but it may also be smoothening irregularities in the speaker itself. Which is it? Should I be using Dirac at all in my reviews? I don’t think there’s a right answer to this question.
While applying Dirac Live room correction is an option with analog active speakers like my Elac Navis ARF-51s, it’s often impractical with DSP-based active speakers with built-in streamers, or speakers that work with wireless transmitters. In those situations, I make an educated guess about the speaker’s low-frequency behavior by comparing it with another speaker whose sound is familiar to me. Or if I’m using Roon to stream music to the speakers, I can use Roon’s parametric equalization function to attempt to counteract those room modes. Should I apply EQ, or is this cheating? Again, I don’t think there’s a right answer to this question.
The last word
I also want to comment briefly on the music I use for my reviews. I try to choose selections that will reveal different aspects of the equipment I’m testing—timbral accuracy and consistency, bass and treble extension, micro- and macrodynamics, imaging and soundstaging, and so on. I favor music with voices and acoustic instruments, as opposed to amplified or electronic instruments, as this makes it a lot easier to get a handle on timbral accuracy. Moreover, my preferred genres—classical, jazz, and roots—eschew amplified and electronic instruments for the most part.
That doesn’t mean that listeners who prefer other genres can’t draw conclusions from my reviews. If a speaker can cope with the dynamic demands of the “Tuba Mirum” from Hector Berlioz’s Grande Messe des Morts, in which the composer is evoking the day of judgment, it can surely cope with the dynamic demands of AC/DC’s “Hells Bells.”
In a field as complex and subjective as audio, no single review can deliver the last word on any product. All reviewers work under conditions that affect their conclusions, and each brings a certain methodology and a set of preconceptions to their work. Now you know mine.
. . . Gordon Brockhouse