"Log sine sweeps rather than linear sine sweeps were employed to allow verification that non-linear distortion components were virtually absent."
And with that, this study is bullshit.
Human beings don't listen to linear sine sweeps. We listen to music. Recorded music has 8+ octaves of frequency range (the bottom octave plus a little extra is almost always rolled off in real-world recordings, to ease stress on downstream components that can't reproduce such low frequencies anyway), and 20-50db of useable dynamic range.
Sine wave measurements of audio gear ignore impulse response, intermodulation distortion, phase shift, and a host of other real-world physical device responses to real-world musical signals. Scientific, reductionist thinking is inadequate to get an accurate picture of the factors that matter to human listeners.
Frequency response and total harmonic distortion aren't measured in these cases because they're useful or relevant. They're measured because they're easy to measure. It's like looking in the wrong place, because the light is better there. And the results? It's like measuring a car's performance by how well it can drive in a straight line at 60mph. Acceleration, braking, and turning are too hard to measure, so we ignore them...
I'm a musician and record producer. I've engineered and produced numerous albums, and rely on multiple different types of headphones for different purposes. The article's claim that one headphone can be easily morphed into another through mere equalization is, frankly, bullshit. The two headphones I rely on the most (Beyerdynamic DT880 and AKG K240) sound wildly different. Neither is "accurate". Neither are the Tannoy System 12 DMT midfield studio monitors I use for mixing, or the stock Subaru car speakers I use for reference to check the mixes from the Tannoys.
Audio reproduction is incredibly complex and difficult stuff. Trying to isolate one factor and saying "That explains everything!" is bad thinking.
> Human beings don't listen to linear sine sweeps. We listen to music. ...
This is irrelevant. They're measuring frequency response, not trying to map the entire world of psychoacoustics. Log sin wave sweeps are a perfectly adequate way of measuring a transducer.
> Sine wave measurements of audio gear ignore impulse response, intermodulation distortion, phase shift, and a host of other real-world physical device responses to real-world musical signals.
No, when you do a sin sweep and measure the impedance you can recover all that. Pick up a basic textbook on loudspeaker testing.
> Scientific, reductionist thinking is inadequate to get an accurate picture of the factors that matter to human listeners.
Learning from it will require you to shed the chip on your shoulder, but it will put you miles ahead of your peers in understanding what's actually going on when we listen to music vs all the commonly repeated mythology.
Sine sweeps are a good method to measure frequency response. That's what they were trying to do after all: measure frequency response. That was the stated goal of the test: find a correlation between frequency response and another parameter. They were playing a signal into a dummy, capturing the sound and measuring it; they were not trying to determine how pleased the dummy would be with the beautiful music.
However, I share your overall assessment that present-day transducers are far from perfect accuracy.
It absolutely looks to me like you're just trying to keep this field unscientific. Misrepresenting the article, then throwing out a bunch of jargon without explanation, then coming to the conclusion that people here shouldn't even try to understand.
The study doesn't say "that explains everything!". It measures one component in isolation. That's what good science does. The existence of other factors in headphone performance is irrelevant.
Besides, would you really not consider it news if 0-60 time in cars didn't correlate with price? Yes, a luxury car can justify its price in many other ways, but, on average, there should still be a correlation.
I'm not going to disagree with you on the technical details, since I'm not knowledgeable enough in the area. But I'm going to take issue with one thing:
"Scientific, reductionist thinking is inadequate to get an accurate picture of the factors that matter to human listeners"
I think you're noting the difference between good and bad science -- science is certainly capable of putting together a picture of the factors that matter to human listeners. It may not have done so, but it's not a failure of the scientific process, it's a failure of the study(ies) involved.
I... don't know what you're talking about. Apply a log scale to linear data and it too represents a log scale.
> Recorded music has 8+ octaves of frequency range (the bottom octave plus a little extra is almost always rolled off in real-world recordings, to ease stress on downstream components that can't reproduce such low frequencies anyway), and 20-50db of useable dynamic range.
So? We all know that listening to an instrument live sounds "different" from a recording, and it's up to us to figure out how to improve both recording and listening fidelity.
Now an argument can be made that because audio formats today tend to have caps on fidelity that headphones only be measured against the maximum fidelity that the recording produces, but that is neither the argument that you are making nor is this a weakness in the data presented by the paper.
> Frequency response and total harmonic distortion aren't measured in these cases because they're useful or relevant. They're measured because they're easy to measure.
Yes and what should we measure? Phase graphs? Third order components? You're not making a meaningful argument here.
> The article's claim that one headphone can be easily morphed into another through mere equalization is, frankly, bullshit.
First of all this paper used novel methods to back the original assertion made in [1]. Also, let me quote the exact line from the conclusion:
"PCA can account for 90% of the variance across all measured headphones with six eigenvectors. The first eigenvector is similar to published target responses, while the second eigenvector represents a global spectral tilt."
> Audio reproduction is incredibly complex and difficult stuff. Trying to isolate one factor and saying "That explains everything!" is bad thinking.
And so is your appeal to non-authority. The gist of your entire argument is that linear sine sweeps are useless, and therefore the entire validity of the paper is moot and your non-scientific opinion is now superior.
The frequency response and the impulse response are correlated. You can just get one from the other through fourier transformation. Therefore it's perfectly fine to measure whatever is easier to measure, which is frequency response through sine waves for audio equipment. Phase shift can also be measured through this approach (and is required for calculating impulse reponse).
Sure, nonlinear distortions are not covered and also sure that the measurements won't cover everything which will influence the perceived sound. But nevertheless frequency response measurements are a very solid tool which allows objective comparions between various systems.
> It's like looking in the wrong place, because the light is better there.
Yes, and actually one of the challenges physicists face, is that looking by the lamp is not self evidently reasonable for most people. Of course you start with the easy case and then you build on that foundation, otherwise your analysis just hangs in thin air. (Plus you have to deal with the easy case anyhow if you want to do comprehensive work, you can as well start were you have a chance.)
I once heard a story in audiophile community, I don't recall the particularities, though. Guy went to audio exhibition and there was one booth with curtains covering the actual product. This is a new WIP product, we want to hear feedback without you actually seeing what it is to get a bit more objective response. Audience listened to some music playing from behind the curtains, discussed various strengths and weaknesses of the audio system. After a while curtains were raised to reveal live acoustic band playing.
Instruments themselves are imperfect, there is long chain of devices (recording, mixing, playback) until the signal reaches speakers (and then room until signal reaches ears), at higher quality levels it is quite often the case of how well all the inaccuracies of the system interact with each other, rather than how accurate certain components of the system are.
>> Frequency response and total harmonic distortion aren't measured in these cases because they're useful or relevant. They're measured because they're easy to measure.
We've evolved our senses around the signal components the pay off the best: the "easy" ones. Thus it isn't surprising that "subjective quality is mostly correlated with linear (spectral) attributes instead of non-linear (distortion) metrics," a claim supported by peer reviewed papers that have vastly greater credibility than you and your anecdotal vitriol.
They're doing a comparative study. To a large extent, it doesn't need to reflect reality. It just needs to be good enough to make a determination within the given sample.
> It's like measuring a car's performance by how well it can drive in a straight line at 60mph.
No, it's like comparing several cars and their MRSP based on their 0-60 performance, which is certainly a reasonable thing to do. Just because they didn't measure stopping distance, it doesn't invalidate the entire comparison.
Question: when checking the reference output, do you sit in the Subaru, or do you just have the speakers in a room with you? I would imagine the acoustics are fairly different.
I've spent some time on frequency correction for headphones and reference monitors in my home studio. If you'd like awesome headphones that have a truly flat frequency response, that you can then adjust with EQ to your taste, one option is to get Sony MDR 7506's and run the audio output through a VST plugin (Using soundflower, ableton, etc) which corrects the EQ. You can either buy precalibrated headphones from sonarworks or use a generic but headphones specific calibration profile for the plugin.
It's really cool hearing what they heard in the studio control room for the final mix. And often surprising.
You can get a range of other precalibrated pro audio headphones or correction profiles from sonarworks.
Consumer headphones are just silly IMHO. Artificially boosted frequencies with prices up to $400. A set of precalibrated MDR7506's is around $220.
If you don't care about truly flat response with correction, you can get a set of AKG K240's for $100 bucks and they're super comfy, amazing sound and loved universally by audio pros.
- Someone with online alias NwAvGuy put the whole AV industry (ok maybe not the whole, but some big players) in a loop by showing in online forums that a totally inexpensive DIY DAC (with a free design he/she shared) could be built with quality rivaling elite products worth thousands of dollars. [1] (well a hazy version of the story goes that he/she exposed various audiophile review sites and forums as being full of sponsored reviews, and that eventually lead to his/her ban from head-fi.org I think)
- As for capsule mics (commonly known as condenser mic), market is flooded with DIY designs and DIY kits which let you build/buy one for $200-$400 (the dominant cost being that of the capsule itself) that will rival the quality of multi-thousand dollar mics. They go by the names Neumann clones, etc. [2] (no affiliation), [3].
In retrospect, and given the shady things AV sellers do, like trying to sell you a USB or HDMI with gold-plated pins, claiming it to be superior, it should come as no surprise.
Though, no offense, but audiophile consumer base is filled to the brim with hipsters who judge the quality of a product by its price (and some of the "experts" were busted after they failed blind tests; I think opus vs flac, I'm mixing a lot of things now).
Of course not! For one thing, as one's earning power increases, one's high frequency hearing deteriorates. So market forces could well be emphasizing features and capabilities other than frequency response. Fashion, build quality, social signals...these are all very significant factors in something you wear, the practical priorities of audiophiles and enthusiasts notwithstanding. In fact, those are probably stronger factors for that set of people! (Of which, I am a member.)
Headphones also have a serious empiricism issue. You can probably pass off one high end Sennheiser for another in an A/B test. But you couldn't pass off an Audeze for one and have a valid A/B test. Also, you will often read or hear an expert say, if the measurements say something is bad, but it sounds good, or vice versa, then it means we're measuring the wrong things. I'm not saying that the Harman response curve isn't valid. It's just not the whole story.
Things are going to change in significant ways in the future as the price of signal processing, compensation, and active correction drops, however. Combining those with advances in the cheaper manufacturing of better drivers will result in the headphones of 10 years from now making the high end headphones of today seem "meh" and today's typical headphones seem trashy.
DSLRs got the same problem : just compare the Canon 70D ($900) with a Nikon D3300 ($400) on DxOMark.
The Nikon has better image quality despite its low price and bad reviews.
We need objective benchmarks for everything. Especially when marketing is growing bigger each year. Even "Tech websites" are biased and not objective anymore.
It would be nice if we knew which headphones they tested. Since so much of a headphone's reputation these days relies on largely anecdotal evidence from self-professed audiophiles, some kind of objective rating on frequency response for major brands or well-known cans would be highly welcomed in the audio world.
It's very easy to say, "I can hear so much more of the song out of my ATH-M50's than I can a pair of Beats", and you may be right. But something objective to back it up would be great, too.
For a lot of people Beats sound good enough, they are affordable (but still expensive), and built fairly well. Clearly, it's subject but to lots of people they also look good. For just about anything worn on one's head, aesthetics and comfort will be the dominating factors.
When I was a kid, my friends and I would kill time in the library by flipping through Car & Driver, Motor Trend, and Road & Track magazines trying to figure out what car is best. The Camaro might be two tenths faster in the quarter mile than the Mustang, so clearly anybody that buys a Mustang is an idiot, right? This is what a lot of headphone evaluations sound like to me.
I don't buy headphones super often, and I'm not super obsessive over it, but the people at headphone.com have always made me feel at least a little better informed:
http://graphs.headphone.com
Frequency and phase response are just linear characterisations, they don't model any non-linear effects.
That being said, speakers and headphones all have a sound (unlike any properly designed amplifier); there is hardly any right or wrong there, and while the best approximation of a flat response might seem technically most correct, people will probably prefer different sounds.
That only works if your objective measurements map well to subjective listening experience. Frequency response does not map well. Neither does THD. They're measured not because they're useful, but because they're easy to measure and sound "objective" and "scientific". It's an appeal to authority fallacy with a graph.
But something objective to back it up would be great, too.
That brain-training game on the Nintendo DS -- the one that had you distinguish two voices saying things simultaneously -- your score on that game is a great way to evaluate headphones for vocal accuracy. Be careful though: the DS can't adequately drive all headphones, so you'll need an amp for high impedance headphones.
I predict a lot of wrong conclusions will be drawn from this. This paper does not preclude the possibility that there exist high-priced headphones with better-than-average or even spectacularly good frequency response. It only says that if you bin together all of the high priced items, their aggregate quality is no better than any other price bin.
"Interestingly, sound quality does not seem to be a major attribute for purchase decisions."
This is a silly assumption, and easily explained.
1. Most headphone purchases aren't and cannot be made by comparing sound quality. Reviews of sound quality are so universally understood to be subjective that most consumers probably ignore those details.
2. There is no one subjective or objective standard that is meaningful for all listening material. Podcasts, modern pop music, older pop music, classical recordings, television shows, and movies all have wildly varying acoustic profiles between and among each genre.
3. The vast majority of headphones have Good Enough sound quality for the vast majority of consumers. Sound quality is highly unlikely to be the primary reason most consumers buy a set of headphones, and it's unlikely to be the reason they are dissatisfied with certain headphones.
4. Headphone design, form factor, build quality, fit, feature-set, and even color are all much more important factors in terms of consumer satisfaction with headphones. They are, after all, a highly noticeable part of your ensemble. They are intimately in contact with your body. And you want them to work without thinking about it too hard. In addition to being more important, most of these factors are far easier for consumers to judge between headphones than sound quality, so again it's no surprise that an arbitrary single standard of sound quality would fail to correlate with perceived value.
In other words, this is silly for reasons that have nothing to do with technical arguments about actual sound quality, whatever that means.
> Price has never been correlated with quality, for any product, ever.
That is an incredible assertion.
I am willing to bet $10,000 that I can find a product where price is correlated with quality (assuming we can agree on a definition of "quality" for a particular product). How about GPUs with "quality" measured by average FPS in a selection of AAA games at 4K?
> Price has never been correlated with quality, for any product, ever.
Tools.
Musical instruments.
For both of those there is strong correlation between price and quality, and the users of both of those are picky for a good reason: the quality of the work they produce depends to some extent on the quality of the tools they use.
Now, bad craftspeople and bad musicians will produce crap no matter what the tool, but good craftspeople and good musicians can definitely produce something better with good tools and instruments and are prepared to pay for that.
Yes, it's definitely a horrible use of the word 'correlated', because it doesn't say what I think you're intending to say. Derived is a much better word.
I believe that what you are meaning to say is that price responds to demand, and demand correlates to the perceived value of the product. In turn, perceived value is often but not inherently responsive to information about the 'quality' of the product.
Instead, perceived value is a summation of various forms of utility, only some of which are affected by 'quality'.
Furthermore, any determination of 'quality' is rarely made upon information that is as complete as the information available to an attempt to methodically determine the relative 'objective' quality of substitutable products, because of the disutility of conducting such an assessment. More often, the determination is based upon perceived indicators of quality, which are known to be unreliable.
This diminishes the relative importance of 'quality' in the formation of the perceived value of the product, which accounts for the importance of alternate, more reliable information about the product, such as the social signalling value, and especially easily manipulable information such as indicators of "style" and the narrative about the product; this in turn explains the willingness of product vendors to invest in advertising.
With regards to the use of the word 'correlate': price is of course correlated with demand, with correlates to perceived value, which tends to correlate to perceived quality, which tends to correlate to quality. Since correlation requires transitivity, price does then tend to correlate to quality. So what you actually said is incorrect.
I believe that your intended assertion (as I understand it) is also incorrect and over-broad, because in many instances a highest-information assessment of quality is the sole determining factor of the perceived value of the product. This is often true in business to business transactions such as part sourcing, where a person with specialized information is in charge of purchasing, and with products in a commodity market such as grain or beans, where a standardized determination (and guarantee) of quality has been established.
(Thanks for the opportunity I've taken to sit for a while and practice arranging my thoughts, which is the reason that my reply is so extensive.)
This makes for a cute soundbite, but it doesn't mean what it implies. You could, for example, have a bunch of expensive headphones with frequency response that varies randomly in the [-1,+1] dB range, and a bunch of cheap headphones that are in the [-10,+10] dB range -- that would also show up as uncorrelated.
Indeed, they did find a significant difference in magnitude response _error_, although the effect was quite small.
"Nevertheless, assuming that the perceived audio quality is largely determined by the spectral magnitude response of headphones..."
This is a very wrong assumption.
Audio component designers have more or less a hard time picking up which measurements can correlate with audio quality. And frequency response measurements using sine sweeps, like in the cited study, are almost of no value for discriminating between two transducers (headphones, speakers) with regarding to 'audio quality'.
Also, the fact that one headphone can extend beyond 20KHz or that it can go below 20Hz will give zero guarantee of better audio quality.
Frequency response measurements using white/pink noise can give a slightly better hint because they can take a look at resonant peaks that might be annoying to the listener, but even this is not a law set in stone*
* Impulse measurements (and waterfall plots) can give you a clearer idea of how clear is the sound going to be; but then you can have a transducer with a fairly good impulse response but a slight resonant peak somewhere --- OR you can have sometimes a transducer which shows pretty flat frequency response but bad impulse response.
A good test for intermodulation distortion (the big white elephant in the audio room) will REALLY give you a hint of which headphone will be least annoying to the ear when listening to loud complex music like classical music, vocal music, etc.
It seems that the article has been written by experts in acoustics, but not really in "audio".
TL;DR: Freq response measured with sine sweeps can't really tell you anything helpful to discriminate headphones with regard to sound quality.
The citation provided further down in the article is [1]. Dr. Olive is thorough in attempting to remove sources of bias and conducts a set of fully double blind listening tests in order to come to the conclusion: "The results provide evidence that trained listeners preferred the headphones perceived to have the most neutral, spectral balance".
> A good test for intermodulation distortion (the big white elephant in the audio room) will REALLY give you a hint of which headphone will be least annoying to the ear when listening to loud complex music like classical music, vocal music, etc.
Many expensive headphones are overpriced, but there is a very obvious difference in sound quality between very cheap headphones and medium priced ones. Either they're measuring the wrong thing or their headphone sample isn't what I'd expect.
Yeah I'm not losing any sleep over the obvious subjective difference between crap headphones and medium- to high-end ones. If the metric du jour isn't picking up any difference then, sorry brother, probably not a good metric.
"New study shows no difference in speed of blue vs. red light!" We always ascribe waaay to much importance to what we can measure easily.
FWIW, it looks like half of the prices were within $10 to $100 USD, the bulk of the rest were >100 and <1000. About 1 was <10 and a few were >1000.
I don't think they're measuring the wrong thing, although a human might be interested in more things than frequency response.
Just goes to show that perhaps the most important thing in selecting headphones is comfort and the context in which they will be used.
For "audiophile listening experiences" an equalizer and CA-style cans might be a good idea and give a lot more bang for the buck than shelling out several hundred just for the cans.
"However, the variance in low-frequency response seems to decrease with increasing price, indicating an improved bass response measurement consistency across headphones in the higher price range."
I wonder if there's any effect that in-ear headphones are cheaper to produce but have advantages in accurate low frequency response?
Of course all this is confounded by the fact that music will tend to sound best on speakers/headphones with a response curve most like the speakers/headphones that the mastering engineer used (or more accurately, the set of speakers/headphones that the engineer compromised among). You will probably tend to have the best experience listening to music with the popular devices within a given musical subculture, because mastering engineers will be targeting those devices.
> The target function suggested by Olive and Welti (2015) is fairly similar to the average headphone response found in this study, with the exception of a deviation of up to about 5 dB for frequencies between 50 Hz and 2 kHz.
I find little fault with the arguments laid out supporting the paper's thesis.
For those commenters making the jump to "sound quality" (which is not the topic of this paper), the quoted observation above conclusively proves that these headphones have differing tonal qualities. Even a casual listener will be able to hear a difference of 5dB in the critical freq range of human speech.
I always buy studio oriented gear for listening to music. If it's good enough to mix the record on, then probably I'll hear enough detail as well. Speakers, headphones, amps. Still there's a difference between regularly priced headphones and the really expensive ones. They tend to be a bit too "honest" for some people, more tiring to listen to. They also might hurt your fashion senses.
I don't care about the accuracy of their response curve (I know it isn't flat) after I found Grado[1] headphones. They are the only headphones I've found that don't add a "headphone" quality to the sound. It's hard to describe what I mean - it's that most headphones don't sound like a proper set of (quality) speakers. I've speculated it's something to do with most headphones not being able to move enough air. Grado uses very large drivers (voice coil is about 4cm in diameter) in a supra-aural (open back) design, which may move more air? Whatever the reason, Grado Labs has discovered a design that I consider categorically better[2] than everything else.
What's funny is that people tend to buy headphones with insane top frequencies (20-22 kHz), even if most humans cannot really hear sound of such frequency. When you are a teenager and have right genetics then there's a chance that you might hear 19kHz tone. If you are over 30 years old you are probably limited to 17 kHz already. Of course it gets even worse with age.
This shows the limits of quantitative studies. Just because you don't know how best to measure doesn't mean it doesn't matter, and there are strong preferences among people who are professionals in this area - of which I am one. I have spent years of my life listening to dialog on film sets; I like my favorite headphones because they have the least difference between how things sound with and without wearing them.
I could quantify that, but why bother unless I'm getting paid a hell of a lot to to do it? I don't see anyone here who's championing this naive approach offering to pay for a study designed by an experienced professional, so don't complain about a lack of scientific rigor if you're not prepared to pay for it. I prefer the more concrete feedback of people telling me it's the best soundtrack material they've ever received in post production.
You can talk about the scientific method all you like. I'm very fond of the scientific method. But rigorous testing costs money. If you're not willing to put your money where your mouth is, then accept the opinion of people who do this kind of thing for a living.
[+] [-] beat|8 years ago|reply
And with that, this study is bullshit.
Human beings don't listen to linear sine sweeps. We listen to music. Recorded music has 8+ octaves of frequency range (the bottom octave plus a little extra is almost always rolled off in real-world recordings, to ease stress on downstream components that can't reproduce such low frequencies anyway), and 20-50db of useable dynamic range.
Sine wave measurements of audio gear ignore impulse response, intermodulation distortion, phase shift, and a host of other real-world physical device responses to real-world musical signals. Scientific, reductionist thinking is inadequate to get an accurate picture of the factors that matter to human listeners.
Frequency response and total harmonic distortion aren't measured in these cases because they're useful or relevant. They're measured because they're easy to measure. It's like looking in the wrong place, because the light is better there. And the results? It's like measuring a car's performance by how well it can drive in a straight line at 60mph. Acceleration, braking, and turning are too hard to measure, so we ignore them...
I'm a musician and record producer. I've engineered and produced numerous albums, and rely on multiple different types of headphones for different purposes. The article's claim that one headphone can be easily morphed into another through mere equalization is, frankly, bullshit. The two headphones I rely on the most (Beyerdynamic DT880 and AKG K240) sound wildly different. Neither is "accurate". Neither are the Tannoy System 12 DMT midfield studio monitors I use for mixing, or the stock Subaru car speakers I use for reference to check the mixes from the Tannoys.
Audio reproduction is incredibly complex and difficult stuff. Trying to isolate one factor and saying "That explains everything!" is bad thinking.
[+] [-] jasonwatkinspdx|8 years ago|reply
This is irrelevant. They're measuring frequency response, not trying to map the entire world of psychoacoustics. Log sin wave sweeps are a perfectly adequate way of measuring a transducer.
> Sine wave measurements of audio gear ignore impulse response, intermodulation distortion, phase shift, and a host of other real-world physical device responses to real-world musical signals.
No, when you do a sin sweep and measure the impedance you can recover all that. Pick up a basic textbook on loudspeaker testing.
> Scientific, reductionist thinking is inadequate to get an accurate picture of the factors that matter to human listeners.
You are completely mistaken. Here's the research itself: http://www.aes.org/e-lib/browse.cfm?elib=12847 You can find blog posts covering a lot of the same material informally on Sean Olive's blog: http://seanolive.blogspot.com/
I strongly recommend you pick up the book Dr Toole wrote for a non technical audience: https://www.amazon.com/Sound-Reproduction-Psychoacoustics-Lo...
Learning from it will require you to shed the chip on your shoulder, but it will put you miles ahead of your peers in understanding what's actually going on when we listen to music vs all the commonly repeated mythology.
[+] [-] Florin_Andrei|8 years ago|reply
Sine sweeps are a good method to measure frequency response. That's what they were trying to do after all: measure frequency response. That was the stated goal of the test: find a correlation between frequency response and another parameter. They were playing a signal into a dummy, capturing the sound and measuring it; they were not trying to determine how pleased the dummy would be with the beautiful music.
However, I share your overall assessment that present-day transducers are far from perfect accuracy.
[+] [-] twhb|8 years ago|reply
The study doesn't say "that explains everything!". It measures one component in isolation. That's what good science does. The existence of other factors in headphone performance is irrelevant.
Besides, would you really not consider it news if 0-60 time in cars didn't correlate with price? Yes, a luxury car can justify its price in many other ways, but, on average, there should still be a correlation.
[+] [-] dgacmu|8 years ago|reply
"Scientific, reductionist thinking is inadequate to get an accurate picture of the factors that matter to human listeners"
I think you're noting the difference between good and bad science -- science is certainly capable of putting together a picture of the factors that matter to human listeners. It may not have done so, but it's not a failure of the scientific process, it's a failure of the study(ies) involved.
[+] [-] Karrot_Kream|8 years ago|reply
I... don't know what you're talking about. Apply a log scale to linear data and it too represents a log scale.
> Recorded music has 8+ octaves of frequency range (the bottom octave plus a little extra is almost always rolled off in real-world recordings, to ease stress on downstream components that can't reproduce such low frequencies anyway), and 20-50db of useable dynamic range.
So? We all know that listening to an instrument live sounds "different" from a recording, and it's up to us to figure out how to improve both recording and listening fidelity.
Now an argument can be made that because audio formats today tend to have caps on fidelity that headphones only be measured against the maximum fidelity that the recording produces, but that is neither the argument that you are making nor is this a weakness in the data presented by the paper.
> Frequency response and total harmonic distortion aren't measured in these cases because they're useful or relevant. They're measured because they're easy to measure.
Yes and what should we measure? Phase graphs? Third order components? You're not making a meaningful argument here.
> The article's claim that one headphone can be easily morphed into another through mere equalization is, frankly, bullshit.
First of all this paper used novel methods to back the original assertion made in [1]. Also, let me quote the exact line from the conclusion:
"PCA can account for 90% of the variance across all measured headphones with six eigenvectors. The first eigenvector is similar to published target responses, while the second eigenvector represents a global spectral tilt."
> Audio reproduction is incredibly complex and difficult stuff. Trying to isolate one factor and saying "That explains everything!" is bad thinking.
And so is your appeal to non-authority. The gist of your entire argument is that linear sine sweeps are useless, and therefore the entire validity of the paper is moot and your non-scientific opinion is now superior.
[1]: http://asa.scitation.org/doi/10.1121/1.4984044#
[+] [-] Matthias247|8 years ago|reply
Sure, nonlinear distortions are not covered and also sure that the measurements won't cover everything which will influence the perceived sound. But nevertheless frequency response measurements are a very solid tool which allows objective comparions between various systems.
[+] [-] chiph|8 years ago|reply
For the curious - here's the story of how the budget NS-10 ended up as a reference standard: http://www.soundonsound.com/reviews/yamaha-ns10-story
[+] [-] yk|8 years ago|reply
Yes, and actually one of the challenges physicists face, is that looking by the lamp is not self evidently reasonable for most people. Of course you start with the easy case and then you build on that foundation, otherwise your analysis just hangs in thin air. (Plus you have to deal with the easy case anyhow if you want to do comprehensive work, you can as well start were you have a chance.)
[+] [-] friendzis|8 years ago|reply
Instruments themselves are imperfect, there is long chain of devices (recording, mixing, playback) until the signal reaches speakers (and then room until signal reaches ears), at higher quality levels it is quite often the case of how well all the inaccuracies of the system interact with each other, rather than how accurate certain components of the system are.
[+] [-] rhizome|8 years ago|reply
Is this what they're doing, though? Seems to me their conclusion is that frequency response doesn't explain price.
[+] [-] cnnsucks|8 years ago|reply
We've evolved our senses around the signal components the pay off the best: the "easy" ones. Thus it isn't surprising that "subjective quality is mostly correlated with linear (spectral) attributes instead of non-linear (distortion) metrics," a claim supported by peer reviewed papers that have vastly greater credibility than you and your anecdotal vitriol.
[+] [-] deelowe|8 years ago|reply
> It's like measuring a car's performance by how well it can drive in a straight line at 60mph.
No, it's like comparing several cars and their MRSP based on their 0-60 performance, which is certainly a reasonable thing to do. Just because they didn't measure stopping distance, it doesn't invalidate the entire comparison.
[+] [-] shawnz|8 years ago|reply
It's not sufficient that a car can drive well in a straight line, but it's necessary. Isn't that at least one of the things you should measure?
[+] [-] derefr|8 years ago|reply
[+] [-] mmaunder|8 years ago|reply
It's really cool hearing what they heard in the studio control room for the final mix. And often surprising.
You can get a range of other precalibrated pro audio headphones or correction profiles from sonarworks.
Consumer headphones are just silly IMHO. Artificially boosted frequencies with prices up to $400. A set of precalibrated MDR7506's is around $220.
If you don't care about truly flat response with correction, you can get a set of AKG K240's for $100 bucks and they're super comfy, amazing sound and loved universally by audio pros.
[+] [-] fizixer|8 years ago|reply
- Someone with online alias NwAvGuy put the whole AV industry (ok maybe not the whole, but some big players) in a loop by showing in online forums that a totally inexpensive DIY DAC (with a free design he/she shared) could be built with quality rivaling elite products worth thousands of dollars. [1] (well a hazy version of the story goes that he/she exposed various audiophile review sites and forums as being full of sponsored reviews, and that eventually lead to his/her ban from head-fi.org I think)
- As for capsule mics (commonly known as condenser mic), market is flooded with DIY designs and DIY kits which let you build/buy one for $200-$400 (the dominant cost being that of the capsule itself) that will rival the quality of multi-thousand dollar mics. They go by the names Neumann clones, etc. [2] (no affiliation), [3].
In retrospect, and given the shady things AV sellers do, like trying to sell you a USB or HDMI with gold-plated pins, claiming it to be superior, it should come as no surprise.
Though, no offense, but audiophile consumer base is filled to the brim with hipsters who judge the quality of a product by its price (and some of the "experts" were busted after they failed blind tests; I think opus vs flac, I'm mixing a lot of things now).
[1] http://spectrum.ieee.org/geek-life/profiles/nwavguy-the-audi...
[2] https://microphone-parts.com/
[3] https://www.youtube.com/watch?v=DtNH46jpwJo
[+] [-] stcredzero|8 years ago|reply
Headphones also have a serious empiricism issue. You can probably pass off one high end Sennheiser for another in an A/B test. But you couldn't pass off an Audeze for one and have a valid A/B test. Also, you will often read or hear an expert say, if the measurements say something is bad, but it sounds good, or vice versa, then it means we're measuring the wrong things. I'm not saying that the Harman response curve isn't valid. It's just not the whole story.
tl;dr -- Buy the cheapest headphones that you really like, and ignore whatever your coworkers say. ( Hell, there are actually Beats that are good headphones! https://www.innerfidelity.com/content/time-rethink-beats-sol... )
Things are going to change in significant ways in the future as the price of signal processing, compensation, and active correction drops, however. Combining those with advances in the cheaper manufacturing of better drivers will result in the headphones of 10 years from now making the high end headphones of today seem "meh" and today's typical headphones seem trashy.
[+] [-] arnaudsm|8 years ago|reply
We need objective benchmarks for everything. Especially when marketing is growing bigger each year. Even "Tech websites" are biased and not objective anymore.
[+] [-] AdmiralAsshat|8 years ago|reply
It's very easy to say, "I can hear so much more of the song out of my ATH-M50's than I can a pair of Beats", and you may be right. But something objective to back it up would be great, too.
[+] [-] josefresco|8 years ago|reply
http://www.consumerreports.org/cro/headphones.htm http://www.consumerreports.org/cro/headphones/buying-guide
[+] [-] criddell|8 years ago|reply
When I was a kid, my friends and I would kill time in the library by flipping through Car & Driver, Motor Trend, and Road & Track magazines trying to figure out what car is best. The Camaro might be two tenths faster in the quarter mile than the Mustang, so clearly anybody that buys a Mustang is an idiot, right? This is what a lot of headphone evaluations sound like to me.
[+] [-] genka|8 years ago|reply
[+] [-] dom0|8 years ago|reply
That being said, speakers and headphones all have a sound (unlike any properly designed amplifier); there is hardly any right or wrong there, and while the best approximation of a flat response might seem technically most correct, people will probably prefer different sounds.
[+] [-] beat|8 years ago|reply
[+] [-] stcredzero|8 years ago|reply
That brain-training game on the Nintendo DS -- the one that had you distinguish two voices saying things simultaneously -- your score on that game is a great way to evaluate headphones for vocal accuracy. Be careful though: the DS can't adequately drive all headphones, so you'll need an amp for high impedance headphones.
[+] [-] londons_explore|8 years ago|reply
[+] [-] calichoochoo|8 years ago|reply
[+] [-] skywhopper|8 years ago|reply
This is a silly assumption, and easily explained.
1. Most headphone purchases aren't and cannot be made by comparing sound quality. Reviews of sound quality are so universally understood to be subjective that most consumers probably ignore those details.
2. There is no one subjective or objective standard that is meaningful for all listening material. Podcasts, modern pop music, older pop music, classical recordings, television shows, and movies all have wildly varying acoustic profiles between and among each genre.
3. The vast majority of headphones have Good Enough sound quality for the vast majority of consumers. Sound quality is highly unlikely to be the primary reason most consumers buy a set of headphones, and it's unlikely to be the reason they are dissatisfied with certain headphones.
4. Headphone design, form factor, build quality, fit, feature-set, and even color are all much more important factors in terms of consumer satisfaction with headphones. They are, after all, a highly noticeable part of your ensemble. They are intimately in contact with your body. And you want them to work without thinking about it too hard. In addition to being more important, most of these factors are far easier for consumers to judge between headphones than sound quality, so again it's no surprise that an arbitrary single standard of sound quality would fail to correlate with perceived value.
In other words, this is silly for reasons that have nothing to do with technical arguments about actual sound quality, whatever that means.
[+] [-] FfejL|8 years ago|reply
Price is correlated with perceived value, which includes quality, brand recognition, brand opinion, current style, and a long list of other factors.
(And, yes, this is a horrible use of the word 'correlated.' 'Derived from' or 'based on' would be much better.)
[+] [-] bryik|8 years ago|reply
That is an incredible assertion.
I am willing to bet $10,000 that I can find a product where price is correlated with quality (assuming we can agree on a definition of "quality" for a particular product). How about GPUs with "quality" measured by average FPS in a selection of AAA games at 4K?
[+] [-] jacquesm|8 years ago|reply
Tools.
Musical instruments.
For both of those there is strong correlation between price and quality, and the users of both of those are picky for a good reason: the quality of the work they produce depends to some extent on the quality of the tools they use.
Now, bad craftspeople and bad musicians will produce crap no matter what the tool, but good craftspeople and good musicians can definitely produce something better with good tools and instruments and are prepared to pay for that.
[+] [-] riskable|8 years ago|reply
Though I suppose "purity" is considerably more exact than, "quality".
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] escape_goat|8 years ago|reply
I believe that what you are meaning to say is that price responds to demand, and demand correlates to the perceived value of the product. In turn, perceived value is often but not inherently responsive to information about the 'quality' of the product.
Instead, perceived value is a summation of various forms of utility, only some of which are affected by 'quality'.
Furthermore, any determination of 'quality' is rarely made upon information that is as complete as the information available to an attempt to methodically determine the relative 'objective' quality of substitutable products, because of the disutility of conducting such an assessment. More often, the determination is based upon perceived indicators of quality, which are known to be unreliable.
This diminishes the relative importance of 'quality' in the formation of the perceived value of the product, which accounts for the importance of alternate, more reliable information about the product, such as the social signalling value, and especially easily manipulable information such as indicators of "style" and the narrative about the product; this in turn explains the willingness of product vendors to invest in advertising.
With regards to the use of the word 'correlate': price is of course correlated with demand, with correlates to perceived value, which tends to correlate to perceived quality, which tends to correlate to quality. Since correlation requires transitivity, price does then tend to correlate to quality. So what you actually said is incorrect.
I believe that your intended assertion (as I understand it) is also incorrect and over-broad, because in many instances a highest-information assessment of quality is the sole determining factor of the perceived value of the product. This is often true in business to business transactions such as part sourcing, where a person with specialized information is in charge of purchasing, and with products in a commodity market such as grain or beans, where a standardized determination (and guarantee) of quality has been established.
(Thanks for the opportunity I've taken to sit for a while and practice arranging my thoughts, which is the reason that my reply is so extensive.)
[+] [-] svantana|8 years ago|reply
Indeed, they did find a significant difference in magnitude response _error_, although the effect was quite small.
[+] [-] flavio81|8 years ago|reply
Study says:
"Nevertheless, assuming that the perceived audio quality is largely determined by the spectral magnitude response of headphones..."
This is a very wrong assumption.
Audio component designers have more or less a hard time picking up which measurements can correlate with audio quality. And frequency response measurements using sine sweeps, like in the cited study, are almost of no value for discriminating between two transducers (headphones, speakers) with regarding to 'audio quality'.
Also, the fact that one headphone can extend beyond 20KHz or that it can go below 20Hz will give zero guarantee of better audio quality.
Frequency response measurements using white/pink noise can give a slightly better hint because they can take a look at resonant peaks that might be annoying to the listener, but even this is not a law set in stone*
* Impulse measurements (and waterfall plots) can give you a clearer idea of how clear is the sound going to be; but then you can have a transducer with a fairly good impulse response but a slight resonant peak somewhere --- OR you can have sometimes a transducer which shows pretty flat frequency response but bad impulse response.
A good test for intermodulation distortion (the big white elephant in the audio room) will REALLY give you a hint of which headphone will be least annoying to the ear when listening to loud complex music like classical music, vocal music, etc.
It seems that the article has been written by experts in acoustics, but not really in "audio".
TL;DR: Freq response measured with sine sweeps can't really tell you anything helpful to discriminate headphones with regard to sound quality.
[+] [-] davidrhunt|8 years ago|reply
The citation provided further down in the article is [1]. Dr. Olive is thorough in attempting to remove sources of bias and conducts a set of fully double blind listening tests in order to come to the conclusion: "The results provide evidence that trained listeners preferred the headphones perceived to have the most neutral, spectral balance".
> A good test for intermodulation distortion (the big white elephant in the audio room) will REALLY give you a hint of which headphone will be least annoying to the ear when listening to loud complex music like classical music, vocal music, etc.
Where is this proven?
[1] http://seanolive.blogspot.com/2013/04/the-relationship-betwe...
[+] [-] rb808|8 years ago|reply
[+] [-] 09bjb|8 years ago|reply
"New study shows no difference in speed of blue vs. red light!" We always ascribe waaay to much importance to what we can measure easily.
[+] [-] crispyambulance|8 years ago|reply
I don't think they're measuring the wrong thing, although a human might be interested in more things than frequency response.
Just goes to show that perhaps the most important thing in selecting headphones is comfort and the context in which they will be used.
For "audiophile listening experiences" an equalizer and CA-style cans might be a good idea and give a lot more bang for the buck than shelling out several hundred just for the cans.
[+] [-] GuB-42|8 years ago|reply
"However, the variance in low-frequency response seems to decrease with increasing price, indicating an improved bass response measurement consistency across headphones in the higher price range."
[+] [-] jmileham|8 years ago|reply
Of course all this is confounded by the fact that music will tend to sound best on speakers/headphones with a response curve most like the speakers/headphones that the mastering engineer used (or more accurately, the set of speakers/headphones that the engineer compromised among). You will probably tend to have the best experience listening to music with the popular devices within a given musical subculture, because mastering engineers will be targeting those devices.
[+] [-] untangle|8 years ago|reply
I find little fault with the arguments laid out supporting the paper's thesis.
For those commenters making the jump to "sound quality" (which is not the topic of this paper), the quoted observation above conclusively proves that these headphones have differing tonal qualities. Even a casual listener will be able to hear a difference of 5dB in the critical freq range of human speech.
[+] [-] dep_b|8 years ago|reply
[+] [-] pdkl95|8 years ago|reply
[1] http://www.gradolabs.com/headphones/prestige-series/item/1-s...
[2] In terms of music quality. Other use cases may prefer designs that focus on other features.
[+] [-] mamon|8 years ago|reply
[+] [-] fffernan|8 years ago|reply
[+] [-] anigbrowl|8 years ago|reply
I could quantify that, but why bother unless I'm getting paid a hell of a lot to to do it? I don't see anyone here who's championing this naive approach offering to pay for a study designed by an experienced professional, so don't complain about a lack of scientific rigor if you're not prepared to pay for it. I prefer the more concrete feedback of people telling me it's the best soundtrack material they've ever received in post production.
You can talk about the scientific method all you like. I'm very fond of the scientific method. But rigorous testing costs money. If you're not willing to put your money where your mouth is, then accept the opinion of people who do this kind of thing for a living.
[+] [-] swayvil|8 years ago|reply