On 2014-06-10 13:49:57, user Authors of the manuscript wrote:
Dear Mike X Cohen,
this kind of personal commenting is much more helpful and constructive for the authors than the anonymous peer-review process and we thank you for taking your time to write this comment. We respond to some of your points in the following:
MXC: “It is not always clear whether the authors are criticizing the biophysical interpretation of CFC analyses, or the mathematical foundations of CFC methods. Perhaps it would be useful for the authors to define the situations under which CFC could be validly interpreted, and what exactly the neurobiologically meaningful interpretation would be.<br />
Concerning the former, the authors accurately state that relatively little is understood about the neural mechanisms that could produce CFC, and this may impede interpretations of empirical findings (the same criticism applies to most macroscopic measures of brain activity, including ERPs, time-frequency power, most measures of functional connectivity, the FMRI BOLD response, etc.).”
Authors:
We agree with this comment in the sense that indeed many measures in Neuroscience depend on an interpretational step. However, in contrast to the current handling of CFC, these aspects are well acknowledged for measures like BOLD and ERP. In addition there have been intense efforts to disentangle various generating mechanisms of BOLD signals and ERPs. (For the origin of the BOLD signal, the role of astrocytes, lactate, and calcium see for example: Niessing et al, Science, 2005; Logothetis et al., Nature, 2001; Barros, TINS, 2013; Petzold&Murthy, Neuron, 2011; Iadecola&Nedergaard, Nat Neurosci, 2007 . For generating principles of the ERP see for example: Mazaheri & Jensen, J Neurosci, 2008; Turi et al. NeuroImage, 2012; Telenczuk et al, J Neurophysiol, 2010, and references therein).
In these fields, the variety of generating mechanisms is typically discussed and wording is carefully chosen. With respect to the interpretation of CFC measures, this care is often lacking. Moreover, the mathematical methods of CFC are more involved compared to standard BOLD-fMRI or ERP analyses. Therefore, plain technical errors in published work occur more frequently than in either ERP or BOLD fMRI studies.<br />
_____
MXC: “Their suggestion for researchers to label their CFC analyses as relatively exploratory vs. confirmatory and as a marker vs. biophysical understanding (figure 5) is also sensible (this suggestion also could be applied to most or perhaps all measures of brain activity). The reliance on DCM should be cautioned against the over-parameterization and opaqueness of DCM models used in practice.”
Authors:
We agree with this comment insofar as the mathematics involved in DCMs is necessarily much more involved than that in the current standard CFC analyses. In our opinion however, this is outweighed by the advantage to be able to state the relative odds for and against the presence of a CFC mechanism in the data. Moreover, we also agree that the mathematical complexity of model specification indeed results in a certain opaqueness, especially to the lay.
We disagree with the criticism of over-parametrization, as models selected by Bayesian model comparison need two properties: (1) the ability to explain the data well, and (2) generalizability. The latter is ensured by automatically favoring models that explain the data well without using an excessive number of parameters, thus implementing Occam's razor. However, it is indeed necessary to carefully specify models for comparison, that are plausible a priori, based on existing knowledge (Lohmann et al, NeuroImage, 2013; comments by Friston et al, NeuroImage, 2013; Breakspear, NeuroImage, 2013; reply by Lohmann, NeuroImage, 2013). This requirement may mean that DCMs of CFC will have to wait until the mechanisms underlying CFC are spelled out more explicitly using interventions.<br />
____
MXC: “the general point is that methods for assessing CFC are not necessarily confounded just because their results can be difficult to interpret from a neurophysiological perspective. Let me explain this by analogy: Imagine comparing ten randomly selected negative numbers with ten randomly selected positive numbers. A t-test would indicate statistical significance, but this significance is uninterpretable. However, the reason that the result is uninterpretable is not due to a confound of the t-test, but rather, due to the assumptions underlying the data collection. Imagine you received the same numbers but were told that they reflected measurements of relative alpha-band power in conditions A and B. Now the same result would be interpretable.”
Authors:
Indeed, in some sense the whole first part of our paper illustrates the variety of different but equally plausible reasons behind a CFC signature, or different possible interpretations if you wish. So, why do we call them "methodological confounds"?
Taking an analogy with the t-test might help us here, though we think that the analogy provided by MXC is slightly misleading and prefer a different version of the analogy. Namely, when you make a t-test, the un-interpretability is not only about the "origin of the data" (as in the example of MXC), but also (and actually even more) about the "nature of the data".
T-test makes specific assumptions on the underlying probability distribution (e.g. normality) and when these assumptions do not hold, the p-value obtained might very well just reflect the fact that the underlying distribution did not match well.
This is similar to CFC - we do not claim that the CFC measures are wrong, but in some sense show that the underlying assumption that there is real coupling in the data might well be doubted (for several reasons explained in the text). We show how alternative assumptions (i.e. non-linearity, common drive etc) could as well account for high CFC values. I.e. the CFC measure describes the amount of coupling only if we already assume the existence of this coupling, and the absence of the other mechanisms, or their constancy over experimental conditions.
Maybe "methodological confounds" sounds more appropriate if one keeps also this analogy in mind - if the methodology is applied in case of doubt with assumptions, the results are not interpretable. It is the same with the T-test - applying it to any distribution, one is not able to draw conclusions. This is not a fault of the T-test. However we would end up with a possible confound if we DID not know what the underlying distribution is, but still applied the T-test. In the case of CFC analysis we do not have a good understanding of underlying biophysics, but still apply the CFC measure and try to interpret it.
It might be useful to compare two different possibilities of expanding the acronym CFC - either Cross-Frequency Correlation or Cross-Frequency Coupling. The latter indicates biophysical interaction and even causality and is the one used now in the literature. Our article discusses at length why in fact we should rather hold to Cross-Frequency Correlation. Moreover, we explain that even in this case it is important to try to partial out the effects that could diminish the specificity of CFC as a marker.<br />
______
MXC: “Their first example is the van der Pol oscillator. The authors claim that CFC here reflects a confound, because (page 3) “there is no simple physical interpretation for the different frequency components of the oscillator.” The interpretation depends entirely on the assumptions of the signal. If this were a neural signal, one might interpret that certain phases of the lower frequency oscillation regulate the variability of faster activity (as an aside, the lack of band-limited activity in Figure S1 is a classic situation of when *not* to interpret results as reflecting an oscillation; this has been discussed since the 1990’s by, among other researchers, Singer, Tallon-Baudry, Pfurtscheller, Miller). This is readily apparent by plotting the van der Pol signal along with its rectified derivative, which can be obtained with the Matlab code below:
ode = @(t,y)
vanderpoldemo(t,y,1);
[t,y] = ode45(ode,[0 20],[2 0]);
plot(t,y(:,1)), hold on
plot(t(1:end-1),abs(diff(y(:,1)))*8,'r')
The problem here is not with the measure of CFC. In fact, I do not see a problem at all; the authors simply tested a method on simulated data and got a result, much like a t-test on signed random numbers would produce a result. Here is another, even more striking, example:
t=0:1/1000:1;
plot(t,sin(2*pi*40*t) .*sin(2*pi*t))
As with the van der Pol illustration, one can say that CFC here is uninterpretable because there is no interaction amongst subsystems; there is simply a 40-Hz sine wave multiplied by a 1-Hz sine wave (this could occur from two independent systems with wave cancelation at the recording electrode). Again, the problem is not with the CFC measure, but that the simulated data do not lend themselves to a neurobiological interpretation of CFC.”
Authors:
Indeed, “the simulated data do not lend themselves to a neurobiological interpretation of CFC”, and neither do the neurobiological data at the moment. This is one of the main points of the manuscript.
The problem is that for now, the neurobiological measurements might not lend themselves to the “coupling” interpretation of CFC. The CFC analysis has been adopted and is used with a certain aim and interpretation. Thus it seems fair to say that if the methodology does not provide answers and interpretations it should, we deal with "methodological confounds".
The examples brought up show that without further assumptions and knowledge of the underlying neurobiology, current methodology is unable to discriminate between various basic but very different interpretations. In analogy with the T-test example above, similar other toy examples treated with a T-test would illustrate what could happen if the underlying distribution did not match the assumptions (i.e. normality) - and why a T-test is not applicable without checking its assumptions first.
As we mention in several places, this is not a problem when one tries to use the CFC measure only as a MARKER, however the problem comes when one goes one step further in the interpretation, trying to give a particular (physiological) meaning to CFC (“high frequency oscillations modulated by low frequency phase” or something along these lines).
Also, notice that your second example (modulated sinusoids) does tell you something about which parameters (in terms of bandwidth) should be used so that the CFC measure would be closer to its desired interpretation.<br />
____
MXC: “Their other examples are also not compelling as identifying any confounds with CFC measures. Prime numbers are nonrandom sequences with a periodic structure (http://xxx.lanl.gov/pdf/cond-m... and anyway, true random sequences can appear non-random at small N. A more serious concern is that the authors are interpreting CFC in random data or in ECoG data with non-linearity introduced (Figure S6) without performing any statistics to justify the interpretation of CFC. Analogously, a t-statistic on random numbers is unlikely to be exactly 0; it is only through evaluation of that t-statistic with respect to a null hypothesis distribution that a t-value of, say, 1.5 can be interpreted.”
Authors:
Interestingly enough, prime-numbers, when one partials out the fact that there is only one even prime number, one prime number that is divisible by three etc, seem to be best described as what are called pseudo-random numbers. (See for example any of Terence Tao’s blog posts or presentations on “primes and pseudorandomness”.) So at least for now, to our knowledge, there seems to be no reason to believe that there is cross-frequency coupling behind any process we might expect to generate prime numbers. ;) But of course this is just an illustration of how hard it is to conclude anything about mechanistic processes by just using a CFC measure. As a side note, one should also not forget that still some care is needed when interpreting such statistics, i.e. recall the numerical information on the change of sign between \pi(x) and li(x) and Skewes’ numbers. ;) But probably none of us is an expert on primes and knows exactly why they give rise to a high CFC index. We reason in the article that even in the case of the CFC measured from the brain, this “why” still continues to have a multitude of possible answers.
Now, more seriously, in the ECoG or random data we use the exactly same procedure as is usual in the CFC analysis. Indeed, we used the code provided by Tort for the modulation index, and the code provided by Canolty et al. from their Science paper and hence, their respective surrogate analysis (and in our text it was indicated that the results were significant). In addition, for the non-linearity case we even provided a simple example (supplementary material) where we derived analytically that quadratic non-linearities lead to CFC. <br />
____
MXC: “Another issue identified by the authors is the potential confound of co-occurring but independent low-frequency phase and high-frequency power dynamics. This is a potential confound (discussed in Cohen, 2014, Analyzing Neural Time Series Data; figure 30.7) but is fairly easy to identify and address (including: avoiding interpreting CFC from immediate post-stimulus periods, removing the phase-locked time-domain signal before computing CFC, and inspecting whether the time-course of CFC differs from the time-course of phase clustering). Perhaps the authors have additional suggestions?”
Authors:
As we note in our manuscript “if a brain area under a recording electrode receives time-varying input from any other brain area, this input might generate similar dependencies across frequency components (Figure 4A). The problem is that usually one has no control over the timing of the internal input to the examined brain area (Figure 4B). Thus, phase-amplitude coupling measured anywhere in the brain can be potentially explained by common influence on the phase and amplitude, without the phase of a low frequency oscillation modulating the power of high frequency activity.” The improvements mentioned in your commentary do not help to identify and address the problems with INTERNAL input, where we have no idea about the onset time (see Figure 4). <br />
____
MXC: “Later, they write (pages 9-10 and figure 4) "If a brain area under a recording electrode receives time-varying input from any other brain area, this input might generate similar dependencies across frequency components." This does not seem to be a confound, but rather, a description of CFC: low-frequency oscillations from a distal brain region modulate local activity, as manifest in higher frequency oscillations. Perhaps if the authors would identify a mechanism/consequence of CFC for neural activity it would be easier to understand whether/how this is a confound.”
Authors:
There is a misunderstanding here. We would not NOT agree with the interpretation that “low-frequency oscillations from a distal brain region modulate local activity, as manifest in higher frequency oscillations”. Instead we clearly write in our manuscript that “non-stationary input to a given area simultaneously affects the phase of a low frequency component and increases high-frequency activity (common drive to frequency components of the same signal).” This means that the low frequency phase is modulated and the high frequency component is influenced by the same common drive to the area. As we conclude: “In this case, high-frequency amplitude increases occur preferentially for certain phases of slow oscillations even without any need of interaction between the two rhythms.” (See also Figure 3). Again, we would agree on this point if CFC would stand for Cross-Frequency Correlation rather than Cross-Frequency Coupling, as the latter indicates interaction or causality.
____
MXC: “On page 6, the authors write “The main conclusion is – not that surprisingly - that a clear peak in the power spectrum of the low frequency component is a prerequisite for a meaningful interpretation of any CFC pattern.” The justification does not follow. If one is interested in *phase* dynamics, why does there need to be a peak in *power*? Assuming that phase reflects the timing of neural populations while power reflects their spatial coherence at the LFP level, why is spatial coherence considered a prerequisite for investigating timing? In real EEG data, power and phase dynamics are often independent of each other.”
Authors:
It is here not at all necessary to think about which neural processes the phase or power variable could reflect. The reason for why a peak in the power spectrum is a prerequisite for a meaningful interpretation of phase (as an index that is a parameter of an oscillation) is well known in the physics/electrical engineering community and simply comes from the signal processing perspective: phase can be meaningfully defined only for narrow-band (and slowly frequency-varying) oscillatory signals for which the phase grows monotonically (please see page 35 of the manuscript: Supplementary discussion - conditions for a meaningful phase). Note that although narrow-band filtering a signal enhances smooth dynamics of its phase, it does not improve its physical interpretability.
____
MCX: “A related discussion is potential differences in power across conditions. CFC methods generally measure the relationship between power and phase, not the magnitude of power. Appropriate permutation-based statistical corrections will account for differences in the magnitude of power (Cohen, 2014, chapter 30).”
Authors:
Yes, we agree that this is something that one indeed can control for and just point out that this is not always done in the literature. (See literature review).<br />
____
MCX: “The potential confound of low power for estimating phase (Muthukumaraswamy & Singh, 2011) applies only for very low SNR; in real EEG data, power and phase dynamics are often easily disambiguated and unrelated to each other.”
Authors:
The level of SNR for EEG is dependent on the frequency band considered and stimulation elicited by the experimental protocol. Here the main point is that many studies compare CFC between conditions that elicit very different power in a given band (e.g. peak vs no peak). Thus there is straight away a bias in the reliability of the phase estimation and therefore of the phase-amplitude coupling. How big this effect is should be assessed for each dataset. In addition, the amplitude and phase defined by the analytical signal approach (using Hilbert transforms) are not fully independent and even a nominal change in one of them induces a perturbation in the other (Supplementary Figure 7B).
____
MXC: “Table 1 should include citations of the papers surveyed; otherwise independent verification is not possible.”
Authors:
we feel that the description preceding the literature review enables anyone to find the respective papers (as the years, journals and search criteria have been mentioned, a simple PUBMED search can provide the explicit list of papers considered). The magic paper is the one we added manually, which we indeed can identify here - Saalmann et al., 2012 in Science. The literature review covers papers up to January 2014 (included).