On 2018-12-11 22:42:03, user Andrew Leifer wrote:
Thank you for reading our manuscript closely and for sharing your comments with the community. We welcome a robust scientific discussion about our findings. In fact, this is one of the primary reasons why we post to the bioRxiv. Your group, in particular, has done pioneering work in this area and we value your thoughts. Below we provide a brief response to your note, highlight areas where we disagree, and discuss specific analyses to support our claims.
- The major concern expressed in your comment relates to noise in our measurements in (Scholz et al., 2018). The strongest argument that counters concerns about noise is that our neural recordings predict the animal's behavior in held out data, while control GFP recordings do not (Fig 2G). Thus, noise in our recordings are not sufficiently strong to swamp out relevant behavior signals in moving animals nor are they strong enough to mimic those signals in control animals.
Extrapolating from your pioneering work, we had expected to see a dominant behavior signal in the first three PCs of neural activity, but we did not find such a signal. You express concern that perhaps noise may be present to such an extent during movement that it precludes drawing any conclusions from our PCA analysis. We do not think that is the case. Nonetheless, it is worth imagining what such noise would have to look like for it to both invalidate our PCA analysis yet simultaneously preserve our ability to successfully predict behavior. To be consistent with our measurements, such noise would have to have the following properties: 1) Be comprised of at least three independent orthogonal components that are the most dominant features in the recording. 2) Be distributed across many neurons (because otherwise these signals would not dominate in PCA, which involves z-scoring each neuron's signal). 3) Not overpower a signal that we observe in the first three PCs that is slightly predictive of the animal's velocity in GCaMP worms but absent in GFP control worms (Fig 2G) and 4) still preserve our ability to predict velocity and turning from the activity of a subset of neurons on held out data. While we cannot rule out noise with such unique properties, we think a much simpler explanation is that the first three PCs are not dominated by noise. We therefore merely conclude that the first three modes lack predominant behavior signals. In retrospect, it may not be surprising that moving worms have other signals dominating their neural dynamics. These could, for example, be related to sensory signals or to internal states.
- You also express concern about our ability to observe the manifold that you report in immobile conditions (Kato et al., 2015). We agree that perhaps the recording shown in Fig 1F is too short to clearly see multiple cycles on the manifold. We chose this recording because it allowed us to directly compare moving and immobile states in a single trial. Longer recordings provide a better example. When we look at longer recordings (BrainScanner20171017_184114 from Table 3) we clearly recover a very similar manifold to the one your group published (see Comment Figure 1, below, two views of same recording). Thank you for urging us to push this comparison further, we will include this plot in future versions of the manuscript.
https://uploads.disquscdn.c...
- You also express concern that our velocity prediction is dominated by switches between positive and negative velocity. Comment Figure 2, below, shows that this is not the case (the same trace is shown here as is in Fig 2). The fit is not dominated by forward or backwards velocity, but rather accurately fits and predicts intermediary velocities. We will add these plots to future versions of the manuscript. Thank you for encouraging us to look more critically at this.
https://uploads.disquscdn.c...
-
We are aware that estimating neural IDs is extremely challenging. We have tried to be very transparent in our estimates. Table S2 and the supplementary methods give details. We are also happy to answer any specific questions you might have.
-
Sparse models have been successfully used before in neuroscience (Pillow et al., 2008; Tankus et al., 2012). We are aware of potential concerns with fitting sparse models. We mitigate them by 1) using elastic net which is suitable for highly collinear datasets such as ours, 2) using cross-validation to evaluate the robustness of our fits (Roberts et al., 2017), and 3) assessing model performance on held-out data, which we note is a higher standard than typically used for linear regression in the field. <br />
In fact, reference (Wu et al., 2007) that you mention in your note claims that LASSO performs best for datasets like ours where the variables have correlations, and we note that our elasticnet model incorporates the LASSO penalty.
-
Thank you for finding the typo in Fig 2D. We will fix it in future versions of the manuscript.
We appreciate your comments as they help us to strengthen the manuscript and anticipate reviewer comments.
Sincerely,<br />
Monika Scholz and Andrew Leifer
REFERENCES
Kato, S., Kaplan, H.S., Schrödel, T., Skora, S., Lindsay, T.H., Yemini, E., Lockery, S., and Zimmer, M. (2015). Global brain dynamics embed the motor command sequence of Caenorhabditis elegans. Cell 163, 656–669.
Pillow, J.W., Shlens, J., Paninski, L., Sher, A., Litke, A.M., Chichilnisky, E.J., and Simoncelli, E.P. (2008). Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999.
Roberts, D.R., Bahn, V., Ciuti, S., Boyce, M.S., Elith, J., Guillera-Arroita, G., Hauenstein, S., Lahoz-Monfort, J.J., Schröder, B., Thuiller, W., et al. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography 40, 913–929.
Scholz, M., Linder, A.N., Randi, F., Sharma, A.K., Yu, X., Shaevitz, J.W., and Leifer, A. (2018). Predicting natural behavior from whole-brain neural dynamics. BioRxiv 445643.
Tankus, A., Fried, I., and Shoham, S. (2012). Sparse decoding of multiple spike trains for brain–machine interfaces. J. Neural Eng. 9, 054001.
Wu, Y., Boos, D.D., and Stefanski, L.A. (2007). Controlling Variable Selection by the Addition of Pseudovariables. Journal of the American Statistical Association 102, 235–243.