Published:
Sometimes you think you understand a basic statistics problem. And then, you look closer and realize that you don’t understand it as well as you thought. This happened to me recently with Linear Discriminant Analysis (LDA). In this post, I discuss some interesting and lesser known aspects of LDA that I learned when diving deeper into this method, and that seem to often cause confusion.
Published:
Covariance matrices are one of the most important objects in statistics and machine learning, essential to many algorithms. But estimating covariance matrices can be difficult, especially in high-dimensional settings. In this post we introduce covariance shrinkage, a technique to improve covariance matrix estimation. We also provide a PyTorch implementation of a popular shrinkage technique, the Oracle Approximating Shrinkage (OAS) estimator.
Published:
If you are a scientist who uses code in your research, you might have experienced the following scenario. You wrote some code, let’s say in directory my_code/, or GitHub repository https://github.com/my_user/my_code. You use this code in a certain project and analyses are currently working. Then, you have the idea of modifying my_code, for example, to add a new feature required by a new project. However, the new modified code might (and probably will) break the old project.
Published:
Often times, we want to optimize some model parameter while keeping it constrained. For example, we might want a parameter vector to have unit norm, a set of vectors to be orthogonal with respect to each other, or a matrix to be symmetric positive definite (SPD). For the specific cases where the constraint is for the parameter to be on a manifold, a common approach is to use Riemannian optimization. However, there is a simpler and often more efficient way to do constrained optimization: we can use a technique called parametrization.
Published:
In this blogpost, we ask a question ancient as philosophy itself: is the world as we see it? However we approach this question from the view of biology and neuroscience. We discuss three different perspectives on why the world is or is not as we see it: An evolutionary perspective, a resource optimization perspective, and the perspective of ambiguity of sensory information. With this question as guiding thread, this post introduces several major concepts of perception and neuroscience to a general audience.