Autism and L1 Regularisation
A topic of special interest recently is the relationship between autism, ADHD, schitzophrenia, and OCD, in the context of machine learning and information theory. Specifically, it seems like we can create two spectra of autism/schitzophrenia and ADHD/OCD and map out at a high level the relationship between these mental conditions using concepts from ML and information theory. I am still working out precisely how to structure these concepts.
As a start to this idea, consider autism and L1 regularisation. Autistic people generally have the following traits:
Overly mechanistic thinking
Extremely precise analytical thinking
Limited ability to generalise outside a few very specific domains
Obsession with a handful of specific concepts
Consider also the impact that L1 regularisation has on a ML model (or even a simple regression model). L1 regularisation forces a model to focus on a limited number of inputs, by penalising the model based on how many inputs it has with a non-zero importance. This leads to a model which focuses on a limited number of inputs. Therefore the model behaves very mechanistically using only a handful of specific inputs, and usually ends up with quite precise rule-based behaviour. If you regularise too far with an L1 penalty which is too high, you end up with a limited ability to generalise outside a few very specific domains.
I am not saying that this tells us anything about how the particular configuration of neurons in an autistic brain is structured, as if they behaved like an artificial neural net with L1 regularisation. But at a level higher, it seems like a similar kind of effect is happening. Something like L1 regularisation is happening in autistic brains.
This prompts the following questions:
Is this a fundamental explanation of autism or a consequence of a fundamental explanation?
What is the analogue of other types of regularisation, e.g. L2 or dropouts?
What is happening with schitzophrenia, ADHD, and OCD?
Maybe next time we will find out.