Wow it’s a real boost of creativity to read this.

Some thoughts about the different systems. Regarding System 1, I think that this can bring to a new trend of weighting samples in a dataset by “importance". If it has a capacity of X samples, we need to choose them carefully and leave the rest for System 2.

Also, given a sample, how do you decide which system is more appropriate? I guess you try system 1, and depending on confidence of the prediction we can hand it over to the next system. Even though this could be parametrised too, which leads be to a hybrid with reinforcement learning of some sort.

I wonder if system 2 can be somewhat “AI’sh” too, database retrieval that is indexed with/by embeddings from layer 1.

I totally agree with the approach and the changing tides, especially due to my NLP experience and the effort it takes to get to NLU from masked neural language models.

Very interesting, thank you for posting!