Simple Example where Statistics Reduces / Erases Data

Average of 2,3,10 is 5.

Standard practices records the 5 and "forgets" the data samples that were utilized to calculate 5.

Initially the "10" sample looks like an outlier.

If next sample is 11 then 10 no longer looks like an outlier.

Associative Memories and SDR - Sparse Data Representations

Associative Memories

Complex domains require humans to help discover new relations, find existing relations or find similar relations within diverse information sources.  Information sources might include text, audio, images, video, social networks, simulations, sampled sensor systems (IOT) and/or correlations across all these types.  Humans are good correlation engines.


Numenta's, Jeff Hawkins co-founder has written "On Intelligence" a small book separated into three primary sections: (1) theory of brain structure, (2) evidence supporting brain structure theory and (3) applications that can be built if humans follow this design.  Jeff Hawkins utilizes the acronym, HTM - Hierarchical Temporal Memory as Numenta's primary data structure that mimics the associative memory correlation capability within human brains.  Numenta has open sourced some of it's source code as a means for further exploration of these concepts.  Cortical.io is a technical partner to Numenta and has gained considerable experience of their own in memory based applications.



Saffron Tech, a competitor to Numenta utilizes compressed multiple linked matrices to represent their associative memory capabilities.  Their api documentation can be seen here:
https://saffrontech.atlassian.net/wiki/display/DOC/Saffron+Documentation+10.x
The Boeing team worked with Saffron Tech over many years to help them create a data driven, web enabled configuration utility that allows for ease of mapping existing data sources into an associative memory:





Utilizing these commercial based infrastructures from my personal experience at Boeing, designing complex, domain specific solutions to help resolve complex problems can be resolved with less effort than traditional software infrastructures and/or compared to "writing your own from scratch" associative memory infrastructure.

An open source, implementation called S-Space for Semantic Space provides a rudimentary implementation for Natural Language Processing.
https://github.com/fozziethebeat/S-Space


Google, of course, also has something like an associative memory embedded within their infrastructure that we utilize on a daily basis when we type in queries and "it" suggests adjacent queries.

Co-mentions within books also reveals interesting relations between characters:


Many implementations utilize Sparse Data Representations



Many software infrastructures now contain SDR data structures including Scala, Python, Java, C, C++, R - you just need to know where  to look; that will be another post.