training_phases.txt

Phase 1: The Digital Anatomy (Architecture Setup)
Before teaching the child, we must build the brain. A single HTM layer is not enough for language mastery; you need a Deep Hierarchical HTM Network.

The Input Sensor (The Retina for Text):

Objective: Convert raw text into Sparse Distributed Representations (SDRs).

Mechanism: Do not use semantic encoders (like Word2Vec) pre-trained by others. That is "cheating" by giving the brain adult knowledge.

Action: Create a Scalar/Category Encoder for raw ASCII/Unicode characters. The brain sees "A", "b", "c" as raw signals, not knowing they form words.

The Cortical Hierarchy:

L1 (V1 - Primary Cortex): Receives character SDRs. Handles spelling and character transitions.

L2 (Association Cortex): Receives output from L1. Recognizes "chunks" (morphemes/words).

L3 (Higher Order Cortex): Receives output from L2. Handles grammar and sentence structure.

L4 (Prefrontal/Semantic Cortex): Handles long-term coherence, paragraphs, and "meaning."

Phase 2: Infancy (Orthography & Pattern Recognition)
Goal: Learn the "physics" of text. Pass the Spelling Bee.

The Curriculum:

Data: Feed the model purely random character streams at first, then simple phonetically consistent gibberish, then simple word lists.

Spatial Pooling (SP) Training:

The L1 Spatial Pooler must learn to form stable representations of frequent character combinations (n-grams like "ing", "th", "qu").

Analogy: This is the baby babbling and realizing that "T" and "H" often go together.

Temporal Memory (TM) Training:

Train L1 TM to predict the next character.

Milestone: The model should be able to autocomplete words. If you input "A-P-P", it predicts "L-E".

Spelling Bee: Freeze the L1 layer. Test it on unseen words. It should flag "anomaly" on misspelled words because they violate the temporal patterns it learned.

Phase 3: Toddlerhood (Morphology & Simple Syntax)
Goal: distinct word recognition and basic 2-3 word sentences.

The Curriculum:

Data: "Baby books." Dr. Seuss, nursery rhymes, high-repetition simple sentences ("The cat sat.", "Run dog run.").

Hierarchy Activation:

Pass the active cells or prediction error from L1 up to L2.

L2 begins to see a sequence of characters ("D-O-G") as a single stable pattern (an Object).

Concept Formation:

L2 learns that "The" is usually followed by a noun. It doesn't know what a noun is, but it learns the statistical structure.

Milestone: The model expects a word category. If you say "The cat runs," and "The dog runs," it learns that "cat" and "dog" are functionally similar (synaptic overlap).

Phase 4: Childhood (Grammar & Rule Acquisition)
Goal: Fluency in sentence structure and basic context.

The Curriculum:

Data: Elementary school textbooks, Wikipedia articles (Simple English), structured grammar exercises.

Feedback Loops (Apical Dendrites):

This is crucial for "Understanding." In standard htm.core, data flows up. In a real brain, expectation flows down.

Connect L3 (Grammar) back down to L2 (Words).

L3 predicts the type of phrase coming next (e.g., a verb phrase). This biases L2 to recall verbs, narrowing the search space.

Milestone: The model can generate grammatically correct sentences that it has never seen before, simply by following the hierarchical rules it has learned.

Phase 5: Adulthood (Semantics, Reasoning & Creativity)
Goal: Long-term dependency, semantic grounding, and beating the benchmark.

The Curriculum:

Data: Complex literature, philosophy, scientific papers, debate transcripts.

Semantic Folding:

The L4 layer operates on very slow-changing patterns. It represents the "Topic" or "Intent" of a paragraph.

If the topic is "Cooking," L4 sends feedback signals down that prime words like "knife," "pan," and "boil" in L2/L3, while suppressing words like "algorithm" or "tire."

Creativity (The Generative Mode):

To be creative, you unclamp the input.

You stimulate a high-level concept in L4 (e.g., the pattern for "Melancholy").

The brain lets the feedback loops cascade down, generating a sequence of words that fits that high-level abstract concept.

This differs from LLMs (which predict the next token). Your HTM will predict the concept, then the grammar, then the word, then the characters.

Phase 6: The Tuning & Benchmarking
Goal: Prove superiority.

Anomaly Detection as "Confusion": Use HTM's anomaly score to measure "perplexity." A lower anomaly score on complex, unseen text means better understanding.

The Turing Test:

Generate a paragraph.

Ask: "Does this sentence follow a logical chain of thought?" (HTM excels at maintaining state over time, whereas Transformers have a fixed context window).