The Future of Information Retrieval

A Deep Dive into RAG

Barry S. Stahl

Solution Architect & Developer

@bsstahl@cognitiveinheritance.com

https://CognitiveInheritance.com

Favorite Physicists

1. Harold "Hal" Stahl
2. Carl Sagan
3. Richard Feynman
4. Marie Curie
5. Nikola Tesla
6. Albert Einstein
7. Neil Degrasse Tyson
8. Niels Bohr
9. Galileo Galilei
10. Michael Faraday

Other notables: Stephen Hawking, Edwin Hubble, Leonard Susskind, Christiaan Huygens

Favorite Mathematicians

1. Ada Lovelace
2. Alan Turing
3. Johannes Kepler
4. Rene Descartes
5. Isaac Newton
6. Emmy Noether
7. George Boole
8. Blaise Pascal
9. Johann Gauss
10. Grace Hopper

Other notables: Daphne Koller, Grady Booch, Leonardo Fibonacci, Evelyn Berezin, Benoit Mandelbrot

• Hear my thoughts on why a thriving Fediverse is so critical on David Giard's Technology and Friends show.

• Learn more about the Fediverse at fediverse.party.

• Get your own Mastodon account at Join Mastodon.

  • Don't stress on what server to use, just pick one and go. You can always move later.

• Follow me on Mastodon at @bsstahl@CognitiveInheritance.com.

• Tokenization

  • Break into lower-case tokens
  • best,ways,to,learn,about,my,problem,domain

• Remove Stop Words

  • Words like "to," and "my" are removed
  • best,ways,learn,problem,domain

• Stemming/Lemmatization

  • Reduce tokens to their root form
  • "runner" => "run", "children" => "child"

• Inverted Index Lookup:

  • Find documents from words

• What are Embeddings?
• How does Vector Search leverage Embeddings to find relevant info?
• How does RAG build on top of Vector Search?
• How do we overcome the limitations of these models?
• How do we leverage these tools for the benefit of our Users?

• A point in multi-dimensional space
• Mathematical representation of a word or phrase
• Encode both semantic and contextual information

• Model: text-embedding-ada-002
• Vectors normalized to unit length
• Use 1536 dimensions

Creating Order from Chaos

• Unstructured string => Structured float[]
• Allows mathematical operations
  • Cosine Similarity & Distance
  • Nearest Neighbor Search
  • Clustering
  • Vector Addition & Subtraction
  • Dimensionality Reduction (e.g., PCA)
  • Anomaly Detection

• Interface to language & embedding models
• Fine-tune, evaluate, and integrate models into apps
• Local server for testing models and applications
• Models can be downloaded from HuggingFace

• Local Models

  1. Load Model in LM Studio

  2. Start Local Inference Server

  3. Issue HTTP request for embedding

  4. Retrieve embedding from response

• Remote Models

  1. Deploy model API

  2. Issue HTTP request for embedding

  3. Retrieve embedding from response

Relate vectors based on the angle between them

• Cosine Similarity ranges from -1 to 1, where:

  • +1 indicates that the vectors represent similar semantics & context
  • 0 indicates that the vectors are orthogonal (no similarity)
  • -1 indicates that the vectors have opposing semantics & context

• Cosine Distance is defined as 1 - cosine similarity where:

  • 0 = Synonymous
  • 1 = Orthogonal
  • 2 = Antonymous

Note: For normalized vectors, cosine similarity is the same as the dot-product

• Store/retrieve high-dimensional vectors
• Values are retrieved using similarity searches
• Leverage data-structures such as K-D Trees
• Examples
  • Azure AI Search
  • Redis
  • Qdrant
  • Pinecone
  • Chroma

• "Binary" Search across K-Dimensions
  • O(n log n) for construction
  • O(log n) for search
• Construction
  • Recursively partitioning dataset
  • Rotate the dimension for each depth level
• Nearest Neighbor Search
  • Recursively search for nearest neighbor
  • Backtrack to find additional neighbors

Vector Search: critical to modern information retrieval systems

• Closest vectors to a query vector
  • Identifies embeddings that are similar to the query
  • Requires the same embedding model for query and search vectors
• Works with nearly any type of data
  • Images, text, audio, etc.
• Still a stochastic process
  • May vary between executions
  • Will vary between models
  • May produce unexpected results

• Armando's Resume: 12.7% "worse"
  • More distant from a match to the job listing
  • A 95 for Jonathon's resume ≈ an 84 for Armando's
• If Armando had 2 additional years of experience
  • His score only increases by ≈ 1 point to 85

• Language
  • English (different dialects) vs Spanish
• Historical Ties
  • USA and UK - Political (colonial), economic and cultural ties
• Cultural Similarities
  • USA and UK - shared media, literature, entertainment, etc.
• Political Systems
  • USA and UK - Democracies w/ common roots
• Other differences
  • Economic Ties
  • Geopolitical Alliances
  • Educational Systems
  • Legal Systems

• Clustering
  • Unsupervised machine learning technique
  • Clusters form around centroids (the geometric center)
  • Data points are grouped (clustered) based on similarity
  • Minimize the error (distance from centroid)
• Resume Clustering
  • Identify a group of relevant resumes
  • Not for ranking, just filtering
  • Cluster the resumes, the GPT summaries, or both
  • Still not perfect

• Combines the benefits of retrieval-based and generative models

• Identify and retrieve relevant information

  • Agument context of the generative models
  • Generative responses based on the augmented context

• Potential uses include

  • Explore large documentation sets
  • Summarize articles in light of known relevant information

Don't leave important information "on the table"

• Leverage contextual data to enrich the user experience
  • Ensure interactions are relevant
• Always Maintain high standards
  • User Privacy
  • User Data Protection
  • Consent where appropriate

Use Responsibly - Be careful to respect user privacy

• Time of Day & Week: Routine vs urgent
• Mouse & Eye Movements: Regions of interest
• Device & Platform: Accessibility preferences
• Location: Geographical relevance
• Browsing History: Interests
• Social Media: Personal interests
• Purchases: Preferences and future needs
• Content Consumption Rate: Casual vs focused
• Feedback: Satisfaction and preferences

• Role: Operations Manager
• Location: Tolleson Dealership
• Form of Address: Bentley
• Pronouns: He/Him
• Date Format: American (M/D/Y)
• Time Format: 12-hour (1:45 pm)
• Time Zone: Arizona (MST)
• Info Format: Bullet-Points

Including details of the user's role allows the model to make better predictions about what is important to that user.

• Key Responsibilities
• Primary Goals and Metrics
• Factors that impact decisions
• Common Challenges
• Tools and Technologies Used

Including additional context allows the model to make predictions about how this information might impact the user's activities and experiences

• Location
  • Including local weather
  • Traffic if relevant
• Current Situation
  • Upcoming events and requirements
  • Current state of the network
• Other possibilities
  • Purchase propensity
  • User Survey Information

Allowing the model to make predictions about what information this user most needs to know, we can improve the user's experience and the relevance of our application's content

• Better Awareness of Issues
• Improved Decision-Making
• Greater Efficiency
• Improved User Satisfaction
• Easier Adaptability

• I heard you like 'cooking' so I calculated some recipes from your shopping list so you so you can turn that grocery haul into a Michelin-star meal
• I heard you like 'apple products' so I ordered you a ladder so you can reach those elevated features
• I heard you like coffee, so I scoured local social-media to make you a map so you can find all the best hidden cafes in town
• I heard you like 'JavaScript' so I created a playlist for you composed entirely of loops so you can get into the proper frame-of-mind for coding

• Allow the host agent to determine when and how to use our data
  • Based on our instructions and descriptions
  • Can also be used to take actions on the user's behalf

• This Presentation - Web | PDF
• These Demos - Code | Docs
• The Depth of GPT Embeddings
• Programmers -- Take Responsibility for Your AI’s Output
• Experiments in Reconstructing Text from Embeddings
• Azure OpenAI Client
• Semantic Kernel
• Carl Sagan - Reasoning on Higher Dimensions (YouTube)

• Arrays of 1536 floating-point values
• Numeric data representing unstructured data
• Representations of the semantics and context of data
• Vectors that support standard mathematical operations

Embeddings can be used directly, or as an input to other models

• Direct Usage
  • Measuring Semantic Distance
    • Quantify similarity between pieces of text
    • Useful for tasks like semantic search
  • Clustering for Pattern Discovery
    • Discover groupings in the data
    • Useful when categorizing user comments or other textual characteristics
• Indirect Usage
  • Inputs to traditional ML models
  • Input to Transformer Attention Mechanisms
    • Dynamically adjusted by attention blocks
    • Powers text generation tasks

• Combines the benefits of retrieval-based and generative models

• Identify and retrieve relevant information

  • Agument context of the generative models
  • Generative responses based on the augmented context

• Potential uses include

  • Explore large documentation sets
  • Summarize articles in light of known relevant information

• This Presentation - Web | PDF
• These Demos - Code | Docs
• The Depth of GPT Embeddings
• Programmers -- Take Responsibility for Your AI’s Output
• Experiments in Reconstructing Text from Embeddings
• Azure OpenAI Client
• Semantic Kernel
• Carl Sagan - Reasoning on Higher Dimensions (YouTube)