Inside AI

Josh Rickel

thumbnail_PZcTayoI_400x400.jpg

PZcTayoI_400x400.jpg

## **Introduction**

In an era where artificial intelligence can generate entire codebases and deep fakes are indistinguishable from reality, the landscape of product development is undergoing a seismic shift. At a recent Perigon board meeting, we discussed how these themes reshape our industry. The question on everyone's mind is: How do we navigate this rapidly evolving terrain?

Consider this: By 2025, [the World Economic Forum predicts](https://www.forbes.com/sites/forbestechcouncil/2024/03/12/the-future-of-work-embracing-ais-job-creation-potential/#:~:text=The%20World%20Economic%20Forum's%20Future,between%20humans%2C%20machines%20and%20algorithms.) that 97 million new roles may emerge due to the division of labor between humans, machines, and algorithms. This staggering statistic underscores the urgency of adapting our product development strategies.

In this post, we'll explore nine key themes that influence and revolutionize how startups, product managers, designers, and engineers approach their craft. From AI's transformative power to value attribution's intricacies, we'll delve into the challenges and opportunities ahead.

## Key Themes

### 1. Generative AI: The Innovation Accelerator

The rise of generative AI is the most transformative trend in product development in our lifetime. Tools like OpenAI's GPT-4, Google's Gemini, and Anthropic's Claude are not just assisting in development; they're fundamentally changing how we conceive and execute ideas.

- **Faster Prototyping and Development**: AI can now generate [product requirement documents](https://www.chatprd.ai/), [initial code](https://www.cognition.ai/blog/introducing-devin), [design mockups](https://www.curated.design/), and even complete user stories, dramatically reducing time-to-market.
- **Enhanced Creativity**: AI is a powerful brainstorming partner, offering novel solutions and perspectives that human teams might overlook.
- **Challenges to Address**:
    - We are ensuring the ethical use of AI-generated content.
    - We are maintaining human oversight for quality control.
    - We are balancing AI capabilities with human expertise and creativity.

**Example**: According to a study by GitHub, GitHub's Copilot, powered by OpenAI technology, has increased [developer productivity by up to 55%](https://github.com/features/copilot) for specific tasks.

**Implication for Product Teams**: Integrate AI tools throughout the development process, from ideation to testing. However, companies must establish clear guidelines for AI usage to ensure originality and quality.

### 2. Information Overload and the Trust Crisis

In an age where information is abundant but truth seems scarce, products that can effectively filter, verify, and contextualize data are becoming increasingly valuable.

- **The Scale of the Problem**: The [2024 Edelman Trust Barometer report](https://www.edelman.com/sites/g/files/aatuss191/files/2024-02/2024%20Edelman%20Trust%20Barometer%20Global%20Report_FINAL.pdf) found that 63% of respondents believe government leaders purposely mislead people by saying things they know are false or gross exaggerations. The percentage for business leaders is slightly lower, 61%, and higher for journalists, 64%.


![Paste Image (1).png](https://gawq.s3.us-east-2.amazonaws.com/Paste_Image_1_9bfd442260.png)


- **Impact on Product Development**:
    - Prioritize robust content verification systems.
    - Implement transparent sourcing and fact-checking mechanisms.
    - Develop interfaces that highlight credibility indicators.
- **Example**: Our products emphasize attribution, source citation, and labels that make the sources of information consumed as transparent as possible and put this information front and center in our applications.


![Perigon-Real-Time-AI-Powered-Contextual-Intelligence-Solutions.png](https://gawq.s3.us-east-2.amazonaws.com/Perigon_Real_Time_AI_Powered_Contextual_Intelligence_Solutions_4bd2da1fe0.png)


- **Implication for Product Teams**: Incorporate trust-building features from the ground up. These features could include clear data provenance, user-friendly fact-checking tools, or AI-powered content verification systems.

### 3. The Rise of In-House Development and AI-Driven Customization

The traditional SaaS model faces challenges as more companies build or test custom in-house solutions, leveraging AI and low-code platforms.

- **Driving Factors**:
    - There is a growing desire for greater control and customization.
    - Advancements in low-code and no-code platforms (e.g., Bubble, Webflow) make development more cost and time-efficient.
    - The democratization of AI tools.
- **Example**: Airbnb developed its [AI-powered design system, Lottie](https://airbnb.io/lottie/#/), which is now open-sourced and used by companies worldwide.
- **Implication for Product Teams**: Focus on creating modular, API-first products that can be easily integrated and customized. Provide robust documentation and developer tools to support in-house adaptations.

### 4. Demand for Flexible, Multi-Purpose Enterprise Solutions

As businesses seek to streamline their tech stacks, there's a growing demand for versatile tools that can address multiple needs across departments.

- **Key Considerations**:
    - Design for scalability and cross-functional use.
    - Implement robust integration capabilities.
    - Prioritize UX design that accommodates diverse user roles.
- **Example**: Notion has successfully positioned itself as a flexible workspace for notes, projects, and team collaboration, challenging traditional single-purpose tools.
- **Implication for Product Teams**: Adopt a platform mindset. Design core functionalities that extend and serve various use cases within an organization.

### 5. The Long Tail Market: Democratizing Powerful Tools

There's an emerging opportunity to provide enterprise-grade capabilities to SMBs and individual users, challenging the traditional enterprise-focused model.

- **Opportunities and Challenges**:
    - Develop tiered pricing strategies that are easy to get users in the door.
    - Balance feature sets for diverse user needs.
    - Leverage cloud technologies for scalable, cost-effective solutions.
- **Example**: Canva has successfully brought professional-grade design tools to a broad user base, from individuals to large enterprises.
- **Implication for Product Teams**: Consider how to scale down or modularize your product to serve smaller businesses or individual users without compromising on core functionality.

### 6. The Self-Serve Revolution in SaaS

Users increasingly prefer self-serve experiences in software adoption and usage, impacting everything from sales to customer support.

- **Key Focus Areas**:
    - Develop intuitive, user-friendly interfaces.
    - Create comprehensive, easily accessible documentation.
    - Implement AI-powered chatbots for instant assistance.
- **Example**: Slack's self-serve model allows teams to start quickly, driving rapid adoption and expansion within organizations.
- **Implication for Product Teams**: Prioritize user onboarding and self-help resources. Consider implementing in-app tutorials, contextual help, and AI-powered assistance to support the self-serve model.

### 7. Shifting Focus: From User Metrics to Value Attribution

There's a growing emphasis on demonstrating tangible value and outcomes across organizations, moving beyond traditional user-centric metrics.

- **Key Strategies**:
    - Develop robust analytics and reporting features highlighting ROI.
    - Design products with clear, measurable impact on business processes.
    - Implement features for value tracking and attribution.
- **Example**: Salesforce's [Einstein Analytics](https://www.salesforce.com/au/artificial-intelligence/) provides AI-powered insights that directly tie CRM activities to business outcomes.

**Implication for Product Teams**: Integrate value attribution features into your product, including customizable dashboards, AI-powered predictive analytics, and direct integrations with business intelligence tools.

### 8. The Complexity of Pricing in the AI Era

As AI capabilities become integral to products, traditional pricing models are being challenged, particularly with token-based AI models.

- **Strategies to Consider**:
    - Develop hybrid pricing models balancing predictability with usage-based components.
    - Provide transparency in AI-related costs and usage.
    - Offer tools for customers to monitor and optimize their AI usage.
- **Example**: OpenAI's tiered pricing model for GPT-4, which includes subscription and usage-based elements, illustrates the complexity of pricing AI-powered services.
- **Implication for Product Teams**: Work closely with finance and sales teams to develop flexible pricing models. Consider building usage monitoring and optimization tools directly into your product. Be bold and ask your customers what makes it easy for them so you can find a win-win structure that delivers value to both sides as usage scales.

### 9. AI Benchmarking and Optimization: The New Frontier

As AI becomes central to products, effective benchmarking and optimization systems are crucial for performance and cost management.

- **Key Focus Areas**:
    - Develop robust AI performance metrics and benchmarking tools.
    - Implement continuous optimization algorithms for AI models experimenting with closed and open source models.
    - Create user-friendly interfaces for monitoring and fine-tuning AI performance.
- **Example**: [Google's MLOps tools, such as Vertex AI](https://cloud.google.com/vertex-ai/docs/start/introduction-mlops#:~:text=MLOps%20is%20a%20set%20of,%2C%20diagnosis%2C%20and%20actionable%20explanations.) and [DataDog's LLM Observability](https://www.datadoghq.com/product/llm-observability/) product, provide comprehensive solutions for deploying, monitoring, and optimizing machine learning models in production.
- **Implication for Product Teams**: Integrate AI performance monitoring and optimization features into your product, including dashboards for tracking AI metrics, tools for fine-tuning models, and automated optimization systems.

## Conclusion

The explored themes reflect a more dynamic and complex product development landscape than ever before. At Perigon, we're observing these trends and shaping our strategy to harness their potential.

These trends underscore the need for adaptability, continuous learning, and a holistic approach to product development for product managers, designers, and engineers. The successful products of tomorrow will effectively leverage AI, prioritize user trust, offer flexibility and customization, and deliver measurable value across organizations.

Looking ahead, Perigon is committed to:

1. Integrating AI capabilities throughout our product suite
2. Enhancing our self-serve features and documentation
3. Developing more robust value attribution tools
4. We are exploring innovative pricing models that align with AI-driven functionalities and the value we deliver for our customers, enterprises, and SMBs.

We invite you to join this critical conversation. How are you addressing these challenges in your work? What other trends do you see shaping the future of product development?

Together, let's embrace these challenges and opportunities to drive innovation in our industry, create products that meet current needs, and shape the future of technology.

Navigating the Generative AI Evolution in Product Development: Key Themes and Strategies for Future Success

large_lacey-blog.png

medium_lacey-blog.png

small_lacey-blog.png

thumbnail_lacey-blog.png

lacey-blog.png

Technology and Software Themes Shaping Product Development in 2024

## Introduction

Imagine a world where you have the power of advanced AI at your fingertips without needing external servers or an internet connection and without the fear of compromising your privacy. This is the promise of local large language models (LLMs), whether they are on your phone, laptop, or desktop. This blog post explores why using local LLMs is becoming increasingly common and what it might mean for you.

## Safeguarding Your Data: The Privacy and Security Advantages of Local LLMs

**For Individuals:** In an era where data privacy is paramount, local LLMs offer a secure option. By keeping sensitive information on your device, you maintain control and minimize the risk of data breaches. Additionally, this is the best way to ensure your information isn't used to train proprietary models.

**For Businesses:** Local LLMs provide a more secure solution for enterprises dealing with proprietary data, confidential information, and intellectual property. There will be no question of maintaining confidentiality, and you'll better ensure compliance with stringent data protection regulations*.* For heavily regulated industries, this might be the only way they can fully take advantage of these technologies.

## Tailored to You: The Power of Customization and Fine-Tuning

**For Individuals:** You can fine-tune local LLM models to align perfectly with your unique preferences and writing style. Local LLMs are adaptable to your needs, whether you're a writer crafting compelling stories or a developer working with specific coding languages.

**For Businesses:** Unlock the full potential of AI by customizing local LLMs to fit your industry-specific requirements. Fine-tune models on your proprietary datasets and gain a competitive edge with highly accurate and relevant outputs. From streamlined processes to data-driven decision-making, more tailored local LLMs can unlock unforeseen advantages for your business.

## Lightning-Fast Performance: Unleashing the Speed of Local LLMs

**For Individuals:** Local LLMs can offer significantly faster response times than cloud-based models. Depending on the specific hardware you are using, response time and interaction times can be quicker since there isn't a dependency on sending and receiving information from an outside provider. That said, some high-end cloud-based systems with powerful GPUs can still outperform some local devices.

**For Businesses:** In sectors like finance or healthcare, where split-second decisions can have significant impacts, the speed advantage of local LLMs is a game-changer.

## Always Available: The Resilience and Reliability of Local LLMs

**For Individuals:** Local LLMs function independently of internet connectivity, ensuring you can leverage AI capabilities without a reliable connection. Local LLMs provide uninterrupted access to AI-powered tools, whether in a remote area, on a flight without wifi, or facing network disruptions.

**For Businesses:** With local LLMs, critical operations can continue uninterrupted, even during network outages or cloud service disruptions. Local LLMs offer a stable and reliable alternative for industries like manufacturing, logistics, and emergency services, where continuous operation is vital.

## Sustainable AI: Reducing the Environmental Impact

**For Individuals:** By leveraging existing local hardware, you can reduce your carbon footprint and contribute to more sustainable computing practices.

**For Businesses:** Deploying local LLMs helps reduce reliance on energy-intensive data centers and lower overall carbon emissions, enhancing your company's reputation as an environmentally responsible entity and demonstrating your commitment to sustainable practices.

## Addressing Potential Challenges

While local LLMs offer numerous benefits, they are not without their challenges. Deploying local LLMs may require technical expertise and adequate hardware resources, which might be beyond the realm of knowledge for some individual users. Additionally, as new models get trained, you will have to update them locally. You will also need to consider the hardware you are using - some local models require significant storage space (over 4GB) and a minimum of 16GB of RAM.

Like cloud-based LLMs, local LLMs are trained on large datasets before deployment, and once deployed, they do not require access to vast amounts of data for inference if the data was included in its training. That said, their knowledge is locked in time like other models and [may require access to up-to-date sources of information to answer some queries](https://www.youtube.com/watch?v=zmoWkpW0Lg4&themeRefresh=1). Nevertheless, for most use cases, privacy, customization, and low latency advantages make local LLMs a compelling choice. Currently, no open-source models can compete with the best frontier models offered by providers like [OpenAI](http://www.openai.com/) and [Anthropic](http://www.anthropic.com/).

## Taking the Next Step

If you are ready to embark on your local LLM journey, you can start by exploring popular frameworks like those available on [Hugging Face](https://huggingface.co/). You can download pre-trained models and fine-tune them to suit your needs.

Consider using powerful software tools like [AnythingLLM](https://useanything.com/) or [LM Studio](https://lmstudio.ai/) to make the most of local LLMs. These platforms provide intuitive interfaces and advanced features to help you build and deploy local LLMs effortlessly and create personalized vector databases.

## Conclusion

Local LLMs are not just a passing trend; they represent a paradigm shift in how we harness the power of AI. By prioritizing privacy, enabling customization, offering cost efficiency, and promoting autonomy, local LLMs are transforming the landscape for individuals and businesses.

Discover the benefits of using local large language models (LLMs) for your business. Learn how they enhance privacy, performance, and control over your data.

large_Monochromatic Laptop Search.jpg

medium_Monochromatic Laptop Search.jpg

small_Monochromatic Laptop Search.jpg

thumbnail_Monochromatic Laptop Search.jpg

Monochromatic Laptop Search.jpg

Top Reasons to Consider Using Local LLMs

Islem Maboud

large_Islem Maboud Headshot SMALL.jpg

medium_Islem Maboud Headshot SMALL.jpg

small_Islem Maboud Headshot SMALL.jpg

thumbnail_Islem Maboud Headshot SMALL.jpg

Islem Maboud Headshot SMALL.jpg

## Introduction

With recent AI advancements and the introduction and daily evolution of new large language models, AI significantly influences how we interact, consume information, work, and search for information online. However, like all technologies, AI also has limitations that hinder us from fully harnessing its potential.

## LLM limitations

Large language models are fascinating, trained on massive data, and know almost anything. The main two problems of LLMs are:

1. **They do not have access to real-time information.**
2. **They are prone to [hallucinations](https://www.digitalocean.com/resources/article/ai-hallucination).**

When we want to introduce a new knowledge base to an LLM, to make it up-to-date, we have to fine-tune or train the model from scratch with the latest data, and both of these methods require a lot of resources and time. 

On the other hand, hallucination is a side-effect of when the LLM doesn’t have the knowledge or when the training data is biased to answer your prompt, so it starts making things up that are either false or do not exist, which can lead to the generation of false or misleading information. You can learn more about AI hallucinations from DigitalOcean by reading this [post](https://www.digitalocean.com/resources/article/ai-hallucination).

## Introducing RAG

RAG stands for **Retrieval-Augmented Generation**. It's an artificial intelligence technique used to solve two previous problems of LLMs. RAG can be added to any LLM (ChatGPT, LLaMa, Mistral…), taking its simple architecture of providing the missing context by looking at a vector database and feeding the context to the LLM through a specific prompt template.

We've recently created a new YouTube video explaining how RAG works and how to build a simple RAG using LangChain. You can watch it from [here](https://youtu.be/zmoWkpW0Lg4).

<iframe width="100%" height="392" src="https://www.youtube.com/embed/zmoWkpW0Lg4" title="RAG Explained with Simple Examples in 10min" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

## How RAG Works?

In simple terms, RAG is an architecture you add to your LLM pipeline to provide the model with the proper up-to-date context (fetched from your database) to answer the user’s prompt correctly.

Depending on how your current pipeline is structured, where you fetch the up-to-date context data, and how you process it, you can build an RAG with many architectures and variations. 

Let’s look at a simple RAG architecture diagram from when the user sends a prompt to receive a response.

![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/b8795f2c-3d5a-4e73-9ecb-046f94e02ad3/c84e7fa4-0f6a-4cd9-95cb-a8decf854885/Untitled.png)

Here is the diagram explained in simple steps:

1. The user sends a request/prompt to our servers.
2. The prompt passes through the RAG implementation for semantic searching of the context database, where the up-to-date data resides. A vector database is used for the semantic search.
3. The prompt and retrieved context are provided to the LLM using a specific template to ensure optimal response and utilization of the provided context, allowing the LLM to respond accurately to the prompt.
4. The LLM response is returned to the user (or might be used for another processing task).

## Building a RAG with LangChain

[LangChain](https://python.langchain.com/v0.2/docs/introduction/) is a framework for developing and deploying applications powered by large language models (LLMs). It streamlines development with open-source tools, simplifies productionization through monitoring and evaluation, and facilitates deployment by turning any chain into an API using LangServe.

The LangChain framework will help us assemble the pieces to build our RAG architecture.

### Prerequisites

Before starting, ensure you have the required libraries installed. Install them using the following commands:

<CodeSyntax>
```bash
pip install openai langchain nest_asyncio playwright html2text sentence-transformers qdrant-client requests
pip install groq python-dotenv langchain-groq langchain-openai
```
</CodeSyntax>

If this is your first time using Playwright on your machine, run the following command to install a Chromium-compatible browser:

<CodeSyntax>
```bash
playwright install
```
</CodeSyntax>

### Step 1: Prepare the Environment

First, we must load the environment variables from a `.env` file. This file should contain your API keys for services like OpenAI, Perigon, and Groq.

<CodeSyntax>
```python
from dotenv import load_dotenv
import os

load_dotenv()
```
</CodeSyntax>

### Step 2: Load Data

1. **Load data from the web**

Let's grab the data from the web using Playwright and transform it into a format suitable for processing. This step is optional since you should already have the data you want to use as a context for feeding it into the LLM. This is just a simple example of fetching the latest news and data.

We want to fetch the latest information about the upcoming (June 1, 2024) UEFA Champions League final match for Borussia Dortmund vs. Real Madrid.

<CodeSyntax>
```python
from langchain.text_splitter import (
    CharacterTextSplitter,
    RecursiveCharacterTextSplitter,
)
from langchain.document_loaders import AsyncChromiumLoader
from langchain.document_transformers import Html2TextTransformer

def scrape_documents_from_web():
    articles = [
        "https://www.uefa.com/uefachampionsleague/news/028d-1af2b2b6fa83-105b56c607b6-1000--champions-league-final-preview-dortmund-vs-real-madrid-wh"
    ]

    loader = AsyncChromiumLoader(articles)
    docs = loader.load()

    html2text = Html2TextTransformer()
    docs_transformed = html2text.transform_documents(docs)

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=0, separators=[" ", ",", "\\n"]
    )
    chunked_documents = text_splitter.split_documents(docs_transformed)

    return chunked_documents
```
</CodeSyntax>


1. **Load data using an API**

Also, you can fetch articles via an API provider like Perigon.

The Perigon API allows you to fetch up-to-date, real-time news and article data powered by AI and contextual intelligence. In this case, we can easily query using the `q` field for `Dortmund> vs Real Madrid 2024` to get the latest articles. Then, we pre-process and convert them into LangChain-compatible documents ready for embedding.

Please provide your valid Perigon API key to access the API. You can find your API key in the [dashboard](https://www.goperigon.com/account/api-key).

<CodeSyntax>
```python
import requests
from langchain.docstore.document import Document

def get_documents_from_perigon():
    url = "<https://api.goperigon.com/v1/all?q='Dortmund> vs Real Madrid 2024' AND 'Dortmund vs Real Madrid line-ups'"

    payload = {}
    headers = {"x-api-key": os.environ.get("PERIGON_API_KEY")}

    response = requests.request("GET", url, headers=headers, data=payload)

    articles_transformed = [
        article["content"] for article in response.json()["articles"]
    ]
    docs_transformed = [
        Document(page_content=article_content)
        for article_content in articles_transformed
    ]

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=0, separators=[" ", ",", "\\n"]
    )
    chunked_documents = text_splitter.split_documents(docs_transformed)

    return chunked_documents
```
</CodeSyntax>

### Step 3: Embed Documents into the Vector Database

**Embedding** is a technique used in natural language processing (NLP) and machine learning to convert textual data into numerical representations. These representations capture the semantic meaning and context of the text.

We use a vector database like [Qdrant](https://qdrant.tech/) to store embedded documents as vectors, which makes it easier for the RAG to fetch the appropriate context for the received prompt. 

<CodeSyntax>
```python
from langchain.vectorstores import Qdrant
from langchain.embeddings.huggingface import HuggingFaceEmbeddings

import nest_asyncio

nest_asyncio.apply()

docs_from_web = scrape_documents_from_web()
docs_from_perigon = get_documents_from_perigon()

db = Qdrant.from_documents(
    docs_from_web,
    HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2"),
    location=":memory:",  # Local mode with in-memory storage only
    collection_name="my_documents",
)

retriever = db.as_retriever(search_type="similarity", search_kwargs={"k": 4})
```
</CodeSyntax>

We take all the data documents we pre-processed before and embed them into our Qdrant database.

### Step 4: Initialize the LLM Model of choice

Choose an LLM model, such as LLaMa (e.g., using Ollama), Groq LLaMa, or OpenAI.

<CodeSyntax>
```python
from langchain_community.llms import Ollama
from langchain_groq import ChatGroq
from langchain_openai import OpenAI

llama3_llm = Ollama(model="llama3-8b-instruct")
groq_llama3_llm = ChatGroq(
    temperature=0.4,
    groq_api_key=os.environ.get("GROQ_API_KEY"),
    model_name="llama3-8b-8192",
)
openai_llm = OpenAI(openai_api_key=os.environ.get("OPENAI_API_KEY"))
```
</CodeSyntax>

If you prefer running your local LLM model, Ollama is recommended for quickly downloading and serving many open-source LLMs (e.g., LLaMa 3). For example, we are pointing Ollama to use the `llama3-8b-instruct` model, which is fine-tuned for instructions.

### Step 5: Testing Prompt without RAG

Attempt to generate a response without using RAG.

<CodeSyntax>
```python
from langchain.prompts import ChatPromptTemplate

system = "You are a helpful assistant."
human = "{text}"
prompt = ChatPromptTemplate.from_messages([("system", system), ("human", human)])

chain = prompt | groq_llama3_llm
query = "Where to watch, match date and possible line-ups for Dortmund vs Real finale"

for s in chain.stream(query):
    print(s.content, end="", flush=True)
```
</CodeSyntax>

This is the response that you would get.

> The highly anticipated UEFA Champions League Final between Borussia Dortmund and Real Madrid!
> 
> 
> Here's the essential information you need to know:
> 
> - *Match Details:*
> - Date: Saturday, May 28, 2023
> - Kick-off time: 21:00 CEST (Central European Summer Time)
> - Venue: Atatürk Olympic Stadium, Istanbul, Turkey
> - *Where to Watch:*
> 
> The UEFA Champions League Final will be broadcast in many countries worldwide. You can check the official UEFA website for a list of broadcasters and streaming options in your region.
> 
> In the United States, you can watch the match on CBS Sports Network and stream it on Paramount+. In the UK, it will be broadcast on BT Sport 2 and streamed on the BT Sport app.
> 
> - *Possible Line-ups:*
> 
> Please note that these line-ups are subject to change and may not reflect the actual starting XI for the match. Here are some possible line-ups for both teams:
> 
> - *Borussia Dortmund:*
> - Goalkeeper: Gregor Kobel
> - Defenders: Nico Schulz, Mats Hummels, Manuel Akanji, Thomas Meunier
> - Midfielders: Jude Bellingham, Thomas Delaney, Axel Witsel, Julian Brandt
> - *Real Madrid:*
> - Defenders: Dani Carvajal, Éder Militão, Raphaël Varane, Ferland Mendy
> - Midfielders: Casemiro, Luka Modrić, Toni Kroos, Federico Valverde
> - Forwards: Karim Benzema, Vinícius Júnior, Rodrygo Goes

If you look at the [official UEFA article](https://www.uefa.com/uefachampionsleague/news/028d-1af2b2b6fa83-105b56c607b6-1000--champions-league-final-preview-dortmund-vs-real-madrid-wh), you’ll know that everything the LLM stated above is wrong and outdated (since it doesn’t have the latest knowledge).

### Step 6: Preparing Prompt Template for RAG

To provide the fetched context regarding the user’s prompt, we must prepare a prompt template that includes the main instruction, the retrieved RAG context, and the user’s original question (prompt).

<CodeSyntax>
```python
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
from langchain.schema.runnable import RunnablePassthrough

prompt_template = """
### [INST]
Instruction: Answer the question based on your latest knowledge

{context}

### QUESTION:
{question}

[/INST]
"""

prompt = PromptTemplate(
    input_variables=["context", "question"],
    template=prompt_template,
)

llm_chain = LLMChain(llm=groq_llama3_llm, prompt=prompt)
```
</CodeSyntax>

### Step 7: Testing Prompt with RAG

Now, let’s see our RAG implementation in action, giving it the same prompt as before, asking about `Where to watch, match date, and possible line-ups for Dortmund vs Real Madrid final`

<CodeSyntax>
```python
from IPython.display import display_markdown

rag_chain = {"context": retriever, "question": RunnablePassthrough()} | llm_chain

query = "Where to watch, match date and possible line-ups for Dortmund vs Real Madrid final"

for s in rag_chain.stream(query):
    display_markdown(s['text'], raw=True)
```
</CodeSyntax>

And yes, as you may have already guessed, the LLM returned the correct, up-to-date answer about the match:

> According to the article, here are the answers to your question:
> 
> 
> **Where to watch the Champions League final:** Fans can find their local UEFA Champions League broadcast partner(s) here.
> 
> **Match date:** Saturday 1 June.
> 
> **Possible line-ups:**
> 
> **Dortmund:** Kobel; Ryerson, Hummels, Schlotterbeck, Maatsen; Can, Sabitzer; Adeyemi, Brandt, Sancho; Füllkrug
> 
> **Real Madrid:** Courtois; Carvajal, Éder Militão, Rüdiger, Mendy; Valverde, Kroos; Camavinga; Bellingham; Rodrygo, Vinícius Júnior
> 

If you double-check the [official UEFA article](https://www.uefa.com/uefachampionsleague/news/028d-1af2b2b6fa83-105b56c607b6-1000--champions-league-final-preview-dortmund-vs-real-madrid-wh), you’ll notice that everything is correct, from the match date to the possible line-ups 😎

RAG is amazing, isn’t it?

## Project Source Code

You can access the source code of the implemented RAG above in this [repo](https://github.com/goperigon/simplified-rag).

## Conclusion

Following these steps, you should have a working RAG system that augments your LLM's question-answering capabilities with relevant context retrieved.

Please feel free to explore and expand upon this foundation to develop more sophisticated information retrieval and generation systems!

RAG: The Key to Enhancing AI's Real-Time Information and Reducing Hallucinations.

large_rag-explained-for-beginners

medium_rag-explained-for-beginners

small_rag-explained-for-beginners

thumbnail_rag-explained-for-beginners

rag-explained-for-beginners

RAG Explained - Building a RAG using Python, LangChain, Ollama and Groq

## Introduction

It's clear that a significant shift is underway online: we are transitioning from the Information Age, where data accessibility reigned supreme, to the Knowledge Age, where AI is making data more valuable and actionable. The [recent announcements at Google I/O](https://arc.net/l/quote/lpnkmrjt) and by [OpenAI](https://openai.com/index/spring-update/) underscore that information accessibility no longer meets consumer expectations. We've reached a point where systems can proactively seek relevant information and transform data for specific purposes to support our actions, decisions, and curiosity. The promise of these new technologies is the freedom to focus on what we do best as humans - creativity, strategy, reasoning, and social connection.

### The Broken Web: Navigating the Information Age's Pitfalls

The web's initial magic lay in its expansive access to information, removing barriers to connection and enabling the widespread distribution of ideas. Those of us old enough can nostalgically recall the melodic beeping of our 14k modems signaling access to a vibrant world of discovery, where our Encyclopedia Britannica and Encarta could no longer keep pace. However, this also led to a tsunami of content. [This content explosion spawned history's most profitable business models](https://www.forbes.com/sites/bernardmarr/2023/03/14/the-7-most-successful-business-models-of-the-digital-era/?sh=2b76971e5617), from search to social platforms. The smartphone era further accelerated this trend, enabling 24/7 content creation from anywhere and introducing the addictive lure of "the feed."

While vibrant, this creator economy brought many challenges downstream social concerns:

- Information surplus and overload
- The rapid spread of misinformation on social media
- Difficulty discerning quality and truth amidst noise and influence
- Sensationalized feeds designed to maximize attention
- Inbox clutter from spam, newsletters, and ads
- A broad increase in mental health concerns

These challenges have led some to say that the "[web was broken](https://www.nytimes.com/2024/03/13/opinion/internet-web-3-tech.html)." Tim Berners-Lee, who initially proposed the concept of the World Wide Web, said that the web was originally intended "to be a tool to empower humanity" and "was decentralized with a long tail of content and options." Since then, he has written that "instead of embodying these values, the web has played a part in eroding them." All is not lost, however.

## Welcome to the Knowledge Age

The introduction of AI might be just the catalytic technology to help restore some of the web's potential to be "a tool to empower humanity." I believe it has the potential to transition us from the Information Age to the Knowledge Age, where we can quickly seek out answers, feed our curiosity, distill insights from the noise, better identify content designed to mislead or manipulate and establish a new relationship with our digital world. This new age prioritizes relevance and efficiency over monetization, transforming unfathomable amounts of data into meaningful insights versus sponsored links and algorithmically delivered results. AI can parse datasets, identify patterns, predict trends, and offer recommendations, interpreting more in a matter of seconds than we could hope to consume in a lifetime with diverse expertise and objectivity. Combining that with the ability to take action from what it shares with us, AI unlocks unprecedented value.

### The Role of User Experience in the Transition

The way we interact with the web is also changing. We aren’t bound by text input into a search box. [AI is multimodal](https://www.theverge.com/2024/5/20/24159258/microsoft-recall-ai-explorer-windows-11-surface-event), unlocking more natural human interactions. Voice and vision opportunities to provide AI with contextual information to better understand the user's world and setting, enabling more natural ways to process data, interpret requests, and prescribe actions. This contextual awareness allows AI to be more helpful in delivering the answers we seek or take actions we direct.

AI's integration into existing workflows creates more intuitive, interactive experiences that support decision-making rather than simply presenting data that require further research and analysis. That said, this comes with the burden of consistent reliability and trust. While AI can now autonomously handle complex tasks, boost efficiency and effectiveness across business intelligence, increase personal productivity, and more, it must build trust before we empower it to do all it may do.

### Cautious Optimism for the Knowledge Age

While optimistic, I fully recognize that AI is creating new paths to further manipulation, fueling disinformation, and making it even easier to generate synthetic content, including voice impersonation, deep image and video fakes, and supporting generation. With every new technological innovation, there will always be bad actors. We must ensure that the benefits outweigh the costs and that we consciously incorporate appropriate safeguards for these technologies while balancing a push forward.

In addition to content concerns, the Knowledge Age fractures the web's business models, mainly digital advertising and search. As someone who has spent much of my life in and around the media business, I recognize that this change will fundamentally rewrite the internet's economics, with far-reaching consequences for publishers and content creators.

The traditional media landscape, which has long relied on advertising revenue, may face significant challenges as AI becomes an intermediary between users and content through search-generated experience (SGE). This inevitably raises the question: will we reach a point where we rarely need to dive deeper into the actual content to confirm AI-generated insights? If so, what does this mean for journalism's future and content creators' role in an AI-driven world?

Lastly, the shift to AI-powered knowledge systems will likely require new sacrifices, acceptance, and expectations regarding privacy in the pursuit of more personalized experiences. As aptly stated in [a recent article](https://nymag.com/intelligencer/article/google-ai-overviews-search-engine.html#/) by John Herrman*,* "These are companies whose existing businesses were built on the acquisition, production, and monetization of large amounts of highly personal data about their users; meanwhile, the secret to unlocking the full, glorious potential of large-language-model-based AI, at the level of the personal assistant or in service of achieving machine intelligence, is, according to the people building them, simply more access to more data." This point highlights the inherent tension between our desire for personalized, AI-driven experiences and the need to protect individual privacy.

As we navigate this new landscape, it will be essential to strike a balance between harnessing the power of AI and safeguarding user data. Transparency and user control over their information will be critical in building trust and fostering widespread adoption of AI-driven knowledge systems. Additionally, we must consider the potential impact on content creators and develop new models that ensure their contributions are valued and rewarded in this new age.

Ultimately, while the AI-driven Knowledge Age promises to revolutionize how we access and utilize information, we must approach this transition with a clear understanding of the costs and challenges involved.

## Perigon's Place in the Knowledge Age

Where does Perigon fit in this? We play a vital role in this shift by equipping AI with dynamic, real-time information access. With this, [AI can generate responses based on current, dynamic information from a broad set of sources, building trust](https://www.goperigon.com/blog/can-ai-search-live-up-to-its-hype-the-critical-role-of-contextual-intelligence). Transparency and control are critical, as users want visibility into what informs AI outputs. As we build applications on our API, users will gain access to information and reliable, timely, and actionable insights.

## Conclusion

In the Knowledge Age, AI will help us extract relevant, timely signals from noise, delivering personalized, efficient, and helpful information. It can fix some of the World Wide Web's underlying problems and restore it to Berners-Lee's original vision, where access to information fuels critical decision-making.

Perigon is committed to leveraging cutting-edge technologies to transform raw data into knowledge and empower users to be better critical thinkers. By proactively addressing these concerns and working towards solutions that benefit all stakeholders, we can unlock the full potential of AI-powered knowledge systems while preserving the integrity of our digital landscape.

Transitioning from the Information Age to the Knowledge Age with AI: Empowering Humanity through Enhanced Digital Experiences.

large_Untitled-min.jpeg

medium_Untitled-min.jpeg

small_Untitled-min.jpeg

thumbnail_Untitled-min.jpeg

Untitled-min.jpeg

From Information to Knowledge: AI's Role in Restoring the Web's Original Vision

Media

## Introduction

Artificial intelligence (AI) is significantly transforming our view of creativity, raising questions about its impact on creative thought and expression. With tools like [ChatGPT](https://chatgpt.com/) for text, [DALL-E](https://openai.com/index/dall-e-3/), [Midjourney](https://www.midjourney.com/home), [Imagen 3](https://deepmind.google/technologies/imagen-3/) and [Firefly](https://www.adobe.com/products/firefly.html) for images, [Sora](https://openai.com/index/sora/) and [Veo](https://deepmind.google/technologies/veo/) for video, and [Suno](https://suno.com/), [Udio](https://www.udio.com/) and [Google’s Music AI Sandbox](https://youtu.be/-dPqc7l2zu8) for audio, AI is reshaping how we create content and the time it takes to produce it. In addition, it’s becoming much easier to incorporate this collaboration into our existing workflows with [the release of GPT-4o, a native application for MacOS, and the availability of GPT-4 for all ChatGPT users](https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-free/). This post delves into the implications of AI on human creativity, explores the balance between innovation and authenticity, the democratization of creative tools, and the potential for a new era of human-AI creative collaboration.


## The AI-Creativity Paradigm Shift

### Understanding AI's Role in Creativity

AI tools [are revolutionizing the creative process by making artistic expression more accessible to individuals with minimal training](https://hbr.org/2023/07/how-generative-ai-can-augment-human-creativity). This democratization allows people who previously lacked access or training to create, iterate, and distribute high-quality content. For instance, despite not being trained in Python, I’ve built simple applications using AI in the past year. Similarly, without expertise in video creation or graphic design, I can produce high-quality images within moments. This shift has the potential to foster a more inclusive creative landscape, where diverse voices and perspectives can flourish.

However, the ease of creation may lead to more homogenized content, influenced by the data and inputs used to train these AI models. Balancing the benefits of democratization with the need to maintain diversity and originality is a critical challenge for the future of creativity. AI models are built on extensive datasets of historical text, images, video, and audio, sparking ethical debates about originality, legality, and the evolving value of human-created content. These discussions prompt us to redefine what constitutes human creativity.

![chatgpt.png](https://gawq.s3.us-east-2.amazonaws.com/Untitled_ff479746e1.png)


Historically, creativity has been viewed as a slow, deliberate process, with time and iteration being essential to developing unique and imaginative works. AI’s ability to generate content swiftly challenges this notion, potentially diminishing the perceived value of the creative process. However, creativity can also be seen as a form of compound interest, where ideas build upon each other over time, gaining complexity and value. In this analogy, AI accelerates the accumulation and transformation of ideas, potentially enriching the creative landscape if managed thoughtfully. Yet, it is important to remember that the true driver of compound interest is patience.

Art, music, and film often draw inspiration from the visionaries and innovators of the past, whose styles we model or derive from. We pay homage to these creators through imitation or by putting a unique spin on their original works, which were likely inspired by others. AI tools may allow us to leverage the genius of these past creators more rapidly, but does this shortcut diminish our understanding and tribute to their vision?

**Human Agency and Authenticity**

A central concern is maintaining human agency and authenticity in a world dominated by AI-generated content. Starting with AI might mean ceding significant influence over the final product. While AI can produce technically proficient and visually appealing works, there is a risk that the unique perspectives and cultural diversity inherent in human creativity may be muted. Ensuring that AI-generated content reflects a wide range of perspectives and cultural backgrounds is essential to preserving the richness of human creativity. This requires models trained in multiple languages, diverse authors, and a broad array of mediums, incorporating not just basic vision and hearing, but also an understanding of the context and inspiration behind each piece.

## The Future of Human + AI Creativity

### New Ideas Inspired by AI

AI has the potential to innovate beyond human imagination. This is exemplified by its unique contributions, such as the [strategic moves in the game of Go that surprised even seasoned experts](https://www.scientificamerican.com/article/ais-victories-in-go-inspire-better-human-game-playing/), [developing new approaches to gene editing](https://www.nature.com/articles/d41586-024-01243-w#:~:text=This), or [millions of new materials](https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/). These capabilities suggest that AI can offer new perspectives and methods that may not be immediately apparent or scalable by human creators.

<iframe width="100%" height="392" src="https://www.youtube.com/embed/6ydG1mqID-s" title="&quot;Right There&quot; (Music AI Demo) - Wyclef Jean" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

### Human-AI Collaboration

AI has the potential to significantly enhance human creativity, offering new tools and expanding the boundaries of what is possible. [By embracing AI collaboration and iteration, human creators can explore new horizons and alternative paths](https://www.sps.nyu.edu/homepage/emerging-technologies-collaborative/blog/2023/embracing-creativity-how-ai-can-enhance-the-creative-process.html) while experimenting with ideas and concepts that might previously have been difficult (even impossible) or cost prohibitive.

<iframe width="100%" height="392" src="https://www.youtube.com/embed/dKAVFLB75xs" title="Filmmaking with Donald Glover and his creative studio, Gilga | Veo" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

### Expansion of Creative Contributions

Copyright law, which protects original works of authorship, incentivizes creators to produce unique content rather than simple derivative works. Copyright ensures that creators retain control over their original expressions and benefit from their creative efforts. We can’t simply recycle existing concepts; instead, [we can use AI to assist in the creative process to generate genuinely new ideas and contributions to the creative landscape](https://www.reuters.com/legal/legalindustry/copyright-conundrum-protection-ai-works-2023-11-28/). The techno-optimist in me hopes this will foster a richer and more diverse array of creative works by allowing more people to explore and make unique contributions to the world. This approach honors the craftsmanship and unique contributions of artists, recognizing their invaluable role in enriching our society.

## Conclusion

AI has the potential to revolutionize creativity but must be balanced with human agency and authenticity. By pushing the boundaries of what is possible, AI offers unprecedented opportunities for innovation and expression. However, it is essential to embrace AI as a tool for enhancing creativity while advocating for responsible and ethical development practices. While AI can accelerate creativity and unlock new paths, the value of thoughtful, deliberate creation and iteration remains paramount.


Exploring the Balance Between Human Innovation and Machine Efficiency

medium_DALL·E May 14 Sketch (1).webp

small_DALL·E May 14 Sketch (1).webp

thumbnail_DALL·E May 14 Sketch (1).webp

DALL·E May 14 Sketch (1).webp

Creativity in an AI First World

Using Perigon

## Introduction

Artificial Intelligence (AI) is evolving at a remarkable pace, reshaping how we interact with technology and make decisions. What was once a simple co-pilot approach, where humans directed technology, has evolved into more autonomous "autopilot" functionality or what some are now calling “agentic.” In this landscape, AI is assuming greater responsibilities, shifting our roles from active participants to agent managers, editors, and conductors. At Perigon, we see this shift as an opportunity to connect next-generation agents with the right information, equipping them to take actions or perform analyses with comprehensive perspectives and insights.

## From Assistants to Agents: The Evolution of AI

AI technology is progressing from being reactive assistants to proactive agents, capable of autonomously handling a broad set of tasks. It’s become increasingly common for us to bring one-off tasks to ChatGPT, Claude, or MetaAI for support. While assistants respond to these user commands in a very transactional way, [agents are designed to act independently](https://aws.amazon.com/what-is/ai-agents/), using machine learning to anticipate user needs, understand multi-step operations, and ultimately proactively take action based on a desired outcome. These agents aren't just booking reservations or finding availability for meetings; they're now interpreting desired goals, laying out a path to achieve them independently, and addressing strategic challenges.

## The Agentic Mindset: Proactive Intelligence in Action

An AI agent or series of agents can interpret a concept for new software, write the code, formulate a marketing plan, recommend pricing, and even suggest a name and build the website. This comprehensive approach, integrating a variety of activities, brings AI closer to achieving much more impactful results. Moreover, AI agents learn from human feedback—without becoming defensive—on their decisions and can evolve to enhance their performance. With this approach, agents can quickly become integral parts of collaborative decision-making processes and streamline workflows.

A standout feature of AI agents is their ability to be more personalized and proactive. By analyzing user behavior, preferences, and historical data, these agents tailor recommendations and actions based on what has worked previously or what they have learned from human feedback. Additionally, they can uncover insights or strategies that humans might have otherwise overlooked. The value of these agents lies in their efficiency—saving time, identifying hidden patterns, and reinforcing feedback.

For a deeper dive into the practical application of these concepts, consider the insights from Joon Sung Park, as discussed [in a recent Forbes article on building generative AI agents](https://www.forbes.com/sites/joannechen/2024/02/20/how-to-build-generative-ai-agents-with-joon-sung-park/?sh=5c87ffa6137b). This resource offers valuable perspectives on the complexities and innovative strategies involved in developing AI systems that can autonomously perform a variety of tasks.

## Enhancing Decision-Making through Tailored Intelligence

Imagine a real estate investor receiving tailored alerts about local market trends, a new headquarters being announced or changes to local laws. Maybe a CEO receives daily briefing that synthesizes and distills information from news, podcasts, and financial reports that sparks a potential strategic partnership, makes here aware of a big customer’s strategic shift, or alerts her to an emerging competitor gaining traction. This could take a human hours, every day, to parse through, interpret, write, and update. These agents can quickly consume information, alert and brief people in the style they prefer, answer follow-up questions, help them evaluate a decision, and  adapt future collaborations accordingly. In business, these agents enhance productivity and decision-making by providing insights and recommendations across different departments and functions, freeing up time for other creative and strategic tasks.

## Building Trust and Oversight: The Path to Autonomy

While AI agents hold great promise, trust remains a concern. We're not ready to take our hands completely off the wheel, but with close oversight and exploration, we can gradually trust AI to act on our behalf. Currently, we're comfortable with AI making recommendations and describing its next steps. If we agree, we might allow it to act—maybe in the future, automatically. However, for now, keeping a human in the loop is crucial. This oversight allows AI to learn how humans might react in various situations and to build trust. Additionally, we want to understand what knowledge an AI agent relied upon to do its work.

A live example of this is [Tesla's revised strategy for full self-driving](https://www.autoweek.com/news/a46535912/tesla-fsd-ai-neural-networks-update/). Previously, they had taken an approach to hard code decisions which led to some ethical and moral debates. Now, with the growing power of compute, they can take an alternative approach that has already proved to be much more effective, where the AI makes decisions based on how good drivers have reacted in similar circumstances based on thousands of hours of recorded video captured by Teslas in the real world.

We want AI to do what we would do or find valuable, while also discovering new insights we might have missed. By incorporating human oversight and feedback, AI agents can learn and improve, building trust and enhancing autonomy.

To further understand the nuances of building trust in AI systems, [this article](https://hbr.org/2023/11/how-companies-can-build-trustworthy-ai-assistants) from Harvard Business Review offers an in-depth look at how companies can create trustworthy AI assistants. This resource explores methods and practices that ensure AI systems are reliable and transparent, key factors that are essential for fostering user trust. 

## Perigon's Vision: Making Information Actionable

Every day at [Perigon](https://www.goperigon.com/about), we work to make the world’s information more useful and actionable. We provide better insights, eliminating noise and focusing on relevance with intentional applications of artificial intelligence and machine learning. [Our technology](https://www.goperigon.com/products/news-api) aligns with an agentic mindset, equipping AI with the knowledge to be relevant, timely, and trustworthy, empowering humans to make informed decisions with ease. We bridge the gap between data and action, enhancing daily decision-making through intelligent, context-driven insights.

## Conclusion

The future of AI is rapidly evolving, with agents becoming increasingly intelligent and autonomous. This shift towards an agentic mindset in an AI-first world promises to revolutionize our interaction with technology, enhancing productivity, personalization, and innovation. At Perigon, we're excited to be at the forefront of this transformation, enhancing decision-making through superior insights.

AI evolves from assistants to proactive agents, handling tasks autonomously and anticipating user needs.

medium_robot-human-handshake-min.png

small_robot-human-handshake-min.png

thumbnail_robot-human-handshake-min.png

robot-human-handshake-min.png

What is Agentic AI?

## Introduction

As anticipation builds around an [AI-driven search engine from OpenAI](https://www.theinformation.com/articles/openai-develops-web-search-product-in-challenge-to-google?rc=nhuxem) and questions grow about [how search generative experience (SGE) may upend Google's search dominance](https://searchengineland.com/search-generative-experience-impacting-ads-437989), it’s crucial to understand both the transformative potential and the nuanced challenges these technologies face. Unlike traditional search engines, AI-driven generative search offers unique capabilities that could redefine information retrieval, discovery and understanding. This is particularly relevant in an era where the volume and complexity of available information continue to grow exponentially.


## Understanding How Search Impacts Answers

[AI excels in managing complex, buried information queries](https://www.theverge.com/24111326/ai-search-perplexity-copilot-you-google-review) that demand a deep understanding beyond simple keyword matches—these queries involve understanding how events, individuals, companies, regulations all connect and require heavy emphasis on recency and temporal relevance. Some early AI products still struggle with this. Here’s an example where both OpenAI and Perplexity fail to identify current regulations limiting AirBnBs in Amsterdam.

![screenshot_2024-05-02_at_9.17.04_am.png](https://gawq.s3.us-east-2.amazonaws.com/screenshot_2024_05_02_at_9_17_04_am_0926f47f98.png)

![capture-2024-05-03-103406.png](https://gawq.s3.us-east-2.amazonaws.com/capture_2024_05_03_103406_3750b277ec.png)


In fact, Perplexity’s first citation is **from 2 years ago**.

![capture-2024-05-03-103520.png](https://gawq.s3.us-east-2.amazonaws.com/capture_2024_05_03_103520_0c97512e21.png)

An AI search experience that is powered by contextual intelligence will capture this nuance and inform the user.

![capture-2024-05-03-103913-min.png](https://gawq.s3.us-east-2.amazonaws.com/capture_2024_05_03_103913_min_02e21d6e18.png)

Despite these advantages, AI also faces notable challenges. Particularly with navigational searches where users seek specific websites or products. This gets to the old adage “if it's not broken don’t fix it.” AI isn’t necessary to take you to your email inbox or to the latest NFL scores. Traditional search nails this. This highlights the need for AI search to categorize queries effectively into navigational, general informational, or deep exploratory types. Each category demands a different strategic response, ensuring AI search applications optimize their strengths and mitigate their weaknesses. This doesn’t mean that we won’t use the same interface for navigation, it just means that this experience might not need to be reimagined like deeper discovery searches.

## Why Keyword Search Doesn’t Work for AI

The current paradigm for keeping AI up to date or filling knowledge gaps with contextual understanding is browsing the internet. This exposes AI search to SEO manipulation from the results of keyword browsing. SEO manipulation of search engine results is nothing new and with [the increased use of AI is only increasing](https://www.theverge.com/2024/3/5/24091099/google-search-high-quality-results-spam-ai-content).

To address these challenges, AI search must:

- **Overcome Algorithmic Manipulations**: AI-driven search must advance beyond traditional keyword indexing to understand the broader context of queries. This involves developing algorithms and techniques capable of discerning genuine content relevance from SEO-enhanced visibility. This ensures that the information retrieved has authority, is current, and is critical and valuable for the response rather than what the manipulated rankings dictate.
- **Leverage Trusted Sources**: AI can significantly benefit from a programmed preference for information from trusted, high-authority sources. This mimics the human approach to research, where credibility plays a crucial role in source selection, thereby enhancing the reliability and credibility of AI search results. Furthermore, giving the user this control based on a specific inquiry can unlock a more collaborative approach to AI search.
- **Identify Non-Obvious Connections**: One of AI’s most potent capabilities is its ability to detect subtle, non-obvious connections between data points. This allows AI to offer novel insights and uncover relationships that disrupt traditional search methods, providing a richer, more comprehensive understanding of query topics.


## Harnessing Contextual Intelligence

At Perigon, we leverage Contextual Intelligence to enhance AI’s ability to understand and fill knowledge gaps with the most pertinent information for each inquiry. By [applying vector databases, retrieval augmented generation (RAG)](https://www.goperigon.com/data-solutions/use-cases/ai-llms), and our proprietary machine learning algorithms to [hundreds of thousands of public data sources daily](https://www.goperigon.com/products/news-api), we ensure that our responses are not only timely and relevant but also deeply insightful. This approach allows us to deliver responses that demonstrate a sophisticated understanding of user intent and the contextual relevance of the information. Experience this first hand [through our chat experience](https://www.goperigon.com/ai-chat). 


## Conclusion

The future of AI search and SGE depends heavily on integrating advanced contextual intelligence to transcend the limitations of traditional search engines. By deepening AI's understanding of query context and improving its ability to assess content relevance and authority, we can significantly enhance the effectiveness of AI-driven search solutions. Perigon is at the forefront of this exciting frontier, pioneering new ways to drive utility and action in an AI-first world.


Exploring the application of AI in search to revolutionize data retrieval from conventional methods to more sophisticated and refined approaches.

medium_dall__e_may_3_header.webp

small_dall__e_may_3_header.webp

thumbnail_dall__e_may_3_header.webp

dall__e_may_3_header.webp

Can AI Search Live Up to Its Hype? The Critical Role of Contextual Intelligence

## Intro to Perigon

Perigon specializes in analyzing, distilling, and connecting vast amounts of dynamic information and breaking news, which we call **"contextual intelligence"**. Our technology is designed to enhance both consumer and business applications by providing high-fidelity, structured data that is globally covered and refined by AI, making it suitable for a variety of users including startups, major corporations, researchers, and government entities. Take a look at the [About](https://www.goperigon.com/about) page to learn more about the company.

![About Perigon.png](https://gawq.s3.us-east-2.amazonaws.com/About_Perigon_442e4f40e0.png)

> **Perigon analyzes and distils more than 1,000,000+ articles per day from more than 145,000 different sources.**

## Account Setup

We recently made our new user onboarding process public for everyone, so now, you can easily create an account and receive free API Requests to try out Perigon API. To get started, [create a new user account](https://www.goperigon.com/sign-up) and confirm your email address (an email with a confirmation link will be sent to your email inbox after account creation). You can now [login](https://www.goperigon.com/sign-in) using your account credentials.

As part of the Perigon trial, you will receive **15 days** of free trial and a limit of **150 API requests**. 

### Access your API Key

An API key is required for authenticating with the Perigon API and getting responses from the different available endpoints! Once you're logged into your account, you can view and copy your API key from your account [dashboard](https://www.goperigon.com/account/api-key). 

> Please note, that your email needs to be verified before your API key will work.

![image (20).png](https://gawq.s3.us-east-2.amazonaws.com/image_20_df00aa3f20.png)

## Perigon API

The Perigon API serves as the portal through which you can easily access global data via simple API requests.

### API Authentication

Perigon supports three types of authentication, to access the API, you need to make sure to provide the API key with each HTTP request. Here are the different types that you can use to authenticate:

1. As the `?apiKey=<API_KEY>` query string parameter within the URL.
2. As the `Authorization: Bearer <API_KEY>` header.
3. As the `x-api-key: <API_KEY>` header.

Make sure to replace `<API_KEY>` with your API key which you can *copy and paste* from your [dashboard](https://www.goperigon.com/account/api-key).

Here is an example of how to put the API Key on the URL:

<CodeSyntax>
```javascript
https://api.goperigon.com/v1/articles/all?sortBy=date&title="Rivian R2"&apiKey=<API_KEY>
```
</CodeSyntax>

You can check out the [Authentication](https://docs.goperigon.com/docs/getting-started#authentication-) Docs Page to learn more.

### API Endpoints

Perigon has 6 public endpoints, that allow you to access the world's data through contextual Intelligence with AI.

The base URL for all endpoints is: `https://api.goperigon.com`.

1. **[Articles](https://docs.goperigon.com/docs/overview) - [`/v1/people/all`](https://docs.goperigon.com/reference/all-news)** 
This endpoint offers features for searching and filtering through all news articles accessible via the API. The outcome comprises a collection of individual articles that meet your specified criteria.

2. **[Stories](https://docs.goperigon.com/docs/stories-overview) - [`/v1/stories/all`](https://docs.goperigon.com/reference/stories-1)**
The /stories endpoint filters related articles for top or trending stories by topics, categories, countries, and locations, with pagination and sorting. Results feature story details, metrics on articles, and updated statistics.

3. **[People](https://docs.goperigon.com/docs/people-data) - [`/v1/people/all`](https://docs.goperigon.com/reference/people)** Get more info on individuals in Perigon's database, sourced from Wikidata. Our database has 650,000+ people worldwide, refreshed often. It includes a wikidataId for further lookup on Wikidata's site.

4. **[Sources](https://docs.goperigon.com/docs/searching-sources) - [`/v1/sources`](https://docs.goperigon.com/reference/sources-media)** The Perigon Sources API offers an endpoint for searching and fetching extra details about the 130,000+ media sources supported by our APIs. More sources are being added frequently.

5. **[Journalists](https://docs.goperigon.com/docs/journalist-data) - [`/v1/journalists`](https://docs.goperigon.com/reference/journalist-detail)** Perigon Journalist API provides endpoints for accessing extra journalist details, like their title, Twitter handle, bio, and location. Lookup journalists using their ID or broader search attributes. Our database has over 230,000 journalists worldwide, refreshed regularly.

6. **[Companies](https://docs.goperigon.com/docs/company-data) - [`/v1/companies/all`](https://docs.goperigon.com/reference/companies)** Search for additional details on companies in Perigon's entity database, referenced in article responses. Our database holds over 80,000 public and private companies worldwide, refreshed often, and sourced from public records and Wikidata.

You can check the full list of the endpoints with an API playground to test it on right on your browser on the [API Reference](https://docs.goperigon.com/reference/all-news) page.

### Postman API Playground

We created a Postman collection with all the public endpoints that are available to use from Perigon, this will help you see all the available endpoints in action with examples and documentation of every available parameter for each of the endpoints.

[<img src="https://run.pstmn.io/button.svg" alt="Run In Postman" style={{width: "128px", height: "32px"}}/>](https://app.getpostman.com/run-collection/32876362-1613d2e8-0bd8-4a6e-bf7c-bf5e4545bddf?action=collection%2Ffork&source=rip_markdown&collection-url=entityId%3D32876362-1613d2e8-0bd8-4a6e-bf7c-bf5e4545bddf%26entityType%3Dcollection%26workspaceId%3D6bb53bc3-e5a0-4660-a432-caac93d0a03d)

Once you click on the **"Run In Postman"** button, it will ask you to fork the Perigon collection to start making API calls. If the above button doesn't work for you, you can access the Postman collection through this [link](https://www.postman.com/goperigon/workspace/perigon-public/collection/32876362-1613d2e8-0bd8-4a6e-bf7c-bf5e4545bddf?action=share&creator=32876362). Please make sure to sign in using your Postman account to be able to view and make API calls in the *Perigon Public workspace*.

## Working with the API

Perigon's sophisticated API empowers developers to construct excellent applications with ease, as it provides JSON-encoded data in the response for efficient management and processing of comprehensive news updates using advanced contextual intelligence. Let's learn how you can make your first API request and the different parts of it.

### Making your first API call

When you can query the world's data from an API, there are a lot of ideas you can explore and get help with. Since Electric Vehicles are a trend this year, especially with the new Rivian EVs releases and hype, let's make an API call to get the recent stories about "electric vehicles".

<CodeSyntax>
```javascript
https://api.goperigon.com/v1/stories/all?q="electric vehicles"&expandArticles=true&apiKey=<YOUR_API_KEY>
```
</CodeSyntax>

We are using the stories endpoint to get the recent timeline of stories about **"electric vehicles",** which we put as the query parameter, and set `expandArticles=true` to preview the first 5 articles from the stories cluster. Make sure to pass in your API key either as a **apiKey** query parameter or using the other supported [Authentication methods](https://docs.goperigon.com/docs/getting-started#authentication-).

Here's a sample of the response JSON, the response is truncated, showing only one preview article instead of 5 (for the sake of making this more readable).

<CodeSyntax showLineNumber={true} maxHeight="800px">
```javascript
{
  "status": 200,
  "numResults": 4607,
  "results": [
    {
      "createdAt": "2024-03-26T10:56:57.318197+00:00",
      "updatedAt": "2024-03-26T16:52:54.419274+00:00",
      "initializedAt": "2024-03-26T16:48:58.098581+00:00",
      "id": "7421411190754f14a71190be0934a852",
      "name": "Toyota to Manufacture Electric Hilux in Thailand by 2025",
      "summary": "Toyota Motor Corp plans to commence mass production of an electric Hilux pickup truck by the end of 2025, spearheading a growing momentum toward electric vehicle adoption in Thailand. The plans follow Isuzu Motors' recent declaration to manufacture its battery-powered D-MAX pickup in Thailand by 2025, signifying a push by Japanese automakers to assert leadership amidst the rise of electric vehicles in the region.",
      "summaryReferences": [],
      "keyPoints": [],
      "selectedArticles": [
        {
          "url": "https://finance.yahoo.com/news/toyota-plans-mass-produce-battery-104914997.html",
          "authorsByline": "Reuters",
          "articleId": "314d78491fff4a6ab91b7d6c608473f8",
          "clusterId": "7421411190754f14a71190be0934a852",
          "source": {
            "domain": "finance.yahoo.com",
            "location": null
          },
          "imageUrl": "https://media.zenfs.com/en/reuters-finance.com/5dc69f9a0320e8d78185cd09d054df59",
          "country": "us",
          "language": "en",
          "pubDate": "2024-03-26T10:49:14+00:00",
          "addDate": "2024-03-26T11:10:49.731303+00:00",
          "refreshDate": "2024-03-26T11:10:49.731306+00:00",
          "score": 1.0,
          "title": "Toyota plans to mass produce battery Hilux pickup truck by 2025",
          "description": "Japan's Toyota plans to mass produce a battery electric Hilux pickup truck by the end of 2024, the president of its Thailand unit, Noriaki Yamashita, told reporters on the sidelines of the Bangkok International Motor Show. In Thailand, Toyota expects to sell 250,000 vehicles this year, he said. He did not say where the Hilux truck would be made.",
          "content": "BANGKOK (Reuters) - Japan's Toyota plans to mass produce a battery electric Hilux pickup truck by the end of 2024, the president of its Thailand unit, Noriaki Yamashita, told reporters on the sidelines of the Bangkok International Motor Show.\n\nIn Thailand, Toyota expects to sell 250,000 vehicles this year, he said. He did not say where the Hilux truck would be made.",
          "medium": "Article",
          "links": [],
          "labels": [],
          "matchedAuthors": [],
          "claim": "",
          "verdict": "",
          "keywords": [
            {
              "name": "battery Hilux pickup truck",
              "weight": 0.1613403
            },
            {
              "name": "Noriaki Yamashita",
              "weight": 0.12375605
            },
            {
              "name": "the Bangkok International Motor Show",
              "weight": 0.11771043
            },
            {
              "name": "a battery electric Hilux pickup truck",
              "weight": 0.10707974
            },
            {
              "name": "Hilux",
              "weight": 0.10379325
            },
            {
              "name": "Thailand",
              "weight": 0.102032855
            },
            {
              "name": "reporters",
              "weight": 0.098259225
            },
            {
              "name": "the Hilux truck",
              "weight": 0.08051304
            },
            {
              "name": "Toyota",
              "weight": 0.07688871
            },
            {
              "name": "its Thailand unit",
              "weight": 0.0690571
            }
          ],
          "topics": [],
          "categories": [
            {
              "name": "Auto"
            }
          ],
          "entities": [
            {
              "data": "Toyota",
              "type": "ORG",
              "mentions": 3
            },
            {
              "data": "Reuters",
              "type": "ORG",
              "mentions": 1
            },
            {
              "data": "Hilux",
              "type": "PRODUCT",
              "mentions": 3
            },
            {
              "data": "BANGKOK",
              "type": "GPE",
              "mentions": 1
            },
            {
              "data": "Japan",
              "type": "GPE",
              "mentions": 1
            },
            {
              "data": "Thailand",
              "type": "GPE",
              "mentions": 2
            },
            {
              "data": "Noriaki Yamashita",
              "type": "PERSON",
              "mentions": 1
            },
            {
              "data": "the Bangkok International Motor Show",
              "type": "EVENT",
              "mentions": 1
            }
          ],
          "companies": [
            {
              "id": "1a841c614e014e3da7fcba199f1e28a0",
              "name": "Toyota UK",
              "domains": ["toyota.co.uk"],
              "symbols": []
            }
          ],
          "sentiment": {
            "positive": 0.5155631,
            "negative": 0.044742495,
            "neutral": 0.4396944
          },
          "summary": "Toyota plans to mass produce a battery electric Hilux pickup truck by 2025, according to the president of its Thailand unit, Noriaki Yamashita. This announcement comes as Toyota plans to sell 250,000 vehicles in Thailand this year alone, with the target of 250,500 in total. Yamashiti did not specify where the Hilux truck would be made.",
          "translation": "",
          "translatedTitle": "",
          "translatedDescription": "",
          "translatedSummary": "",
          "locations": [
            {
              "country": "th",
              "state": "10"
            }
          ],
          "reprint": true,
          "reprintGroupId": "f99d8d8666b64218b7577869fd0c593c",
          "places": [],
          "people": []
        }
      ],
      "sentiment": {
        "positive": 0.5555712,
        "negative": 0.055060733,
        "neutral": 0.38936806
      },
      "uniqueCount": 5,
      "reprintCount": 14,
      "totalCount": 19,
      "countries": [
        {
          "name": "us",
          "count": 10
        },
        {
          "name": "sg",
          "count": 2
        },
        {
          "name": "gb",
          "count": 2
        },
        {
          "name": "in",
          "count": 1
        },
        {
          "name": "th",
          "count": 1
        },
        {
          "name": "ca",
          "count": 2
        }
      ],
      "topCountries": ["us"],
      "topics": [
        {
          "name": "Markets",
          "count": 2
        }
      ],
      "topTopics": [
        {
          "name": "Markets"
        }
      ],
      "categories": [
        {
          "name": "Auto",
          "count": 19
        },
        {
          "name": "Business",
          "count": 4
        },
        {
          "name": "Finance",
          "count": 2
        }
      ],
      "topCategories": [
        {
          "name": "Auto"
        }
      ],
      "people": [],
      "topPeople": [],
      "companies": [
        {
          "id": "1a841c614e014e3da7fcba199f1e28a0",
          "name": "Toyota UK",
          "domains": ["toyota.co.uk"],
          "symbols": [],
          "count": 19
        },
        {
          "id": "4abf84190e76494d89fc757f69f0f022",
          "name": "Isuzu Truck South Africa",
          "domains": ["isuzu.co.za"],
          "symbols": [],
          "count": 1
        },
        {
          "id": "b5f3525b8af942f395b25af76d242b2e",
          "name": "BYD Company Limited",
          "domains": ["byd.com"],
          "symbols": ["1211.HK", "002594.SZ", "BY6.F", "BYDDF", "BYDDY"],
          "count": 2
        },
        {
          "id": "ea3755a3201c4fc99889242e26c2117e",
          "name": "Toyota Motor Corporation",
          "domains": ["global.toyota"],
          "symbols": ["7203.T", "TYT.L", "TM"],
          "count": 2
        }
      ],
      "topCompanies": [
        {
          "id": "1a841c614e014e3da7fcba199f1e28a0",
          "name": "Toyota UK",
          "domains": ["toyota.co.uk"],
          "symbols": []
        }
      ],
      "locations": [
        {
          "state": "10",
          "count": 7
        },
        {
          "state": "NCO",
          "count": 6
        },
        {
          "state": "20",
          "city": "Pattaya City",
          "count": 1
        }
      ],
      "topLocations": [
        {
          "state": "10"
        },
        {
          "state": "NCO"
        }
      ]
    }
  ]
}

```
</CodeSyntax>

The response has many crucial fields and each one helps you explore more details about the stories, for instance, the `name` field is for the main theme of the story, the `summary` field is for getting a quick AI generated summary about the current story and it's articles, `selectedArticles` array is the preview list of the 5 first articles for the current story.

## Example App built with Perigon API

To showcase how you can utilize the Perigon API to build real-world web apps, we have created a sample React app to render the most recent stories about **Electric Vehicles (EVs)** around the glob. The app is hosted on codesandbox to make it easier for anyone to play around with the Demo without the need to run the code locally.

To view the demo app, you need to enter your Pergion API key in the `<YOUR_API_KEY>` input.

<iframe
  src="https://codesandbox.io/embed/tht47h?view=Editor+%2B+Preview&module=%2Fsrc%2FApp.tsx"
  style={{ width: "100%", height: "500px", border: 0, borderRadius: "4px", overflow: "hidden" }}
  title="perigon-stories-sample-app"
  allow="accelerometer; ambient-light-sensor; camera; encrypted-media; geolocation; gyroscope; hid; microphone; midi; payment; usb; vr; xr-spatial-tracking"
  sandbox="allow-forms allow-modals allow-popups allow-presentation allow-same-origin allow-scripts"
></iframe>

**Application structure & how it works**:
1. As soon as you enter your Perigon API key into the input, an API request will be made with your API key to fetch recent stories querying about "Electric Vehicles".
2. All stories will be displayed as a grid layout.
3. If you click on a story card, it will take you to the story timeline page where the 5 recent articles (`story.selectedArticles` array) for the selected story will be previewed on a timeline sorted by recent *article publication date* (`article.pubDate`).
4. Each article card on the timeline shows relevant information about the article. **Article title**, **description**, **article country**, **author** and **related companies**. All of this data was returned by the Perigon API for each of the selected articles.
5. If you click on the Article card, you will be redirected to the **article web page** for reference.

## Conclusion

In summary, Perigon offers advanced contextual intelligence by analyzing over 1,000,000 daily articles from diverse sources worldwide. With an intuitive user onboarding process, users can access the Perigon API for a 15-day trial period, enabling exploration of its capabilities. The API provides easy access to global data through various endpoints, including articles, stories, people, sources, journalists, and companies. A sample React app demonstrates the practical use of the API by showcasing recent stories on Electric Vehicles. Perigon's technology facilitates informed decision-making and innovation across industries.

Everything you need to know to get started with Perigon and using the AI powered API

large_marvin-meyer-SYTO3xs06fU-unsplash.jpg

medium_marvin-meyer-SYTO3xs06fU-unsplash.jpg

small_marvin-meyer-SYTO3xs06fU-unsplash.jpg

thumbnail_marvin-meyer-SYTO3xs06fU-unsplash.jpg

marvin-meyer-SYTO3xs06fU-unsplash.jpg

Getting Started with Perigon API

Linzi Nunes

large_PXL_20230705_172553506.PORTRAIT~2 (1).jpg

medium_PXL_20230705_172553506.PORTRAIT~2 (1).jpg

small_PXL_20230705_172553506.PORTRAIT~2 (1).jpg

thumbnail_PXL_20230705_172553506.PORTRAIT~2 (1).jpg

PXL_20230705_172553506.PORTRAIT~2 (1).jpg

A reflection of our 72-Hour Hackathon

large_IMG_0671.HEIC

medium_IMG_0671.HEIC

small_IMG_0671.HEIC

thumbnail_IMG_0671.HEIC

IMG_0671.HEIC

From Zero to MVP: How ChatGPT Guided Our C-Suite's Rapid Tech Transformation

Radhika Rajkumar

thumbnail_008A2427 Small.jpeg

008A2427 Small.jpeg

How to find the best news API for your needs 

large_misinformation—1920x1080.png

medium_misinformation—1920x1080.png

small_misinformation—1920x1080.png

thumbnail_misinformation—1920x1080.png

misinformation—1920x1080.png

What to Know When Choosing a News API

Our VP of Engineering on the future of coding with AI 

medium_The Wire - Inside AI - Gold (1).png

small_The Wire - Inside AI - Gold (1).png

thumbnail_The Wire - Inside AI - Gold (1).png

The Wire - Inside AI - Gold (1).png

Inside AI: Interview with Linzi Nunes

Press Releases

Our first news product partnership

medium_The Wire - Inside AI - Gold-1 (1).png

small_The Wire - Inside AI - Gold-1 (1).png

thumbnail_The Wire - Inside AI - Gold-1 (1).png

The Wire - Inside AI - Gold-1 (1).png

James Borton Launches South China Sea NewsWire with Perigon News API

Monitor your operations on the ground with news and other event data

large_supply chain illo—1920x1080.png

medium_supply chain illo—1920x1080.png

small_supply chain illo—1920x1080.png

thumbnail_supply chain illo—1920x1080.png

supply chain illo—1920x1080.png

How News Data Can Inform Supply Chain Logistics

Automatically convert search terms into 25 languages

large_S-Wide.png

medium_S-Wide.png

small_S-Wide.png

thumbnail_S-Wide.png

S-Wide.png

Using Search Translation

Perigon VP of Product on building safe, reliable news tools with AI

medium_The Wire - Inside AI - Gold.png

small_The Wire - Inside AI - Gold.png

thumbnail_The Wire - Inside AI - Gold.png

The Wire - Inside AI - Gold.png

Inside AI: Interview with Zach Bartholomew

Misinformation

What to know about AI-generated content

medium__1b5c2c22-481c-4984-94f5-377175e3a4ed.jpeg

small__1b5c2c22-481c-4984-94f5-377175e3a4ed.jpeg

thumbnail__1b5c2c22-481c-4984-94f5-377175e3a4ed.jpeg

Bing_Image_Creator_62923.jpeg

What is Synthetic Media?

Our founder & CEO on the impact of the AI revolution, and what he hopes it brings us

medium_Inside AI_Template

small_Inside AI_Template

thumbnail_Inside AI_Template

Inside AI_Template

Inside AI: Interview with Joshua Dziabiak 

What we deliver and why it matters

large__70e1bee4-ecb8-436d-8d0c-77b3f02a7d53.jpeg

medium__70e1bee4-ecb8-436d-8d0c-77b3f02a7d53.jpeg

small__70e1bee4-ecb8-436d-8d0c-77b3f02a7d53.jpeg

thumbnail__70e1bee4-ecb8-436d-8d0c-77b3f02a7d53.jpeg

_70e1bee4-ecb8-436d-8d0c-77b3f02a7d53.jpeg

What is Contextual Intelligence?

Context (and people) are about to become even more important

medium_AI-MEDIA-PERIGON-ILLO

small_AI-MEDIA-PERIGON-ILLO

thumbnail_AI-MEDIA-PERIGON-ILLO

AI-MEDIA-PERIGON-ILLO

Why the Future of News Needs Perigon

A new knowledge graph for smarter and faster content search

small_Wire-Entities.png

thumbnail_Wire-Entities.png

Wire-Entities.png

About Entities

Find themes, events, top stories and trending headlines using the Stories endpoint

small_feedback_graphic

thumbnail_feedback_graphic

feedback_graphic

Using the Stories API

Jimmy Pribble

small_20220127_113153.jpg

thumbnail_20220127_113153.jpg

20220127_113153.jpg

You may have heard the term "fake news.” The term is becoming increasingly prevalent, but what does it mean? The basic definition of fake news is - a false or misleading news story that is published and spread deliberately with the intent to deceive. These news stories might contain some truth, but often without verifiable sources, supporting quotes, or attribution. In other cases, the stories might be completely fabricated.

Fake news is often used as nothing more than obvious clickbait to get readers to click on a link to a news article to generate more page views, without any intent to harm. For example, satire and parody news sites can be considered fake news. However, fake news can be used as part of a larger disinformation campaign used to influence and harm. To further complicate matters, the term fake news has itself become politicized and is often used to dismiss or discredit legitimate news stories that support an opposing viewpoint.   

Fake news has existed for as long as people have been able to communicate with each other. But today it's easier than ever before to spread misinformation quickly and globally thanks to the internet and social media. 

In recent years, the spread of fake news has been exacerbated by the popularity of social media in recent years. According to a study conducted by researchers at MIT, fake news is 70 percent more likely to be retweeted on Facebook and Twitter than accurate news. Co-author Sinan Aral, a professor at the MIT Sloan School of Management, says, *"We found that falsehood diffuses significantly farther, faster, deeper, and more broadly than the truth, in all categories of information, and in many cases by an order of magnitude."*  The study shows that we tend to believe things that fit with our existing beliefs or with what we think other people believe. This means that if your friends share articles from sites you trust like CNN or Fox News, then those stories are probably going to seem trustworthy and accurate—even if they're not actually true! The more people that see a story, the more likely it is to be believed—and vice versa.

### How to Spot Fake News

Fake news is a big problem and that means we all need to learn how to spot fake news if we want to make sure our information remains accurate and trustworthy. With so much false information floating around the internet on so many topics, how can you know what's real and what's not?

- Check the source. Who's publishing it? Are they a credible news organization or a source with an agenda?
- Look for a balanced perspective. If you're reading something that seems one-sided, look for another viewpoint on the topic.
- Check if any supporting evidence is given to back up claims being made in the article—and whether that evidence appears to be accurate and unbiased. Look for quotes from authorities or experts on the subject.
- Consider who benefits from this story going viral; often fake news stories have an underlying agenda or political motive behind them (for example: clickbait articles about celebrities or sports teams who 'hate' each other). This suggests that whoever wrote the piece has an interest in getting as many people talking about their story as possible—so perhaps think twice before sharing!
- Use multiple sources. When you're trying to figure out if something is true or not, look for multiple sources that say the same thing. If it's just one source saying it and no one else, there's a good chance that they could be wrong, or at the very least their source might be flawed. By looking at more than one source, you can get a better idea of what is true and what isn't.
- Use watchdog and fact-checking sites, such as: FactCheck.org, Politifact.com, ProPublica.org, and Snopes.com.

Fake news is a growing problem that has become an important topic in our political discourse. While it may seem difficult to identify fake stories, there are ways to tell what's real from what isn't and it is our responsibility to not participate in the spreading of harmful fake news.


You may have heard the term "fake news.” The term is becoming increasingly prevalent, but what does it mean? 

A magnifying glass highlighting fake news.

large_iStock-1179996571.jpg

medium_iStock-1179996571.jpg

small_iStock-1179996571.jpg

thumbnail_iStock-1179996571.jpg

iStock-1179996571.jpg

What is Fake News?

How to use and understand Perigon's sentiment analysis data

small_Disney Stories graph

thumbnail_Disney Stories graph

Disney Stories graph

About Sentiment Analysis

Data Reports

The Perigon Data Studio