Perplexity in AI tools

Amazon announced a new generative AI tool that makes it easier for sellers to write engaging, effective product listings by providing a URL to their website and having the AI generate a product description.

A team of former Twitter engineers, led by Sara Beykpour and Marcel Molina, have launched a new startup called Particle.news. Particle is an AI-powered news reader that aims to provide a personalized, multi-perspective news experience by using AI to summarize news articles.

Microsoft's increasing reliance on automation and ai over human editors to curate its homepage has led to the amplification of false and bizarre stories on its homepage, including conspiracy theories and obituaries that describe nba players as useless.

Pharmaceutical company Moderna announced a partnership with OpenAI to leverage generative AI for drug discovery and development. The AI models will be used to analyze biological data and generate new drug candidate molecules.

Snowflake released their SnowCat language model tailored for querying and analyzing data in their cloud data platform.

Inkline.ai, A new fast minimalist AI writing tool. It can used by almost anybody for whatever type of writing you would like. I am currently using this tool for my creative writing work.

AI models like DeepCRISPR, CRISTA, DeepHF are being used to predict optimal guide RNAs (gRNAs) for CRISPR systems by analyzing factors like genomic context, protein type, on/off-target scores. Researchers at ORNL developed an "explainable AI" model that provides insights into the biological mechanisms driving gRNA efficiency for microbes. NYU 7 used deep learning to predict on/off-target activity of RNA-targeting CRISPR systems.

The Black Swan is just a very good read. I am rereading this currently and I am still learning things I missed the first time I read through the book.

Perplexity in AI tools

What is perplexity in AI?

In practical terms, perplexity in AI is a measure of how well an AI language model, like GPT-3 or BERT, can predict the next word in a sequence of text. Perplexity is a metric used to evaluate how well a language model can predict the next word in a sequence. A higher perplexity indicates a more complex or ambiguous language model, while a lower perplexity suggests the model is better at predicting the next word.

Think of it as a game of "guess the next word." If I give you the sentence, "The cat sat on the ____," you'd probably guess "mat" or "couch" as the next word. That's because, based on your understanding of English, those words fit the context well, so you have low "perplexity" or uncertainty about what word should come next.

Now, if an AI language model is given a large amount of text to train on, it learns patterns and relationships between words. When given a new piece of text, it uses this knowledge to predict the likelihood of each possible next word.

The core idea: Perplexity essentially reflects how surprised a language model is by the words it encounters.

• Lower perplexity: Imagine the model is reading a text and can easily predict the next word with high probability. This indicates the language is familiar and the model is confident in its predictions. A lower perplexity score translates to a less complex or ambiguous model.

• Higher perplexity: Conversely, if the model is constantly surprised by the next word and assigns low probabilities, the text is likely more complex or uses less common words. This results in a higher perplexity score, suggesting a more challenging model for the system.

It's derived from the concept of entropy, which measures the uncertainty in information. Perplexity is essentially the exponential of the average per-word entropy. In simpler terms, it represents the average number of possible next words the model considers before settling on the most likely one.

What are the Limitations of perplexity:

While perplexity is a helpful metric, it has its limitations:

• Doesn't guarantee meaningful output: A low perplexity score might just mean the model is good at predicting generic, common phrases. It doesn't necessarily translate to generating creative or informative text.

• Sensitive to data: The training data used for the model can significantly impact perplexity. A model trained on a specific domain might have a lower perplexity score for that domain but perform poorly on others.

  pen_spark

How can we use perplexity to get better output from AI Chatbots?

Perplexity itself isn't directly controllable by users of AI tools like Bard or Perplexity.ai (the search engine). However, by understanding perplexity, you can indirectly influence the output you get in a few ways:

1. Crafting better prompts: Knowing that perplexity reflects the model's surprise at the input, you can craft prompts that provide more context and guide the AI in the direction you want.

• Be specific: Instead of a vague prompt like "write a poem," try "write a haiku about a cat napping in the sun."

• Provide background information: If you're researching a complex topic, give the AI a starting point with a summary or key concepts.

2. Choosing the right tool: Some AI tools might be better suited for specific tasks based on their training data.

• If you need a factual summary of a topic, Perplexity.ai (the search engine) might be a good choice due to its focus on curated information.

• If you want creative text generation, a tool like Bard might be better suited, even if its perplexity score might be higher due to the nature of creative tasks.

3. Evaluating and refining prompts:

• Analyze the AI's output. See if it aligns with your intent.

• If the output is generic or off-topic, consider revising your prompt to provide more clarity or direction. You can use perplexity as an indirect indicator. If the output seems nonsensical, it might suggest the prompt led the model down a very high perplexity path (unfamiliar territory).

Remember, perplexity is a behind-the-scenes metric. You can't directly control it, but by understanding how it works and using the tips above, you can influence the AI to generate outputs that are more aligned with your needs.