Solve with Google.

Using Generative and Hybrid Search in the Medical Domain

introduction.

Victoria Slocum

@victorialslocum

/in/victorialslocum

Machine Learning Engineer

MY

TEAM

Weaviate

The AI Native Vector Database

AI as part of our world

(responsibly)

struggles 

successes

a part of the whole

It's often hard to get structured, reliable outputs with LLMs in a way that you can ensure complete accuracy. They are statistical models.

New AI developments, like LLMs and vector databases have made problems that have historically been very hard to solve or do well much easier.

I believe in the importance of building trustable and transparent systems, tools that can easily integrate with each other, and providing best-practice education. 

The Weaviate Journey

From beginning to end

Involving the community

Being an ecosystem

Offer support for projects of all sizes, from different index constructions, compression features, easy deployment options, search types.

 

Everything has good defaults to get started with something that works right away, but many options for further customizability. 

We love our community - offering support through tons of educational resources, an open roadmap where you can vote for features, and even incentive programs like the Weaviate Hero.

We provide a ton of easy integrations to other companies and products, like OpenAI, LlamaIndex, LangChain, Haystack, Cohere, ...

 

Part of being an AI company means recognizing the power of an ecosystem of amazing products that can work together.

What can you...

?

Verba

We 

Hackathons

Building an

advanced

medical RAG 

use case

Can we build an AI powered doctor assistant?

  • Easily understand new information about a patient
  • Reduce time to review patient notes
  • Review ailment suggestions based on specific patients
  • Make sure no data goes to waste

search.

Semantic search

Understands meaning, rather than just exact matching

advanced technique

Hybrid Search

Medicine has a lot of specific keywords you might want to use in the search for relevant cases/articles

    reviews = client.collections.get("WineReviewNV")
    response = reviews.query.hybrid(
        query="A French Riesling",
        target_vector="title_country",
        limit=3
    )

    for o in response.objects:
        print(o.properties)

embeddings.

MedCPT Query Encoder: compute the embeddings of short texts

 

MedCPT Article Encoder: compute the embeddings of patient cases & articles

Advanced technique

LLMs for responses

If you use a LLM fine-tuned on medical domain data it can perform better

 

Meditron-70B open-source medical LLM: trained on 48.1B tokens from the medical domain.

Advanced technique

    task = "What do these animals have in common, if anything?"

    jeopardy = client.collections.get("JeopardyQuestion")
    response = jeopardy.generate.near_text(
        query="Cute animals",
        limit=3,
        grouped_task=task
    )

    # print the generated response
    print(response.generated)

generative search.

advanced technique #1

autocut.

Get more specific results, instead of just top k

advanced technique #2

query rewriting.

We can use LLMs to re-write both the prompt (DSPy) and the query to the vector DB

advanced technique #3

reranking.

Sift through top returned patient cases and re-rank them based on relevance!

    jeopardy = client.collections.get("JeopardyQuestion")

    response = jeopardy.query.near_text(
        query="flying",
        limit=10,
        rerank=wvc.query.Rerank(
            prop="question",
            query="publication"
        ),
        return_metadata=wvc.query.MetadataQuery(score=True)
    )

    for o in response.objects:
        print(o.properties)
        print(o.metadata.score)

Cohere & Sentence Transformers

DEMO

resources.

questions?