{"id":466,"date":"2024-11-25T15:38:38","date_gmt":"2024-11-25T07:38:38","guid":{"rendered":"https:\/\/my.di.cloudns.asia\/?p=466"},"modified":"2024-11-25T15:38:38","modified_gmt":"2024-11-25T07:38:38","slug":"building-private-healthcare-ai-assistant-for-clinics-using-qdrant-hybrid-cloud-jwt-rbac-dspy-and","status":"publish","type":"post","link":"https:\/\/my.di.cloudns.asia\/index.php\/2024\/11\/25\/466.html","title":{"rendered":"Building Private Healthcare AI Assistant for Clinics Using Qdrant Hybrid Cloud (JWT-RBAC), DSPy and\u2026"},"content":{"rendered":"
\n

\u672c\u6587\u7531 \u7b80\u60a6 SimpRead<\/a> \u8f6c\u7801\uff0c \u539f\u6587\u5730\u5740 readmedium.com<\/a><\/p>\n

Introduction: Security in AI Healthcare<\/p>\n<\/blockquote>\n

With the latest advancements in AI, we often overlook the security aspect while thinking of solutions. For example, suppose you find a new tool that can simplify your chatbot pipeline but the catch is it\u2019s not secure and your organization does not approve its usage.<\/p>\n

In my experience of working with a US-based healthcare company, I have observed how seriously we take data security because of the Health Insurance Portability and Accountability Act (HIPAA) which aims to protect Protected Health Information (PHI) like account numbers, addresses, phone numbers, social security numbers, and so on. Generally speaking, any business that serves users should definitely adhere to privacy rules and take measures to prevent data leak.<\/p>\n

And with the increase in AI solutions in healthcare, especially in the domain of NLP, AI security is bound to grow as well.<\/p>\n

\"\"<\/div>https:\/\/www.unite.ai\/wp-content\/uploads\/2023\/08\/ai-crucial-for-healthcare-cybersecurity-feature.jpg<\/a><\/p>\n

As an experiment, we will build a Private AI Assistant for clinics and hospitals which fetches patient data and answers questions on top of that data.<\/p>\n

But, before proceeding, let\u2019s take a look at the flow diagram:<\/p>\n

\"\"<\/div>https:\/\/github.com\/sachink1729\/AI-Assistant-Clinics-Medical-Data-Qdrant-Dspy-Groq<\/a><\/p>\n

Briefly:<\/p>\n

    \n
  1. Dataset:<\/strong> We will be working on a healthcare dataset that contains the patient\u2019s data, including details about name, illness, medication, bills, hospital name, etc. One thing to be noted is, datasets like these are rarely available online; this dataset also is not real and is generated digitally: originally, it\u2019s a multi-label classification dataset and can be downloaded from Kaggle here: \ud83e\ude7aHealthcare Dataset \ud83e\uddea<\/a><\/li>\n
  2. DSPy:<\/strong> DSPy (or Declarative Sequencing Python framework) is a game-changing framework for algorithmically optimizing LM prompts instead of manual prompting. I have covered it in detail in one of my blogs; take a look: Prompt Like a Pro Using DSPy: A guide to build a better local RAG model using DSPy, Qdrant and Ollama | by Sachin Khandewal | Mar, 2024<\/a><\/li>\n
  3. Qdrant Managed Cloud<\/strong>: Qdrant<\/a> is a lightweight vector database that recently started their managed cloud services, which let you use a free cluster for trial and the option to upgrade as you use more features. We will use it to store our dataset in the form of vectors.<\/li>\n
  4. Groq<\/strong>: Groq is building an AI accelerator application-specific integrated circuit (ASIC) which they call the Language Processing Unit<\/a> (LPU) and related hardware to accelerate the inference performance of AI workloads. They provide access to latest models like Llama3 free of cost (it\u2019s limited), but it\u2019s enough for our use case.<\/li>\n<\/ol>\n

    Now let\u2019s set up Qdrant cloud.<\/strong><\/p>\n

      \n
    1. Go to https:\/\/cloud.qdrant.io\/login<\/a>, create a new account, and proceed.<\/li>\n
    2. After signing in, on the left hand side you\u2019ll see clusters; click on that \u2014 you can select the free or the paid version based on your needs.<\/li>\n
    3. After successful creation, you\u2019ll see your cluster on the dashboard:<\/li>\n<\/ol>\n

      \"\"<\/div><\/p>\n
        \n
      1. The next thing is creating an API Key for the usage; click on the API key below Cluster0 and copy it for future use.<\/li>\n<\/ol>\n

        \"\"<\/div><\/p>\n
          \n
        1. You will also see the Python code snippet to access this container and create collections; but we will explore that later.<\/li>\n<\/ol>\n

          Before environment setup, let us make sure you can access Groq:<\/strong><\/p>\n

            \n
          1. Go to https:\/\/groq.com\/<\/a> to sign up.<\/li>\n
          2. After that, go to https:\/\/console.groq.com\/keys<\/a> to create or manage API keys; copy the API key and keep it with you.<\/li>\n<\/ol>\n

            \"\"<\/div><\/p>\n

            Environment Setup:<\/strong><\/p>\n

            Install the required packages using:<\/p>\n

            pip install qdrant-client groq sentence-transformers dspy-ai fastembed gradio --upgrade\n<\/code><\/pre>\n
            Before coding, make sure you download the Kaggle dataset from this link: https:\/\/www.kaggle.com\/datasets\/prasad22\/healthcare-dataset<\/a><\/p>\n
            Your setup is done!<\/p>\n
              \n
            1. Let\u2019s explore the dataset and preprocess it according to our use case. You will see that the dataset is in tabular format and so we need to format it accordingly. Let\u2019s follow the steps below:<\/li>\n<\/ol>\n
              import pandas as pd\ndf = pd.read_csv("healthcare_dataset.csv")\n<\/code><\/pre>\n
              If you take a close look, you will see that this contains a lot of information which we can use!<\/p>\n
              \"\"<\/div><\/p>\n
              Let\u2019s format these rows into snippets of text.<\/p>\n
              def format_row(row):\n    return (\n        f"Name: {row['Name']}, Age: {row['Age']}, Gender: {row['Gender']}, "\n        f"Blood Type: {row['Blood Type']}, Medical Condition: {row['Medical Condition']}, "\n        f"Date of Admission: {row['Date of Admission']}, Doctor: {row['Doctor']}, "\n        f"Hospital: {row['Hospital']}, Insurance Provider: {row['Insurance Provider']}, "\n        f"Billing Amount: {row['Billing Amount']}, Room Number: {row['Room Number']}, "\n        f"Admission Type: {row['Admission Type']}, Discharge Date: {row['Discharge Date']}, "\n        f"Medication: {row['Medication']}, Test Results: {row['Test Results']}"\n        "\\n\\n".lower()\n    )\n\ndf['formatted_text'] = df.apply(format_row, axis=1)\n\ntext_data = df['formatted_text'].tolist()\n<\/code><\/pre>\n
              For the sake of experiment, I will use only 128 data rows, since we are using the free-tier Qdrant Cloud Cluster.<\/p>\n
              from random import shuffle\nsampled_dataset = text_data[:128]\nshuffle(sampled_dataset)\n<\/code><\/pre>\n
              See how these rows look now:<\/p>\n
              which gives:<\/p>\n
              ['name: heather miller, age: 76, gender: male, blood type: a+, medical condition: diabetes, date of admission: 2021-04-17, doctor: scott grant, hospital: powell ward, and mercado, insurance provider: aetna, billing amount: 3908.9465679463137, room number: 428, admission type: elective, discharge date: 2021-05-10, medication: lipitor, test results: inconclusive\\n\\n',\n 'name: connor hansen, age: 75, gender: female, blood type: a+, medical condition: diabetes, date of admission: 2019-12-12, doctor: kenneth fletcher, hospital: powers miller, and flores, insurance provider: cigna, billing amount: 43282.28335770435, room number: 134, admission type: emergency, discharge date: 2019-12-28, medication: penicillin, test results: abnormal\\n\\n',\n 'name: daniel schmidt, age: 63, gender: male, blood type: b+, medical condition: asthma, date of admission: 2022-11-15, doctor: denise galloway, hospital: hammond ltd, insurance provider: cigna, billing amount: 23762.203579059587, room number: 465, admission type: elective, discharge date: 2022-11-22, medication: penicillin, test results: normal\\n\\n',\n 'name: david higgins, age: 49, gender: female, blood type: b-, medical condition: arthritis, date of admission: 2021-03-05, doctor: erin henderson md, hospital: evans and hall schneider,, insurance provider: medicare, billing amount: 24948.47782402692, room number: 361, admission type: emergency, discharge date: 2021-03-20, medication: penicillin, test results: abnormal\\n\\n',\n 'name: lindsey lambert, age: 82, gender: female, blood type: a+, medical condition: hypertension, date of admission: 2021-11-19, doctor: christopher guerra, hospital: and brown oneal, shah, insurance provider: medicare, billing amount: 23067.672165245425, room number: 307, admission type: elective, discharge date: 2021-12-12, medication: ibuprofen, test results: normal\\n\\n']\n<\/code><\/pre>\n
              Now that we have our sample_dataset,<\/strong> the next step is to generate embeddings for these sentences in order to store them in a vector DB.<\/p>\n
              from sentence_transformers import SentenceTransformer\nmodel = SentenceTransformer("BAAI\/bge-large-en-v1.5", device='cuda')\nvectors = model.encode(sampled_dataset)\n<\/code><\/pre>\n
              Remember the size (columns or features) of this vector; we need it while creating the vector DB collection.<\/p>\n
              Which gives:<\/p>\n
              (1024,)<\/p>\n
              Now remember Qdrant\u2019s API key and your own cluster\u2019s URL \u2014 we need them now:<\/p>\n
              import os\nos.environ['QDRANT__SERVICE__API_KEY']=<your api key>\n\nfrom qdrant_client import QdrantClient\nfrom qdrant_client.models import Distance, VectorParams\n\nclient = QdrantClient(\n    url=<your cluster\u2019s url>,\n\n    api_key=os.environ['QDRANT__SERVICE__API_KEY'],\n<\/code><\/pre>\n
              Note: Typically the API key should come from a config server that you have to set up, which is internal to your organization or services only. In that way, only people who have access to that config server will get access to the API key, so not everyone can access your cluster and your data is safe!<\/strong><\/p>\n
              But since we\u2019re primarily showing the functionality, we will proceed like this.<\/p>\n
              Create a collection named: phi_data<\/p>\n
              client.recreate_collection(\n    collection_,\n    vectors_config=VectorParams(size=1024, distance=Distance.COSINE),\n)\n<\/code><\/pre>\n
              Now comes the main part \u2014 upload this collection to the cloud cluster:<\/p>\n
              client.upload_collection(\n    collection_,\n    ids=[i for i in range(len(sampled_dataset))],\n    vectors=vectors,\n    parallel=4,    \n    max_retries=3,\n)\n<\/code><\/pre>\n
              That\u2019s about it!<\/p>\n
              Qdrant also provides the option of access control via JWT, to replicate it here it has to be done on local mode, to do that simply install docker and run these 2 lines:<\/p>\n
              docker pull qdrant\/qdrant\n<\/code><\/pre>\n
              docker run -p 6333:6333 -p 6334:6334 \/\n-e QDRANT__SERVICE__API_KEY=eXaMplE12345Key67890Api \/  \n-e QDRANT__SERVICE__JWT_RBAC=true \nqdrant\/qdrant\n<\/code><\/pre>\n
              Note: this will be a single line so delete \u2018\/\u2019 in above code.<\/strong><\/p>\n
              After that, let\u2019s start up the root_client and create a dummy collection:<\/p>\n
              root_client = QdrantClient(\n    url="http:\/\/localhost:6333",\n    api_key="eXaMplE12345Key67890Api",\n)\n\nroot_client.recreate_collection(\n    collection_,\n    vectors_config=VectorParams(size=1024, distance=Distance.COSINE),\n)\n\nroot_client.upload_collection(\n    collection_,\n    ids=[i for i in range(len(sampled_dataset))],\n    vectors=vectors,\n    parallel=4,\n    max_retries=3,\n)\n<\/code><\/pre>\n
              After that, let\u2019s create a user and limit their access to read only mode using JWT, it creates a temporary key that is linked to the original key.<\/p>\n
              import jwt\nimport time\n\napi_key = 'eXaMplE12345Key67890Api'\n\ncurrent_time = int(time.time())\n\npayload = {\n    'exp': current_time + 3600,  \n    'value_exists': {\n        'collection': 'demo_collection',\n        'matches': [\n            {'key': 'user', 'value': 'John'}\n        ]\n    },\n    "access": [\n    {\n        "collection": "demo_collection",\n        "access": "r",\n        "payload": {\n            "user": "John"\n      }\n    }\n  ]  \n}\n\nencoded_jwt = jwt.encode(payload, api_key, algorithm='HS256')\n\nprint(encoded_jwt)\n<\/code><\/pre>\n
              Which gives:<\/p>\n
              eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MTYzMTA2MDgsInZhbHVlX2V4aXN0cyI6eyJjb2xsZWN0aW9uIjoiZGVtb19jb2xsZWN0aW9uIiwibWF0Y2hlcyI6W3sia2V5IjoidXNlciIsInZhbHVlIjoiSm9obiJ9XX0sImFjY2VzcyI6W3siY29sbGVjdGlvbiI6ImRlbW9fY29sbGVjdGlvbiIsImFjY2VzcyI6InIiLCJwYXlsb2FkIjp7InVzZXIiOiJKb2huIn19XX0.7ald90UIk7hI5d57S0vDfo_bdatNsi20XURlhUee_Nw\n<\/code><\/pre>\n
              This key will only give a \u201cread\u201d access to the collection we created \u2014 \u201cdummy\u201d.<\/p>\n
              Even if you try to upload a new data point into this collection you won\u2019t be allowed to, to test it use this:<\/p>\n
              from qdrant_client import QdrantClient, models\nimport numpy as np\n\nclient = QdrantClient(\n    url="http:\/\/localhost:6333",\n    api_key=your_role_key,\n)\n\ndata = np.array(list([0.1]*1024))\nprint(data.shape)\n\nclient.upload_points(\n    collection_,\n    points=[\n        models.PointStruct(\n           ,\n            vector=data,\n        )])\n<\/code><\/pre>\n
              Which will give:<\/p>\n
              UnexpectedResponse: Unexpected Response: 403 (Forbidden)\n<\/code><\/pre>\n
              Now let\u2019s set up our prompting (programming, rather) tool, DSPy!<\/p>\n
              from dspy.retrieve.qdrant_rm import QdrantRM\nqdrant_retriever_model = QdrantRM("phi_data", client, k=3)\n<\/code><\/pre>\n
              Let\u2019s initialize DSPy \u2014 Groq\u2019s integration using Groq\u2019s API key:<\/p>\n
              import dspy\nllama3 = dspy.GROQ(model='llama3-8b-8192', api_key = <your groq api> )\n<\/code><\/pre>\n
              The next step tells the system to use Qdrant as the retriever model and Groq as the LLM.<\/p>\n
              dspy.settings.configure(rm=qdrant_retriever_model, lm=llama3)\n<\/code><\/pre>\n
              Let\u2019s set up our CoT (chain of thought) modules and signatures using DSPy:<\/p>\n
              class GenerateAnswer(dspy.Signature):\n    """Answer questions with logical factoid answers."""\n\n    context = dspy.InputField(desc="will contain phi medical data of patients matched with the query")\n    question = dspy.InputField()\n    answer = dspy.OutputField(desc="an answer between 10 to 20 words")\n<\/code><\/pre>\n
              We define a function get_context to get the 3 best matching data points for the query:<\/p>\n
              def get_context(text):\n    query_vector = model.encode(text)\n\n    hits = client.search(\n        collection_,\n        query_vector=query_vector,\n        limit=3  \n    )\n    s=''\n    for x in [sampled_dataset[i.id] for i in hits]:\n        s = s + x\n    return s\n<\/code><\/pre>\n
              After that, define the main class that handles the RAG pipeline.<\/p>\n
              class RAG(dspy.Module):\n    def __init__(self, num_passages=3):\n        super().__init__()\n\n        self.retrieve = dspy.Retrieve(k=num_passages)\n        self.generate_answer = dspy.ChainOfThought(GenerateAnswer)\n\n    def forward(self, question):\n        context = get_context(question)\n        prediction = self.generate_answer(context=context, question=question)\n        return dspy.Prediction(context=context, answer=prediction.answer)\n<\/code><\/pre>\n
              These may not make sense if you are encountering these concepts for the first time. Want to understand how everything works? Hop on to my article that covers it beautifully! \u2014 Prompt Like a Pro Using DSPy: A guide to build a better local RAG model using DSPy, Qdrant and Ollama | by Sachin Khandewal | Mar, 2024<\/a><\/p>\n
              To respond to the queries, use:<\/p>\n
              rag = RAG()\ndef respond(query):\n    response = rag(query)\n    return response.answer\n<\/code><\/pre>\n
              This is not visually pleasing, so let\u2019s build a very simple chatbot using my favorite framework Gradio!<\/p>\n
              import gradio as gr\n\nwith gr.Blocks() as demo:\n    chatbot = gr.Chatbot()\n    msg = gr.Textbox()\n    clear = gr.ClearButton([msg, chatbot])\n\n    def respond(query, chat_history):\n        response = uncompiled_rag(query)\n        chat_history.append((query, response.answer))\n        return "", chat_history\n\n    msg.submit(respond, [msg, chatbot], [msg, chatbot])\n<\/code><\/pre>\n
              Finally, let\u2019s start the app!<\/p>\n
              If you are running this on Colab or on any other cloud service, consider using:<\/p>\n
              After starting the app, let\u2019s query and see how this works.<\/p>\n
              Based on the random shuffle, you may or may not see these results; that\u2019s why find a data point in sampled_dataset and use a name from it to get started. In my case, a person named kayla padilla<\/strong> showed up and I will query on her to get some information.<\/p>\n
                \n
              1. kayla padilla\u2019s details?<\/strong><\/li>\n<\/ol>\n
                Which gives:<\/p>\n
                \"\"<\/div><\/p>\n
                2. kayla padilla\u2019s doctor\u2019s name?<\/strong><\/p>\n
                \"\"<\/div><\/p>\n
                Amazing right?!<\/p>\n
                  \n
                1. Now let\u2019s query something more complex:<\/li>\n<\/ol>\n
                  Highest bill by Blue Cross Insurance?<\/strong><\/p>\n
                  \"\"<\/div><\/p>\n
                    \n
                  1. You can run a series of questions like this as well; you don\u2019t necessarily have to specify \u201cdoctor\u201d.<\/li>\n<\/ol>\n
                    \"\"<\/div><\/p>\n
                      \n
                    1. You can even ask questions according to the illness concerned:<\/li>\n<\/ol>\n
                      Young people with hypertension<\/strong><\/p>\n
                      \"\"<\/div><\/p>\n
                      And a lot more!<\/p>\n
                      With just 100 or so lines of code, you can build such amazing assistants while keeping your data safe!<\/p>\n
                      Go ahead and play around with it.<\/p>\n
                      This article covered:<\/p>\n