1 of 63

Jugalbandi Docs 1.2

Getting Started

About Jugalbandi Manager

Jugalbandi, at its core, is an information and action tool.

In today's information age, access to justice relies on the availability of quick, accurate, and verified information—be it regarding rights, entitlements, policies, or laws or any other matter that significantly impacts our lives. Yet many times, the intricate language and interconnected nature of such information scattered across various documents, departments, and institutions makes it intimidating for anyone to access it without formal assistance or training. Even those with formal training may struggle, losing precious time navigating the complex landscape to find the right information. Overall, this results in significant financial, opportunity, and time costs for everyone involved.

Institutions around the world, both private and government, have tried to tackle these issues across various domains, aiming to provide accessible and actionable information to all. However, these efforts are often limited in terms of both reach and resources, leaving many without the benefits they can potentially offer.

More importantly, information is only as powerful as the actions it can enable/inspire; without an added capability to act on the provided information, its potential remains unrealized. This is where Jugalbandi comes into play. Jugalbandi offers the potential of a transformative AI technology stack, leveraging conversational AI to make information on anything and everything - be it policies, laws, rights, or simply information required to navigate any task, both accessible and understandable. This stack allows the user to ask any query directly through an interface such as WhatsApp, in their native language. For instance, a user can ask a question in Malayalam using their voice and receive a response in Malayalam voice, making the interaction as natural as speaking to a human assistant.

Jugalbandi doesn’t stop at providing information. If a citizen wishes to take action based on the answer to their query, such as booking an appointment with a service provider or even initiating legal proceedings, such as filling out a legal notice or complaint, they can successfully do so through the same WhatsApp interface. Any entrepreneur, working in any field, can leverage this stack to build their conversational and action services at scale. Developers can pick and choose the components of Jugalbandi most suitable for them to build or improve their web apps, mobile apps, IVRS, etc. We will detail out these components in sections to follow.

Introduction
- About OpenyAI
- What is Jugalbandi?
- Why dive in: How will this playbook help you?
- Importance and Relevance of Jugalbandi
- Key Principles powering Jugalbandi
- Demystifying Jugalbandi: What Jugalbandi isn’t?
Breaking Down Jugalbandi's Anatomy
- Jugalbandi Manager
- Jugalbandi’s Key Services
- Key Features for Enhancing Security and Privacy
Building with Jugalbandi
- Detailed Technical Documentation
- Getting Started
- Local Development / Writing your own bot
- Dependencies
- Configuring the FSM to modify flows
- Cost of building with Jugalbandi
- Tips and Best Practices / FAQs
Use Cases of Jugalbandi
- Jugalbandi for Access to Legal Information
- Jugalbandi for Govt. Schemes
- Accessing Legal Services
- Jugalbandi for Grievance Redressal
- Climate Action with Wildlife First
- Affordable Housing with Bandhu
- Paralegal Services with Nyaya Mitra
- UNESCO X Jugalbandi
Is JB Applicable to Your Use-Case?
- The versatile applications of Jugalbandi
- Some other example use cases
Possibilities and Future Directions
- Potential Technological Advancements
- Jugalbandi Studio: Building with AI has never been more accessible
Community and Collaborations
- Understanding the community-led spirit of our mission
- Our current collaborators and community
Get Involved
- How can volunteers contribute?
- How can maintenance of the stack be open-sourced?
- Technical Support and Contact Information

Introduction

About OpenyAI

Established in 2021, OpenNyAI is a community of legal professionals, technologists, entrepreneurs, policy practitioners, students among others, with a common vision to advance AI for Justice in India. This community is anchored by Agami, EkStep, Microsoft, Thoughtworks & National Law School India University, Bangalore. By enabling creation of the necessary tools, infrastructure, and testbeds, OpenNyAI is making it easier and more affordable for organisations to demonstrate proof of concept across various knowledge bases, extending it beyond just law and justice.Driven by the OpenNyAI initiative, Jugalbandi's mission is to enable social entrepreneurs to harness AI for social impact.

What is Jugalbandi?

Jugalbandi, at its core, is an information and action tool.

Why dive in: How will this playbook help you?

Empowerment: This playbook serves as a comprehensive guide to Jugalbandi, enabling both functional and technical professionals from various backgrounds to grasp its features, components, and potential applications.

Deepen understanding: We also aim to engage people towards a deeper understanding of the capabilities, challenges, ethical considerations, and opportunities surrounding not just Jugalbandi but also other similar conversational AI stacks, particularly in the context of social impact initiatives.

Innovation: By democratising the know-how of Jugalbandi's capabilities, we aim to inspire professionals to explore innovative use cases and integrations, fostering continuous innovation on Jugalbandi.

Collaboration: Encouraging collaboration across different domains, the playbook facilitates knowledge sharing and collective problem-solving to maximise Jugalbandi's impact.

Ecosystem Expansion: Equipped with essential knowledge and tools, professionals can contribute to expanding the current Jugalbandi ecosystem, attracting more stakeholders to further enhance the platform.

Accessibility: it's not just the Jugalbandi stack that is accessible, the design principle cuts across all our efforts - documentation, benchmarking & dataset creation.

Importance and Relevance of Jugalbandi

The current approach to AI-based solutions has two issues: the proprietary nature of the tools being developed and the lack of collaboration among creators. This results in AI solutions that require considerable private investment and are often kept behind closed doors. As a result, entrepreneurs, particularly in the development sector often start from scratch, leading to inefficiencies and higher costs. To overcome this, OpenNyaAI mooted the design and development of AI public goods like baseline models, datasets, and benchmarks, which, despite being expensive and time-consuming to develop, offer high repeat value. These public goods can be used to quickly build POCs that can aid any social entrepreneur to garner funding and support for various applications.

While the Jugalbandi stack is not restricted to any particular sector or domain, it was initially devised to make law, justice and government schemes more accessible to Indian citizens. With the low proliferation of legal services in the hinterlands, with some reports suggesting that only about 0.5% of the population having access to legal aid, this was the ideal domain as a unit of change. But the original proposition of accessing critical information in a simplified manner in your own language was powerful enough to not be bound by only a specific sector. Additionally, as the capability of the LLMs kept improving exponentially, it became increasingly clear that the applications of Jugalbandi that utilised these LLMs could be sector agnostic.

Although LLMs like GPT have been advancing rapidly from GPT-3 to newer versions like GPT-4, and GPT-4o, their quick development rate poses a challenge. While the digitally equipped can fully utilize these technologies, people lacking digital and language skills risk being left behind, widening the gap between those who are digitally literate and those who aren't. Additionally, these LLMs many times run the risk of hallucination which is an unaffordable flaw when providing with crucial information that directly impacts the actions that a user may take, Jugalbandi’s value add to address this problem is three-fold:

Make the conversational interfaces accessible across many more Indian languages
Restrict the possibility of the LLMs hallucinating. Overcoming a key roadblock to ensure that AI based solutions are more reliable for someone without the time or capability to immediately fact check the information provided to them
Through the use of finite-state machines and support across multiple channels, make various services available to the user beyond conversing with the LLMs, on any medium of their choosing

Key Principles powering Jugalbandi

Jugalbandi is a versatile, scalable, and secure framework that democratises access to actionable information across languages, integrates with multiple LLMs, and ensures affordability and accuracy, promoting widespread adoption and innovation.

Accessibility Across Languages: In diverse regions with numerous spoken languages, breaking down language barriers is important to democratise access to critical information effortlessly.

Actionable Information: Information in itself is only valuable if it empowers users to take meaningful actions. By integrating capabilities to perform actions like booking appointments or initiating legal proceedings directly through the interface, Jugalbandi ensures that users can act on the information provided, thus maximizing its utility and impact.

Platform-Agnostic Framework: Users interact with technology through various platforms. Being platform-agnostic allows Jugalbandi to be deployed across multiple channels such as WhatsApp, Telegram, mobile apps, and web portals, ensuring that it meets users on their preferred platforms, improving user engagement and adoption.

Scalability: High user demand can strain systems, but Jugalbandi Manager is designed to handle a wide range of users, from just 10 to over a million. This scalability ensures that the framework can be effectively utilized by both small and large user bases, adapting to various levels of demand. The only limit is the infrastructure provided by the adopter of the stack (such as cloud services and the ability to incur any costs imposed in the usage of LLMs), making it a versatile solution for diverse organizational needs.

Integration with Multiple LLMs and Services: Different scenarios require different AI capabilities. By supporting integration with various language models like GPT, Llama, and Phi, as well as services like Bhashini, Azure, and Google, Jugalbandi offers flexibility and ensures that developers can choose the most suitable tools for their specific needs.

Accuracy and Reliability via Retrieval-Augmented Generation (RAG): Ensuring the accuracy and reliability of information is crucial, RAG in Jugalbandi helps in generating responses based on verified data from the knowledge base, minimizing the risk of misinformation and enhancing user trust.

Security and Privacy: Protecting user data is paramount. Features like sensitive information redaction, encryption, and controlled AI model access ensure that user data is secure and private, maintaining confidentiality and fostering trust among users.

Ease of Deployment and Customization: To encourage widespread adoption, Jugalbandi is designed to be easy to deploy and customize. This ensures that organizations and entrepreneurs, regardless of their technical expertise, can quickly implement and tailor the framework to suit their specific requirements, thereby accelerating innovation and impact.

Support for Finite-State Machines (FSM): FSMs provide structured and predictable conversational flows, essential for handling complex interactions and ensuring smooth user experiences. This design principle allows Jugalbandi to manage conversations effectively, providing users with coherent and logical interactions.

Open Source with Community Support: Open-source availability ensures transparency, fosters collaboration, and accelerates innovation. By providing the Jugalbandi code on GitHub, the framework invites contributions from a diverse community of developers, legal professionals, and social entrepreneurs, enriching the ecosystem and driving continuous improvement.

Affordability and Cost-Effectiveness: High costs can be a barrier to adoption, especially for social impact projects. Jugalbandi aims to be affordable by utilizing open-source components and minimizing additional charges. This ensures that even organizations with limited resources can leverage the framework to drive positive change.

Resilience Against AI Hallucinations: Ensuring the reliability of AI-generated responses is crucial, especially in high-stakes scenarios. By incorporating mechanisms to reduce the risk of hallucinations in LLMs, Jugalbandi ensures that users receive accurate and trustworthy information, enhancing the framework's credibility and effectiveness.

Demystifying Jugalbandi: What Jugalbandi isn’t?

Jugalbandi leverages pre-trained models via RAG, extends beyond government schemes and chatbots, is platform-agnostic, requires customization, and offers free code but may contain some cost-bearing components.

As Jugalbandi continues to evolve and expand its reach, we think it’s helpful to clarify some common misconceptions to understand its true potential and capabilities better. Here are the key clarifications:

Jugalbandi does not require training an AI model

It's important to know that Jugalbandi doesn’t involve developing an AI model from scratch. Rather, it leverages pre-trained models such as GPT-4, Llama, Phi by integrating them into various applications. The accuracy offered by Jugalbandi is not due to the fine tuning of these models either, rather it relies on an approach called Retrieval Augmented Generation (RAG) that removes the need of training an AI model from scratch or fine tuning a model.

Jugalbandi application is not limited to government schemes

While Jugalbandi’s first application was a bot built for seeking information on government schemes, its utility extends well beyond this realm. Its conversational and multilingual interaction capabilities make it suitable for diverse sectors.

Jugalbandi is not confined to chatbot functionality on WhatsApp and Telegram

Jugalbandi is platform-agnostic, capable of integrating with a wide array of digital platforms, including mobile apps, web portals, and other messaging services. This flexibility ensures that Jugalbandi can meet users on their preferred platforms, providing consistent and accessible support across multiple channels.

Jugalbandi is not a simple plug and play API

Jugalbandi implementation goes beyond simple plug-and-play APIs that require zero to minimal coding for integration. While it offers robust functionalities and flexibility, integrating Jugalbandi into applications often involves tailoring and customization for specific usecases.

Jugalbandi code is free to use

If you wish to deploy Jugalbandi for your use case, the code is available freely on GitHub. One thing to note, however, is that all the components of the Jugalbandi stack are not entirely free of any cost. While many elements are open-source or low-cost, certain components like GPT models and cloud services may incur additional charges to be borne by the entrepreneurs building out their instance of Jugalbandi.

Breaking Down Jugalbandi's Anatomy

Jugalbandi Manager: Your companion for building and managing AI services

Jugalbandi Manager is a bot building framework that is agnostic of platform, LLM or hosting services.

Jugalbandi (Jugalbandi) Manager is a full AI-powered conversational and action bot-building framework that is platform agnostic and can serve multiple channels such as WhatsApp, Telegram, or custom web interfaces. Jugalbandi also supports the integration of any language model, including Bhashini, Azure, and Google. It is also LLM agnostic, providing integration options for both open source and proprietary models like GPT, Llama, Phi etc.

For those interested in using Jugalbandi to build their services, Jugalbandi manager makes it very easy for any organisation or entrepreneur to deploy their application to enable:

Simplifying access to knowledge - what started as an effort to simplify the language of the law, can now be used to summarise, categorise or explain any information in a language and format of the user’s choice.
Taking action - by plugging in with interoperable services, Jugalbandi bots can facilitate the user to take action w.r.t KYC, signing up for government schemes, payments, e-signing, availing dispute resolution services, discovering & ordering through e-commerce platforms etc.

Jugalbandi’s Key Services

While the technical architecture of deploying and using Jugalbandi is slightly more complicated, we have tried to simplify the core features and relate them to everyday objects around us so that we can easily understand how each component functions and interacts within the system.

1. Communication Channels: Think of this as the part of Jugalbandi that talks to different apps and systems. Whether it’s WhatsApp, Telegram, a website, or even business software like CRM systems, this component makes sure the messages are formatted correctly for whichever platform you’re using. It connects through the API service of Jugalbandi.

2. Database (DB): We can imagine this as a cabinet where all the messages and information that pass through Jugalbandi are stored. It has instructions on how to set up, update, and manage the cabinet. Depending on the needs of the organisation using Jugalbandi (including their specific use case, geography and regulations), the way this data is handled can be customised.

3. Indexer: Imagine the indexer as a librarian who organizes all the books in the library as per some categorisation. Similarly,the indexer arranges the information into neat, searchable chunks. This helps in finding the right information quickly. It breaks down the knowledge fed into Jugalbandi into smaller parts and adds labels to each part, making it easy to search and relate to other information.

4. Retriever: The retriever is like a search engine within Jugalbandi. When you ask a question or look for information, the retriever searches through the organized chunks created by the indexer (the librarian) to find the relevant answers. It’s also responsible for making sure the search operations are efficient and tailored to specific needs.

5. Retrieval-Augmented Generation (RAG): RAG is a key feature of Jugalbandi that allows any organization to conversationally query large datasets. It ensures that the answers are generated exclusively from the provided data, reducing the risk of misinformation and enhancing the accuracy of the responses.

A Typical RAG Pipeline ; Source: Hirokazu Suzuki for Beatrust

6. Flow Manager: This component acts like a guide or a manager of conversations. It controls how the interaction flows, making sure the conversation is smooth and logical. For example, if you ask a vague question, it will prompt you for more details. It can categorize your needs, ask for more specific information, provide the right answers, and handle unexpected inputs. It’s highly customizable to fit different sectors and use cases.

7. Language: The language component takes care of anything related to language. This includes translating languages, converting speech to text, and text to speech. It works with various services like Bhashini, Azure, or Google to handle these tasks.

Key Features for Enhancing Security and Privacy

The Jugalbandi framework incorporates some core features to ensure the security and privacy of the users. Some of these features are already in place, while others are planned for future implementation. Here, we elaborate on each feature with examples to show their practical application.

A.Redaction of Sensitive Data:

Sensitive Information Redaction: When a user inputs sensitive information such as an Aadhar number or phone number into the Jugalbandi system, this data is immediately flagged as sensitive.Before any data is stored in the database or sent to an LLM for processing, the Jugalbandi system automatically redacts sensitive information. This involves replacing the sensitive data with placeholders or masking it (for example, replacing an Aadhar number like 1234-5678-9123 with X's or other characters). Only the redacted data is stored in the database or transmitted to the LLM servers. This ensures that sensitive information is not exposed even if the database is compromised during transmission.When the LLM receives and processes the user's request, it does so without access to the actual sensitive information, working instead with the redacted data.
Regex Pattern Matching: The Jugalbandi framework employs "Regex" (short for Regular Expressions) to automatically identify and remove sensitive data or profanity from user inputs by recording certain patterns. These patterns serve as templates for identifying specific sequences of characters that match the criteria for sensitive data or offensive language. When a user inputs a message, the Jugalbandi system scans the text using the predefined Regex patterns. This allows the system to identify any parts of the message that match the patterns for sensitive data or profanity.

Once a match is found, the system automatically redacts or removes the identified sensitive data or offensive language.The cleaned message, free from sensitive data and profanity, is then logged and processed by the Jugalbandi system.

B. Encryption:

Encryption of Sensitive Keys and Media: Jugalbandi encrypts sensitive keys, audio messages, and media messages, especially when migrating databases to the cloud.

When a user sends an audio message through the Jugalbandi bot on WhatsApp, this message is encrypted before being stored or transmitted. Even if intercepted, the encrypted message remains unreadable without the decryption key, protecting the user's privacy. Encryption is different from redaction in that redaction removes sensitive information from a dataset to prevent unauthorized access while encryption just transforms it into unreadable format.

Profanity Filter: If a user types a message containing offensive language, the profanity filter would detect and replace inappropriate words with asterisks or any other symbol, ensuring the conversation remains respectful and appropriate.

C. Controlled AI Model Access:

Restricted LLM Access: Jugalbandi ensures that OpenAI models only access prompts and data provided by the organization implementing the Jugalbandi framework, keeping sensitive information like WhatsApp or OS configurations secure.

For example, when using Jugalbandi to interact with an OpenAI model, the model only processes the user's questions and responses provided by the bot. It does not have access to the user's WhatsApp configuration or other underlying system details, preserving user confidentiality.

D. Strong Prompts: Jugalbandi uses strong prompts and prompt engineering techniques to control the output of large language models (LLMs) like GPT, Phi, and Llama, mitigating the risk of providing harmful or inappropriate responses. The aim is to ensure that the AI-generated responses remain accurate, relevant, and safe for users. By using well-engineered prompts, Jugalbandi ensures that the AI models produce outputs that are in line with the intended use case and user expectations. This reduces the likelihood of the AI generating inappropriate, irrelevant, or harmful responses. Additionally, we prefer integrating with proprietary LLMs like GPT, Phi, and Llama over open-source models. Proprietary models often come with additional safeguards, extensive training on diverse datasets, and better support for implementing prompt engineering techniques.

Below is an example of a strong prompt used by Jugalbandi:

=GPT("You are grievance description writing bot whose job it is to write humanly readable, exhaustive and understandable descriptions of a category of grievance that may come under a particular Departments/Ministry of the Government of India. You will be provided a category of grievance in the form of “<MINISTRY> | <CATEGORY> | <SUB CATEGORY> | <SUB CATEGORY>… Sometimes there may be multiple SUB CATEGORY levels. You will include in your description what kind of Grievances the Sub Cattegory/Categories pertain to in such a way that a bot’s RAG system will be able to identify given a user query. You will only output a description and no other prefixes or sufixes. You will also include the different examples of ways in which people may refer to this grievance, for example, phone network could also be called network coverage and other terms. Be as exhaustive and all encompassing as possible. The Grievance category you must address is: " & J15251)

The prompt clearly defines the role of the LLM as a "grievance description writing bot," setting a specific context and purpose for the task. This helps the LLM understand the scope and limits of its response, ensuring it focuses on the task at hand.
The prompt provides comprehensive instructions on what the LLM should do, including writing "humanly readable, exhaustive and understandable descriptions," and specifying the structure of the input ("<MINISTRY> | <CATEGORY> | <SUB CATEGORY> | <SUB CATEGORY>").
By instructing the LLM to include different examples of how people might refer to a grievance, the prompt ensures the responses are relevant and cover a wide range of possible user queries. This reduces the chances of the AI providing irrelevant information.
The prompt's structure and specificity helps reduce the risk of generating harmful or inappropriate responses. By focusing on specific grievance categories and subcategories, the AI is less likely to produce outputs that are off-topic or inappropriate.
The direction to "only output a description and no other prefixes or suffixes" ensures the AI's response is concise and to the point, avoiding unnecessary or irrelevant information that could confuse users or dilute the response's quality.

E. Data Residency Guarantees: All the data provided by Indian users reside within the boundaries of the country ensuring data sovereignty (in the case of deploying on Azure, the data usually resides within their South Indian data centres)

Building with Jugalbandi

Technical Guide

Jugalbandi (JB) Manager is a full AI-powered conversational chatbot platform. It's platform agnostic and can serve multiple channels such as WhatsApp or custom web interfaces. It can handle conversations in both text and voice across any language. It comes with Bhashini Speech models out of the box and can failover to Azure.

The technical documentation is organized as follows:

Tutorials
1. Quickstart -- Start Here
How to guides
1. Local Development
2. Write Your Own Bot
References
Explanations
1. JB Manager Architecture

explanations

JB Manager Architecture

This document explains the inner workings of the JB Manager.

Architecture

All the services are generic i.e. they are designed for multiple bots or use cases. The only customization that exists lies in the tiny bot code (finite state machine code) which is published by the Studio.

DB Structure

Bot

bot_id (e.g. "12345") is at the highest level. All other entries like person, sessions are against a particular bot. We support multiple bots from the same backend.

User

A user needs to be registered for the bot. Phone Number & Bot ID make the user unique.

Session

This table stores the state (stage in the workflow) and variables for every user. We maintain session for 24 hours. After that a fresh conversation starts.

Turn

Every message received from the user creates a new turn in the conversation. Against one user message, we can have multiple bot responses.

Message

Every message received or sent is logged to the DB.

Services

API

Provides APIs to interact with JB Manager. Here are the top buckets of APIs:

Callback from channels like WhatsApp or Telegram
JB Studio like publishing the bot
Managing bots changing their states and variables
Viewing chat logs and analytics
Triggering indexing jobs

Callback

Figure out bot_id based on the callback
Create person if doesn't exists. Fetch pid
Create session_id if session has expired (> 24 hours).
Creates turn_id for the new conversation
Create msg_id

Channel

Responsible for both input and output communication with the channel (WhatsApp / Telegram). It needs to handle channel specific API calls and rendering of UI messages (e.g. WhatsApp Flow, List & Buttons)

If media present in the message, fetch and upload to Azure.
Provides standard dialogues (list/buttons) for features that are common to all bots. It handles rendering (outbound) and the related user action. It let's Flow service know that the user has completed a dialogue.
1. Language - It shows the dialog and saves the user's language preference
2. Reset (not yet there)

In the inbound scenario:

It mostly enqueues messages for the Language service
Unless the message related to a dialogue (e.g. language), WhatsApp Flow or Button. In this case it directly talks to the Flow service.

In the outbound scenario:

It mostly receives messages from the Language service.
Unless the message is related to WhatsApp flow in which case Flow directly writes to it's queue.

Language

There are two flows:
1. Receiving users message
2. Sending the response to the user.

It's responsible for translating labels in case of interactives messages e.g. Lists & Buttons

We assume that all messages sent by the user obey the user's preferred_language.

For outbound messages, Language service will look at the message type from the Flow Service and then decide on the following:

If text
1. Then fetch details about the turn -- In what mode did the user ask their question:
  1. If user asks in text / form / interactive, we respond in text
  2. If user asks in voice, we respond in voice + text -- Language service creates two inputs for Channel service
  3. In all other cases, send text
If document,
1. Translate the caption
image
Translate the caption
If interactive
1. Translate the text, header, footer
2. Translate the labels for buttons

Flow

Retrieve session from DB based on session_id. This provides it memory of the current state (stage of the workflow), variables.
Process the message via a finite state machine code (this code is generated in the Studio and pushed to the JB Manager)
1. It can send multiple messages for every user message
2. It can make API calls or use plugins
3. The responses that it sends can include the following:
  1. Text
  2. Interactive - WhatsApp flow, List, Buttons
  3. Documents, Images
  4. Dialogues -- it uses keywords language to trigger Channel service into sending a standardized dialogue.
On complete (or logical pause in the workflow e.g. user input or plugin input), save the state and variables.

Retrieval

This fetches data from the Vector DB (PG Vector / Postgres) based on query and metadata if any. The metadata is used to create filter query to restrict the similarity search even further.

Indexer

This service receives input files (urls from Azure) and pre-processes them and indexes them into the Vector DB.

Explanations

how-tos

Local Development

If you are new here, read everything and follow the steps the end.

Develop and Test Locally

We have written a script that makes it easy to run this on any local machine (Mac, WSL2). Only deploy the services you need. This script depends on the env values in .env-dev.

$ ./scripts/run.sh <one or more service names that you are testing>
# names of the services - api, channel, language, flow, retriver, indexer

It's important that we test the data contracts (input / output JSONs) for our service. Run the service in one window, send JSON message to Kafka via another and read the messages generated by your service in another.

# create topics (queues) on kafka
./scripts/create-topic.sh topic_name

# send message to your service
./scripts/send-message.sh topic <json>

# read messages from your service
./scripts/read-message.sh topic

Setting up Local DB

# Updating your local database
./scripts/upgrade-db.sh

# Adding an alembic migration
./scripts/create-migration.sh <description of the changes>

Connecting to database

sudo apt-get install -y postgresql-client
psql -Upostgres -hlocalhost

It will prompt for password.

flow - adding a user

 insert into jb_users (pid,first_name, last_name) values ('test4','a','b');

flow - checking state stored in the db

select * from jb_fsm_state;

Data Contracts

See the Pydantic models lib/data_models.py

API

curl http://localhost:8000/callback -H 'content-type:application/json' -d '{"object": "whatsapp_business_account", "entry": [{"id": "112776635030672", "changes": [{"value": {"messaging_product": "whatsapp", "metadata": {"display_phone_number": "919711028566", "phone_number_id": "116346771524855"}, "contacts": [{"profile": {"name": "Test User"}, "wa_id": "919999999999"}], "messages": [{"from": "919999999999", "id": "wamid.HBgMOTE5ODg2Njg5NzU0FQIAEhgUM0E0RjQ2QzQyMUUxREYyODcxNjQA", "timestamp": "1705550692", "text": {"body": "Hey"}, "type": "text"}]}, "field": "messages"}]}]}'

Indexer

topic: indexer message:

'{"collection_name":"KB_Law_Files", "files":["combine.pdf", "Property Law.xlsx", "Information Technology Laws.xlsx", "Family Laws for Succession and Inheritance.xlsx", "Senior Citizens Rights.xlsx", "Motor Vehicles Act.xlsx", "Income Tax Compliance.xlsx", "Cyber Crimes.xlsx", "Disability Rights.xlsx", "Women Rights.xlsx", "Law related to SC and ST.xlsx", "Medical Termination of Pregnancy.xlsx", "Mental Health.xlsx", "Narcotics and Psycotropic Substances.xlsx", "Formation of Company and Corporate Social Responsibility.xlsx", "POCSO Act.xlsx", "SEBI Regulations.xlsx", "Legal Services in India.xlsx", "Transgender Rights.xlsx", "Farmer Rights.xlsx", "Arbitration Conciliation.xlsx", "Labour Laws.xlsx"]}'

Steps

Create environment by copying .env-dev.template to .env-dev. Update the values if required. You can look at the larger list of constants in .env
Bring up the service e.g. api

$ ./scripts/run.sh -e .env-dev api

Make any modifications to the database

$ ./scripts/upgrade-db.sh

Setup a listener to the queue

$ ./scripts/read-message.sh <topic name>

Test the data contracts by sending a message to the queue or api

# For API
$ curl http://localhost:8000/callback ....

# For all
$ ./scripts/send-message.sh <topic> <JSON message>

Check the database by using DBeaver

Reseting Dev Environment

Often there are times when it's better to start from scratch.

# bring down whatever is running
$ docker compose down

# following will delete all your containers - be careful
$ docker volume ls | awk '{print $2}' | xargs docker volume rm

# if you don't want clutter, you can delete all containers
$ docker ps -a | awk '{print $1}' | xargs docker rm

Write Your Own Bot

Bots in JBManager follow a finite state machine paradigm, using the transitions library. It is recommended to refer transitions documentation to understand the basic concepts of states, transitions & conditions.

Each Bot is represented by a python class which is a child class of AbstractFSM class present in jb-manager-bot module.

class BotCode(AbstractFSM):
    states = []
    transitions = []
    conditions = {}
    output_variables = set()

    def __init__(self, send_message: callable, credentials: Dict[str, Any] = None):
        if credentials is None:
            credentials = {}
        self.credentials = {}
        super().__init__(send_message=send_message)

credentials in constructor contains all the credentials that will be required by the bot. You can define these credentials while installing the bot in JBManager UI. Values of credentials are passed while configuring the bot in JBManager UI. These credentials are passed in the credentials dict in the constructor, with credential name as key and credential secret as value. Save this credential as an instance variable credentials, after performing sanity check if the credentials are present.

A new instance of Bot is initialised for each user session, maintaining seperation between user session.

States

Each processing step in the Bot flow is represented be a state. This is same as state defined in the transitions library.

Declaration: To declare state in Bot code in JBManager, add the state name in states list class variable of your bot code.

class BotCode(AbstractFSM):
    states = [
        ...
        "new_state_name",
        ...
    ]

Definition: To define the state, create a instance method of name on_enter_<state_name> in your Bot class.

class BotCode(AbstractFSM):
    states = [
        ...
        "new_state_name",
        ...
    ]

    def on_enter_new_state_name(self):
        self.status = Status.WAIT_FOR_ME
        ...
        # State processing logic
        ...
        self.status = Status.MOVE_FORWARD

Status

Each bot running on JBManager can have 4 different status which controls the flow of the bot. The various status are described below:

WAIT_FOR_ME - It should be used on the top of each state definition representing processing logic of a state is currently executing and should wait for processing to finish before making transition to any other state.
MOVE_FORWARD - It represents that current processing is finished and the FSM is ready to transition to next state.
WAIT_FOR_USER_INPUT - It represents that FSM should wait for user input before making transition to another state. It is used in cases where the FSM wants to capture the user input. The next state after this state should contain the logic to handle the user input.
WAIT_FOR_CALLBACK - It represents that FSM should wait for external before making transition to another state. It is used in cases where the FSM wants to capture the input from 3rd party webhook APIs. The next state after this state should contain the logic to handle the callback input.
WAIT_FOR_PLUGIN - For internal use only.
END - It represents the end of the bot flow. It should be added at the last state of the bot flow.

NOTE: WAIT_FOR_ME should only be used at start of the state definition. MOVE_FORWARD, WAIT_FOR_USER_INPUT, WAIT_FOR_CALLBACK and END should be used at the end of state definition. WaIT_FOR_PLUGIN should never be used in any custom FSM.

Transitions

The switch from one state to another state is managed by defining transition in Bot class. After completing the current state, the bot will switch to the state as defined by the transition.

class BotCode(AbstractFSM):
    states = [
        ...
        "first_state",
        "second_state"
        ...
    ]
    transitions = [
        ...
        {
            "source": "first_state",  # Source State
            "dest": "second_state",  # Destination State
            "trigger": "next"   # Trigger, value will always be next for JBManager
        }
        ...
    ]

In the above example, after completing the first_state, the bot will switch to executing second_state. In case the bot is defined to wait for user input or webhook callback after first state (by setting appropriate status in the end), the bot will wait after executing first_state and only after recieving user input or webhook callback(whichever applicable), it will switch to second_state.

Conditional Transition

If the next state to switch depends on a boolean condition, conditional transition can be defined to switch. We need define the transition with conditions key and appropriate condition should also be declared and defined.

class BotCode(AbstractFSM):
    states = [
        ...
        "first_state",
        "second_a_state",
        "second_b_state",
        ...
    ]
    transitions = [
        ...
        {
            "source": "first_state",
            "dest": "second_a_state",
            "trigger": "next",
            "conditions": "is_second_a",   # using condition a
        },
        {
            "source": "first_state",
            "dest": "second_b_state",
            "trigger": "next",
            "conditions": "is_second_b",   # using condition b
        },
        ...
    ]
    conditions = {
        "is_second_a",  # Declaring condition a
        "is_second_b"   # Declaring condition b
    }

To define the condition, create an instance method with the same name as condition name. The method should return boolean True value denoting the condition to be satisfied, else False denoting the condition is not satisfied.

class BotCode(AbstractFSM):
    states = [
        ...
        "first_state",
        "second_a_state",
        "second_b_state",
        ...
    ]
    transitions = [
        ...
        {
            "source": "first_state",
            "dest": "second_a_state",
            "trigger": "next",
            "conditions": "is_second_a",   # using condition a
        },
        {
            "source": "first_state",
            "dest": "second_b_state",
            "trigger": "next",
            "conditions": "is_second_b",   # using condition b
        },
        ...
    ]
    conditions = {
        "is_second_a",  # Declaring condition a
        "is_second_b"   # Declaring condition b
    }
    
    ...

    def is_second_a(self):
        ...
        # condition a logic here
        ...
        return is_condition_a_satisifed
    
    def is_second_b(self):
        ...
        # condition b logic here
        ...
        return is_condition_b_satisifed

In the above example, after finishing the first_state, bot will call the conditions associated with first_state as source. Whichever condition returns True, bot will take that transition and switch to appropriate state defined in the dest of the transition. If multiple conditions are satisfied, bot will take the transition which is defined first in the transitions list. For more details on the working of transitions, see transitions documentation.

Variables

To retain information across the states, use variables dictionary defined in the AbstractFSM, with variable name as key in string format and value for the variable data.

class BotCode(AbstractFSM):
    states = [
        ...
        "new_state_name",
        ...
    ]

    def on_enter_new_state_name(self):
        self.status = Status.WAIT_FOR_ME
        ...
        # State processing logic
        
        self.variables["my_variable"] = 10 # Here we are setting my_variable value to 10
        your_variable = self.variables("your_variable") # Here we are accessing your_variable set in any previous state
        
        # State processing logic
        ...
        self.status = Status.MOVE_FORWARD

NOTE: Add only json serializable keys and values to the variables dictionary.

User Input

To access input by user inside a state, use the instance variable current_input defined in the AbstractFSM class. This variable contains the value of current user input in str data type format.

Note: Variable current_input will contain the value of user input within the scope of state which comes next to input state i.e state ends with WAIT_FOR_USER_INPUT status. It contains None in any other state.

class BotCode(AbstractFSM):
    states = [
        ...
        "user_input_processing_state",
        ...
    ]

    def on_enter_user_input_processing_state(self):
        self.status = Status.WAIT_FOR_ME
        ...
        user_input = self.current_input
        # user input processing logic
        ...
        self.status = Status.MOVE_FORWARD

Webhook Input

To access input by an 3rd party webhook api inside a state, use the instance variable current_callback defined in the AbstractFSM class. This variable contains the value of current webhook data in str data type format (You might need to parse it to json).

Note: Variable current_callback will contain the value of webhook data within the scope of state which comes next to callback state i.e state ends with WAIT_FOR_CALLBACK status. It contains None in any other state.

class BotCode(AbstractFSM):
    states = [
        ...
        "webhook_processing_state",
        ...
    ]

    def on_enter_webhook_processing_state(self):
        self.status = Status.WAIT_FOR_ME
        ...
        webhook_data = self.current_callback
        webhook_data = json.loads(webhook_data)
        # webhook processing logic
        ...
        self.status = Status.MOVE_FORWARD

Sending Message to User

To send a message back to user from the bot, use send_message method defined in the AbstractFSM class. send_message method takes FSMOutput object as an arguement. Populate the object of FSMOutput class according to the message you want to send to user.

class BotCode(AbstractFSM):
    states = [
        ...
        "greet_user",
        ...
    ]

    def on_enter_greet_user(self):
        self.status = Status.WAIT_FOR_ME
        self.send_message(
            FSMOutput(
                message_data=MessageData(
                    body="Hello! How are you?" # Message Text
                ),
                type=MessageType.TEXT, # Text Message
                dest="out", # Destination of the message
            )
        )
        self.status = Status.MOVE_FORWARD

For more details, see FSMOutput documentation.

Possible destinations

out - The message will follow translation(if applicable) and converted to speech and sent it to user.
channel - There are some messages which don't require translation and should be interpreted as a special message. Those could be either flags for language selection or metadata for the forms which will be displayed to user.
rag - The message will go to RAG component of JBManager instead of user.

Handling Files

To handle Files in the FSM, supplied in the form of bytes. You can access the file bytes using the current_input variable if the state is waiting for user input or current_callback if waiting for a webhook callback.

class BotCode(AbstractFSM):
    states = [
        ...
        "file_processing_state",
        ...
    ]

    def on_enter_file_processing_state(self):
        self.status = Status.WAIT_FOR_ME
        ...
        file_bytes = self.current_input  
        # Process the imafilege bytes
        ...
        self.send_message(
            FSMOutput(
                message_data=MessageData(
                    body="Thanks for sharing the file."  # Acknowledgment message
                ),
                type=MessageType.TEXT,
                dest="out",
            )
        )
        self.status = Status.MOVE_FORWARD

Note:

Follow the below steps to ensure th FSM receives the file.

Modify the process_incoming_messages function inside the incoming file under Channel section to create ImageMessage or DocumentMessage data object accordingly
Modify the handle_user_input function inside the bot_input file under flow section to add file data received from the user.
- Ensure that the fsm_input object is populated with the file data.

How to Guides

Local Development
Write Your Own Bot

references

Dependencies

Storage
Speech and Translation
Whatsapp
Telegram

FSMOutput

The purpose of FSMOutput is to give a structured and defined data structure to communicate between Bot code and JBManager framework. Instance of FSMOutput is passed as an arguement while calling the send_message function.

class FSMOutput:
    dest: str = "out"
    type: MessageType = MessageType.TEXT
    message_data: MessageData
    options_list: Optional[List[OptionsListType]] = None
    media_url: Optional[str] = None
    file: Optional[UploadFile] = None
    whatsapp_flow_id: Optional[str] = None
    whatsapp_screen_id: Optional[str] = None
    menu_selector: Optional[str] = None
    menu_title: Optional[str] = None
    form_token: Optional[str] = None
    plugin_uuid: Optional[str] = None
    dialog: Optional[str] = None

dest - This represents the corresponding layer in JBManager to which the message needs to be sent. Possible destinations:
- out - The message will follow translation(if applicable) and converted to speech and sent it to user.
- channel - There are some messages which don't require translation and should be interpreted as a special message. Those could be either flags for language selection or metadata for the forms which will be displayed to user.
- rag - The message will go to RAG component of JBManager instead of user.
type - Represents the type of the message which needs to be send to user. It will be of enum of type MessageType. See MessageType for info regarding the supported message types.
message_data - Body of the message which will be sent to the user.
options_list - List of options which will be presented to user. Only used in case of interactive message.
media_url - Url of the file(image/audio/document) which needs to be send to the user along with message body. Only used in case of image, document and audio message.
file - Document to send to the user along with message body. Only used in case of document message.
whatsapp_flow_id - Flow ID required to identify the form to send to user in whatsapp. Only used in case of form message.
whatsapp_screen_id - Screen ID required to identify the form screen to send to user in whatsapp. Only used in case of form message.
menu_selector - In case options are displayed in form of list to the user(number of options > 3), this represents the text on the button which opens the list of options. Only used in case of interactive message.
menu_title - In case options are displayed in form of list to the user(number of options > 3), this represents the text on the panel showing list of options. Only used in case of interactive message.
form_token - Token to identify the form to send to user.
plugin_uuid - This represents the uniqueID used to require to identify the current running session in case of an 3rd party webhook input. Here, the expectation is that this unique ID is used to call the 3rd party plugin and this will be returned in the webhook call.
dialog - It represents various signals sent to JBManager framework which will trigger certain existing flows which are a part of JBManager. Currently supported options:
- language - Represents the signal to send the language selector option to the user on which user will select the preferred language.

class MessageData:
    header: Optional[str] = None
    body: str
    footer: Optional[str] = None

header - Header of the message body. Only used incase of interactive and form message.
body - Message text which needs to be displayed.
footer - Footer of the message body. Only used incase of interactive and form message.

class OptionsListType:
    id: str
    title: str

id - Option Identifier. This value of the selected option will be returned to the bot code.
title - This value will be displayed to the user as text on the option.

Supported Message Type

The following message types are supported as defined by the enum MessageType

class MessageType(Enum):
    """
    Enum class to define the type of message.
    """
    INTERACTIVE = "interactive"
    TEXT = "text"
    AUDIO = "audio"
    DOCUMENT = "document"
    FORM = "form"
    IMAGE = "image"
    DIALOG = "dialog"

References

Dependencies
Output from Bot
Example: Grievance Bot

Speech and Translation

Bhasini Translation API

JB Manager depends on Bhasini translation and speech processing functionalites for processing messages (both text and audio) in Indic languages. To access Bhasini API, register at Bhashini. Detailed steps on prerequisite is available on Bhasini Onboarding Page. After registration, USER_ID, API_KEY and PIPELINE_ID will be required. You can find PIPELINE_ID here.

Azure Speech and Translation API

JB Manager depends on Azure speech and translation API for processesing non-Indic languages as well as fallback to Bhasini API.

Create a Speech resource in the Azure portal.
Create a Translation resource in the Azure portal.
Procure the Speech and Translation resource key and region. After your resource is deployed, select Go to resource to view and manage keys. For more information about Azure AI services resources, see Get the keys for your resource.

Azure Storage

JB Manager leverages Azure Blob Storage for the persistent storage of input and output files through which user interacts with the bot. Alternatively, we have the flexibility of switching to a local storage as well.

In this document, we have covered the below-mentioned aspects for utilizing either of the storages effectively:

Introduction
- Preferred Scenario
- Prerequisites
Step-by-step Configuration

Introduction

Azure Blob Storage

Preferred Scenario:
- Scalability Needs: When you need to scale storage capacity quickly and efficiently without worrying about hardware limitations
- High Availability and Durability: When you require high availability and durability with data redundancy across multiple locations
- Managed Services: When you want to offload hardware maintenance, updates, and critical issue management to a cloud provider
Prerequisites:
- Azure Storage Account: First, you will need to setup an Azure Storage Account on the portal (refer the link)
- Containers: Next, you will create a container which will be used to store the files

Local Storage

Preferred Scenario:
- High-Speed Access Required: When applications require very low latency and high-speed access to data
- Complete Data Control: When you need full control over data storage, management, and security (highly confidential files)
Prerequisites:
- Hardware Requirements: Ensure you have sufficient hardware resources, such as hard drives (HDDs) or solid-state drives (SSDs), to meet your storage needs
- Backup and Redundancy: Implement robust backup and redundancy strategies to protect against data loss due to hardware failures or other issues

Configuration

Setting up Environment Variables

To use Azure Blob Storage, you need to update the following lines in .env-dev:

   STORAGE_TYPE=azure
   AZURE_STORAGE_ACCOUNT_URL=<your_storage_account_url>
   AZURE_STORAGE_ACCOUNT_KEY=<your_storage_account_key>
   AZURE_STORAGE_CONTAINER=<your_container_name>

To use Local Storage, you need to update the following lines in .env-dev:
```
  STORAGE_TYPE=local
  PUBLIC_URL_PREFIX=<tunnerl_url>
```
By default, the JB Manager utilizes Local Storage unless specified using the above pointers

FAQs:

What is the significance of PUBLIC_URL_PREFIX?
- To refer and access the inputted files across services in the JB Manager i.e. Language service in case of audio files
What are the default storage paths in both cases?
- In case of Azure Blob Storage, default location is /tmp/jb_files within the defined container
- While, in case of local storage, default location is /mnt/jb_files
I am already utilizing the tmp folder in the Azure Blob storage container in a different process. So is there a way to change the default path?
- Yes, you could define the custom path in azure_storage.py at jb-lib/lib/file_storage/azure in case of Azure option
```
    class AzureAsyncStorage(AsyncStorage):
       __client__ = None
       tmp_folder = "/tmp/jb_files" # Update the path here
```
- While in case of local storage, update the local_storage.py at jb-lib/lib/file_storage/local/
```
   class LocalAsyncStorage(AsyncStorage):
      tmp_folder = "/mnt/jb_files" # Update the path here
```

For conversation with Bot, currently JB Manager depends on whatsapp access provided by Pinnacle.

Steps to follow

a. Create a Telegram Bot:

Use the BotFather chatbot on Telegram to create your bot
Acquire the token provided by BotFather. This token will be used to set up the webhook

b. Create a JB Manager Bot:

Set up a bot as described in the quickstart documentation
Select type as telegram and appropriate details while creating the channel

c. Set-up the Webhook:

Set the webhook by sending a POST request to https://api.telegram.org/bot<token>/setWebhook with the URL of the endpoint as the body under the url key. The format should be: <tunnel_url>/v2/callback/telegram/<app_id>.

Example using curl:

curl -X POST "https://api.telegram.org/bot<token>/setWebhook" -d "url=<tunnel_url>/v2/callback/telegram/<app_id>"

Note:

BotFather: This is an official Telegram bot that helps you create and manage your Telegram bots. You can find it by searching for "BotFather" in the Telegram app
Webhook URL: The <tunnel_url> should be the URL where your API is hosted. Ensure it is accessible from the internet for Telegram to send updates to your bot
By following these steps, you can set up, integrate and test your Telegram-based JB Manager bot.

example-grievance-bot

Example Grievance Bots

This is an example of building a grievance bot. We consider an organization like CPGRAMS that allows Indian citizens to lodge complaints with any department of the government.

Review the notbooks indexer.ipynb and retriever.ipynb in this directory for the entire source code.

Final User experience

Step 1: Data Indexing Pipeline

Intial Data

We scrape the following information from the CPGRAMS website:

Ministry

Data Augmentation

We have generated an elaborate description for each grievance by using GPT-4, with appropriate system prompt and providing the ministry name, category and subcategories. The generated description is in following format:

Ministry

Schema for Vector DB

We need to make this available in a VectorDB to fetch the data based on similarity. We create the following schema. embedding is the vector representation of description column.

class GrievanceCategory(Base):
    __tablename__ = "grievance_category"
    id = Column(String, primary_key=True)
    ministry = Column(String)
    category = Column(String)
    subcategory = Column(ARRAY(String))
    description = Column(String)
    embedding = mapped_column(Vector(1536))
    fields = Column(ARRAY(JSON))

fields stores an array of fields that the user needs to provide to lodge their compliant.

Data Indexing

Step 2: Data Retriever Pipeline

Conversation so far,

llm_system_prompt = sm(system_prompt)
messages = conversation_history + [llm_system_prompt]
pprint(messages)

[
    {
        "content": "You are a helpful assistant, the CPGRAMS bot, working with the Prime Minister of India's office. Your role is to help users share their grievances on a grievance redressal platform for the Government of India. You will not offer any solutions for their grievance. You will merely try to extract as much information as possible to allow a grievance officer to deal with the grievance effectively. Current date: 2024-04-25",
        "role": "system"
    },
    {
        "content": "Hi, How can I help you?",
        "role": "assistant"
    },
    {
        "content": "I am unable to access my PAN card from digilocker",
        "role": "user"
    },
    {
        "content": "Analyze the above conversation in detail and give your analysis. Your analysis should be detailed enough such that it could be used to determine the grievance category from a database. Do not try to determine the grievance category in this step. Current date and time: 2024-04-25 15:16:17. Process any date and time related information in the conversation.",
        "role": "system"
    }
]

Summarizing details provided so far

summary, cost = llm(messages, model="gpt-4-turbo-preview")
print(summary)

The user's grievance revolves around an issue with accessing their Permanent Account Number (PAN) card from DigiLocker. DigiLocker is a digital platform launched by the Government of India aimed at ensuring that citizens have access to authentic digital documents in a digital wallet provided by the government. The user's inability to access their PAN card from this platform can be attributed to several potential causes, such as technical glitches within the DigiLocker system, issues related to the linking of the PAN with the user's DigiLocker account, or discrepancies in the user's personal information that may be causing a mismatch.

Key elements to extract and analyze for effectively categorizing and addressing this grievance include:

1. **Nature of the Issue**: The problem is specifically related to accessing a government-issued document (PAN card) through a government-provided digital service (DigiLocker). This indicates that the grievance falls within the domain of digital governance and e-services.

2. **Services Involved**: The grievance involves two critical services - the issuance and management of PAN cards, which is typically overseen by the Income Tax Department, and the DigiLocker service, which is an initiative under the Ministry of Electronics and Information Technology (MeitY). Understanding the interplay between these services is crucial for addressing the grievance.

3. **Possible Causes**: The issue could stem from technical problems (e.g., server downtime, app malfunctions), user account issues (e.g., incorrect login credentials, PAN not linked to DigiLocker), or data mismatches (e.g., discrepancies in personal details between PAN records and DigiLocker).

4. **Information Required for Resolution**: To effectively address the grievance, additional information might be needed, such as error messages received, steps already taken by the user to try and resolve the issue, and confirmation of personal details for verification purposes.

5. **Impact on the User**: The inability to access the PAN card digitally can affect the user's ability to perform various financial transactions and comply with statutory requirements, highlighting the urgency of resolving the issue.

6. **Potential Departments for Escalation**: The grievance may need to be escalated to the Income Tax Department for issues related to PAN details and to the technical team behind DigiLocker for problems related to the digital platform itself.

This detailed analysis helps in understanding the complexity of the issue, which spans across technical, procedural, and personal information verification aspects, requiring a coordinated response from multiple government departments.

Fetching matching records from VectorDB

top_k = 10
relevant_categories = session.scalars(
    select(GrievanceCategory)
    .order_by(GrievanceCategory.embedding.l2_distance(summary_embedding))
    .limit(top_k)
).all()

[<GrievanceCategory(ministry=Unique Identification Authority of India, category=Linking of Aadhaar related issues, subcategory=['Linking of Aadhaar with PAN']>,

<GrievanceCategory(ministry=Central Board of Direct Taxes (Income Tax), category=PAN Issues, subcategory=['Other']>,

<GrievanceCategory(ministry=Central Board of Direct Taxes (Income Tax), category=PAN Issues, subcategory=['Delay in PAN issues']>,

<GrievanceCategory(ministry=Central Board of Direct Taxes (Income Tax), category=PAN Issues, subcategory=['Wrong PAN number allotted']>,

<GrievanceCategory(ministry=Central Board of Direct Taxes (Income Tax), category=PAN Issues, subcategory=['Mistakes in PAN card']>,

<GrievanceCategory(ministry=Central Board of Direct Taxes (Income Tax), category=PAN Issues, subcategory=['Same PAN numbers allotted to multiple users']>,

<GrievanceCategory(ministry=Ministry of Electronics & Information Technology, category=Digital Services (CSC/MyGov/NeGD/NIC), subcategory=['NeGD', 'DigiLocker']>,

<GrievanceCategory(ministry=Labour and Employment, category=Compliance related Issues, subcategory=['UAN related issues/KYC related issues.']>,

<GrievanceCategory(ministry=Telecommunications, category=Mobile Related, subcategory=['AADHAR Linking/Documents verification']>,

<GrievanceCategory(ministry=Unique Identification Authority of India, category=Linking of Aadhaar related issues, subcategory=['Linking of Aadhaar with Bank Account']>]

Ask clarifying question or are we done?

def determine_grievance_bucket_and_followup_questions(
    conversation_history, possible_buckets
):
    system_prompt = """You are a helpful assistant, the CPGRAMS bot, working with the Prime Minister of India’s office. Your role is to help users share their grievances on a grievance redressal platform for the Government of India.
    Here is a set of possible grievance buckets in which the user could raise a grievances. 
    Possible grievance buckets:
    {options}
    
    Analyze the conversation history and the user input. 
    Your primary task is to determine most relevant grievance bucket in which the user query can fall.
    Do not recommend grievance buckets if you do not have enough required information or you are not sure about. 
    Do not recommend grievance buckets that do not satisfy all the categories and subcatgories according to the information provided by user.
    If you think you do not have enough information, generate a short follow up question that can be asked to the user whose response can help in determining the grievance bucket.
    Also, generate an brief empathetic prefix message to show to the user before asking the followup question.
    Do not recommend grievance buckets if multiple grievance buckets from the list satisfing the criteria given by required information. Ask the user to provide more information.
    The follow up question should be very diverse. 
    You can ask upto 1 follow up question, not more than 1. 
    The follow up question should be very basic, do not ask question that you think a 15 year old would not be able to answer.
    You will not offer any solutions for their grievance. You will merely try to extract as much information to exactly determine the grievance bucket.
    Since you are not offering any solutions, you can ask any question that you think can help you determine the grievance bucket.
    Analyze and tell why you think the grievance falls under the grievance bucket. If you don't think any grievance bucket from the provided list is relevant, tell the reason for it.


    Give the output in the following json format:
    {{
        "grievance_bucket": 
            {{
                "bucket_id": "bucket_id",
                "ministry": "ministry",
                "category": "category",
                "subcategory": ["subcategory"],
                "description": "description"
            }}
        ,
        "follow_up_questions": [
            "question 1"
        ],
        "user_message": "Empathetic prefix message to show to user in a conversational manner. The message should be short and include emojis to show empathy and make the user feel comfortable."
    }}
    Provide empty list if no follow up questions are mentioned.
    Provide none if no grievance buckets are mentioned by user. Do not generate new grievance buckets. Only consider the grievance buckets provided.
    """
    system_prompt = sm(system_prompt.format(options=possible_buckets))
    buckets_and_questions, cost = llm(
        messages=conversation_history + [system_prompt],
        model="gpt-4-turbo-preview",
        response_format={"type": "json_object"},
    )
    response = json.loads(buckets_and_questions)
    return response, cost

grievance_buckets, _ = determine_grievance_bucket_and_followup_questions(
    conversation_history, possible_buckets_text
)
pprint(grievance_buckets)

{
    "follow_up_questions": [],
    "grievance_bucket": {
        "bucket_id": "6",
        "category": "Digital Services (CSC/MyGov/NeGD/NIC)",
        "description": "The DigiLocker service provided by the National e-Governance Division (NeGD) under the Ministry of Electronics & Information Technology allows users to store, access, and share digital copies of important documents and certificates. Users may have the following grievances related to DigiLocker:\n1. Difficulty in accessing or signing up for DigiLocker account\n2. Issues with uploading or downloading documents on DigiLocker\n3. Concerns about the security and privacy of personal information stored on DigiLocker\n4. Problems with linking DigiLocker to other government services or platforms\n5. Unauthorized access to DigiLocker account\n6. Technical glitches or errors while using DigiLocker\n7. Unclear guidelines or instructions on how to use DigiLocker effectively. Alternate terms for DigiLocker may include digital locker, online document storage, digital document repository, etc.",
        "ministry": "Ministry of Electronics & Information Technology",
        "subcategory": [
            "NeGD",
            "DigiLocker"
        ]
    },
    "user_message": "I'm really sorry to hear that you're facing issues with accessing your PAN card from DigiLocker. Let's get this sorted."
}

Asking for details required to lodge the complaint.

def generate_field_questions(grievance_bucket, required_fields, conversation_summary):
    system_prompt = """
    You are a helpful assistant, the CPGRAMS bot, working with the Prime Minister of India’s office. Your role is to help users share their grievances on a grievance redressal platform for the Government of India.
    
    You'll be grievance bucket, containing ministry name, category, sub categories and description of grievance in which user can file grievance.
    You're also provided the fields required to file the grievance.
    You'll be provided the conversation summary of the user.
    Analyze the conversation summary and determine the value of fields. 
    Some fields might require description of the grievance, so infer the description on your own instead of asking the user.
    Analyze and determine if value of all the mandatory fields are present. 
    Analyze the grievance bucket, and generate an brief empathetic prefix message to show to the user before asking the followup question.
    Everything should be brief. 
    If value of all mandatory fields are present, set the value of mandatory_data_present to True, else set it to False.
    If value of all mandatory fields are present, you can tell user in the brief emphatic message that all the required fields are present and ask them to confirm the values of the fields to proceed filing the grievance. 
    Do not generate more than 1 follow up question.
    Your task is to restructure the follow up in json format as shown below:
    {{
        "grievance_bucket": 
            {{
                "bucket_id": "bucket_id",
                "ministry": "ministry",
                "category": "category",
                "subcategory": ["subcategory"],
                "description": "description"
            }}
        ,
        "follow_up_questions": [
            "question 1"
        ],
        "user_message": "Empathetic message to show to user in a conversational manner. The message should be short and include emojis to show empathy and make the user feel comfortable."
        "field_data": 
            {{
                "field_1": "field_1_value", # field_1 is the name of field and field_1_value is the value. use none if the value is not present.
                "field_2": "field_2_value"
            }}
        ,
        "mandatory_data_present": True/False # Set it to True if value for all the mandatory fields are present, else set it to False.
    }}

    Provide empty list if no follow up questions are needed.
    
    """
    user_message = f"""
    Grievance bucket:
    {grievance_bucket}

    Required fields:
    {required_fields}

    Conversation Summary:
    {conversation_summary}
    """
    response, cost = llm(
        messages=[sm(system_prompt), um(user_message)],
        model="gpt-4-turbo-preview",
        response_format={"type": "json_object"},
    )
    response = json.loads(response)
    return response, cost

field_questions, _ = generate_field_questions(user_choice, required_fields, summary)
pprint(field_questions)

{
    "field_data": {
        "Device_Used": "none",
        "Document_Type": "PAN card",
        "Error_Message": "none",
        "Internet_Connection": "none",
        "Issue_Details": "Inability to access PAN card on DigiLocker, potentially due to technical issues, account problems, or discrepancies in linking PAN with DigiLocker account."
    },
    "follow_up_questions": [],
    "grievance_bucket": {
        "bucket_id": "6",
        "category": "Digital Services (CSC/MyGov/NeGD/NIC)",
        "description": "The user is experiencing difficulty accessing their Permanent Account Number (PAN) card on DigiLocker. This issue may be due to technical glitches, user account problems, or discrepancies in linking the PAN with the DigiLocker account. Access to the PAN card is crucial for the user for various financial transactions and identity verification processes.",
        "ministry": "Ministry of Electronics & Information Technology",
        "subcategory": [
            "NeGD",
            "DigiLocker"
        ]
    },
    "mandatory_data_present": true,
    "user_message": "We understand how important it is for you to access your PAN card on DigiLocker and the inconvenience this issue may be causing you. Rest assured, we're here to help. All the required details for filing your grievance are present. Could you please confirm the details so we can proceed with filing your grievance?"
}

tutorials

Tutorials

Quickstart

Quickstart

This quickstart is designed for linux based systems such as Linux distros and Mac. If using windows, Use WSL terminal to follow the quickstart. For installation instruction, refer to WSL Installation Guide.

Prerequisite

Docker - Ensure that your system has docker engine installed and running. For installation, please refer to docker engine installation instruction.
- The recommended way is to install docker desktop for your system. If using WSL with Windows, Enable docker for you WSL distro. Please refer to docker desktop installation for more details.
Docker Compose - Ensure docker compose is enabled along with docker engine. Please refer to docker compose installation instruction.
Tunnel - Tunneling exposes your local system to the services on the web by tunneling the external calls to your local system. Either of these two popular services can be used:
- Loophole - Please refer to the Loophole Quickstart Guide for installation.
- Ngrok - Please refer to the Ngrok Quickstart Guide for installation.
Translation and Speech API - JBManager uses speech processing API to handle audio and translation API to handle user input in various languages. Please refer to Translation and Speech API setup guide for more details.
Whatsapp Interface - This quickstart will focus on setup your own through whatsapp as channel. Please refer to channel setup guide for more details.

Running JB Manager

Clone and Change the directory to the project root.

$ git clone git@github.com:OpenNyAI/Jugalbandi-Manager.git
$ cd Jugalbandi-Manager

Copy the contents of .env-dev.template file to .env-dev in the same directory.

$ cp .env-dev.template .env-dev

Update the values of translation and speech fields in the .env-dev file.

# Bhasini Keys
BHASHINI_USER_ID=
BHASHINI_API_KEY=
BHASHINI_PIPELINE_ID=
# Azure Speech Keys
AZURE_SPEECH_KEY=
AZURE_SPEECH_REGION=
# Azure Translation Keys
AZURE_TRANSLATION_KEY=
AZURE_TRANSLATION_RESOURCE_LOCATION=
# Whatsapp API URL
WA_API_HOST=

Generate an Encryption key using the following command

$ dd if=/dev/urandom bs=32 count=1 2>/dev/null | openssl base64

and add it to ENCRYPTION_KEY in .env-dev file.

# Encryption key for storing credentials
ENCRYPTION_KEY=

Note: Remember to enclose value of ENCRYPTION_KEY within single quotes.

Create a folder media in the project root, and start a http server on port 5000 inside the media directory.

$ mkdir -p media
$ cd media
$ python3 -m http.server 5000

Start a new shell session and start tunnel for port 5000 on your system
- For Loophole, use the following command:
  $ ./loophole http 5000
- For ngrok, use the following command:
  $ ngrok http 5000

PUBLIC_URL_PREFIX= # Set Tunnel URL if using local storage

Start Kafka and Postgres services using the following command.

$ bash scripts/run.sh kafka postgres

Start JB Manager

To start the JB Manager, you can use the following command with or without the --stage option:

With --stage option:
- When you use the --stage option, the script will leverage docker-compose in conjunction with docker-compose.stage.yml file which uses existing Docker images without rebuilding them, which will save time. This is useful for staging environments where you might want to use different configurations or services compared to the development environment.
```
$ bash scripts/run.sh --stage api channel language flow frontend
```
Without --stage option:
- When you do not use the --stage option, the script will build the Docker images from scratch and run docker-compose with the default docker-compose.yml file. This is typically used for development environments where you want to ensure that you are working with the latest code changes.
  $ bash scripts/run.sh api channel language flow frontend

Once the services are up and running, you can access the JB Manager UI at http://localhost:4173.

Bot Installation and Go Live

1. Name [Mandatory] is the name of the bot. It should be same as the name of class defined within below-mentioned bot code (fsm.py). For this example, use CarWashDealerFSM.
2. Code [Mandatory] is the fsm.py file python code. Copy the contents of python file and paste it.
3. version [Mandatory] - version of the bot. Put 1.0.0.
4. required_credentials [Mandatory] - Credentials required by the bot to access various external services. Enter the following: AZURE_OPENAI_API_KEY,AZURE_OPENAI_API_VERSION,AZURE_OPENAI_API_ENDPOINT,FAST_MODEL,SLOW_MODEL , so put these keys in this section seperated by comma.
5. Click on Save button.
Start a new shell session and start tunnel for port 8000 on your system
- For Loophole, use the following command:
  $ ./loophole http 8000
- For ngrok, use the following command:
  $ ngrok http 8000

Add this url to register the callback URL for Whatsapp service provider. Your callback url will look like this <Tunnel URL>/v2/callback/pinnacle_whatsapp/<Whatsapp business account number>.

For this tutorial, we are using the shell script to add the callback URL using Whatsapp API host. Run the script on a new shell session with the appropriate values to register the callback URL.

#!/bin/bash

WEBHOOK_URL="<Webhook url here>"
WA_API_HOST="<API host here>"
WABA_NUMBER="<Whatsapp business account number>"
WA_API_KEY="<API Key here>"

BODY='{"webhook_url": "'$WEBHOOK_URL'"}'
echo $BODY
curl -k "$WA_API_HOST/v1/setwebhooks" \
--header "wanumber: $WABA_NUMBER" \
--header "apikey: $WA_API_KEY" \
--header 'Content-Type: application/json' \
--data-raw "$BODY"

Your bot is running. Send a Hi message to whatsapp business number to start conversation with the bot.

Quickstart FAQs

Cannot install loophole on Apple Silicon-based Mac?
Try setting up ngrok with a free tier subscription. Creating a single tunnel does not require any paid plan.
Which OpenAI Model to use?
You can use any model just make sure it supports generating output in JSON mode.

Getting Started

The quickstart guide within the technical documentation of Jugalbandi manager has a detailed step-by-step process for a developer to run their own instance, and begin building their services using Jugalbandi.

Local Development / Writing your own bot

If the application to be built using Jugalbandi is a conversational bot, it’s 5-steps for the bot to go-live:

Launch Jugalbandi Manager UI. Typically this would be in: http://localhost:4173/
Fill in some details like the name of the bot, version and the entire contents of the FSM file. Click on "Install new bot".
Provide required data to create your bot. This may include credentials to access LLMs (if applicable, as in the case of OpenAI services).
If Whatsapp is the chosen channel of communication for the bot, enter the number of an active Whatsapp Business Account (WABA). Since the service is running on a local HTTP server, the callback url for the tunnelling service (such as ngrok) needs to be registered with the service provider.
Activate the bot by clicking the play icon. Verify that the status of the bot has been changed to ‘active’.

Dependencies

While each use case will have different dependencies based on the services they choose to provide,and the medium through which Jugalbandi has to operate, some of the dependencies that are common across all implementations are:

Kafka: Manages message exchange between different parts of the system for real-time data flow.
Postgres Database: Stores user data and conversation history for reliable data access.
Bhashini Service: Handles language translation to support multilingual interactions. Or, Azure Speech: Converts spoken words to text and text to spoken words for voice interactions.
Whatsapp/Telegram API or API management services: Ensure the API host is correctly set up and accessible.

Configuring the FSM to modify flows

As described previously, FSMs in Jugalbandi manager help define the conversation flow and manage state transitions. This component is vital in customising Jugalbandi for each use-case, and defines terms under which external plugins may be used, and controls the information to be fed into and interpreted from the LLMs. To achieve this, the FSM uses 3 concepts:

State: Represents a specific point in the conversation flow. (Eg: Initial state, greeting state, information retrieval state, end state etc.) Each state is typically associated with actions that the application is supposed to undertake, this can include asking the user questions, or processing the user-input or sending messages to a specified channel (such as whatsapp, telegram or any CMS or CRM system) The ‘send_message’ function allows the application to message the user within each state. A very simple FSM flow would include the following states:
Initial State: The bot waits for user input.(such as ‘Hi’)
Greeting State: The bot greets the user.
Info State: The bot provides the requested information.
End State: The bot ends the conversation.

Global Variables: Global variables are defined in the FSM to cover core fields such as complainant name, address, dispute details, etc. These variables facilitate transitions between states. Status Indicators: Used to indicate the flow manager's actions:

Wait for User Input
Move Forward
Wait for Callback (e.g., waiting for an API response in PULSE)
Transition: The switch from one state to another.
Condition: Determines whether a transition should occur, and specifies the logic to trigger said transition.
This refers to the prompts which give the LLMs context about their specific tasks in the application that’s utilising them.
Lookup changes in the how-to guide of the indexer if any modifications are required in how the application would have to process the provided knowledge base.

When configured properly, the FSM allows any developer to manage how their application interacts with users, in order to provide a structured and predictable user experience. Detailed explanations on how the FSMs can be built can be found in the how-to section of the technical documentation, along with a reference implementation of an FSM.

Cost of building with Jugalbandi

Accessing the Jugalbandi stack is and will remain free, through the project’s GitHub repository. To be able to utilise the stack to build and offer solutions, any individual or organisation needs to consider the cost of operating the application and the associated data & infrastructure.

The key considerations in budgeting for a conversational service is as follows:

Estimating the usage and plan to scale

The amount of people using the service, how often they use it and the complexity of their queries will determine the associated costs of the cloud infrastructure and services that will be required for solutions such as Jugalbandi to operate
For example, if 100 users send 30 messages each day, an estimated ₹656 per month would have to be budgeted for processing these messages using LLMs such as GPT-4o mini or 3.5.
Start small and add more resources as needed. This helps control costs and scale efficiently.
Depending on the nature of the use case, cloud service providers are known to offer negotiated contracts to access their services at subsidised rates.

Monitoring data transfer and storage needs

Calculate how much data will need to be stored including user data, message logs and media files. This should also include the costs of encrypting and masking sensitive information.
The data in and out of cloud services, would have to be routinely monitored in accordance with the regional regulations on data protection and privacy.

Human Resources

Depending on the complexity of the use case, data scientists, data engineers and full stack developers may be needed to implement the solution as per the requirements.

Communication Costs

Based on the medium of communication with users, there may be additional costs incurred through third party service providers.
While the exact cost will vary based on the chosen service provider, at an estimated Rs. 0.3 per session, the same example from above of 100 users sending 30 messages a day would cost an additional ~Rs. 1000 a month over whatsapp.

The following table indicates the costs associated with deploying and operating a platform with Jugalbandi manager:

Cost Category

Service Required

Indicative Cost (as of July 2024, and all costs are subject to the terms of the service provider)

Notes

Cloud Infrastructure

Compute

~Rs. 10 per hour per VM

Running compute resources. The cost will vary based on usage

Storage

~Rs. 2 per GB

Storing user data and media. Estimated: ₹166.71 per month (assuming 100 GB)

Event Hubs for Kafka

~Rs. 120 per processing hour

Queue management for messaging between the various services of the platform.

Vector Databases

~Rs. 25 per hour

Scalable databases for AI models

Landing Zones

Can vary based on the chosen service providers

A one-time cost for the initial setup of cloud resources

Embedding Models

₹0.008336 to ₹0.010836 per 1000 tokens

Creating chunks of the provided knowledge bases to be consumed by AI models

Speech Processing

₹83.35 to ₹1,250.27 [per million characters or audio hours]

Converting speech to text, text to speech, and processing text.

LLMs

GPT-3.5 Turbo Tokens

₹0.125 (Input) + ₹0.167 (Output) per 1000 tokens

Processing user messages for AI responses. Estimated: ₹656 per month for a 100 users with 30 messages each

Technical implementation team: Data Scientists, Data Engineers & Full-Stack Devs

Whatsapp/Communication Channel Integration and Communication Services

Data handling, transfer, security and compliance costs

Tips and Best Practices / FAQs (Including best practices for creating a knowledge base)

How does Jugalbandi Manager work?

Jugalbandi manager is an AI-powered stack designed to process and respond to user queries by leveraging RAG over a predefined set of documents and data.It uses LLMs to generate accurate and contextually relevant answers. It is designed to be highly-customisable and can be configured to be interoperable with a wide range of services.

What is a knowledge base?

A knowledge base (KB) is a structured repository of information that can be queried and used to provide answers or insights. It serves as the primary source of data that the Jugalbandi Manager uses to generate responses to user queries.

How can Jugalbandi Manager benefit my organisation?

Jugalbandi enables any organisation to converse with users in their preferred channel, language and mode (text or voice based) of communication. It can streamline information retrieval, reduce the time spent on searching for answers, for just about any problem faced by an organisation or the stakeholders of their programs.

How to create a knowledge base?

While most LLMs can consume documents and catalogues without any modifications, as the information fed to them becomes larger and more complex, the accuracy and reliability of the answers generated may tend to falter. Organising the information before they’re presented to the AI models can help overcome their limitations as they scale.

Some things to keep in mind while creating a KB:

A. Identify the scope and categories

Determine the scope and main categories for the knowledge base. For example, for legal information, categories might include "Arbitration," "Contracts," "Dispute Resolution," etc. Setting a scope with appropriate version controls for each phase of development of an AI service allows for the testing of the generated answers to be more comprehensive. Organising the dataset by categories can help the team building the service and the LLMs differentiate between the datasets, and trace the origin of faulty answers.

B. Structure the data

Use structured formats like CSV or Excel or JSONs/XMLl to store the data. Ensure each entry is clearly labelled with metadata such as titles, authors, dates, and keywords. Providing variations of questions that may be asked by different types of users can help the LLMs relate the questions to the appropriate answer. One example of this format is as follows:

FAQ_ID

QUESTION V1

QUESTION V2

QUESTION V3

ANSWERS

LAW NAME 1

LAW NAME 2

LAW NAME 3

C. Create Cross-References:

Link related documents or sections to each other through cross-references, similar to hyperlinks on a website, to guide users and AI models through related content.

D. Create embeddings:

Use pre-trained models to generate embeddings for documents or sections. Embeddings are numerical representations of the data that capture semantic meaning, enhancing information retrieval. (Jugalbandi currently uses the model ‘text-embedding-ada-002’ and defaults to OpenAIEmbeddings) These embeddings are stored in a vector database (which are optimised for quick similarity searches) but can also be stored in a relational, noSQL databases or CSV/JSON/HDF5 files, although their performance may be relatively slower.

What are the limitations of using Jugalbandi Manager?

The accuracy of answers provided by Jugalbandi is dependent on the quality and comprehensiveness of the knowledge base it is built upon. Additionally, while the scope of AI is constantly widening, not every problem requires an AI solution. Each adopter needs to evaluate the necessity of using Jugalbandi, or any AI solution to their workstream, consider the costs of scaling a POC to serve their user base and alternative tech and non-tech interventions to address their problem.

Can Jugalbandi Manager be integrated with other systems?

Yes, Jugalbandi Manager can be integrated with various existing systems and platforms through APIs (and other integration tools such as webhooks, zapier, custom scripts and ETL tools), making it adaptable to the specific needs of different organisations.

What kind of support and training is available for adopters of Jugalbandi Manager?

Support and training are available in the documentation of Jugalbandi manager, replete with reference implementations, how-to guides and explanations of the stack and its operations. The OpenNyAI team is also available to facilitate teams to quickly adopt and start using the stack, and anyone can readily seek support through the discord channel.

Use cases of Jugalbandi

The selected use cases of Jugalbandi in this section illustrate its applications across various domains, including those at Proof of Concept (POC) and implementation stages. Collaborations on these span across national and international governments, civil society, and regulatory bodies. This section does not cover all the use cases. The intent behind covering specific use cases is two fold:

Jugalbandi’s ability to ease access to justice. For instance, initial use cases like NHRC Bots, JIVA, Nyaya Setu, JAGRIT, and SEBI's Investor Knowledge Portal highlight its role in providing quick, verified, and contextualised information, including legal and regulatory details and scheme-related information. Beyond information provision, other use cases show how Jugalbandi supports grievance redressal by helping users structure grievances effectively and connect with relevant departments or dispute resolution services via open protocols like PULSE.

Access to Legal Information - NHRC, JIVA , Nyaya Setu, JAGRIT, SEBI

Government Schemes- MySchemes

Grievance Redressal At Scale - CPGRAMS

PULSE

Outlawed

Highlight applications beyond justice-related functions, such as service provision for marginalised populations (e.g., Bandhu's housing use case) and addressing environmental justice and climate action (e.g., Wildlife First use case). These instances demonstrate Jugalbandi's flexibility and its ability to integrate with diverse services, such as using Bhashini for Indian languages in India-based use cases and Microsoft Azure for international applications, ensuring adaptability across operational needs.

Access to Housing for marginalised populations Bandhu

Environmental Justice / Climate Action - Wildlife First

International use cases - UNESCO

Jugalbandi for Access to Legal Information

Community members involved

NHRC, Karnataka High Court, SEBI and the Department of Justice

Problem statement

The language of the law has often been inaccessible to the average citizen, regardless of the domain the law caters to. Interpreting the nuances of legal language and its linkages to related policies and other laws, has been the cornerstone of legal advice, which is almost never free. This complexity can reduce the law’s ability to protect its intended beneficiaries, hinder the ability of the agents of the state to enforce it and deter its effectiveness in curbing malpractices.

Even for institutions that serve, protect and enforce the law, their operations and administration can be eased with quicker access to legal research. The judicial system in India often faces prolonged case durations, taking an average of six to nine months to resolve a case. This lengthy process places a significant strain on judges, lawyers,litigants and the staff involved. A major contributing factor is the outdated and traditional methods of information retrieval used in the judiciary. Judges and court staff spend considerable time manually searching through legal documents and references, which delays the resolution process. For regulatory bodies, while detailed awareness and cautionary material have been publicly released, their accessibility can still be improved. In the case of financial markets, SEBI’s investor awareness and support portal sits on a wealth of such information. However, it doesn’t yet have the ability to be queried conversationally.

Another avenue of the law where there’s a pressing need for credible legal information is in the case of India’s migration to the new criminal laws, wherein the three primary criminal laws of the country were replaced. The Bharatiya Nyaya Sanhita (BNS) replaces the Indian Penal Code (IPC), the Bharatiya Nagarik Suraksha Sanhita (BNSS) replaces the Criminal Procedure Code (CrPC), and the Bharatiya Sakshya Adhiniyam (BSA) replaces the Indian Evidence Act (IEA). With the new laws set to come in place on the 1st July 2024, a solution was sought that could allow agencies of the state (such as the police), lawyers etc. to quickly map their understanding of the old laws to the newer provisions, and quickly get up to speed with the changes in the newer laws.

How the problem was addressed

Through the Jagrit telegram bots, Jugalbandi aimed to create a more accessible and user-friendly legal information system to deal with matters of motor vehicle accidents and domestic violence. The solution involved converting legal documents into a question-and-answer format, allowing users to ask specific legal questions and receive direct answers. Particularly in the case of domestic violence queries, Jagrit was designed to be sensitive to the nature of queries and to not store or process any information that could be used to identify the person seeking legal help.

For such complex issues, users may also choose to be connected with legal advisors through services such as TeleLaw.in, within the regional language chat interface.

JIVA, short for Judges Intelligent Virtual Assistant, is a Jugalbandi-powered tool designed to aid judges in their search for laws and legal information. With just a few keywords or a specific query, JIVA enables judges to swiftly access and retrieve the relevant PDFs of legislation or specific sections. This advanced tool has been fine-tuned to cater to Karnataka-specific state laws and specific central laws. By focusing on these specific legal domains, JIVA ensures that judges in Karnataka have access to accurate and up-to-date information that is relevant to their jurisdiction, which could be queried via text or voice.

Jugalbandi was configured to assist SEBI by extracting relevant information from regulatory filings and investor awareness documents. Users could then conversationally go through these documents or ask pointed questions that they may have in their dealings with securities & financial assets.

An instance of Jugalbandi was created with the knowledge base that included a comparative study of the IPC, CrPC, IEA & BNS, BNSS and BSA acts. This process involved preparing and cleaning an excel sheet that mapped the relevant sections and subsections of the older laws to the newer laws.This data was then structured into chunks, with each chunk containing relevant sections, titles, content, and consequences from both old and new laws. This structured data was used to create a repository of information that the bot could query.

User Journey

For the conversational bots, the user could use it in one of two ways:

Mention the section/sub-section of the relevant laws and get information about related acts or newer versions of the law
Ask general queries about matters related to criminal law, and the application would provide the answers
In use cases where the Jugalbandi service were integrated with services such as the DoJ’s Nyaya Setu, the service could connect users to legal advisors registered on the platform, ensuring the users receive comprehensive assistance.

Some of these implementations also took the shape of a dashboard. The user could use said dashboard to:

Retrieve the appropriate legal document relating to a matter
Retrieve the relevant section/sub-section of a legal document
Avail the services of a chatbot, as with the previous example

5. What’s happening in the backend?

Prompts and Query Handling: Legal documents are converted into an FAQ format, ensuring complete and coherent chunks of information. Jugalbandi first processes the user's query, identifying sections and subsections if specified, after matching it to the relevant FAQs fed as knowledge bases while building the service.

Metadata Filtering: When a user submits a query, Jugalbandi extracts relevant metadata such as section numbers and titles.This metadata is then used to enhance the semantic search process, making it more precise. The metadata includes details like the source document, section numbers, titles, and any related subsections. An example of how this metadata has been organised for searching for titles or specific sections of a document has been indicated in the appendix.

Retrieval: Jugalbandi retrieves the top five relevant chunks from the knowledge base based on the query and metadata. These chunks contain the necessary information to answer the user's query.

Answer Generation using LLMs: The language model (typically GPT-4o mini or 3.5) processes the retrieved chunks to extract the most suitable and relevant answers, which are then presented to the user.

Accessing information from third party services: Where necessary, the Jugalbandi service can retrieve information from a trusted information and services provider. For example: contact details about lawyers who specialise in the types of cases relevant to the user’s grievance or disputes.

Implementation requirements Tools:

API services to communicate with a whatsapp business account
Indexed legal datasets such as FAQ database for domestic violence and motor vehicle laws.

Team:

Developers to handle the technical development and maintenance
Functional/product personnel to develop scripts, manage knowledge bases (with one spreadsheet mapping the changes from the old laws to the new for each of the three criminal laws), oversee testing, and maintain communication with the developer.

Timeline

The development of the POC of a judge’s dashboard took approximately 2-3 months, including data modelling and testing of the service. Building out the other POCs took 2 weeks including:

Status

In the case of the bot to better understand the transitioning to the newer criminal laws, the application will undergo further testing and feature development, where another organisation will be refining its functionalities and ensuring it meets the users' needs. The OpenNyAI development team will continue providing clarifications and support during this phase.

Roadblocks & Learnings

Some applications/intent classification required multiple calls to the LLM. Making them more cost and resource intensive

Jugalbandi for Govt. Schemes

Community members involved

MySchemes, Digital India Corporation

Problem statement

Citizens often face difficulties in accessing relevant information about various government schemes due to the complexity of navigating through numerous central and state schemes. This makes it challenging for them to understand which schemes they qualify for and how to benefit from these government services.

How the problem was addressed

A service was developed that could accept open ended queries from users, understand their needs, and map these to the relevant government schemes. The service continuously asks follow-up questions until it has gathered enough information to provide accurate scheme recommendations. It can provide information about which scheme is most relevant for the user, benefits of the scheme, eligibility criteria, documents required.

What’s happening in the backend?

The jugalbandi stack facilitated the following tasks:

Data Scraping: Extracting data from the MySchemes website to gather information about various government schemes.
Data Structuring: Organising the scraped data into a predefined format for easy querying, which serves as the knowledge base for the service.
Intent classification: Handling user queries and prompt engineering to ensure comprehensive information retrieval.
Integration with Bhashini Models: Uses Bhashini models for speech-to-text, translations, and text-to-speech functionalities.
Feedback Loop: Continuously refining the system based on user interactions to improve accuracy.
Answer Generation: Based on the highest ranked chunks from the provided knowledge base to provide the most relevant information to the user.

An illustration of Jugalbandi’s RAG pipeline

Implementation Requirements

Team:

2 Functional Personnel: To develop scripts, manage the knowledge base, oversee testing, and communicate with the developer.

Timeline:

Total 5 weeks including:

Initial Setup and Data Scraping: 1 week

Data Manipulation and Organization: 1 week

Development of Query Handling and Follow-up Mechanisms: 1 week

Testing and Iteration: 1 week

Deployment and Monitoring: 1 week

Roadblocks and Learnings

Scalability: The initial versions were not highly scalable, but this has been improved with the Jugalbandi manager, which uses microservices and Kafka queues for better handling of multiple requests.

User Interaction: Designing the chatbot to ask the right follow-up questions was crucial for understanding user needs and providing accurate scheme recommendations.

Accessing Legal Services

Community members involved

CADRE, CORD, Presolv 360, Sama, Beckn, Agami, Thoughtworks, PUCAR

Problem statement Most Indian citizens face significant challenges in accessing effective dispute resolution services, particularly through institutions that operate independent of courts. Traditional legal processes can be complex, time-consuming, and expensive, making it difficult for people to resolve disputes efficiently. This inaccessibility often leaves individuals without the necessary support to address their legal issues, such as cheque bouncing disputes, which can have serious financial and legal implications.
How the problem was addressed

The project aimed to create a solution that simplifies and makes the services of Online Dispute Resolution (ODR) providers more accessible. Building on the concept of Jagrit bots, which provide users with legal information, the project focused on developing a platform where users can take practical steps based on the information they receive.

Taking the process of dealing with cheque bounce disputes as the unit of change, this solution would allow a user to:

Seek information on legal matters arising from a bounced cheque.
Draft a legal notice demanding payment for the bounced cheque.
Access services of an ODR platform to resolve the dispute.

Jugalbandi could readily deploy a conversational bot that could answer queries based on the provided knowledge base of cheque bounce disputes. For the discovery and availing of legal services, an adaptation of the Beckn protocol was utilised. The Beckn Protocol is an open and decentralised protocol that facilitates seamless transactions across various sectors by enabling the creation of peer-to-peer digital networks. It allows businesses and service providers to be universally discoverable from any Beckn-enabled application, removing the need for centralised platforms.

The PULSE protocol, is an adaptation of Beckn for legal services, it enables any user to discover and avail the services of various legal service providers, but existing implementations of the protocol (on the seeker side, for those who intended to use the protocol to discover legal service providers) operate as web applications that use drop down menus and static buttons, all in English.To make the protocol demonstration more user-friendly, Jugalbandi was configured to enable users to access services, such as those provided by Online DR platforms.

User Journey Information on cheque bouncing disputes that were publicly available from credible sources, as well as PUCAR’s research on such disputes were synthesised into tables that could act as an effective knowledge base. Scripts detailing users' possible journeys were created, along with flow-charts that could help developers quickly familiarise themselves with the configurations required to the flow states of the Jugalbandi manager.

A brief summary of the user journey is as follows, for the more detailed flow refer to the appendix:

User Initiates Interaction:

The user receives a notification about a bounced cheque on the WhatsApp bot.
The bot asks the user for their preferred language, name, and place of residence.

Bot Provides Options:

The bot informs the user it can assist with connecting to a banking representative, providing legal rights information, or seeking legal recourse.
The user opts to get legal information on what can be done about the bounced cheque.

Legal Information and Actions:

The bot explains the legal procedures and consequences of a bounced cheque, including the need to draft a demand notice.
The user can choose to draft a legal notice, consult a lawyer, or use an Online Dispute Resolution (ODR) platform.

Action Execution:

If the user chooses to draft a notice, the bot collects necessary details and uses an API of a third-party service (such as EazyDraft) to create the draft.
If the user needs a lawyer, the bot assists in discovering available legal service providers and scheduling appointments.

What’s happening in the backend?

Understanding User Request/Concern: Jugalbandi parses user inputs regarding cheque bounce disputes. By employing cost-effective LLMs such as GPT- 4o mini or 3.5, it identifies the user's specific needs, whether it is seeking information, drafting a legal notice, or consulting legal services. The prompt that allows Jugalbandi to understand the context of user’s messages for this use case can be found in the appendix.
Optimising Information Retrieval: The knowledge base, consisting of spreadsheets and PDFs, was pre-formatted for optimal use by Jugalbandi. The information is divided into small, easy-to-parse pieces to make searching easier. Through a process of testing and refining, it was decided that the five best chunks of information will be considered, so that it most closely matches the user's question. These 5 chunks are then used to generate answers using LLMs.
API Integrations: This instance of Jugalbandi requires extra configuration to call relevant third party APIs (like PULSE, EazyDraft etc). These APIs provide additional information that Jugalbandi can use as dynamic knowledge bases. With these integrations, Jugalbandi can offer not only the information from its internal RAG pipeline but also provide data and services from various legal platforms and service providers. This makes Jugalbandi more versatile and able to deliver a wider range of information and services.

Implementation Requirements:

Tools:

API services to communicate with a whatsapp business account & a testing environment for PULSE APIs.
Access to the Beckn sandbox to test the PULSE protocol

Team:

Developer to handle the technical development and maintenance.
Functional personnel to develop scripts, manage knowledge bases, oversee testing, and maintain communication with the developer.

Timeline: 2 weeks, including:

Status

Upon development and testing of 2 bots that leveraged the PULSE protocol, ie, the cheque bounce bot and a helper bot to aid small businesses and aspiring entrepreneurs navigate legal/regulatory hurdles. The code repository for the POC can be found in the OpenNyAI Github page.

Roadblocks & Learnings

One of the hurdles encountered during the implementation of this whatsapp bot, was in the usage of forms (or whatsapp ‘flows’), a pop-up template to collect information from the user. This form was used to collect information on the complainant details, respondent details and the dispute details.

While the whatsapp form operated without a glitch for most devices, for some devices (that didn’t have any correlation of the OS used, version of whatsapp etc.) returned the parameters of the form itself, instead of the information entered by the user. Example: The form returned “form.name”, instead of the name interred by the user in the respective field, the field itself was returned to the FSM. While the root issue of this bug hasn’t yet been uncovered, the issue has been escalated to the service provider facilitating managed services for the Meta whatsapp APIs, as well as the team from Meta responsible for the whatsapp flows.
Implementing the PULSE APIs presented challenges, particularly with fields like Dispute details and consent forms, which required the use of X-Forms Getting Started. The Beckn Sandbox did not support rendering these forms for collecting information outside Beckn’s core specifications. After consulting with the Beckn team, it was decided to simulate the collection of this information by creating mock data to represent the required fields. This approach allowed the team to integrate and test the PULSE API flow without relying on the unsupported functionalities of the Beckn Sandbox.

Jugalbandi for Grievance Redressal

Community members involved: CPGRAMS, Bhashini, Microsoft Research, Celebal Technologies

Problem statement:

Grievance redressal systems across various government departments and schemes face significant challenges. Citizens often struggle with the cumbersome process of registering grievances, as they must determine the appropriate department or scheme to address their issue. This complexity is compounded by the fact that each state, department, branch, and at times individual schemes can have their own Grievance Redressal Officer (GRO). For example, different government departments may oversee various hospitals. This lack of clarity about which department is responsible for a specific hospital complicates the grievance filing process.

These systemic issues highlight the need for a more user-centric approach to grievance redressal. The current process can be confusing and time-consuming for citizens, who may provide ambiguous details or be unsure of the correct department for their grievances. This situation underscores the necessity for a solution that can simplify the grievance registration process by accurately determining the relevant department, clearly structuring grievance descriptions for departmental officers, and automatically populating necessary forms based on user-provided descriptions.

How was the problem addressed?

Besides integration with LLM to interpret and classify grievance descriptions shared by the citizen accurately, the proposed solution had two more elements, including:

Intent Classification: Designing the solution to continue asking follow-up questions until it gathers enough details to accurately understand the user's issue.
Retriever-Based Approach: Implementing a retriever system to index and retrieve relevant categories based on user inputs, ensuring precise category selection.
Automated Form Filling: Utilising AI-generated inputs to automate the completion of grievance submission forms, reducing manual effort and improving efficiency.

Scenario based User Journey with CPGRAMS Jugalbandi bot

User: "I have not received my ticket yet."
Bot: "Thank you for reaching out. Could you please provide more details about the ticket you're referring to? Are you referring to a travel ticket such as flight, train etc. or an exam entrance ticket or grievance ticket etc.
User: Travel ticket
Bot: Awesome, please specify mode of transport
User: train
Bot: When and where was it issued, and have you received any confirmation or tracking details?"
User: "I booked a train ticket two weeks ago, but I haven't received any email or physical ticket."
Bot: "Understood. To assist you better, could you please specify the departure date and destination of your journey?"
User: "Sure, the ticket was for a train journey from New Delhi to Mumbai on July 15th."
Bot: "Based on the information provided, it appears that your grievance pertains to a 'Non-receipt of Train Ticket.' I will now categorize your grievance accordingly."

The bot automatically populates the required fields in CPGRAMS' submission form related to complaints about train tickets. It includes details such as the user's personal information, booking reference (if known), departure date, journey details, and specific grievances related to the non-receipt of the ticket. After confirming the details filled by the bot, the user clicks on the "Submit" button.

What's happening in the backend?

Understanding Grievances: When a user submits a grievance, Jugalbandi analyzes the provided details to understand the issue.
Finding Relevant Categories: Jugalbandi searches through a dataset (such as an Excel or document file) with descriptions of different grievance categories to find matches.
Matching Similarities: It compares the user's description with those in its database to identify the most similar categories.
Selecting the Best Fit: The system ranks potential categories based on their relevance and selects the top-ranked category or provides options for the user to choose from.
User Confirmation: To ensure accuracy, the system may ask the user to confirm the selected category or provide feedback, helping to clarify any uncertainties.
Automating Form Completion: Once the category is confirmed, the system fills out the necessary fields in a format suitable for submission to the appropriate portal. (For eg - the CPGRAMS portal or Karnataka - Janaspandana, CM Helplines, IT Grievance etc.)

Implementation Requirements

Tools:

GPT-4 and GPT-3.5 models for parsing existing grievance categories and generating example grievances for each category
Cleaned dataset for storing grievance categories.
Jugalbandi’s Retriever component for category selection.
Third party APIs for submission of grievances to the relevant departments

Team:

Development team for bot implementation, including AI model integration and cloud migration
Functional consultants for domain expertise in grievance management and workflow design.
Quality assurance team for testing and validation of bot functionalities.

Learnings

Retriever-based approaches can play a crucial role in improving how information is categorised and retrieved across various applications.
Incorporating user feedback into the retrieval process can help in identifying and correcting misclassifications or inaccuracies. When user interactions with the system are analyzed, developers can iteratively refine the retrieval methods to better meet user expectations.

Climate Action with Wildlife First

Community: Wildlife First

Problem Statement

Forest rangers in India face challenges in effectively addressing wildlife and conservation issues due to a lack of immediate access to comprehensive legal information. In most of their postings and patrolling areas, they lack the ability to communicate with other rangers, legal experts, or avail online services such as google or legal information forums. They often mistakenly book poachers under less stringent civil acts instead of more severe laws like the Arms Act, leading to inadequate legal action against offenders.

How was the problem addressed?

A dashboard was developed for forest rangers, providing them with easy access to critical legal information and resources. This dashboard facilitates the following:

Information Sharing: The dashboard allows forest rangers to share vital information and collaborate more effectively. They can document their experiences, share tips, and keep track of ongoing issues related to wildlife conservation.
Conversation Threads: Rangers can maintain and revisit conversation threads, ensuring continuity in communication and better follow-up on previous discussions.
Legal Reference: By digitising material on conservation laws (including bare acts and the two books published by Praveen Bhargav from Wildlife First), the dashboard enables rangers to quickly look up legal information. This feature helps rangers understand which laws to apply in various situations, such as distinguishing when to book poachers under the Arms Act instead of less stringent civil acts.
Drafting Legal Documents: The dashboard assists rangers in drafting complaints and FIRs. It ensures that all necessary legal details are included, improving the accuracy and effectiveness of legal actions taken against offenders.

To support these functionalities, Jugalbandi was integrated as the backend system for information retrieval. Jugalbandi accesses the digitised legal texts and provides rangers with relevant sections and laws in Kannada. This integration ensures that rangers have access to accurate and actionable legal information in their native language, which is crucial for effective on-ground implementation.

What’s happening in the backend?

Dashboard Integration: A dashboard was developed to allow forest rangers to share information, maintain conversation threads, and draft legal documents. This ensures effective collaboration and communication among rangers. Jugalbandi requires configuration to communicate with this dashboard.
Legal Information Retrieval: When a ranger queries the system, Jugalbandi searches through the digitised legal texts (including the forest act, forest rights act, arms act and the provided books) to find relevant sections and laws. It retrieves the top five chunks of information that are most relevant to the query.
Metadata Filtering: Jugalbandi uses metadata such as section numbers and titles to enhance the search process, ensuring the retrieved information is precise and contextually relevant.
Language Support: Jugalbandi was configured to provide information in Kannada, ensuring that forest rangers can access legal details in their native language.
Automating Document Drafting: Once the relevant legal information is confirmed, the system assists in drafting complaints and FIRs by automatically filling out necessary fields based on the user-provided details and retrieved legal references.

Implementation Requirements

Tools:

Digitised legal texts of wildlife laws.
API services for information retrieval and dashboard integration.

Team:

Developer for technical development and maintenance.
Functional personnel to manage scripts, oversee testing, and maintain communication with the developer.

Timeline

May: Initial meeting and knowledge transfer session.
June: Deployment of a minor version of Jugalbandi.
July: Further refinement and testing.
August: Expected pilot launch.

Status

The project began in May 2024 with an initial meeting. A version of Jugalbandi was deployed within a month, and further development continued. By July, the team was refining the implementation and expected to pilot the solution by August. The project is currently in the testing and refinement phase.

Roadblocks & Learnings

Bandwidth Limitations: Limited resources and developer availability slowed down the development process.

Affordable Housing with Bandhu

Community Involved

Bandhu: Affordable housing platform focusing on migrant workers.
Microsoft Research: Providing technical expertise and support.
Glyphic: WhatsApp chatbot service provider.
Agami: Facilitated the initial connection between Bandhu and Jugalbandi.

Problem Statement

Migrant workers in India often face significant challenges in finding affordable housing when they move to new cities for work. The lack of accessible information and support systems makes it difficult for them to secure safe and affordable accommodation.

How Was the Problem Addressed?

Bandhu developed a platform to help migrant workers discover and avail affordable housing. The solution involves integrating housing options, support services, and relevant information into a user-friendly platform. Through an API, Jugalbandi accesses housing registries compiled by Bandhu along with other third-party services like payment gateways to streamline the process of securing accommodation.

User Journey

Query Input: Migrant workers enter their housing requirements and preferred locations into the platform.
Information Retrieval: The system searches Bandhu’s database of affordable housing options and related support services.
Result Display: The user receives a list of available housing options, complete with details about rent, location, amenities, and additional services.
Payment & Confirmation: The Jugalbandi-enabled service facilitates secure transactions and official registrations.

What’s Happening in the Backend?

Query Processing: User queries from workers are processed to understand the specific housing requirements.
Third-Party Integrations: Integration with payment portals allows users to make secure payments. Bandhu’s services help with the discovery of resources, official documentation, and registration.
Search and Retrieval: The system retrieves the most relevant housing options based on the user's query.
Response Generation: The platform generates a list of suitable housing options, providing users with detailed information.

Implementation Requirements

Tools:

Database of affordable housing options.
Payment Gateway.

Team:

Developers for technical development and maintenance.
Housing/migration experts to verify the accuracy of the integrated information.
Quality assurance team for testing and validating the platform.

Status

Limited implementation with small groups; still running usability experiments and improving the prompts and APIs.

Roadblocks and Learnings

Complexity in Data Integration: Integrating various third-party services like payment portals and housing registries was challenging. Ensuring seamless communication between these services and the Bandhu platform required significant effort.
Query Processing: Ensuring the chatbot responds intelligently, accurately, and relevantly to user inputs was a challenge. It required refining the length and accuracy of the backend queries to improve response relevance.
User Engagement: Migrant workers were pleasantly surprised by a voice-based solution designed specifically for them, indicating a positive user reception and a need to continue improving the accessibility and usability of the platform.

Paralegal Services with Nyaya Mitra

Community Involved

Outlawed, Nyaya Mitra Paralegals

Problem Statement

Paralegals play a crucial role in providing first-contact legal aid, such as helping with FIR registrations, protection orders, and connecting clients to NGOs and legal services. They gather essential information from clients to assist with legal matters. But the paralegals working with outlawed faced several challenges in their work, including difficulties with data collection due to time constraints and unsuitable user interfaces. Manual follow-ups for collecting complete data are inefficient, leading to incomplete reporting and data loss. There was a need for a system that could efficiently collect, manage, and track case data while being accessible and user-friendly for paralegals.

How was the problem addressed?

To address these challenges, a conversational service was developed to streamline data collection and case management. The service includes functionalities such as voice-to-text conversion, asking follow-up questions to ensure complete data collection, and the ability to handle document uploads. It was designed to be user-friendly and responsive, catering to the specific needs of paralegals. The introduction of a dashboard further facilitated easy tracking and management of case data.

User Journey

Outlawed’s Jugalbandi based service facilitates with the training of the paralegals, helps them log their one-time services for clients in need of legal aid, as well has provides a more comprehensive monitoring technique to follow up on legal matters that may require multiple consultations over a prolonged period.

Broadly, the actions of a paralegal when facilitating their clients through this conversational service would be as follows:

Data Collection: Paralegals use the bot to collect client details, including name, contact number, gender, age, and a description of the legal issue.

Follow-Up Questions: The bot asks follow-up questions if necessary, ensuring all relevant information is collected.

Document Upload: Paralegals can upload documents, such as FIRs or RTI filings, directly through the bot.

Client Database: Once client details are logged, they are stored permanently in the database, allowing for easy follow-up and tracking of repeat clients.

Periodic Follow-ups: After a specified period, the action items for the paralegals are highlighted for them to go back to their clients and reassess the need for any intervention.

What’s happening in the backend?

Understanding Grievances: The bot analyzes user-provided details to understand the issue.

Finding Relevant Categories: It searches through a dataset with descriptions of different legal categories to find matches.

User Confirmation: The system may ask the user to confirm the selected category or provide feedback to ensure accuracy.

Automating Form Completion: Once confirmed, the bot fills out necessary fields for submission to Outlawed’s coda dashboard.

Status

With a launch planned for late August 2024, the bot is currently being tested and refined based on feedback from paralegals. It has shown promise in improving data collection accuracy and efficiency. Discussions are ongoing with cloud service providers for hosting and maintaining the bot.

Implementation Requirements

Team:

1 developer for building, deploying and managing the Jugalbandi based service

2 functional personnel familiar with the operations of paralegals and their community’s requirements to design the flows and oversee testing and refinement

5-8 paralegals to test and provide feedback

Timelines

4 weeks including:

1 week to create user flows and relay them to the development team

1 week to build a preliminary prototype of the indicated flows (including a feature addition of accepting user documents, apart from the KBs that were provided during the creation of the bot)

2 weeks for testing and refinement based on the feedback of the paralegals

Roadblocks and Learnings

Data Collection Issues: Initial form-based data collection was ineffective due to time constraints and user interface challenges, which resulted in inadequate data being captured and multiple follow ups with the client.

Voice-to-Text Difficulties: Ensuring accurate voice-to-text conversion for various languages was challenging, particularly in the matter of interpreting dates from the voice notes.

Training Needs: Initial testing emphasised the importance of training paralegals on the significance of data collection and proper usage of the bot.

UNESCO X Jugalbandi

Is Jugalbandi Manager Applicable to your Use case?

Jugalbandi is a versatile tool that can be applied to a wide array of scenarios as long as we think of it as an “information and action tool”. The term "action" is broad and encompasses a wide range of possibilities, so instead of narrowly defining what actions Jugalbandi can facilitate, we encourage you to consider the potential applications in your context. To assist you in evaluating whether Jugalbandi is a highly applicable solution for your needs or if it requires some modifications, or if another digital tool might serve you better, we have developed a thinking model. This model is designed to guide you through key questions to assess the direct applicability of Jugalbandi for your use case. By answering these questions, you can determine how well Jugalbandi aligns with your requirements and identify any adjustments that may be necessary.

Primary Questions - If your use case fits into any of these questions, its highly applicable for the adoption of the Jugalbandi stack

Provision of quick and verified information

Do your users usually need to search through large amounts of information? Is providing immediate, contextualized, and verified information a critical aspect of your service to the end user?

Multichannel Communication

Do you require a system that can also support multiple communication channels (e.g., WhatsApp, Telegram, web apps, IVRS)?

Multilingual Support

Is your organization looking to provide services or information in multiple languages?

Action Support

Do you need to allow the users to take actions going beyond just provision of information, such as booking appointments, signing up for services, or initiating legal proceedings through a conversational interface?

Secondary Questions - These are important considerations if the primary questions indicate that Jugalbandi is highly applicable. They help to further refine the understanding of how well the Jugalbandi stack can be implemented and tailored to specific needs, but the answers might not be immediately known or might require further assessment.

AI Infrastructure

Can your current IT setup handle AI operations, including data processing and storage? Do you have access to cloud computing platforms like AWS, Google Cloud, or Azure?

Data Privacy Compliance

Is the application of Jugalbandi in congruence with the data and privacy policies/regulations applicable to the organization?

The versatile applications of Jugalbandi

Jugalbandi holds transformative potential across various sectors by bridging language barriers and personalizing the user experience. It could change how we learn, manage health, farm, handle finances, or interact with any organization or government services.

In education, Jugalbandi can be a game-changer. Think about students in remote areas who struggle because educational resources are not available in their native language. Jugalbandi can step in here, providing personalized tutoring and real-time assistance in multiple languages. By integrating with educational platforms, it can adapt learning content to fit each student’s unique needs, making high-quality education accessible to everyone, regardless of where they live.

Healthcare is another domain where Jugalbandi’s potential is immense. Imagine visiting a telemedicine platform and having Jugalbandi help you understand your medical conditions, guide you through medication instructions, and offer advice on lifestyle changes, all in a language you are comfortable with. Imagine a pregnant woman participating in a women's health program, receiving interactive daily advice on nutrition, vaccinations, and prenatal care in her native language, all from the comfort of her home. Jugalbandi can not only provide this personalised information but also serve as an action tool, connecting her with doctors when needed.

For farmers, Jugalbandi can be a valuable tool in increasing productivity and making informed decisions. Picture a farmer receiving timely updates on weather conditions, crop management tips, and market prices, all tailored to their specific region and delivered in their native language. By integrating with platforms like the National Agriculture Market (eNAM) and other agri-tech solutions, Jugalbandi can provide data-driven recommendations that make farming smarter and more efficient.

In the financial sector, Jugalbandi can drive greater inclusion by demystifying complex financial processes for users. Imagine using a digital payment platform where Jugalbandi guides you through transactions, explains financial products, and helps you make informed decisions. This could be especially beneficial for people in rural areas who might not have access to traditional banking education but have mobile phones and digital connectivity.

Governance can also benefit immensely from Jugalbandi’s capabilities. Think about interacting with government portals where Jugalbandi helps you navigate various services, apply for schemes, and provide feedback, all seamlessly. This can streamline administrative processes and ensure that citizens receive accurate and timely information.

Another potential future use case, not very far away with Jugalbandi is AI-led facilitation of mediation between banks and borrowers facing loan repayment issues. Jugalbandi can collect relevant information, offer settlement options, and handle exceptional cases with tailored solutions.

Integrating Jugalbandi with other components of Digital Public Infrastructure (DPI) can further amplify its impact. Imagine linking it with Aadhaar for seamless identity verification, making interactions more secure and personalised. By integrating with India Stack, service delivery can become more efficient, ensuring smooth data flow across various platforms.

Envision Jugalbandi in the domain of beneficiary identification, particularly in a diverse and populous country like India. By engaging beneficiaries in their native languages, Jugalbandi can streamline identification processes, making them more inclusive and accessible. Integration with Aadhaar can verify identities using biometric and demographic data, ensuring benefits reach the right people efficiently.

Beyond identification, Jugalbandi can simplify eligibility assessments for government schemes by cross-referencing multiple databases and records. This can reduce errors and omissions, helping users navigate the application process for schemes smoothly. Integrating financial systems and digital wallets can facilitate direct benefit transfers, ensuring timely and secure delivery of funds. AI-driven analytics can further enhance security by detecting and preventing fraud.

The applications of Jugalbandi are limited only by our imagination. Just like the development and widespread adoption of smartphones led to (and continue to lead to) innovations like ride-sharing apps, mobile banking, and quick commerce apps, the true scope of Jugalbandi's impact will emerge as more people engage with and contribute to its evolution.

Some other example use cases

Do you need to enhance customer support systems for MSMEs or large organizations, including accessing public services such as applying for a subsidy/scheme, permits, and retrieving government & compliance documents?
Do you require a platform to support e-commerce activities such as product discovery, ordering, and customer service through a conversational interface?
Do you need to implement conversational interfaces in the travel and hospitality sector to assist with booking, customer inquiries, and travel information?
Is your organization involved in community development or non-profit work where conversational AI could help in mobilizing resources, managing volunteers, or providing community services?
Do you require AI-powered conversational interfaces for internal use within your organization, such as employee training, HR inquiries, or IT support?

Possibilities and Future Directions

Despite these strides being made, Deloitte’s state of AI in India report has found in a survey of over 200 executives across industries and business functions, of the various use cases of GenAI being applied to automate or streamline routine operational practices, its usage for legal document review still had the fewest adopters (at 33% of the surveyed participants). This presents a considerable opportunity for entrepreneurs to build a reliable solution that leverages AI, for routine business operations, knowledge management, marketing, HR, IT and the plethora of use-cases that’ll emerge.

Potential Technological Advancements

This section details some of the experiments that were carried out that helped make crucial decisions on how the Jugalbandi stack has been configured. The RAG framework has been central to these experiments, given its pivotal role in increasing the reliability of the information provided by any AI solution, including Jugalbandi. These experiments have been carried out through technical evaluation frameworks, legal professionals and law school volunteers.

Experimenting with RAG Evaluation Frameworks

Synthetic dataset creation: Ragas used OpenAI LLMs to generate a varied set of questions from a predefined dataset. These questions could be one of three: simple, questions whose answers require reasoning, and questions whose answers require information from different contexts. The dataset stores the corresponding context from which the questions were generated, and refers to it as the ‘ground truth’.
Jugalbandi answer generation: The questions generated in the synthetic dataset are passed to Jugalbandi to generate answers. Jugalbandi retrieves relevant chunks (or embeddings) of information from the knowledge base using its similarity search algorithms. These chunks are then passed to a generation model (such as GPT-3.5 or GPT-4o) to produce the answers.
Comparison of Jugalbandi’s answers with the ground truth: In the evaluation of Jugalbandi’s RAG pipeline against Ragas’ ground truth, four key metrics were used to assess performance:

Answer Relevance: How relevant the generated answer is to the question.
Faithfulness: How accurately the answer reflects the information in the context (ground truth).
Context Recall: The ability to retrieve relevant chunks from the knowledge base.
Context Precision: The precision of the retrieved chunks in providing the correct answer.

Experiment limitations: While the evaluation framework is a useful tool to assess the performance of Jugalbandi’s chunking and retrieval strategies, some limitations that couldn’t be tested are:

Question Variety: The synthetic dataset mostly includes straightforward questions, whereas user questions can vary significantly in complexity and phrasing. This limitation can affect the evaluation's comprehensiveness. Citizen centric applications will tend to get more open-ended questions, where determining the intent of the user and relating it to the available knowledge base is not always a straightforward task.
Model Dependency: The quality of answers depends on the underlying LLM (e.g., GPT-3.5 or GPT-4). Smaller models like Phi-3 may not perform as well for complex questions, although they are more lightweight and can be deployed on devices with limited resources. Testing with the same LLMs being used both for Jugalbandi and Ragas means that the context may be very similar for the generated and evaluation datasets. In the experiments it was found that answers generated using GPT-4 tended to yield better results in their evaluation, as compared to when GPT-3.5 was used. While this may be attributed to the advances made in the LLM, it’s unclear how it affects the assessment using automated frameworks.
Limited to legal information: The experiments conducted with the evaluation framework were limited to legal datasets. With advances being made in context specific RAG pipelines, the performance metrics generated when evaluating answers from a legal dataset may not apply to other contexts.

Need for manual checks by SMEs: The answers generated by Jugalbandi were compared with the ‘ground truth’ as determined by the frameworks, which in turn is generated using Open AI’s embedding models. It is preferable for the comparison to be made with the ground truth as determined by a subject matter expert for any given context.

While this evaluation framework provides valuable insights, it is one component of what must be a much more comprehensive testing process. This highlights the need for extensive testing of the RAG pipeline by experienced professionals or individuals familiar with the knowledge base.

pRAGyoga - RAG Benchmarking

To address the challenges of AI evaluation, pRAGyoga aims to create a comprehensive legal benchmarking dataset in India. This dataset will be used to test RAG systems on key metrics such as recall, accuracy, and possibly the cost-effectiveness of each RAG pipeline. Recall measures the system's ability to retrieve all relevant information, while accuracy assesses the correctness of the information retrieved.

The creation of this dataset will involve a diverse group of students from various disciplines, including law, social sciences, gender studies, data sciences, and humanities. This approach brings diverse and realistic questions that purely computational methods might overlook. Students can generate varied, real-world queries and provide detailed feedback, which enhances the system's reliability. Additionally, using a crowd-sourced method to create benchmarked data ensures that it reflects a wide range of perspectives, making it a strong standard for evaluating AI systems.

Implementation: The program has conducted pilots with a handful of students, with a scale up with multiple institutions planned. The students go through a diverse set of legal documents, including acts, case laws, judgements etc. This is followed by the following steps:

Using open-source annotation tools, the student volunteers generate questions from the provided documents and annotate the parts of the document containing the answers to their questions.
Students input their questions into the evaluation tool, which uses retrieval methods to pull up relevant information from the documents in its knowledge base. Typically, the system retrieves five chunks of information for each query.
Students review the retrieved information to check for relevance. If the information was relevant, they highlighted the parts. If not, they added necessary details in a free-text box.

Quality Control: Considering that evaluating the quality of evaluations of each student would be difficult as the program scales, the following measures were undertaken:

Peer-Review: Once the annotations are complete, different students review the work of their peers. They check if the questions make sense and if the answers match the questions. They can flag or edit the answers as needed.
Expert Review: Questions and answers flagged or edited by students undergo a final review by experts, who either confirm the annotations or make further edits to ensure accuracy.

Results of the pilot: Initial pilots showed that students could create a diverse and relevant set of questions and answers, as compared to the straightforward and pointed questions generated by automated evaluation frameworks. The exercise has been valuable in highlighting areas where the bot performs well and identifying aspects that need improvement.

Limitations: While this exercise is beneficial, it is time-consuming and depends heavily on the quality of student annotations.Comparing student-generated answers with Jugalbandi's responses is done through additional LLM evaluations, which may introduce some subjectivity, when an LLM is used in the evaluation of another LLMs performance.

Jugalbandi Studio: Building with AI has never been more accessible

Technology around us is rapidly evolving, making the need for intuitive and user-friendly platforms more critical than ever. This is especially true for technology that has the potential to improve lives across various sectors by enhancing access to information and enabling action. Simplifying access to such technology for developers and functional experts is necessary to reduce dependence and roadblocks in development. This is the rationale that inspired the creation of Jugalbandi Studio.

Jugalbandi Studio is an intuitive and user-friendly application designed to simplify bot creation and implementation meant for users with minimal technical expertise. It features a chat-based interface where users can functionally articulate their bot requirements without worrying about the technical know-how.

How is Jugalbandi studio different from Jugalbandi manager?

Jugalbandi Studio is designed to make bot creation accessible to users with minimal technical expertise by offering an intuitive, chat-based interface and guided setup. It simplifies the process of defining bot requirements, customising functionalities, and visualising user journeys, enabling anyone to develop and deploy bots without deep technical knowledge. In contrast, Jugalbandi Manager acts as a managed service that caters to developers, providing a robust, scalable framework for building and managing sophisticated AI-powered bots. It supports multiple platforms and can integrate with any LLM or services, allowing for advanced customization and scalability suitable for larger user bases and more complex applications.

Core Capabilities of Jugalbandi Studio

It has a user-friendly Interface

Jugalbandi Studio features an intuitive chat-based interface where users can describe their bot requirements in natural language. This approach removes the barrier of needing technical knowledge, allowing anyone to create a Jugalbandi service tailored to their needs. For example, users can simply type, "I need a bot to help farmers find government schemes," and the Studio will guide them through the setup.

It allows customization

Jugalbandi Studio includes a tool for developing flow diagrams. These diagrams provide a visual representation of interaction paths, making it easier to structure the chatbot’s functionalities and user journeys. For instance, users can visualise how a farmer might interact with the bot to find relevant schemes and follow-up actions.

You can visualise the interaction paths/user journey

Jugalbandi Studio also has a tool that can develop flow diagrams. These diagrams provide a visual representation of the interaction paths, making it easier to structure the chatbot’s functionalities and user journeys.

Realtime Testing and Debugging

Users can test and debug their applications in real-time, ensuring smooth functionality before deployment. This feature allows users to simulate interactions and make necessary adjustments on the fly before the bot goes live.

It also contains a template Library

The platform offers pre-built templates for common use cases, speeding up the chatbot development process. For example, templates for customer service bots, educational bots, and informational bots are available for quick customization.

It can guide you with best practices

Jugalbandi Studio provides initial guidance on best practices, helping users create effective and efficient bots from the start. This feature is particularly useful for users new to chatbot development, offering tips and examples to ensure optimal performance.

Deployment and Monitoring

Studio facilitates easy deployment of chatbots and provides tools for monitoring their performance and usage. Citizen developers and SMEs building bots using Studio can track interactions and gather analytics to improve their bots over time.

Community and Collaboration

Understanding the community-led spirit of our mission

The OpenNyAI team holds makerspaces annually, where developers and entrepreneurs gather to explore and co-develop AI solutions. These residencies, designed as hackathons, enable participants to innovate and prototype impactful AI-based applications, particularly targeting challenges for access to Justice. Through hands-on sessions/prototyping, mentorship, networking etc., participants refine their ideas and contribute to the creation of accessible, scalable AI tools that can democratize legal assistance and improve judicial processes. This initiative underscores the power of community-driven innovation in addressing societal issues with technology

The Openyai and Jugalbandi mission in its truest sense is community led and community born. In fact, Jugalbandi was first announced to the world at the Agami Summit 2022, presented as an idea in Vivek Raghavan’s (Co-founder, Sarvam AI and a member of the OpenyAI community) Keynote address. It was in the next Agami Summit in 2023, Sameer Segal (Scientist, Microsoft Research) publically announced the work OpenNyAI and the Microsoft Research team had been undertaking with support of several other community members including ……. .in the months leading up to the summit.

Maker residency, as forums aren’t meant to serve as product releases, rather as collectives where the possibilities of AI for increasing access to justice are formulated. They serve as the antithesis of the black box approach that’s been associated with the development and usage of AI. Instead of focusing on the tools, these communities seek to identify the problems that can be reasonably tackled using technology, and to find ways of deploying solutions in an inclusive, transparent, open and collaborative manner.

Our current collaborators and community

Organisations:

TresVista, MSR, Deloitte, AWS, Pinnacle, GCP, Tekdi, Tarento

Individual Contributors:

Get Involved

How can volunteers contribute?

Students - RAG Benchmarking

Students can engage in RAG benchmarking to evaluate the effectiveness and efficiency of AI solutions developed through Jugalbandi. They can provide valuable feedback on the performance and areas for improvement, helping to refine and optimize these AI tools.

Developers - Through C4GT or through creative solutions under Responsible AI

Developers can participate through initiatives like C4GT or they could contribute to responsible AI solutions by coding, testing, and refining AI applications, ensuring they are robust, scalable, and adhere to ethical standards.

Entrepreneurs - Use Case Development

Entrepreneurs can contribute by identifying and developing diverse use cases that Jugalbandi can address. Their insights into various industries and business challenges help in tailoring AI solutions that are practical, innovative, and implementation-ready, expanding the reach and impact of Jugalbandi.

Government - Policy and Implementation

Government bodies can play a pivotal role by facilitating the implementation of AI solutions within public services. They can help in shaping policies that support AI integration, ensuring regulatory compliance, and promoting the adoption of AI tools in areas like public administration, justice, and law enforcement.

Lawyers / Legal Professionals - Responsible AI and Use Cases

Lawyers and legal professionals can contribute by providing expertise on responsible AI practices, ensuring that the AI solutions developed are ethically sound and legally compliant. They can also identify specific legal challenges and use cases that Jugalbandi can address.

While these examples highlight some ways contributors can get involved with Jugalbandi, they are by no means exhaustive. Creative ideas and innovative approaches are always welcome. The roles mentioned serve as a starting point, and we encourage the community to think outside the box and propose new ways to contribute.

How can maintenance of the stack be open-sourced?

The developments to Jugalbandi have evolved from a publicly accessible API (which has been deprecated recently) for information retrieval, to a framework for quickly building applications using a manager, and finally to the no-code Jugalbandi Studio. All of these initiatives have been led and anchored by OpenNyAI which comprises an extensive network of organisations, volunteers and resources made available through value and mission aligned initiatives/contributions to steer this development. As this community of builders and users continues to grow, more aspects surrounding Jugalbandi, such as its management and maintenance will also open up to the community.

While the codebase is already open-sourced, the eventual goal is open-sourcing the ownership, maintenance and governance of the stack, to serve as a DPG in all aspects. A few measures that have been taken to this end are:

In-depth documentation in the codebase: The stack has been designed to be modular, with clear instructions on the workings of each component and how they interact with one another. Interested developers will also find reference implementations and tutorials on how they can quickly start using Jugalbandi for their own service.
Open to issue identification & contributions: The repository for the codebase is publicly accessible through Github. This allows anyone with a registered Github account to raise an issue with the workings of the stack, or make their own contributions to the stack through a pull request. Upon review of the contribution by the repository owners, currently the OpenNyAI team, it may be merged with the main branch of the codebase, provided that it contributes to the working of the stack.
Licensed as open-source: With the project having an Apache License (Version 2.0, January 2004) there are no limitations on who can use, modify, distribute the stack or combine it with their own solution. The only limiting condition imposed by the licence is that when distributing the software, the distributor must include a copy of the licence, provide a notice of any changes made to the original files, and ensure the availability of the source code.
Transparent governance: All key decisions taken in the development and adoption of the stack continue to be documented on the Coda dashboard.
C4GT: C4GT is a one-of-its-kind initiative that aims to build communities around Digital Public Goods (DPG), Digital Public Infrastructure (DPI) & Tech for Good products. It encourage ongoing contributions and strengthen collaboration between DPG/DPI builders, adopters, and contributors (students or working professionals). Jugalbandi has had many volunteers who connected with us to solve unique problem statements through C4GT.
Volunteer Contributions: This has already been detailed out in the previous section on our volunteer organizations and individual contributors.

Technical Support and Contact Information

The discord server would be the fastest way to resolve technical queries, as it houses the entire team who’ve contributed to the development and maintenance of the stack.

For other communications, contact information of the OpenNyAI team from Agami is given below:

Smita Gupta: smita@agami.in
Varun Hemachandran: varun@agami.in
Atreyo Banerjee: atreyo@agami.in