Local Development

If you are new here, read everything and follow the steps the end.

Develop and Test Locally

We have written a script that makes it easy to run this on any local machine (Mac, WSL2). Only deploy the services you need. This script depends on the env values in .env-dev.

$ ./scripts/run.sh <one or more service names that you are testing>
# names of the services - api, channel, language, flow, retriver, indexer

It's important that we test the data contracts (input / output JSONs) for our service. Run the service in one window, send JSON message to Kafka via another and read the messages generated by your service in another.

# create topics (queues) on kafka
./scripts/create-topic.sh topic_name

# send message to your service
./scripts/send-message.sh topic <json>

# read messages from your service
./scripts/read-message.sh topic

Setting up Local DB

# Updating your local database
./scripts/upgrade-db.sh

# Adding an alembic migration
./scripts/create-migration.sh <description of the changes>

Connecting to database

sudo apt-get install -y postgresql-client
psql -Upostgres -hlocalhost

It will prompt for password.

• flow - adding a user

 insert into jb_users (pid,first_name, last_name) values ('test4','a','b');

• flow - checking state stored in the db

select * from jb_fsm_state;

Data Contracts

See the Pydantic models lib/data_models.py

API

curl http://localhost:8000/callback -H 'content-type:application/json' -d '{"object": "whatsapp_business_account", "entry": [{"id": "112776635030672", "changes": [{"value": {"messaging_product": "whatsapp", "metadata": {"display_phone_number": "919711028566", "phone_number_id": "116346771524855"}, "contacts": [{"profile": {"name": "Test User"}, "wa_id": "919999999999"}], "messages": [{"from": "919999999999", "id": "wamid.HBgMOTE5ODg2Njg5NzU0FQIAEhgUM0E0RjQ2QzQyMUUxREYyODcxNjQA", "timestamp": "1705550692", "text": {"body": "Hey"}, "type": "text"}]}, "field": "messages"}]}]}'

Indexer

topic: indexer message:

'{"collection_name":"KB_Law_Files", "files":["combine.pdf", "Property Law.xlsx", "Information Technology Laws.xlsx", "Family Laws for Succession and Inheritance.xlsx", "Senior Citizens Rights.xlsx", "Motor Vehicles Act.xlsx", "Income Tax Compliance.xlsx", "Cyber Crimes.xlsx", "Disability Rights.xlsx", "Women Rights.xlsx", "Law related to SC and ST.xlsx", "Medical Termination of Pregnancy.xlsx", "Mental Health.xlsx", "Narcotics and Psycotropic Substances.xlsx", "Formation of Company and Corporate Social Responsibility.xlsx", "POCSO Act.xlsx", "SEBI Regulations.xlsx", "Legal Services in India.xlsx", "Transgender Rights.xlsx", "Farmer Rights.xlsx", "Arbitration Conciliation.xlsx", "Labour Laws.xlsx"]}'

Steps

1. Create environment by copying .env-dev.template to .env-dev. Update the values if required. You can look at the larger list of constants in .env

2. Bring up the service e.g. api

$ ./scripts/run.sh -e .env-dev api

3. Make any modifications to the database

$ ./scripts/upgrade-db.sh

4. Setup a listener to the queue

$ ./scripts/read-message.sh <topic name>

5. Test the data contracts by sending a message to the queue or api

# For API
$ curl http://localhost:8000/callback ....

# For all
$ ./scripts/send-message.sh <topic> <JSON message>

6. Check the database by using DBeaver

Reseting Dev Environment

Often there are times when it's better to start from scratch.

# bring down whatever is running
$ docker compose down

# following will delete all your containers - be careful
$ docker volume ls | awk '{print $2}' | xargs docker volume rm

# if you don't want clutter, you can delete all containers
$ docker ps -a | awk '{print $1}' | xargs docker rm