My product
BI Operations Agent
Ask business data questions in plain English. A governed semantic layer and a read-only SQL guard sit between the model and your data.
What it is
An agent that lets a manager ask a question like “which store grew fastest last quarter?” and get an answer and a chart, with daily KPI alerts and a weekly summary. It connects to spreadsheets and SQL databases.
The problem
Small and mid-sized businesses keep data in spreadsheets and databases and rarely have an analyst on call. Plain text-to-SQL tools can define the same metric differently from one question to the next, and a careless one can be tricked into writing to or leaking from the database.
What I built
- A governed semantic layer in YAML. It defines approved metrics, dimensions and joins once, so “gross margin” always means the same thing.
- A SQL guard that parses the query into a syntax tree with sqlglot, instead of matching text. It allows only read-only SELECT statements on allowed tables, adds a row limit and enforces a timeout.
- A KPI monitor that flags unusual days using a robust z-score (median absolute deviation) and writes a weekly summary in plain English.
How it’s checked
I wrote a 50-question benchmark with known answers and a set of 12 adversarial prompts (injection and data-leak attempts). The guard has its own tests. I also wrote 25 reworded held-out questions, to check how it copes with wording it was not built around.
What the results do and don’t tell you
On a synthetic two-year Nigerian retail dataset that the repo generates (not real transactions):
- 48 of 50 benchmark questions correct (96.0%). This benchmark was used while building, so it flatters the system.
- 14 of 25 reworded held-out questions correct (56.0%). This is the more honest figure.
- 12 of 12 adversarial prompts blocked, with no unsafe SQL executed.
So the safety guard held up in my tests, but the question-answering is not yet reliable on unseen wording. It needs more work before anyone should rely on it for decisions.
My role
I designed, built and evaluated the whole agent: the semantic layer, the guard, the monitor and the benchmark.
Stack
Python, FastAPI, DuckDB, sqlglot, SQLAlchemy, Streamlit and React, with an LLM tool loop behind adapters.
Where these facts come from
/home/user/AIMLOps-BI-Operations-Agent/README.md/home/user/AIMLOps-BI-Operations-Agent/reports/eval_summary.json/home/user/ai-ml-data-engineering/products/bi-operations-agent/README.md