An internal support assistant that only sees what it should

The problem

Second-line support at a B2B SaaS company answered escalated tickets by searching past tickets, product docs, release notes, the CRM and the billing system. In a four-week sample, it took a median of 25 minutes to find the answer to an escalated ticket. The information existed. It was spread across five tools with five different permission models.

The first prototype, and why we threw it away

In week two we had a working prototype: everything embedded into one vector index, with an LLM answering on top. The answers were good.

In testing, one answer to a question about a customer's invoice cited a ticket from a different customer. The index had no idea who was allowed to see what. Support engineers at the company do not have access to every account, so this would have been a real data leak.

What we built instead

1. 015 sources
2. 02Unity Catalog
3. 03Filtered index
4. 04Retrieval
5. 05Assistant

1. All five sources were landed in the lakehouse first, with account-level permissions in Unity Catalog that mirror the ones in the CRM.
2. Every chunk in the vector index carries the account it belongs to. Retrieval filters by the user's permissions before ranking, not after.
3. Every answer shows its sources, and engineers can open the original ticket or doc in one click.
4. The assistant runs behind the company's existing SSO, so there are no new accounts to manage.

This took four weeks longer than shipping the prototype would have.

Result

Measured on the same kind of four-week ticket sample after launch:

The permission-aware lakehouse built for this is now also used by the analytics team for support reporting.