Secure AccountantAI Chatbot: Lili's Journey with Amazon Bedrock

This post was co-written by Lili's Liran Zelkha and Eyal Solnik.

Small business owners tend to prioritize the operational aspects of their business over administrative tasks such as financial records and accounting. While hiring a professional accountant can provide valuable advice and expertise, it can be cost-prohibitive for many small businesses. Additionally, the accountant's responses may not always meet the owner's immediate needs, leaving questions unanswered and slowing down the decision-making process.

In the rapidly evolving world of large-scale language models (LLMs) and generative artificial intelligence (AI), Lili saw an opportunity to use this technology to address the financial advisory needs of its small business customers. Using Anthropic's Claude 3 Haiku on Amazon Bedrock, Lili developed the intelligent AccountantAI chatbot that can provide on-demand accounting advice tailored to each customer's financial history and unique business requirements. The AccountantAI chatbot acts as a virtual assistant, providing affordable, actionable financial guidance, allowing small business owners to focus on their core expertise while ensuring their financial health.

About Lili

Lili is a financial platform designed specifically for businesses, combining advanced business banking with built-in accounting and tax preparation software.

Lili streamlines and simplifies corporate financial management by integrating financial tools into a user-friendly interface, making it a compelling solution for business owners who want to manage their financial operations centrally and efficiently.

In this post, we explain how Lili, a financial platform designed specifically for enterprises, built a secure and intelligent AccountantAI chatbot for small business owners using Amazon Bedrock. Amazon Bedrock is a fully managed service that offers a choice of high-performance foundational models (FMs) from leading AI companies such as Anthropic, Meta, Mistral AI, Stability AI, Cohere, AI21 Labs, and Amazon in a single API. It also provides a wide range of capabilities required to build generative AI applications with security, privacy, and responsible AI.

Solution overview

The AccountantAI chatbot provides small business owners with accurate and relevant financial accounting advice in a secure manner. To achieve this, the solution is designed to address two key requirements:

Question Validation: Implement guardrails to ensure that user input is a valid, legitimate financial accounting question. This step filters out irrelevant or inappropriate queries and maintains the integrity of the system.
Enhanced context: Augment the user's questions with relevant contextual data, such as up-to-date accounting information or user-specific financial data. This step enables the chatbot's responses to be tailored to each individual user's business and financial situation, providing more personalized, actionable advice.

To address the two key requirements of question validation and context enrichment, the AccountantAI solution employs a two-stage architecture consisting of an ingestion workflow and a capture workflow.

Ingestion Workflow

The ingestion workflow is an offline process that prepares the system to respond to customer inquiries. At this stage, Lili provides a comprehensive Golden Collection A collection of financial accounting questions extracted from common inquiries and real-world questions from our long-standing customer base. This diverse, high-quality collection serves as a reference corpus, enabling our chatbot to handle a wide range of relevant queries. The ingestion workflow converts these curated questions into vector embeddings using the Amazon Titan Text Embeddings model API. This process runs over AWS PrivateLink for Amazon Bedrock, a secured, private connection within your VPC. The vector embeddings are persisted in the application's in-memory vector store. These vectors help validate user input during the ingestion workflow.

Each carefully selected vector embedding is paired with a matching prompt template that was rated as most effective during testing.

Prompt template example

<role>
Provides context about the agent's role as Lili's AI assistant for financial questions and outlines the general guidelines applied to all queries.
</role>

<about>
Provides details on Lili platform.
</about>

<features>
Lists out all of Lili's product features in detail. This section aims to explain Lili's features in detail, ensuring that answers are aligned with the Lili platform. For instance, when addressing questions about tax reduction management, highlight the relevant features that Lili offers, which customers should be familiar with.
</features>

<output_format>
Outlines the required formatting for the response to ensure it meets the expected structure.
</output_format>

<user_data>
Data relevant to answering the customer's question.
</user_date>

<knowledge>
Specific accounting knowledge that is relevant to the question and the model is not familiar with, such as updated data for 2024.
<knowledge>

<question>
Contains the user's actual question.
</question>

<instructions>
Provides the core instructions on how to approach answering the question appropriately and meet expectations. It also defines the steps in providing a detailed and high-quality answer.
</instructions>

<reminders>
Important guidelines to remind the agent and make sure it follows them, such as the exact format of the answer.
</reminders>

Search workflow

Lili's web chatbot web interface allows users to submit queries and receive responses in real time. When a customer asks a question, it is sent to a backend system for processing.

The system first converts the query into vector embeddings using the Amazon Titan Text Embeddings model API, which is securely accessed through PrivateLink.
The system then performs a similarity search against the precomputed embeddings of the golden collection to find the most relevant matches to the user's query. The system evaluates the similarity scores of the search results against a pre-set threshold. If the user's question produces matches with a low similarity score, the question is deemed incorrect or unclear and the user is asked to rephrase or refine the query.
However, if a user's question generates a match with a high similarity score, it is considered a legitimate query. In this case, Lili's backend system will use the golden question with the highest similarity score with the user's query for further processing.
Based on the golden question with the highest similarity score, the system retrieves the corresponding prompt template.

This template is then populated with up-to-date accounting information and customer-specific financial data from external sources, such as Amazon RDS for MySQL. The resulting contextualized prompts are sent to Anthropic's Claude 3 Haiku on Amazon Bedrock to generate customized responses that address the customer's queries within their unique business context.

Amazon Bedrock makes it easy to adopt new advancements in generative AI across multiple model providers, as model providers continuously enhance their services with innovative updates. This approach has demonstrated its benefits from the initial rollout of AccountantAI: Lili migrated from Claude Instant to Claude 3 for Anthropic within two weeks of its official release in the Amazon Bedrock environment and within three weeks of general availability.

Lili selected Anthropic's Claude model family for AccountantAI after reviewing industry benchmarks and conducting its own quality assessment. Anthropic Claude on Amazon Bedrock has consistently outperformed other models in understanding financial concepts, generating consistent natural language, and providing accurate, customized recommendations.

After the initial release of AccountantAI, Amazon Bedrock introduced the Anthropic Claude 3 Haiku model, which Lili evaluated against the Anthropic Claude Instant version. The Anthropic Claude 3 Haiku model showed significant improvements across three key evaluation metrics:

quality – Anthropic Claude 3 Haiku offers more detail and better response than its predecessor, resulting in a higher quality output.
Reaction time – Anthropic Claude 3 Haiku delivered faster performance with a 10-20 percent improvement in response time compared to Claude Instant.
Fee – Amazon Bedrock Anthropic Claude 3 Haiku is the most cost-effective choice, offering higher levels of intelligence and performance at up to 68% lower cost per 1,000 input/output tokens compared to Anthropic Claude Instant. Learn more about Amazon Bedrock Anthropic Claude 3 models.

For customers like Lili, this highlights the importance of access to a fully managed service like Amazon Bedrock, which offers a choice of high-performance underlying models to meet diverse enterprise AI needs. There is no “one size fits all” model, and the ability to choose from cutting-edge FMs is critical for organizations that want to effectively and cost-effectively leverage the latest advancements in generative AI.

Conclusion

Exclusively available to Lili customers, AccountantAI features reduce the need to hire a professional accountant. While professional accountants can provide valuable advice and expertise, their services can be too costly for many small businesses. AccountantAI has already answered thousands of questions, bringing real value to businesses and providing quality answers to financial, tax and accounting inquiries.

Lili uses Amazon Bedrock, which provides easy, secure, and reliable access to high-performance underlying models from leading AI companies, to integrate each client's unique data with accounting knowledge at scale. This innovative solution provides affordable expertise to optimize cash flow, streamline tax planning, and drive growth through informed decisions. AccountantAI fills the accounting resource gap, making quality financial information accessible to every business.

Explore Lili's AccountantAI capabilities, powered by Amazon Bedrock, to get affordable financial intelligence for your business today, or use Amazon Bedrock Playgrounds to experiment with running inference with different models on your data.