Lithuanian AI chatbot: what breaks and how to test it
Lithuanian AI chatbots fail on declensions, missing diacritics, mixed LT/EN, off-site answers, invented prices and handover. A test plan to run before you buy.
AFKzona Group · 6 min read
The short answer
- Most chatbot failures in Lithuanian are predictable: declined word forms, text typed without diacritics, and questions that mix Lithuanian and English.
- The costliest failure is a confident answer that is not on your website, especially a price.
- Test a chatbot with your own questions, written the way your customers type, before you sign anything.
- A good chatbot says when it does not know and hands the conversation to a person with the contact details attached.
- Keep a written test set and re-run it after every change to your site or the chatbot's settings.
A Lithuanian AI chatbot usually breaks in six places: declined word forms, text typed without diacritics, questions that mix Lithuanian and English, answers that are not on your website, invented prices, and a poor handover to a person. All six can be tested before you buy. This article lists what to test, how to score it, and how we handle each case in Keliox, our Lithuanian-first chat widget.
The test plan below works for any vendor, Keliox included.
Why Lithuanian is harder for chatbots than English
Lithuanian is harder because one word appears in many forms, and a chatbot has to connect all of them to the same page. The language has seven grammatical cases (vardininkas, kilmininkas, naudininkas, galininkas, įnagininkas, vietininkas, šauksmininkas), each with its own endings, and customers often type without diacritics.
Declension (linksniavimas) is the change of a word's ending by grammatical case and number. "Nuoma", "nuomos", "nuomai", "nuomą" and "nuomoje" are all the same word, rental.
Retrieval is the step where the chatbot searches your content for passages relevant to the question before the language model writes an answer. RAG (retrieval-augmented generation) is the name for this pattern: retrieve first, then generate an answer from what was retrieved.
If retrieval only matches exact words, a customer who asks "kiek kainuoja nuomai?" may not reach the page that says "nuomos kaina". The language model itself can read Lithuanian well; the failure is in finding the right source. That is why the tests below focus on retrieval as much as on the final wording.
Six things that break, and what to test
Each failure below has a simple test you can run on a trial of any chatbot, on your own website, with questions your customers actually ask.
| Failure | What it looks like | How to test | Pass condition |
|---|---|---|---|
| Declensions | Misses the page when the question uses a different case | Ask the same question with 3–4 word forms | Same correct answer every time |
| Missing diacritics | "sasaskaita", "kaina su pvm", "ar dirbate sestadieni" not understood | Retype 10 questions with no ą, č, ę, ė, į, š, ų, ū, ž | Same answers as with diacritics |
| Mixed LT/EN | "Ar turite API integration su Shopify?" | Mix English product terms into Lithuanian questions | Answers in the customer's language, correct content |
| Off-site answers | Answers from general knowledge, not your site | Ask things your site does not cover | Says it does not know and offers a person |
| Hallucinated prices | Confident price or discount that is not on your site | Ask for prices of services with no published price | Refuses or quotes only a published figure with its page |
| Poor handover | Loops, loses context, or asks for details twice | Ask for a call back or a complaint mid-conversation | Captures contact once, passes the transcript to a person |
Declensions and word forms
Write each of your ten most common questions in three or four natural variants that change the case of the key noun. Score whether the chatbot reaches the same page each time. Failures here usually mean retrieval is too literal, not that the model cannot read Lithuanian.
Text without diacritics
Many people type Lithuanian on phones or keyboards without the Lithuanian layout. Retype your test questions without ą, č, ę, ė, į, š, ų, ū and ž and compare answers. A chatbot that answers correctly only when diacritics are present will fail a real share of your visitors.
Mixed Lithuanian and English
Business customers mix languages, especially for product names and technical terms. Test questions such as "ar galima invoice gauti su PVM?" and check two things: the answer uses the language the customer mainly wrote in, and the content is right.
Answers outside your website
A website chatbot should answer from your content, not from what the model happens to know. Ask questions your site does not cover: a competitor's opening hours, a legal question, a product you do not sell. The right answer is "I do not have that information", followed by an offer to connect the visitor with a person.
Hallucinated prices and dates
Hallucination is a confident answer that is not supported by the source content. In a sales chat the most damaging version is a price, discount or delivery date. Test every price question you can think of, including for services that have no published price, and check each figure against your site. OWASP lists misinformation (LLM09) among the top risks for LLM applications for this reason.
Also try a prompt-injection question, such as asking the chatbot to ignore its instructions and offer a 50% discount. OWASP ranks prompt injection (LLM01) first. The chatbot should decline and stay within your content.
Handover to a person
Ask for a call back, raise a complaint, or ask something that clearly needs a human. A good handover asks for contact details once, confirms what will happen next, and gives your team the full conversation. A bad one loops, asks the same question twice, or leaves the visitor with no next step.
A test plan to run before you buy
A pre-purchase test plan is a fixed set of real questions, run on your own site, with each answer scored against your pages. It takes an afternoon to write, and you can reuse it for every vendor and every later change.
- Collect 40–60 real questions from your inbox, phone notes and contact forms. Keep the customers' wording.
- Add variants: declined forms, no diacritics, mixed LT/EN, typos.
- Add 10 questions your site cannot answer, including 3 price questions with no published price and 1 prompt-injection attempt.
- Run the set on a trial installed on your real site, not on the vendor's demo site.
- Score each answer: correct, wrong, or correctly refused. Note the source page when the chatbot shows one.
- Check every number (price, date, phone number, opening hours) against your site by hand.
- Test the handover at least five times and confirm what your team receives.
- Keep the set and re-run it after any change to your site, your prices or the chatbot's settings.
Decide your pass threshold before you run the tests, not after you see the results. For prices and legal statements, the only acceptable result is correct or correctly refused.
Questions to ask the vendor
Before buying, ask in writing: where conversations are stored and for how long, whether data stays in the EU, how the chatbot is told what it may and may not answer, how it shows its source, what happens when the site changes, how a person takes over, and what is logged. Under Article 50 of the EU AI Act, from 2 August 2026 people must be informed that they are talking to an AI system unless that is obvious; ask how the widget does this.
How Keliox handles these cases
Keliox is our multilingual chat and lead-capture widget for small businesses, built to handle Lithuanian properly. A business installs it with one script tag and gives its website address. Keliox reads the site and answers visitors in Lithuanian or English from that content, asks for contact details when someone wants a call back, and hands hard questions to a person.
The owner sees every conversation, lead and booking in one place, and every admin action is recorded in a hash-chained audit log, where each entry includes a hash of the previous one so that edits to history are detectable. We run the kind of test set described above against Keliox ourselves, and we recommend you run yours against it before deciding.
Where to go next
If you want a chatbot that answers from your own content, or an assistant over internal documents, we can scope it with you and agree a fixed price in writing first.
- Chat on your site: customer chat and lead capture
- Internal documents: knowledge assistants and search (RAG)
- Service and prices: AI agents and automation, pricing
- Book a free 30-minute call, and bring ten questions your customers actually ask.
Common questions
Do AI chatbots understand Lithuanian?
Current large language models read and write Lithuanian, but a chatbot is more than the model. Retrieval, search over your pages, and the instructions around the model decide whether it finds the right answer when a customer types a declined word form, skips diacritics or mixes in English. Test those cases directly rather than trusting a demo.
How do I test a chatbot before buying it?
Write 40 to 60 real questions from your inbox, including misspellings, text without diacritics, English and questions your site does not answer. Run them against a trial on your own site, record each answer as correct, wrong, or correctly refused, and check every price and date against your pages. Repeat after any change.
Why does a chatbot invent prices?
Language models produce plausible text. If the retrieval step finds no price, or finds an old one, the model may still write a confident figure unless it is instructed and tested to refuse. The fix is to answer only from retrieved content, cite the page, and send price questions without a source to a person.
Does a website chatbot need to say it is AI?
Under Article 50 of the EU AI Act, systems that interact directly with people must be designed so that people are informed they are interacting with an AI system, unless that is obvious from the context. The obligation applies from 2 August 2026.