AI31 min readUpdated

How to Choose an AI Development Company: 15 Questions to Ask Before You Sign

15 questions covering evals, who owns the code, handover, model choice, data privacy, how pricing is structured (fixed vs time and materials, without our prices), timeline, post-launch support, references and red flags. For each question say what a good answer sounds like. Add a scoring checklist and a red-flags section.

How to Choose an AI Development Company: 15 Questions to Ask Before You Sign

You're about to spend serious money on an AI project. Maybe it's a support agent. Maybe it's an AI feature inside your SaaS. Maybe it's an internal tool that saves your ops team 20 hours a week.

Whatever it is, you need a reliable partner for production AI development to build it.

And here's the uncomfortable truth: most AI projects don't fail because the AI was bad. They fail because the company building them didn't know how to ship to production. They built a demo that looked great in a sales call, took your money, and delivered something that breaks the moment a real customer asks an unexpected question.

This guide exists to protect you from that.

Below are 15 questions you should ask any AI development company before you sign a contract. For each one, I'll tell you what a good answer sounds like, what a bad answer sounds like, and why it matters. At the end, there's a scoring checklist you can use during sales calls, plus a red-flags section that tells you when to walk away.

No fluff. No "top 10 AI companies" listicle nonsense. Just the questions that actually separate real engineering firms from slide-deck sellers.


Why Choosing the Wrong AI Development Company Is Expensive

Let's talk about what happens when you pick the wrong partner.

They quote you a fixed price that sounds too good. You sign. Three weeks in, they tell you the scope needs to change. The price goes up. Then up again.

They build something that works in testing. You launch it. A customer asks a question the AI wasn't trained on. It hallucinates. It promises a refund your company doesn't offer. This is exactly why we architect production AI support agents with strict human-in-the-loop checkpoints. Now you have an angry customer, a support ticket, and a system you can't fix because the agency used a proprietary framework you don't understand.

They deliver the code. You ask for documentation. They say, "We'll walk your team through it on a Zoom call." Two months later, your lead developer quits. Nobody knows how the system works. You call the agency. They quote you another project.

This isn't a horror story. This is what happens in most failed AI engagements.

The fix isn't to avoid AI. The fix is to ask better questions before you sign anything.

That's what the next 15 questions are for.


Category 1 — Technical Depth & Production Readiness

These first four questions are the most important ones on this list. They tell you whether the company builds AI systems that survive real users, or AI demos that survive a sales call.

If a company can't answer these clearly, stop the conversation. You're talking to a sales team, not an engineering team.


Question 1 — "How do you test and evaluate AI agents before they go to production?"

Why this matters:

Every AI model makes mistakes. Every single one. The question isn't whether the AI will make mistakes. The question is whether the company has a system to catch those mistakes before your customers see them.

What a good answer sounds like:

"We build eval suites. Before any prompt or agent change goes live, it runs against a set of test cases — real customer questions, edge cases, and known failure scenarios. If the new version scores worse than the old one, it doesn't ship. We also track confidence scores, so the system knows when it's unsure and hands off to a human instead of guessing."

What a bad answer sounds like:

"We test it manually." Or worse: "We just check if it works." Or: "GPT-4 is really accurate so we don't worry about that."

The red flag: If they don't mention the words "eval," "regression testing," "confidence scoring," or "test cases," they are not testing. They are guessing. And you will be the one who finds the bugs in production.


Question 2 — "Are your systems locked into one model provider, or are they provider-agnostic?"

Why this matters:

OpenAI raises prices. Anthropic has an outage. Google deprecates a model. If your entire system is hard-coded to one provider, you're stuck. You either eat the cost, go offline, or pay someone to rebuild everything.

What a good answer sounds like:

"We build provider-agnostic. The core logic, the prompts, the tool connections — none of it is tied to one model. We can swap Claude for GPT for Gemini without rewriting your app. This matters the day one provider has an outage, raises prices, or releases a better model."

What a bad answer sounds like:

"We only work with OpenAI." Or: "We're an OpenAI partner." Or: "You won't need to switch."

The red flag: If they're a "certified partner" of one provider and can't explain how they'd work with another, you're not hiring an engineering firm. You're hiring a reseller. And resellers don't build production systems.


Question 3 — "What happens when the AI hallucinates or gives a wrong answer in production?"

Why this matters:

This is the question most companies are afraid to ask. They don't want to hear that AI makes mistakes. But here's the reality: your AI agent will hallucinate. It will happen. The only question is what the system does when it does.

What a good answer sounds like:

"We design for failure. The system has confidence thresholds — if the AI isn't at least 85% confident in its answer, it doesn't respond. It escalates to a human. We also have human-in-the-loop checkpoints for high-stakes actions like refunds, contract changes, or escalations. And we have fallback routing, so if the model fails entirely, the customer still gets a response — just from a human instead."

What a bad answer sounds like:

"We use GPT-4, so it doesn't hallucinate." Or: "We'll fix it if it happens." Or: "Our prompts are really good so it shouldn't be an issue."

The red flag: Any company that says their AI won't hallucinate is lying to you. Every LLM hallucinates. The only acceptable answer is an explanation of how the system handles it safely.


Question 4 — "What observability and monitoring do you set up for the AI system?"

Why this matters:

Once the AI is live, you need to see what it's doing. How many tokens is it burning? How fast is it responding? Is it drifting? Is it making the same mistake over and over? If you can't see it, you can't fix it.

What a good answer sounds like:

"We set up observability from day one. Tools like Langfuse, Helicone, or OpenTelemetry. You get dashboards showing token costs per conversation, response latency, error rates, and prompt drift. We also set up alerts — if the error rate spikes or costs go above budget, you know immediately."

What a bad answer sounds like:

"We'll add logging if needed." Or: "You can check the server logs." Or: "We don't usually set that up unless the client asks."

The red flag: If they treat monitoring as an afterthought, they haven't run AI in production before. Production AI without observability is like flying a plane without instruments. You won't know something's wrong until you've already crashed.

Category 2 — Ownership, Code & Handover

These three questions protect you from the most common trap in agency work: you pay for a system, but you don't actually own it. Or you own it, but nobody on your team understands how it works. Or you own it and understand it, but the moment something breaks, you're calling the agency again and paying another invoice.

If you get these three questions right, you walk away from the engagement with a system your team can run, fix, and improve without the agency. That's the goal.


Question 5 — "Who owns the code, the prompts, and the training data?"

Why this matters:

This sounds like a legal question. It's not. It's a survival question.

If the agency retains ownership of the prompts, you can't change them. If they retain the eval datasets, you can't test changes. If they use a proprietary framework, you can't leave. You're locked in. And locked-in clients get charged whatever the agency wants, forever.

What a good answer sounds like:

"You own everything. The code, the prompts, the eval datasets, the architecture docs, the deployment scripts. All of it transfers to your repository at launch. We use standard frameworks — LangGraph, Next.js, Python — so any competent developer can pick it up. There's no proprietary layer you can't leave."

What a bad answer sounds like:

"We retain rights to the prompt engineering." Or: "The core framework is ours, but you get a license to use it." Or: "That's in the contract, you'd need to check with legal." Or worse — they haven't thought about it and say, "We'll figure that out later."

The red flag: If the word "proprietary" comes up in their answer, or if they say you get a "license" instead of full ownership, walk away. You're not building a system. You're renting one. And the rent never stops.


Question 6 — "What does the handover documentation look like?"

Why this matters:

Every agency says they'll "hand over" the project. Very few of them mean the same thing by that word.

Some agencies mean: "We'll send you the GitHub repo and wish you luck." Others mean: "We'll do a 45-minute Zoom call and record it." Neither of those is documentation. Documentation is what lets your team fix a bug at 2 AM when the agency isn't answering their phone.

What a good answer sounds like:

"You get a full handover package. Architecture diagrams showing how every component connects. Deployment guides so your DevOps team can redeploy without us. A runbook for common issues — what to do if the model API goes down, what to do if costs spike, what to do if a prompt starts drifting. Prompt versioning docs so you know which version is live and why. And an incident response plan for when things break in production."

What a bad answer sounds like:

"We'll walk your team through it." Or: "The code is well-commented, so it should be clear." Or: "We'll be available for questions after launch."

The red flag: If their answer doesn't include the words "runbook," "architecture diagram," or "deployment guide," they haven't done a real handover before. They've just thrown code over a wall and hoped for the best. You will be the one cleaning up that mess.


Question 7 — "Can our internal team maintain and iterate on the system without you?"

Why this matters:

This is the ultimate test. If the answer is no, you haven't hired a development partner. You've created a dependency. And dependencies are expensive. Every small change — a new prompt, a new integration, a bug fix — becomes a new invoice.

The goal of any good agency engagement should be to make themselves unnecessary. Build it, hand it over, train your team, and leave. If they can't do that, they're not building for you. They're building for their own recurring revenue.

What a good answer sounds like:

"Yes. That's the goal. We use standard, widely-known frameworks so your team can hire for them. We write clean, documented code. We set up CI/CD pipelines so your team can deploy changes safely. And before we leave, we run training sessions with your developers — not a slideshow, actual hands-on sessions where they make changes and deploy them while we watch."

What a bad answer sounds like:

"You'll probably want us for ongoing maintenance." Or: "The system is complex, so you'll need our expertise." Or: "We offer a monthly retainer for support."

The red flag: If they pitch you a monthly retainer before the project is even built, they're not planning to hand over. They're planning to keep you. Ask them directly: "If we don't renew the retainer, can our team still deploy changes?" If they hesitate, you have your answer.

Category 3 — Data Security & Compliance

These next two questions are non-negotiable.

A few years ago, AI was just a fun chatbot on a marketing page. Today, AI agents are reading customer support tickets, processing financial documents, and making decisions about user accounts. If your AI touches personal data, and the agency building it doesn't understand security, a single data leak can end your business.

Worse, the legal landscape just changed. With the EU AI Act now in full effect and GDPR strictly enforced, "we'll figure out compliance later" is no longer an acceptable answer.


Question 8 — "How do you handle our customer data during development and in production?"

Why this matters:

To build and test an AI agent, the developers need data. But there is a massive difference between test data and live customer data. If an agency asks you to email them a spreadsheet of your real users so they can "test the prompts," you have a massive security problem on your hands.

Furthermore, if you are building a SaaS product, you need to guarantee that Customer A's AI agent never accidentally reads Customer B's private documents. This is called multi-tenant data isolation, and getting it wrong is a fatal flaw.

What a good answer sounds like:

"Your data never leaves your cloud environment (like your AWS or GCP account). For development and testing, we use anonymized data or synthetic datasets that mimic your real users without exposing private info. In production, if we are building AI integrations for existing SaaS platforms, we use strict Row-Level Security (RLS) in the database and vector stores (like pgvector or Pinecone) so the AI can physically only access the data belonging to the user asking the question."

What a bad answer sounds like:

"Just export your database to a CSV and send it over." Or: "We'll just connect our local development machines directly to your live production database." Or: "We use secure password-protected folders to store your data."

The red flag: If they don't mention "anonymized data," "synthetic data," or "Row-Level Security (RLS)," they are treating your highly sensitive customer data like a high school science project. Walk away immediately.


Question 9 — "What is your approach to AI compliance — EU AI Act, SOC 2, and GDPR?"

Why this matters:

Here is the biggest lie in the AI industry right now: "We use OpenAI/Anthropic, so our app is compliant."

That is false. The model provider secures their servers. You are legally responsible for how your application uses the model. If your AI makes a high-risk decision (like denying a loan, routing a medical query, or firing a warning to an employee), the EU AI Act requires you to have human oversight and an audit trail. Under GDPR, if a user asks to be "forgotten," you must be able to delete their specific data from your AI's memory (vector database).

What a good answer sounds like:

"We build compliance into the architecture. For GDPR, we use vector databases that allow us to delete specific user embeddings when a user requests deletion. For the EU AI Act, we build 'audit trails' using tools like Langfuse. This means if a customer complains about what the AI said, you can pull up the exact log showing the prompt, the context the AI read, and the final answer, so you can prove why the AI made that decision."

What a bad answer sounds like:

"Our legal team handles that." Or: "AI isn't really regulated yet." Or the biggest lie: "OpenAI is SOC 2 compliant, so your app will be too."

The red flag: If they look confused when you mention the EU AI Act, or if they think using a compliant API makes your app compliant, they are building you a legal liability. You need an engineering team that builds audit logs and data-deletion pipelines from day one.

Category 4 — Pricing, Timeline & Process

Here's where most agency relationships go to die.

Not because the code was bad. Not because the AI hallucinated. Because of money and time. The agency quoted a number. You said yes. Then the scope changed. Then the timeline slipped. Then the invoice arrived and it was 40% higher than what you agreed on.

These three questions won't prevent every surprise. But they will tell you immediately whether the company operates with transparency, or whether they plan to bleed you dry through scope creep and vague contracts.


Question 10 — "How is pricing structured — fixed scope, time and materials, or hybrid?"

Why this matters:

There are three ways agencies charge for AI development. Each one has a different risk profile for you.

Fixed price means they quote you one number and build to that number. Sounds great. But if they quote too low, they cut corners. If the scope changes (and in AI, it always changes), they hit you with a "change order" that costs more than the original project.

Time and materials means you pay for every hour they work. Sounds fair. But there's no cap. If they're slow, you pay more. If they're inefficient, you pay more. You're funding their learning curve.

Hybrid means a discovery phase first (usually fixed price), then a build phase (either fixed or T&M based on what discovery revealed). This is usually the healthiest model for AI projects because AI is inherently uncertain. You can't accurately quote a system you haven't investigated yet.

What a good answer sounds like:

"We do a discovery sprint first. It's a fixed price, usually two weeks. We dig into your data, your systems, your constraints, and we produce a technical specification and architecture document. Then, based on that spec, we give you a fixed quote for the build. No surprises. If something changes mid-build, we flag it, discuss it, and agree on the impact before we proceed."

What a bad answer sounds like:

"It's 15,000. Let′s start Monday. "(Nodiscovery, noscoping, just an umber pulled from thin air.) Or:" We charge 15,000.Let's start Monday."(No discovery, no=scoping, just a number pulled from thin air.) Or: "We charge150 per hour and we'll keep you updated." (No cap, no scope, no accountability.) Or: "It depends. Hard to say until we get into it." (They don't know what they're doing.)

The red flag: If they give you a fixed price without doing any discovery or asking about your data, your systems, or your constraints, they are guessing. And when their guess is wrong, you're the one who pays for it. A serious engineering firm will never quote a complex AI system without understanding the problem first.


Question 11 — "What does a realistic timeline look like for a project like ours?"

Why this matters:

Every buyer wants to hear "two weeks." And every dishonest agency will tell you two weeks. Then they deliver a demo that looks good in a Zoom call but falls apart the moment a real user interacts with it.

Production AI takes time. Not because the code is hard. Because the testing is hard. Because the edge cases are hard. Because making an AI agent that handles 95% of queries correctly and safely escalates the other 5% takes real engineering.

What a good answer sounds like:

"It depends on scope, but here's our experience. A focused, single-task AI agent — like a support agent that handles one category of tickets — takes 4 to 6 weeks from discovery to launch. A multi-agent system with voice, tool use, and a managed knowledge base takes 10 to 16 weeks. A full SaaS platform with AI features built in takes 12 to 20 weeks. We'll confirm the exact timeline after the discovery sprint, but those are the ranges we've seen in production."

What a bad answer sounds like:

"We can have it done in 2 weeks." Or: "It's just an API call, should be quick." Or: "We'll figure out the timeline once we start."

The red flag: If they promise less than 4 weeks for anything that touches real customer data, real integrations, or real production traffic, they haven't shipped production AI before. They've built demos. And demos break. The only acceptable answer is a realistic range backed by past experience.


Question 12 — "What does your discovery and scoping process look like before we commit?"

Why this matters:

This is the question that separates professional engineering firms from freelancers with a LinkedIn profile.

Discovery is where the actual work of understanding your problem happens. What data do you have? What systems does the AI need to connect to? What are the failure states? What does "done" look like? If the agency skips this step, they're building in the dark. And you're paying for them to figure it out as they go.

What a good answer sounds like:

"We run a structured discovery sprint. It's usually two weeks. We interview your team, audit your existing data and systems, identify the risks, and define what success looks like. At the end, you get four things: a technical specification, an architecture document, a risk assessment, and a fixed proposal for the build. You own all of it. If you decide not to proceed with us, you keep the spec and can hand it to another team."

What a bad answer sounds like:

"We'll figure it out as we go." Or: "We like to stay agile and adapt." Or: "Discovery is just a formality, we can skip it and start building."

The red flag: If they can't describe what deliverables you get from discovery, they don't have a process. They're improvising. And improvisation in AI development means you'll be paying for their mistakes. A real firm will tell you exactly what you get, how long it takes, and what it costs before you commit to the full build.

Category 5 — Post-Launch Support & Iteration

Here's something most buyers don't think about until it's too late: AI systems don't just "work" after you launch them. They degrade.

Models get updated. APIs change. User behavior shifts. A prompt that worked perfectly in March starts giving weird answers in June because the underlying model changed. Token costs creep up. A new edge case appears that nobody tested for.

If the agency that built your system disappears the day you pay the final invoice, you're on your own. And "on your own" with a production AI system means you'll find out about problems from angry customers instead of monitoring dashboards.

These two questions tell you whether the company plans to stick around, or whether they're collecting your money and running.


Question 13 — "What does post-launch support look like?"

Why this matters:

Launch day is not the finish line. It's the starting line. The first week in production is when you discover all the edge cases your testing didn't catch. The first month is when you realize the AI is handling 80% of queries great but struggling with the other 20%. The first quarter is when you want to add a new feature or integrate a new tool.

If there's no support structure, every one of those moments becomes a crisis. And crises are expensive.

What a good answer sounds like:

"We don't disappear after launch. Here's what you get. Monitoring dashboards are live from day one — you can see token costs, response times, error rates, and conversation quality. We set up alerts so if something breaks or costs spike, we know before your customers do. We have an on-call rotation for critical issues with a defined SLA — usually 4 hours for production-down problems. And we schedule monthly check-ins to review performance, discuss improvements, and plan the next iteration."

What a bad answer sounds like:

"We'll be available if you need us." Or: "Just email us and we'll get back to you." Or: "We offer a monthly retainer if you want ongoing support." (Notice how they're already upselling you before the project is even built.)

The red flag: If they can't tell you their response time for a production outage, or if they don't mention monitoring dashboards, they haven't supported a live AI system before. They've built things and walked away. You don't want to be the one who discovers the system is broken because a customer tweeted about it.


Question 14 — "How do you handle model updates, API changes, and dependency upgrades?"

Why this matters:

This is the question nobody asks, and it's the one that causes the most pain six months after launch.

Here's what happens. OpenAI releases a new version of GPT-4. Anthropic updates Claude. A vector database you're using changes its API. A Python library you depend on pushes a breaking change.

If your system isn't built to handle these updates gracefully, every single one of them is a production risk. The AI starts answering differently. Costs change. Latency shifts. Something that worked yesterday breaks today, and nobody knows why.

What a good answer sounds like:

"We version-control everything. Every prompt is tagged with a version number. Before we switch to a new model or update a dependency, we run the entire eval suite against the new version. If the scores drop, we don't deploy. We also pin dependencies so nothing updates automatically and breaks your system. When a model provider announces a change, we evaluate it, test it, and give you a recommendation before anything touches production."

What a bad answer sounds like:

"We'll update it when it breaks." Or: "The model providers handle backward compatibility, so it shouldn't be an issue." Or: "We'll cross that bridge when we come to it."

The red flag: If they don't mention the words "version control," "eval suite," or "regression testing" in their answer, they have no process for handling change. And in AI, change is constant. OpenAI updates models multiple times a year. Anthropic does the same. If your system can't absorb those updates safely, you're going to have a bad time every single quarter.

Category 6 — Track Record & References

This is the final question. And honestly, it's the one that matters most.

You can ask all the technical questions in the world. You can talk about eval suites, observability, data security, and compliance. But at the end of the day, the only proof that a company can actually deliver is that they've done it before. For someone else. In a situation similar to yours.

This question separates real engineering firms from sales teams with a nice website.


Question 15 — "Can we speak to a past client whose project is similar to ours?"

Why this matters:

Every agency will show you logos on their website. Every agency will have a "Case Studies" page with vague descriptions like "We helped a Fortune 500 company improve efficiency by 40%."

That's marketing. It's not proof.

Proof is getting on a 15-minute phone call with the CTO or product lead of a company that hired this agency, built something similar to what you're building, and shipped it to production. Proof is hearing them say, "Yeah, they delivered on time, the system works, and here's what surprised us."

What a good answer sounds like:

"Absolutely. We'll connect you with two past clients who built systems similar to yours. You can also review our detailed case studies to see the exact systems we've shipped. One is in your industry, the other is in a different vertical but solved the same technical problem. We'll set up a 15-minute call with each. You can ask them anything — about our process, our communication, what went wrong, and whether they'd hire us again."

What a bad answer sounds like:

"We can't share client names due to NDAs." (This is sometimes true, but a good agency will still find a way to connect you with someone.) Or: "We have testimonials on our website." Or: "Our clients are very busy, but I can share a written quote."

The red flag: If they can't provide a single reference — not even one, not even with an NDA — they haven't shipped enough production work to have references. They might have built demos. They might have done small pilots. But they haven't delivered enough real systems to have clients willing to vouch for them. That's a risk you shouldn't take.


The Red Flags — 7 Warning Signs to Walk Away

You've asked the 15 questions. Now here's the quick-reference list of dealbreakers. If you see any of these during your evaluation, stop the conversation. Save your money. Walk away.

1. They can't explain their testing process.
If they don't mention eval suites, regression testing, confidence scoring, or test cases, they are not testing. They are guessing. And you will be the one who finds the bugs in production.

2. They guarantee zero hallucinations.
Every large language model hallucinates. Every single one. If a company tells you their AI won't hallucinate, they either don't understand the technology or they're lying to close the sale. The only acceptable answer is an explanation of how the system handles hallucinations safely.

3. They want access to your production database during development.
Developers should never touch live customer data. They should work with anonymized datasets, synthetic data, or sandboxed environments. If they ask you to email a CSV export of your real users, that's a security incident waiting to happen.

4. They use proprietary frameworks you can't leave.
If the system is built on a framework only they understand, you're locked in. You can't hire another developer. You can't switch agencies. You can't maintain it yourself. You're renting your own product forever.

5. They quote a fixed price without doing discovery.
If they give you a number without understanding your data, your systems, your constraints, and your goals, they're guessing. And when their guess is wrong, you pay for it through change orders, scope creep, and surprise invoices.

6. They can't provide a single reference.
No references means no production deployments. It means they've built demos, maybe run small pilots, but haven't delivered enough real systems to have clients willing to vouch for them. You don't want to be their first real project.

7. They promise delivery in 2 weeks for a complex AI system.
Production AI takes time. A focused agent takes 4 to 6 weeks. A multi-agent system takes 10 to 16 weeks. A full SaaS platform takes 12 to 20 weeks. If they promise less than 4 weeks for anything that touches real customer data and real integrations, they're building you a demo, not a system. And demos break.


The Scoring Checklist — Rate Your AI Development Partner

Print this out. Bring it to your sales calls. Score each category as you listen to the agency's answers. At the end, add up the total.

CategoryQuestionsScore (out of total)Notes
Technical Depth & Production ReadinessQ1–Q4__ / 20
Ownership, Code & HandoverQ5–Q7__ / 15
Data Security & ComplianceQ8–Q9__ / 10
Pricing, Timeline & ProcessQ10–Q12__ / 15
Post-Launch Support & IterationQ13–Q14__ / 10
Track Record & ReferencesQ15__ / 5
TOTAL15 Questions__ / 75

Download the Checklist: Download Now

How to score each question:

  • 5 points: They gave a specific, detailed answer with examples from past projects.
  • 4 points: They gave a good answer but lacked specific examples.
  • 3 points: They gave a vague answer but seemed to understand the concept.
  • 2 points: They gave a weak answer or dodged the question.
  • 1 point: They couldn't answer or gave a red-flag answer.
  • 0 points: They refused to answer or gave a dangerous answer.

How to read your total:

  • 65–75: Elite engineering partner. They know what they're doing. Proceed with confidence.
  • 50–64: Solid, but probe deeper on the weak categories. Ask follow-up questions. Get specifics.
  • 35–49: Significant gaps. Proceed with caution. You'll need to manage this agency closely, and that costs you time and energy.
  • Below 35: Walk away. They're not ready for production work. You'll spend more money fixing their mistakes than you would have hiring someone better.

How Kreability Answers These 15 Questions

We wrote this guide. We believe in it. So here's the fair thing to do: answer every single question for ourselves, transparently, so you can judge us by the same standard we're asking you to apply to everyone else.

If our answers don't meet the bar we just described, don't hire us. But if you want to see how Kreability operates as a true engineering partner, read below. Seriously. Use the checklist. Score us. If we don't hit at least 65 out of 75, walk away.

Here's where we stand.

#The QuestionKreability's Answer
1How do you test and evaluate AI agents before they go to production?We build eval suites for every project. Before any prompt change ships, it runs against a set of test cases — real queries, edge cases, and known failure scenarios. If the new version scores worse than the old one, it doesn't deploy. We also use confidence scoring so the system knows when it's unsure and hands off to a human instead of guessing.
2Are your systems locked into one model provider?No. We build provider-agnostic. The core logic, prompts, and tool connections aren't tied to one model. We can swap Claude for GPT for Gemini without rewriting your app. This matters the day one provider has an outage, raises prices, or releases a better model.
3What happens when the AI hallucinates in production?We design for failure. Confidence thresholds mean the system doesn't answer if it's unsure. Human-in-the-loop checkpoints protect high-stakes actions like refunds or escalations. Fallback routing ensures the customer always gets a response — from a human if the AI can't handle it safely.
4What observability and monitoring do you set up?Langfuse or OpenTelemetry dashboards from day one. You see token costs per conversation, response latency, error rates, and prompt drift. Alerts fire if costs spike or error rates go above threshold. You'll never find out about a problem from an angry customer.
5Who owns the code, prompts, and training data?You do. Everything. Code, prompts, eval datasets, architecture docs, deployment scripts. All of it transfers to your repository at launch. We use standard frameworks — LangGraph, Next.js, Python — so any competent developer can pick it up. No proprietary layers. No lock-in.
6What does the handover documentation look like?You get a full handover package. Architecture diagrams showing how every component connects. Deployment guides for your DevOps team. A runbook for common issues. Prompt versioning docs. And an incident response plan for when things break in production. Not a Zoom call. Actual documentation.
7Can our internal team maintain and iterate without you?Yes. That's the goal. We use standard, widely-known frameworks so your team can hire for them. Clean, documented code. CI/CD pipelines. And before we leave, we run hands-on training sessions where your developers make changes and deploy them while we watch.
8How do you handle our customer data?Your data stays in your cloud. For development, we use anonymized or synthetic datasets. In production, multi-tenant systems use Row-Level Security in the database and vector stores so the AI can only access data belonging to the user asking the question. Your developers never see raw production data.
9What's your approach to AI compliance?We build compliance into the architecture. Vector databases support GDPR deletion requests. We build audit trails using observability tools so you can prove why the AI made any decision. We design with the EU AI Act in mind — human oversight for high-risk actions, transparency for users.
10How is pricing structured?Discovery sprint first — fixed price, usually two weeks. We investigate your data, systems, and constraints. Then we produce a technical spec, architecture doc, and risk assessment. Based on that, we give you a fixed quote for the build. No surprises. If something changes mid-build, we flag it and agree on impact before proceeding.
11What's a realistic timeline?4 to 6 weeks for a focused, single-task agent. 10 to 16 weeks for a multi-agent system with voice and tool use. 12 to 20 weeks for a full SaaS platform with AI features. We confirm the exact timeline after discovery, but those are the ranges we've delivered in production.
12What does discovery look like?Two weeks. We interview your team, audit your data and systems, identify risks, and define success. You get four deliverables: a technical specification, an architecture document, a risk assessment, and a fixed proposal. You own all of it. If you don't proceed with us, you keep the spec.
13What does post-launch support look like?Monitoring dashboards live from day one. Alerts for cost spikes and error rates. On-call rotation for critical issues with a 4-hour SLA. Monthly check-ins to review performance and plan iterations. We don't disappear after launch.
14How do you handle model updates and dependency changes?Everything is version-controlled. Every prompt has a version number. Before switching models or updating dependencies, we run the full eval suite. If scores drop, we don't deploy. Dependencies are pinned so nothing breaks automatically. When a provider announces a change, we evaluate it and give you a recommendation.
15Can we speak to a past client?Yes. We'll connect you with past clients who built systems similar to yours. We'll set up a 15-minute call. You can ask them anything — about our process, our communication, what went wrong, and whether they'd hire us again.

That's the full picture. No spin. No marketing language. Just what we do.

If you want to hold us to these answers, book a discovery call and ask us the 15 questions yourself. We'll answer every one live, on the spot.

Book a Discovery Call →


Frequently Asked Questions

How much does it cost to hire an AI development company?

It depends on scope, but here's the honest breakdown. A discovery sprint where the agency investigates your data, systems, and constraints typically costs between 5,000 and 5,000 and 15,000. A focused, single-task AI agent usually lands between 20,000 and 20,000 and 50,000. A multi-agent system or AI-powered SaaS platform ranges from 50,000 to 50,000 to 150,000+. Be careful with anyone who quotes significantly below these ranges. They're either cutting corners on testing, using junior developers, or planning to hit you with change orders later. Production AI costs money because production AI requires testing, monitoring, and documentation. If it's cheap, something is missing.

How long does it take to build a production AI agent?

A focused, single-task agent like a support bot that handles one category of queries takes 4 to 6 weeks from discovery to launch. A multi-agent system with voice, tool use, and a managed knowledge base takes 10 to 16 weeks. A full SaaS platform with AI features takes 12 to 20 weeks.

If someone tells you less than 4 weeks for anything that touches real customer data and real integrations, they're building a demo, not a system. Demos look great in sales calls. They break in production.

Should we build AI in-house or hire an agency?

Here's the honest answer. Build in-house if you already have 3 or more dedicated AI engineers who have shipped production systems before. You'll move slower, but you'll own the knowledge.

Hire an agency if you need to ship fast, you don't have the team, or you want to avoid the 6-month learning curve of hiring and onboarding AI engineers. A good agency will build the system, hand it over cleanly, train your team, and leave. The goal is to make themselves unnecessary.

What you should never do is hire a single freelancer to build a production AI system. One person can't cover architecture, testing, observability, security, and deployment. You'll end up with a fragile system and no documentation.

What is the difference between an AI development company and an AI consulting firm?

A consulting firm advises. They produce strategy decks, roadmaps, and recommendations. They tell you what to build and why. Then they leave.

A development firm builds and ships. They write the code, set up the infrastructure, test the system, deploy it, and hand it over. They produce a working product, not a PowerPoint.

Some firms do both. Kreability does both but we bias toward building. If you need a 50-page strategy document, hire a consultant. If you need a working AI system that handles real customers and real data, hire a development firm. And make sure they can prove they've shipped one before.



Ready to replace manual work with AI?

Get in touch to explore where AI can have the biggest impact in your business.