12 Questions to Ask a Software Development Consultancy
The questions to ask a software development consultancy that separate real engineering teams from good salespeople — plus the answers that should end the call.
Most Vendor Questionnaires Can Be Passed by Reading the Website
There are a hundred listicles of questions to ask a software development consultancy, and nearly all of them share one defect: every question on the list has a right answer the vendor already knows. "Do you write tests?" Yes. "Is the code ours?" Yes. "How do you handle communication?" Weekly demos and a Slack channel. You have learned nothing, and you have used up an hour of the only diligence window you get.
A useful question has a property the generic ones don't: the answer is checkable, and a weak team cannot fake it without producing a specific, falsifiable claim. You are not trying to find out whether a consultancy says the right words. You are trying to find out whether the words correspond to anything.
At Kuaray we sit on both sides of this. We're asked these questions by prospective clients in the US and Europe, and we're also brought in to do technical due diligence on other people's vendors — usually after a delivery has already gone sideways. The pattern is consistent enough to write down: the failures were visible in the sales conversation, and the buyer asked questions that could not have surfaced them.
Here is the set we'd use, grouped by what each group actually tests. Twelve questions, chosen because a weak team cannot answer them in the abstract.
Group 1: Does an Engineer Answer, or Does an Account Manager?
The first thing to establish is who you are talking to and who will actually do the work. Staffing bait-and-switch is the most common and least discussed failure in software development vendor evaluation.
1. Who exactly writes this code, and can I talk to them before signing? Not "do you have senior engineers." Names, seniority, and whether they are currently on another engagement. A consultancy that is comfortable with its bench will put the lead engineer on a call unaccompanied. If every technical conversation routes through a delivery manager, assume the people on the call are not the people on the keyboard.
2. Walk me through the last project that went badly, and what you changed afterwards. This is the highest-yield question in the set, and almost nobody asks it. Everyone has a disaster. A team with real engineering culture will describe a specific incident, a specific root cause, and a specific process change — because they ran a post-mortem. A team without one will either deny the premise or produce a vague story about "a client with shifting priorities," which is a way of saying the fault was elsewhere.
3. What did you argue with your last client about, and who won? A consultancy that has never pushed back on a client's technical decision is selling capacity, not judgment. You want to hear about a time they said "that will not work" and either prevailed or documented why they disagreed and proceeded anyway. Pure agreeableness is a risk you are paying a premium not to have.
Group 2: Questions to Ask a Software Development Consultancy About Engineering
These are the ones where a vague answer is itself the finding.
4. What breaks your build, and what does it block? Ask for the actual gate list: tests, type checks, linting, dependency audit, migration checks, coverage thresholds. Then ask what happens when the pipeline is red on a Friday with a deadline on Monday. The second half is the real question. Every team has CI. The interesting variable is whether the gate is allowed to lose.
5. How do you test the parts that aren't unit-testable? Unit tests are table stakes and prove very little about integrated behaviour. You want to hear about contract tests against external APIs, ephemeral preview environments per pull request, seeded test data, and some form of end-to-end coverage on the two or three flows that actually generate revenue. "100% coverage" as an answer is a yellow flag — it usually means tests written to satisfy a metric.
6. Show me how an incident gets diagnosed in something you've already shipped. Ask them to describe, concretely, the path from "a user reports the thing is broken" to a root cause: structured logs, trace IDs propagated across services, dashboards, alert thresholds, who gets paged. Observability is the single best proxy for engineering maturity, because it costs real effort, delivers no demo value, and only teams that have been burned bother to build it properly.
7. What does handover look like if we part ways after phase one? The answer should be boring: the repository is yours from commit one, infrastructure is in code in your cloud account under your billing, secrets are in your vault, CI runs in your organisation, and the architecture decision records explain why, not just what. Any dependency on the vendor's own tooling, accounts, or undocumented knowledge is lock-in, whether or not anyone intended it. This is the question that most reliably produces an uncomfortable pause.
Group 3: The Agent-Era Questions Nobody Asked Three Years Ago
Essentially every competent consultancy now ships code written with AI agents. Ours does. The question is not whether they use them — a team that claims it doesn't is either behind or not being straight with you. The question is what sits between the agent and your production branch.
8. What percentage of merged code was agent-authored, and who reviewed it? You are listening for a number and a named human. The specific failure to probe for is review throughput: if agent-assisted output went up and the reviewer headcount didn't, something absorbed the difference, and it is usually review depth. Ask directly how many pull requests merged last month with no human approval.
9. How do you handle licence and provenance risk on generated code? A real answer involves dependency and licence scanning in CI, a policy on copyleft, and an SBOM they can hand you. "Our developers check" is not a control. If you sell into regulated buyers, you will eventually be asked to attest to this, and you want the attestation to be true.
10. If you're building agentic features for us, what stops an agent from doing something irreversible? Scoped short-lived credentials rather than one shared service account, idempotent writes, hard budgets per run, approval gates on anything externally visible, and an independent checker that verifies the work rather than trusting the agent's own summary. Those controls belong in the tool layer, not in a prompt. A vendor who cannot name two of them has shipped a demo, not an operated system.
Group 4: Commercial Shape, in One Question Each
11. How do you price, and what specifically happens when the scope is wrong? Every scope is wrong somewhere. What matters is whether the change path is a half-hour conversation or a three-week amendment cycle — and whether the commercial model punishes the vendor for discovering a better approach mid-build. We've laid out the trade-offs between fixed scope and flat-rate engagements separately; for diligence purposes you only need to hear a mechanism, not a philosophy.
12. Who is legally on the hook, and under which jurisdiction? Entity, governing law, insurance, data processing agreement, sub-processor list, where the data physically sits. Distributed teams are normal and fine. Ambiguity about which legal entity you are contracting with is not.
Weak Answer vs Strong Answer
| Question | Weak answer | Strong answer |
|---|---|---|
| Who writes the code | "Our senior team" | Named engineers, available on a call |
| Last failure | "Client changed priorities" | Specific incident, root cause, process change |
| CI gates | "We have CI/CD" | Named gates, and what happens when they're red |
| Testing | "100% coverage" | Contract tests, preview envs, E2E on revenue flows |
| Incident path | "We monitor everything" | Trace IDs, dashboards, paging rota, a worked example |
| Handover | "We'll document it" | Your repo, your cloud, IaC, ADRs, from day one |
| Agent-written code | "We use AI responsibly" | A percentage, a named reviewer, no-review PR count |
| Provenance | "Developers check licences" | Licence scanning in CI, SBOM on request |
| Scope change | "We're flexible" | A concrete change mechanism with a timebox |
How to Run This in One Week
You do not need a procurement process. You need five working days and the discipline to score answers rather than vibes. Run the questions to ask a software development consultancy in this order and the expensive mistakes surface early, while walking away is still cheap.
- Send questions 1, 4, 7 and 8 in writing before the first call. Written answers are slower to improvise, easier to compare, and the quality of the writing is itself a signal.
- Run a 60-minute technical call with the lead engineer and no account manager present. Spend it on groups 2 and 3. Interrupt when an answer goes abstract and ask for a specific instance.
- Ask for a code artefact you're allowed to see — an open-source repository, a sanitised sample, a public ADR. You are looking at structure, test shape, commit hygiene and whether the README would let a new engineer start.
- Take two references and ask both the same three things: what went wrong, how it was handled, and whether the people who sold the work were the people who did it.
- Buy a small, real, bounded first slice. Two to four weeks of genuine production work beats any amount of diligence. Our MVP approach exists partly for this reason: a paid trial run tells you more about a custom software partner than a reference ever will.
- Score, then decide. Write the scores down before the final call so the most charming vendor doesn't win on recency.
Three Red Flags That Are Worth Ending a Call Over
- No failure story. A team that has never had one is either new or not telling you the truth, and both matter.
- Resistance to putting the lead engineer on an unaccompanied call. There is no benign explanation for this.
- Handover answers that depend on the vendor's own infrastructure. Lock-in built during the honeymoon is the most expensive kind.
The best questions to ask a software development consultancy are the ones a good consultancy enjoys answering. Being asked question 6 by someone who knows what a trace ID is for is a relief, because it means the engagement will be judged on engineering rather than on deck quality. If a vendor seems to resent the diligence, you have learned the most useful thing available in the first hour.
Put these questions to us directly — book a technical call with Kuaray. We'll answer all twelve on the record, with the lead engineer on the call. You can also read how we work and who we are first.