Why "generalist developer" isn't enough anymore

01.Where generalists struggle on AI work

1. Evaluation. Knowing whether a model is good enough needs test sets, metrics and judgement. "It looked right in testing" is not a standard.

A generalist will run a handful of examples, see good answers and ship. A specialist will ask what the test set looks like, how many cases cover edge conditions, how results compare with a baseline, and what happens when the model is wrong. They build an evaluation set before building the feature, and they run it with every change.

2. Data. Most AI quality problems are data problems. Specialists spot them early.

Duplicated records, stale documents, inconsistent labels and missing fields all produce confusing output. A generalist may blame the model and swap it for another. A specialist will look at the inputs first, because that is where the cause usually sits.

3. Production behaviour. Latency, cost and drift matter more than the demo. MLOps experience prevents expensive surprises.

A demo with ten users is cheap. A product with ten thousand has token costs, rate limits, timeouts, caching and fallbacks to design. Models degrade as data changes. Without monitoring, nobody notices until customers do.

4. Agent safety. Agents that take actions need permissions, logs and fallbacks designed in.

An agent that can send emails, update records or issue refunds is not a chatbot. It needs limits on what it may do, human approval for risky steps, a full record of each action and a way to stop it. These are design decisions, and they are expensive to retrofit.

02.What this looks like in practice

Imagine a team adding a support assistant that answers customer questions from internal documents. A generalist builds a first version in a week. It works on the demo questions.

In production, it fails in familiar ways. It cites an outdated policy because old documents were never removed. It answers confidently when the right document does not exist. Costs rise because every message sends a large block of text. Nobody can say whether the new version is better than last week's.

A specialist would have set up retrieval with document freshness in mind, added a "no answer" path, trimmed context to control cost, and built an evaluation set so changes could be compared. None of that is exotic. It is simply experience.

03.A second example: the retrieval problem

Consider a team building a search assistant over contracts. A generalist reaches for a vector store, splits documents into equal chunks and wires up a model. The first demo looks good.

Then a lawyer asks about a termination clause and gets text from the wrong contract. Chunk boundaries cut clauses in half. Similar clauses from different agreements look alike to the search. A specialist would split by document structure, store metadata so results can be filtered by contract, test retrieval separately from generation and measure whether the right passage appears in the top results. Only then would they tune the model's wording.

The lesson is that most of the quality sits in the parts around the model. Knowing that, and where to look first, is what specialist experience buys.

04.Signs you have outgrown generalists

• Quality complaints arrive from users before your own tests catch them.
• Nobody can say whether last week's change helped.
• The cloud bill for AI features is a surprise.
• Prototypes work, but nothing reaches production.
• Prompt changes are made by gut feel and never recorded.
• Nobody owns monitoring for model behaviour.

If three or more are true, a specialist will repay their cost quickly.

05.How to bring a specialist in well

Give them a real problem, not a research brief. Give them access early. Pair them with a generalist who knows the product, and set a short check-in to review decisions. Ask them to write down evaluation methods and monitoring plans in the team's own wiki. A specialist who leaves nothing behind has only solved today's problem.

06.The practical answer

Keep your generalists. They know your product. Add one or two specialists who raise the whole team's level, and let them pair. That mix ships faster than either group alone.

A workable pattern:

• Put the specialist on the design decisions: evaluation, data, deployment approach.
• Have generalists build the surrounding product, with the specialist reviewing.
• Pair on the first production release, so skills transfer.
• Write down the evaluation method and monitoring plan, so it survives staff changes.
• Review after a quarter. Decide whether to keep, reduce or hand over.

07.The cost question

Specialists cost more per day than generalists. They can also cost less overall. A feature that ships with an evaluation set, cost controls and monitoring avoids weeks of firefighting, rewrites and surprise bills. Compare the cost of a specialist for a few months with the cost of a failed launch, a stalled roadmap or a quiet loss of user trust. For many teams the comparison is not close.

There is also an option value. Short, flexible engagements let you try a specialism before committing to a permanent hire. If the work proves central to your product, you will know exactly what profile to recruit for.

08.What generalists do better

It is worth saying plainly where generalists lead. They understand the product, the customers and the codebase. They write the surrounding application, handle the user interface, and make sure the AI feature fits a real workflow. A specialist who builds a technically sound model that nobody can use has not helped. The best results come when both groups respect each other's strengths and review each other's work.

09.Common mistakes

• Hiring one specialist and isolating them from the product team.
• Expecting a data scientist to also run production infrastructure.
• Skipping evaluation because there is a deadline.
• Letting cost per request go unmeasured until the bill arrives.
• Giving agents broad permissions at launch.
• Treating the model as the whole problem.

10.A quick self-check

• Do we have a test set for each AI feature?
• Can we show quality over time?
• Do we know cost per request?
• Do we know what the agent can do on its own?
• Who is paged when the feature misbehaves?

11.Hiring order for a first AI team

If you can add only one or two people, a sensible order is this. First, an ML or LLM engineer who owns evaluation and can build the first production feature. Second, a data engineer or MLOps engineer who makes the pipeline and deployment reliable. Add an architect when several teams start building AI features and need shared standards. Hire for the gap that currently causes the most pain, not the title that sounds most impressive.

12.What to do next

Look at your current AI work. For each feature, ask four questions. How do we know it is good? What does the data look like? What does it cost per request? What happens when it is wrong? Gaps in the answers show you which specialism you need first.

If you need that skill quickly, Brandsmashers places pre-vetted AI/ML engineers who join your team and your sprint. You interview them and make the final call.