What generative AI actually changes for in-house legal teams
By Asım Serdar Yılmaz · Published · Last updated · 6 min read
When our in-house team built generative AI tooling on AWS Bedrock, the choice of model was not the first real decision. That was which single workflow to point it at first, and in what order to add the rest: we wrote down what the system must never do before we wrote down what it could do, then picked one narrow, high-volume task to run first. Sequencing taught me more than any product demonstration has, which is why the questions worth asking here are the narrow ones: what this changes, what it leaves untouched, and how you run it so that it helps rather than quietly creating risk.
What does not change
Start with what stays put, because that is the part people skip. The duty to reach a defensible answer does not move: a model can draft a position, but a lawyer owns it, signs it off, and answers for it if a regulator asks. Privilege, confidentiality and conflicts do not relax because a tool is in the loop, and if anything they get harder, since a poorly configured system can move sensitive material to places it should never be. How far privilege even reaches depends on where you are standing. In-house communications are not protected everywhere, and in a European Commission competition investigation they attract no privilege at all, so a tool that copies such material into new systems only widens the exposure.
Judgment about risk appetite does not change either. How much regulatory ambiguity a business can carry, and on what terms, comes out of context, relationships and history that sit in people’s heads rather than in any model. So do the harder calls, when to escalate, when to say no, when to say yes with conditions attached. Those decisions are the actual job, and I haven’t seen a model come near them.
This is worth saying plainly, because a product demonstration is very good at making you forget it. A regulator does not accept “the tool produced it” as an answer, and neither does a board. The lawyer is still accountable, and that should shape every design choice that follows: put these tools where they support an accountable human, not where they obscure one.
What actually changes
What does change is the work that surrounds those decisions, and there is a great deal of it. Three areas stand out, at least in the work I see.
The first is intake. A large share of an in-house team’s time goes on routing questions, restating them so they make sense, and chasing the facts needed to answer them. A well-scoped model can triage a request, route it to the right team, put the standard ones into a template, bring the matters that genuinely need a lawyer forward, ask for the missing pieces, and hand the lawyer a clean brief instead of a vague email. No decision has been taken at that point, only the friction in front of it cleared away; the legal judgment and the decision remain with the accountable lawyer.
The second is knowledge retrieval. In-house teams sit on years of prior advice, playbooks and precedent, nearly all of it unsearchable at the moment somebody needs it. Retrieval over that material, done carefully, means a lawyer can find how the team handled a similar question two years ago in seconds rather than hours. Nothing new has been created there; the team is just getting its own reasoning back quickly.
The third is first drafts. Standard clauses, routine responses, summaries of long filings and initial cuts of internal memos are all work where a model can produce a serviceable starting point, after which the lawyer edits, corrects and takes ownership of it. With a suitable task and setup, a first draft can emerge within minutes; that is a possibility rather than a measured result here or a delivery guarantee, and the lawyer’s review and correction remain required. Getting past the blank page looks like a small gain until you time it, because on most of this work it is the slowest step there is. When we built generative AI tooling on AWS Bedrock, the first-phase target was a meaningful, measured reduction in manual workload on exactly this kind of task, together with less routine dependence on outside counsel. The point was never to remove the lawyer. It was to stop spending a lawyer’s hours on work that does not need one.
How to run it responsibly
The gap between a useful tool and a liability sits almost entirely in how the thing is run. A few rules have held up so far.
- Set guardrails before capability. Decide what data the system may touch, where outputs may go, and what it must never do, and enforce that in configuration rather than in a policy document nobody reads. Privacy, confidentiality and data residency belong in that first decision too; bolting them on after the tool is running does not work.
- Tie review to risk. High-impact work and anything that leaves the team gets a lawyer’s review before it goes out; low-risk internal steps can carry a lighter touch. Treat that review as the operational approval gate, not the whole of responsibility: accountability runs across the design, the configuration and the sign-off, not one moment at the end. Setting how things run by risk and reversibility reduces both unnecessary load and unseen risk.
- Run measurable pilots. Pick one workflow, define what good looks like before you start, and measure it. A pilot that cannot show whether it saved time or introduced errors has told you nothing, and neither does one that never looks at what happened to outside-counsel spend. Ours went in stages: start narrow, measure, correct, then widen.
- Track errors honestly. Models are confidently wrong in ways that look plausible; the 2024 hallucination studies in the Journal of Legal Analysis and from Stanford document exactly this pattern in legal use. You need a way to catch that, and a team that treats a wrong answer as a finding rather than an embarrassment.
One more point about outside counsel, since that is where the economics are supposed to show up first. A lot of what a legal team sends out is volume rather than hard law: routine research, first-pass review and drafting that leaves the building because there is no internal capacity, not because the question needs a specialist. Bringing some of that back in is what teams mean when they set a target for lower outside-counsel spend. I would treat it as a target and not a promise, since plenty of teams have not seen those savings land, while the genuinely difficult matters still go to the firms that should handle them.
Generative AI has not changed what a legal team is for. It has changed how much of the work around the judgment has to be done by hand, and that is worth a lot if you are honest about where the judgment still sits. The difference it makes is real and it is almost never the one in the demonstration: the road to the judgment gets shorter, and the judgment stays where it was. In my experience the teams that get this right are specific about the workflow before they are enthusiastic about the tool, and they prove each step before they trust it with the next. The ones that go the other way round tend to end up with a pilot nobody can describe.
Sources
- Court of Justice of the European Union, Akzo Nobel Chemicals and Akcros Chemicals v Commission, C-550/07 P (2010)
- Journal of Legal Analysis, Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models (2024)
- Magesh, Surani, Dahl, Suzgun, Manning and Ho, Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies, 2025 (Stanford RegLab and HAI preprint, 2024)
Next: AI governance for in-house legal teams: five minimum controls →