Week one, day one: define what “feasible” means before you write code
The single biggest failure mode in fast prototyping is starting to build before defining what success looks like. Before any code, write down: what’s the riskiest technical assumption in this idea? Is it “can retrieval find the right information in our messy data,” “can an LLM reliably extract structured fields from our specific document format,” or “can we get acceptable latency for a real-time use case”? Name the actual risk. A two-week prototype should be laser-focused on de-risking that one thing, not building a polished end-to-end product.
Also define your acceptance bar numerically if at all possible: “80% of test queries return a relevant answer,” or “extraction accuracy above 90% on a sample of 50 real documents.” Vague success criteria like “see if it feels good” produce prototypes nobody can make a confident go/no-go decision from.
Days 1-2: assemble real data, not synthetic data
Prototypes that get evaluated on made-up or cherry-picked examples give false confidence. Pull 30-50 real examples from your actual domain — real support tickets, real documents, real user questions if you have any, even from a beta or manual process. This is often the most time-consuming part of the two weeks and the part teams are most tempted to skip. Don’t skip it. A prototype validated against synthetic data tells you nothing about whether it’ll work on the messy reality of your actual use case.
Days 3-5: build the thinnest possible version of the real architecture
This is the part people get wrong most often: they build a demo that doesn’t resemble the real system at all (a single unstructured prompt with no retrieval, no validation) and then can’t tell whether a positive result means the idea works or just means the demo was cherry-picked.
Instead, build the actual architecture pattern you’d use in production, just without the polish. If it’s a RAG feature, build real chunking and retrieval over real data — don’t fake it with everything crammed into one prompt. If it’s document extraction, use real OCR and a real structured-output schema, not a hand-picked clean example. Skip the things that don’t affect feasibility: authentication, a polished UI, edge-case error handling, scalability. Keep the things that determine whether the core idea works.
Days 6-8: run your evaluation set and look at failures, not just successes
Run your 30-50 real examples through the prototype and score them against your acceptance bar. The valuable part isn’t the aggregate score — it’s reading every failure individually and categorizing why it failed. Bad retrieval? Bad chunking? Model limitation? Ambiguous source data? This failure analysis is what tells you whether the gap to production quality is a solvable engineering problem or a fundamental feasibility issue.
Days 9-10: make the call, and write down why
By day 10 you should have a clear answer to one of three outcomes: it works well enough that a full build is low-risk, it doesn’t work yet but the failures are clearly fixable engineering problems (better chunking, a different model, cleaner source data), or it’s revealing a fundamental feasibility issue that needs a different approach entirely (maybe the data isn’t there, or the task needs deterministic logic instead of an LLM).
Write this decision down with the evidence behind it. This becomes your scoping document for the full build — a much stronger foundation than a green light based on gut feel from a polished demo.
What to explicitly not do in a two-week prototype
Don’t build authentication, billing, or a production-grade UI — none of that answers the feasibility question. Don’t optimize for cost or latency yet — that’s a production concern, not a feasibility one. Don’t skip the failure analysis to save time — it’s the highest-value part of the exercise. And don’t evaluate on cherry-picked examples just to get a good-looking demo for a stakeholder update — that defeats the entire purpose and sets up a much more expensive failure later in the full build.
Why this matters before you commit real budget
A disciplined two-week prototype turns “we think this AI idea could work” into “we know it works on real data, and here’s exactly what the full build needs to handle.” That’s the difference between a scoped engagement with a clear budget and timeline, and an open-ended build that discovers its hardest problems three months and a lot of runway in.
This is close to how we scope early client engagements — sometimes as a low-cost or free working prototype specifically to prove out the riskiest technical assumption before a client commits to the full build. It’s not the right move for every project, but for a genuinely novel idea, it’s the fastest way to make a confident decision.
CTA: If you have an AI idea and want to know if it’s actually feasible before committing budget, let’s scope a fast prototype together — nextpak.org, short call.