Azure AI Studio, previewed at Ignite in November, is the workbench Microsoft wants you to build generative applications in — model catalog, prompt orchestration, evaluation, content safety, and deployment under one portal. It is genuinely good, and it is also, at this date, still in public preview. That matters, because half the questions we field about it assume a general-availability guarantee that does not exist yet. You can absolutely take work from AI Studio into production right now; you just have to do it with clear eyes about what preview means for your change management, your support commitments, and the surface you are building on shifting under you.
The gap this note is about is not the model's — the models are the easy part. The gap is between a demo that wins a hackathon and a system your on-call engineer can defend at 3am. A demo needs one prompt to work once in front of a friendly audience. A production system needs that prompt to work on the inputs you did not anticipate, to fail safely when it does not, to cost a predictable amount, to keep secrets, and to be debuggable by someone who was not in the room when it was built. Everything below is about closing that gap.
Prompt flow is where the demo becomes a system
The single most useful thing AI Studio gives you over a notebook is prompt flow — a way to express the whole generative pipeline as a graph of steps rather than a wall of glue code. Retrieval, prompt construction, the model call, output parsing, post-processing: each becomes a node you can inspect, version, and test in isolation. The reason this matters is not aesthetics. It is that a demo is almost always a single prompt hiding a dozen implicit assumptions, and prompt flow forces those assumptions into named, testable steps.
The discipline we impose on top of it: treat the flow as source code. It lives in version control, changes go through review, and no one edits the production prompt live in the portal because "it's just a string." That last habit is the one that bites hardest — a prompt is behavior, and unversioned behavior in production is an outage waiting for a bad edit. Prompt flow gives you the seams to run the same flow in a development environment against a test dataset before it reaches anyone real, which is the whole point of having environments at all.
The model catalog is a real decision, not a default
The catalog puts GPT-4, GPT-4 Turbo, and a growing shelf of open and partner models in one place, and the instinct is to reach for the biggest model and move on. Resist it. The model is a cost, latency, and capability tradeoff you make deliberately per use case, and the biggest model is frequently the wrong answer.
GPT-4 Turbo's larger context window buys you room for heavier grounding, which is often more valuable than raw model capability for a retrieval-heavy application — better to feed a competent model good context than to feed a stronger model thin context and hope. For classification, extraction, and routing steps, a smaller and faster model frequently matches the big one at a fraction of the latency and cost, and a well-built flow will use different models at different nodes rather than paying flagship prices for a step that sorts inputs into three buckets. The open models in the catalog are worth a look where data residency or cost pushes you toward hosting your own, though you inherit the operational weight of running them. The point is that "which model" is a design decision with a paper trail, revisited as the catalog shifts under you — and in preview it shifts often.
Evaluations are the gate; vibes are not
This is the discipline that separates teams who ship reliable generative systems from teams who ship demos and then firefight. You cannot improve what you do not measure, and "it seemed good when I tried it" is not measurement. AI Studio's evaluation tooling lets you run a flow against a curated dataset and score outputs — for groundedness, relevance, coherence, and similarity to reference answers, including model-graded evaluations where one model scores another's output against criteria.
The work is building the dataset, and it is unglamorous and non-negotiable. You need a golden set of representative inputs with known-good outputs, and you need adversarial inputs — the questions designed to make the system hallucinate, leak, or wander off task. Then every change to the prompt, the model, or the retrieval runs against that set before it ships, and a regression on the eval blocks the deploy exactly the way a failing unit test blocks a merge. Without this gate you are flying blind, and the day a prompt tweak that fixed one case silently broke five others, you will have no way to know until a user tells you. Build the eval before you build the feature; it is the difference between engineering and gambling.
Content safety and private networking are table stakes
Two platform controls are not optional for anything touching production. Azure AI Content Safety filters both prompts and completions for hateful, violent, sexual, and self-harm content across severity levels, and it should be on by default at the strictest setting your use case tolerates, with a deliberate, documented decision if you loosen it. It also gives you a lever against a class of prompt-injection and jailbreak attempts, which is worth having even when your primary defenses live elsewhere in the flow.
Networking is the control people forget until an auditor asks. By default your Azure OpenAI and AI Studio resources have public endpoints, and for anything handling real data that is the wrong posture. Private endpoints put the resources on your virtual network so they are not reachable from the public internet, managed identity replaces API keys for service-to-service authentication so there are no secrets to leak, and any keys that must exist live in Key Vault rather than in configuration or, worse, in code. This is the same platform hygiene we apply to every Azure workload; generative AI does not get a pass on it because the demo was exciting.
PTU versus pay-as-you-go, before the bill teaches you
The economics of Azure OpenAI have two shapes and choosing wrong is expensive in opposite directions. Pay-as-you-go bills per token, scales to zero, and is right for development, spiky traffic, and anything you cannot yet forecast. Provisioned throughput units — PTU — reserve dedicated capacity for a fixed cost, which buys you predictable latency and throughput and, at sustained high volume, a better effective rate, at the cost of paying for the capacity whether you use it or not.
The trap is committing to PTU before you understand your traffic, and eating a large reservation for a workload that turns out to be bursty and low-volume. Start on pay-as-you-go, instrument token consumption from day one, and let real usage data tell you whether and when the PTU math turns favorable. A worked model we walked a client through recently: their proof of concept looked cheap at a handful of test users, but projecting the same per-interaction token cost across the intended thousand-user rollout turned a rounding-error demo into a five-figure monthly line item that reshaped the whole business case. Better to find that in a spreadsheet than in an invoice, and better to find it before you have promised a launch date.
What preview means for your commitments
Because AI Studio is in public preview, a few things need to be true in your head before you build a business on it. Preview features can change, and interfaces you depend on today may shift before general availability, so isolate your application's logic from portal specifics where you can and expect some churn. Read the preview terms for the specific capabilities you are using, because support and SLA commitments differ from GA services and your enterprise agreement may treat them differently. And be honest with your stakeholders that you are building on evolving ground — which does not mean do not build, it means build with the seams and the eval gates that let you absorb change without a rewrite.
A 5,000-seat logistics firm we advised this quarter did exactly the right thing here: they built their internal document assistant in AI Studio, took it to a genuine limited production for one department, and explicitly scoped it as a preview-based pilot with a re-evaluation checkpoint aligned to the platform's expected general availability. That is the posture that works — real users, real value, real telemetry, and a decision point built into the plan rather than a bet that nothing will move. They got the learning and the early value without pretending the ground under them was as solid as it will eventually be.
The organizational tell we look for is who owns the thing after launch. A demo has a builder; a production system has an on-call owner, a runbook, and a documented answer to "what do we do when it starts giving bad answers at scale." The logistics firm named that owner before the limited production opened, which sounds obvious and almost never happens — the default is that the enthusiast who built the demo is assumed to support it forever, which works until they take vacation or change teams and the system becomes an orphan nobody understands. Preview or GA, the ownership question is the one that decides whether you have built an asset or a future incident, and it is worth settling in the same meeting where you approve the launch rather than the one where you investigate the outage.
If you're facing this
The distance between a generative demo that lands in a steering committee and a system your on-call rotation will not resent inheriting is almost entirely the work above — flow discipline, an eval gate, deliberate model and capacity choices, and the platform controls that keep it safe and private. If you have a promising proof of concept and no clear idea what it takes to make it supportable, that is exactly the conversation we have most weeks right now. Reach out and we will map your path from demo to production.