Azure OpenAI Service, Microsoft's enterprise on-ramp for generative AI, went generally available on January 16 — yesterday, as we write this. It is a small, procedural-sounding line in a Microsoft blog post, but it lands in the middle of the loudest seven weeks our practice has seen since the pandemic sent everyone home overnight. ChatGPT crossed a million users five days after its November 30 launch, and since the new year started we have taken a call almost every week from a client who wants "an AI strategy" before the next board meeting. Scratch the surface and what most of them actually want is narrower: permission to stop worrying, and a plausible answer for the director who asked why competitors are already "doing something with AI."
We are not dismissing the trigger. Generative AI adoption inside a Microsoft-centric enterprise is a real operational question, not a fad to wait out — hybrid identity and cloud email both started as "someone else's fad" once, too. But a strategy deck is the wrong artifact for this particular month. What these engagements actually need in the next thirty days is triage: separate the manufactured urgency from the real exposure, get one sanctioned option in front of a small group before more employees route around IT on their own, and avoid locking into a platform before anyone has run an actual pilot. This note is what that thirty days looks like this week, with the tools that exist today — not the ones everyone assumes are coming.
Panic, prohibition, or pilot
Three responses keep showing up in these conversations, and only one of them survives contact with a real estate.
The panic response treats generative AI like a fire drill. A steering committee forms, a vendor demo gets booked, and someone is told to have "a position" inside two weeks. It produces slide decks and rarely produces an owner. When we ask who is accountable for logging, exceptions, and the eventual bill, the room goes quiet.
The prohibition response blocks the obvious domain at the proxy and calls the problem closed. It fails for a boring reason: most laptops leave the building every night, most phones were never on the corporate network to begin with, and a browser tab on a personal phone is one copy-paste away from whatever was open on the work laptop a minute earlier. Blocking chat.openai.com at the office firewall stops the fraction of usage that was already the easiest to see. It does nothing about the fraction you cannot see, and it answers none of the business pressure that produced the panic response to begin with.
The pilot response is slower to describe in a memo and faster to actually finish: name what is already happening, stand up one sanctioned option, define exactly what a scoped group may try and for how long, and measure it against numbers agreed before the pilot starts. It is a less exciting update for a board meeting. It is the only one of the three we would still defend in an audit a year from now.
Where this actually leaks
The risk driving most of these calls is not exotic. It is an employee with a legitimate task, a deadline, and a browser tab open to a tool that writes competent English faster than they do.
At a regional bank we advise, a commercial lending analyst pasted two paragraphs of a prospective borrower's financials into ChatGPT last week, looking for help tightening a credit memo before a committee meeting. Nobody had told her not to; nobody had gotten around to it yet. At a state agency we work with, a benefits caseworker used it to draft a letter that included a specific resident's case details, because the draft needed to sound right and the deadline was that afternoon. Neither person did anything malicious. Both moved material that belonged inside a compliance boundary into a tool that sits entirely outside every boundary the organization has ever drawn.
That is the actual shape of the risk, and it is worth being precise about why it is real. The public ChatGPT interface carries no enterprise agreement in January 2023. There is no admin console, no single sign-on tied to your Azure AD tenant, no audit log for IT to pull, and no contractual commitment about how a submitted conversation is retained or reviewed on the other end. You cannot verify, from the corporate side, whether someone pasted in a customer record. You can only ask afterward and hope the answer is honest.
None of this requires a villain. It requires exactly what every organization already has: capable people under deadline pressure, reaching for the fastest tool on the desktop. Treat that as the baseline, because the rest of this note is built on it.
The sanctioned path: Azure OpenAI Service in your own tenant
This is the part that changed yesterday. Azure OpenAI Service is the product of a Microsoft-OpenAI partnership that predates ChatGPT by three years, made available as an ordinary Azure resource inside your own subscription rather than a consumer product hosted somewhere else.
One expectation worth setting early: Azure OpenAI Service does not hand you ChatGPT itself. It hands you the GPT-3.5 family it was built from — text-davinci-003 today, plus the Codex models for code — and the pilot will not feel identical to the tab everyone already has open at home. Say that out loud in the kickoff meeting; it saves a week of "why doesn't it sound the same" tickets later.
What actually matters is everything wrapped around the model. Prompts and completions run inside your tenant's Azure boundary instead of a consumer vendor's. Microsoft's terms for the service are explicit that customer data is not used to train or improve the underlying models — a materially different promise than anything the free public chat product offers. Access to the resource is controlled through Azure AD like any other Azure service, so you get RBAC and a real identity story instead of a personal email address at sign-up. Networking can sit behind a virtual network with private endpoints rather than travel the open internet. And because it is an Azure resource, usage shows up in your existing logging and cost management tools, not in a shadow expense report someone finds in March.
None of that is free of friction, and we would rather flag the friction now than have a client discover it mid-pilot. Access still requires filing a Microsoft application describing the intended use case, even after GA — approval is not instant. Content filtering runs on every prompt and completion by default and cannot be switched off from the portal; loosening it for a legitimate business case is its own separate request, a form behind the form. Regional availability is short — a handful of regions, East US and West Europe among them — so a data-residency conversation needs to happen before the pilot, not after. And billing is per-token, which means a pilot with no usage guardrails can produce a surprising line item before anyone notices.
| Public chatbot (consumer) | Azure OpenAI Service (in-tenant) | |
|---|---|---|
| Data boundary | Vendor-hosted, outside your tenant | Your Azure subscription, your region |
| Used to train the model | Not something you control | Contractually excluded |
| Identity | Personal or work email, no SSO | Azure AD, RBAC on the resource |
| Network path | Public internet only | VNet integration, private endpoint |
| Visibility for IT/security | None available | Standard Azure logging and cost data |
Provisioning it, once access is approved, is routine:
az cognitiveservices account create `
--name "contoso-openai-eastus" `
--resource-group "rg-genai-pilot" `
--kind "OpenAI" `
--sku "S0" `
--location "eastus" `
--custom-domain "contoso-openai-eastus"
The resource is the easy part. Deciding who can call it, with what data, is the actual project.
DLP for prompts: teaching Purview about a text box
Blocking a domain at the firewall is a blunt instrument, but it is not useless as a stopgap while the sanctioned path gets built — it just needs a partner that can see more than a URL.
Microsoft Purview's endpoint data loss prevention policies already watch actions like copy-to-clipboard, printing, and uploads from a managed device, the same policy engine we have used for two years to stop source code and customer exports walking out through personal webmail. Pointing that machinery at a public AI chat interface is the same motion: define the site as a monitored or restricted destination, and let the existing sensitive-information-type classifiers judge whether what is being pasted looks like a credit card number, a national ID, or a block of source code, instead of trying to write a rule that understands prose.
We start every one of these policies in audit mode, the same discipline we apply to a brand-new Conditional Access policy. Two weeks of logging tells you which departments are already doing this, at what volume, and with what kind of content, before you block a single paste action and generate a help-desk queue full of confused, well-meaning employees. Microsoft Defender for Cloud Apps' discovery reporting helps here too: it will surface a generative AI domain in your traffic logs as soon as anyone visits it, cataloged or not, the same shadow-IT discovery motion we have run against Dropbox and unsanctioned file-sharing for years.
The point of this layer is not to replace the acceptable-use conversation below. It is to give that conversation evidence instead of assumptions, and to give security a way to learn when a policy is being ignored instead of finding out during an incident review.
An acceptable-use policy that survives contact with reality
Every client asks us for a policy document in the first meeting. Most of the documents we have seen elsewhere are a paragraph of legal caution filed somewhere in a handbook, read once and never again. That is not a policy. It is a liability disclaimer wearing a policy's clothes.
A policy that survives contact with reality does three things a disclaimer does not. First, it draws the line in terms an employee can apply without a lawyer in the room — plain data classes, not abstractions:
Can this text leave the tenant boundary?
Public information only -> public AI tools, no restriction
Internal, non-sensitive -> public AI tools, keep it generic
Confidential, regulated, or PII -> sanctioned in-tenant tool only
Source code or trade secrets -> sanctioned in-tenant tool only
Second, it offers the sanctioned alternative in the same breath as the restriction. A policy that only says no, with nothing to say yes to, guarantees the shadow usage continues — it just gets quieter about it. Every acceptable-use draft we write this month ships alongside a working Azure OpenAI Service pilot request form, so "not that tool" always comes paired with "try this one instead."
Third, it has a named owner and a review date, not just a publish date. We ask clients to name who approves exceptions, who reviews the DLP evidence monthly, and when the policy itself gets revisited — because the model landscape underneath this document is going to keep moving, and a policy that assumes today's tools are the permanent state will be wrong within the quarter.
Designing a first pilot that can graduate
The pilots worth running this quarter share a shape: narrow scope, a named sponsor, a small cohort, and three kinds of success criteria agreed before anyone opens a laptop.
One of our manufacturing clients — call it 6,000 seats, heavy in engineering documentation — is piloting an internal assistant grounded in its own maintenance manuals through Azure OpenAI Service, restricted to a cohort of field engineers small enough to fit around one conference table. No customer data goes near it; that is a written non-goal, not an assumption. Success is judged against three numbers the sponsor agreed to up front: a risk measure (zero confidential records logged in prompts during the pilot), a usefulness measure (time to a usable answer, judged by the engineers actually using it), and a cost measure (token spend per user per week, so nobody discovers the bill in April).
The scope decisions that matter most happen before the first prompt is ever typed. Four to six weeks, not indefinite. A cohort in the double digits, not a company-wide rollout with an opt-out box. An explicit list of what the pilot will not attempt yet — customer-facing use, regulated data, anything a compliance officer has not already reviewed. And a rollback that is genuinely one step: disable the app registration, revoke the key, done, with the disabling step written into the same change record.
If you're facing this
If generative AI has your organization caught between a board-level panic and a quiet prohibition, the way out is a sanctioned in-tenant path and a pilot small enough to judge honestly. We help Microsoft-centric estates stand up Azure OpenAI Service with real data boundaries, write acceptable-use guidance people will actually follow, and design first pilots with success criteria agreed before anyone opens a laptop. Bring us your data classes and your Azure AD tenant, and we will help you find the smallest defensible pilot first.