Where to Start with AI Governance

AI is already being used across teams, tools, and workflows. Waiting to fully understand all of it before acting feels responsible. But in practice, it leads to delay rather than better governance.

In a previous post, we explored why many AI governance decisions are effectively made early, often before organizations realize they’ve made them at all. 

This piece focuses on the practical follow-up: how to start.

Step 1: Don’t start with tools or policies

A common instinct is to begin with lists: which AI tools are approved, which are blocked, and what policy applies to each.

This approach rarely holds up.

AI capabilities are added continuously to tools teams already use. New copilots appear without new deployments. Tool-based governance quickly becomes reactive.

Starting with policy documents has similar limits. Policies written before understanding how AI is actually used tend to be ignored, worked around, or endlessly revised.

Governance that starts too far from real usage doesn’t stick.

Step 2: Define your first exposure boundary

Another assumption that slows teams down is the belief that governance must begin with a complete inventory of AI usage.

That inventory never stabilizes.

AI usage is fluid and constantly evolving. Waiting for a full assessment often creates a loop: governance is delayed in order to understand usage, but usage can’t be fully understood without governance in place.

A more effective starting point is to define where AI usage would matter most if it went wrong, and begin there.

For most organizations, these exposure boundaries are already clear:

• AI interacting with customer or personal data
  
• AI handling source code or proprietary intellectual property
  
• AI used in financial, legal, or strategic workflows
  
• AI influencing or taking automated actions in production systems
  

You don’t need to govern all AI at once.
You need to govern AI when it crosses these boundaries.

Step 3: Start with the right group, not the whole company

“Start small” doesn’t mean starting randomly.

The most effective place to begin is the function where:

• AI usage is already frequent
  
• sensitive data is involved
  
• workflows move fast enough that informal usage exists

In many organizations, that’s one of the following:

• Developers, where AI is used daily and touches source code and automation
  
• HR or People Ops, where personal data and internal policies are involved
  
• Legal or Finance, where sensitive documents and external accountability intersect

These are examples, not prescriptions. The right starting group depends on where AI is already creating exposure today.

What matters is choosing one area where governance is clearly justified, and starting there.

Step 4: Define acceptable AI use before defaults form

Once you’ve defined an exposure boundary and a starting group, the next step is to clarify how AI is allowed to be used within that scope.

This is not about anticipating every behavior or defining static rules. AI usage is inherently variable, and governance that relies on fixed patterns breaks down quickly.

Instead, effective early guardrails focus on intent and boundaries:

• When AI can be used for assistance versus automation
  
• When AI output can inform decisions versus trigger actions
  
• When human review is required before AI-generated results are acted on
  

These guardrails don’t attempt to predict outcomes. They define acceptable use.

Putting these expectations in place early gives teams clarity and reduces friction. Waiting to define them turns governance into negotiation after the fact.

Starting small doesn’t mean being permissive.
It means setting direction before defaults harden.

Step 5: Assign ownership and account for AI you don’t control

AI governance spans security, IT, legal, compliance, and the business. Without clear ownership, progress slows.

That doesn’t mean one team owns everything. It means one function is accountable for coordination, prioritization, and consistency.

In many organizations, security leaders are already being asked to explain AI risk and provide assurance. Formalizing that role early prevents governance from becoming fragmented or reactive.

At the same time, governance can’t stop at AI you directly operate. AI is increasingly embedded inside SaaS platforms and third-party systems, where data flows into models you don’t control.

From an accountability perspective, where the AI runs doesn’t change responsibility. Starting with internal usage alone creates blind spots immediately.

Closing thought

AI governance doesn’t begin with perfect knowledge.
It begins with choosing the right place to focus.

If you want to explore why timing matters so much in these decisions, and why some of them are difficult to revisit later, you can read our earlier post on the AI governance decisions organizations only get to make once.