Access Control
Authentication, Authorization & Permission Management for LLM Applications
Why Access Control in LLM Apps?
Access control ensures that users can only access data, features, and AI capabilities appropriate to their role and permissions. In LLM applications, this is especially critical because AI systems often have access to sensitive data through RAG, tools, and integrations.
"The principle of least privilege should guide AI system design: grant only the minimum access necessary for the intended function. This applies to both users and the AI agents themselves."
Authentication
Verify user identity (Who are you?)
Authorization
Grant permissions (What can you do?)
Auditing
Track actions (What did you do?)
Access Control Models
Role-Based Access Control (RBAC)
Users are assigned roles (Admin, Editor, Viewer) and each role has predefined permissions. Simple to manage but can become complex with many roles.
Attribute-Based Access Control (ABAC)
Permissions based on attributes of user, resource, and environment. More flexible than RBAC for complex scenarios like "department X can access documents from department X only".
Document-Level Access Control
Permissions set on individual documents or data sources. Essential for RAG systems where different users should only retrieve from authorized document collections.
Multi-Tenant Isolation
Complete data isolation between different organizations or tenants sharing the same infrastructure. Critical for SaaS LLM applications.
LLM-Specific Access Control
| Layer | What to Control | Example |
|---|---|---|
| Model Access | Which models users can interact with | Free tier → GPT-3.5 only; Pro → GPT-4 |
| Feature Access | Which capabilities are available | Basic → Chat only; Advanced → + Code execution |
| RAG Documents | Which knowledge bases can be queried | HR team → HR docs; Sales → Sales docs |
| Tool Access | Which tools/functions the AI can call | Viewer → Read-only APIs; Admin → All APIs |
| Usage Limits | Rate limits, token quotas per user/tier | Free → 1000 tokens/day; Pro → Unlimited |
| Data Visibility | What data appears in responses | External users see public info only |
RAG Access Control Pattern
# Document-level access control in RAG
from typing import List, Dict, Optional
class SecureRAGRetriever:
def __init__(self, vector_store, permission_service):
self.vector_store = vector_store
self.permissions = permission_service
def retrieve(
self,
query: str,
user_id: str,
top_k: int = 5
) -> List[Dict]:
"""Retrieve documents respecting user permissions"""
# Get user's accessible document IDs and roles
user_permissions = self.permissions.get_user_access(user_id)
allowed_doc_ids = user_permissions["document_ids"]
user_roles = user_permissions["roles"]
user_clearance = user_permissions["clearance_level"]
# Build metadata filter for vector search
metadata_filter = {
"$or": [
{"document_id": {"$in": allowed_doc_ids}},
{"allowed_roles": {"$in": user_roles}},
{"visibility": "public"}
],
"classification_level": {"$lte": user_clearance}
}
# Search with access filter
results = self.vector_store.similarity_search(
query=query,
k=top_k,
filter=metadata_filter
)
# Log access for audit trail
self._log_access(user_id, query, [r.id for r in results])
return results
Agent Tool Access Control
# Role-based tool access for AI agents
from typing import List, Callable
from functools import wraps
# Define tool permissions per role
TOOL_PERMISSIONS = {
"viewer": ["search_docs", "get_info"],
"editor": ["search_docs", "get_info", "update_record"],
"admin": ["search_docs", "get_info", "update_record", "delete_record", "manage_users"]
}
def require_permission(tool_name: str):
"""Decorator to enforce tool access based on user role"""
def decorator(func: Callable):
@wraps(func)
def wrapper(*args, user_context: dict, **kwargs):
user_role = user_context.get("role", "viewer")
allowed_tools = TOOL_PERMISSIONS.get(user_role, [])
if tool_name not in allowed_tools:
raise PermissionError(
f"User role '{user_role}' cannot access tool '{tool_name}'"
)
return func(*args, **kwargs)
return wrapper
return decorator
# Example tool with permission check
@require_permission("delete_record")
def delete_record(record_id: str) -> dict:
# Only admins can execute this
return {"deleted": record_id}
Authentication Methods
API Keys
Simple key-based auth for programmatic access.
- Simple to implement
- Good for server-to-server
- No user context
OAuth 2.0 / OIDC
Industry standard for user authentication.
- SSO support
- Token-based, scopes
- User identity context
JWT Tokens
Stateless tokens with embedded claims.
- Stateless / scalable
- Contains user info
- Can't revoke instantly
MFA / Passkeys
Multi-factor for sensitive operations.
- Highest security
- Phishing resistant
- User friction
Best Practices
Do This
- Apply least privilege principle
- Filter RAG results by user permissions
- Limit AI tool access based on role
- Implement tenant isolation for SaaS
- Audit all access attempts
- Require MFA for sensitive actions
Avoid This
- Trusting client-side permission checks
- Sharing vector stores without filtering
- Giving AI agents admin credentials
- Hardcoding permissions in prompts
- Ignoring session expiration
- Logging sensitive tokens/credentials