DoCoreAI Developer Docs

Quick Start

DoCoreAI installs in the same Python environment as your application and starts intercepting all LLM SDK calls automatically. No changes to your existing code required.

# Standard
pip install docoreai

# If pip not in PATH (Windows/Python 3.13)
python -m pip install docoreai

docoreai config

# Start
docoreai start

# Stop (graceful shutdown)
Ctrl+C

# Create a venv per client
python -m venv hsbc\venv
python -m venv target\venv

# Install DoCoreAI into each venv
hsbc\venv\Scripts\pip install docoreai
target\venv\Scripts\pip install docoreai

# Start each client — cd into the client folder first
cd hsbc
..\hsbc\venv\Scripts\docoreai start

# In a second terminal
cd target
..\target\venv\Scripts\docoreai start

# Run client apps using their venv's Python directly
hsbc\venv\Scripts\python.exe hsbc\hsbc.py
target\venv\Scripts\python.exe target\target.py

# Dockerfile
FROM python:3.12-slim
WORKDIR /app

# Required — tells DoCoreAI where the environment lives
ENV DOCOREAI_ENV_PATH=/app

COPY requirements.txt .
RUN pip install -r requirements.txt
RUN pip install docoreai

COPY . .

# Start DoCoreAI then run your app
CMD ["sh", "-c", "docoreai start & python your_app.py"]

How It Works

DoCoreAI runs as a sidecar in the same Python environment as your application. It never sits in your call path. Your app calls the LLM directly and unchanged. DoCoreAI listens via SDK patch, extracts metadata locally, and sends only cost and token data to the cloud.

Budget Control

Set a daily spend limit and choose how DoCoreAI enforces it. Six enforcement modes available — from intelligent optimization to hard block.

# Budget configuration example
BUDGET_CONFIG = {
    'daily_budget': 100.00,
    'mode': 'smart_reduce',
    'use_soft_limits': True,
    'use_pacing_engine': True,
    'allow_user_override': False
}

Pacing Engine

The pacing engine spreads your daily budget evenly across 24 hours — like Google Ads pacing. It detects when you are spending faster than expected and applies graduated throttling automatically. No service interruptions. No manual intervention.

Soft Limits

Instead of hard-truncating responses, DoCoreAI injects a concise guidance message into the system prompt — gently guiding the LLM to shorter responses when budget is tight. Quality is preserved. Responses are never cut off mid-sentence.

SOFT_LIMIT_CONFIG = {
    'enabled': True,
    'buffer_pct': 20,
    'min_words': 50,
    'hard_cap_multiplier': 1.2,
    'use_when': {
        'budget_tight': True,
        'high_cost_request': True
    }
}

Governance & PII Detection

DoCoreAI performs governance checks on every request before the LLM call is made. PII detection runs at the edge — inside your environment, before any data leaves your network.

• PII detection — scans for names, emails, phone numbers, credit cards before API call
• Risky prompt detection — flags or blocks prompts matching governance rules
• Block vs. warn mode — configurable per governance rule
• Audit trail — all governance events logged locally, no sensitive content ever sent to cloud

Prediction Engine

The prediction engine uses a LightGBM model to estimate how many tokens each response will actually need — replacing the wasteful default ceiling with a precise prediction.

Auto-Retraining

When prediction accuracy degrades, DoCoreAI automatically detects drift, retrains the LightGBM model on recent telemetry, runs an A/B test against the previous champion, and promotes the better model. No action required from your team.

Supported Providers

DoCoreAI detects and wraps all active SDKs automatically at startup. No provider-specific configuration required.

CLI Reference

All DoCoreAI operations are available via the command line.

Configuration Reference

All configuration lives in phase2_config.py. Key settings grouped by feature area.

# phase2_config.py — key settings

# Budget
BUDGET_CONFIG = {
    'daily_budget': 100.00,
    'mode': 'smart_reduce',
    'use_soft_limits': True,
    'use_pacing_engine': True
}

# Pacing
PACING_ENGINE_CONFIG = {
    'enabled': True,
    'strategy': 'adaptive',
    'throttle_threshold_pct': 20,
    'aggressive_threshold_pct': 40
}

# Auto-Retrain
AUTO_RETRAIN_CONFIG = {
    'auto_trigger': True,
    'cooldown_hours': 24,
    'max_retrains_per_week': 2,
    'min_mae_increase_pct': 20.0
}

# A/B Testing
AB_TESTING_CONFIG = {
    'min_test_duration_hours': 24,
    'challenger_traffic_pct': 0.10,
    'auto_promote': True,
    'auto_rollback': True
}

Troubleshooting

Common issues and their solutions. Click any item to expand.

FAQ