Skip to content

A
aisbf
Commit 483b2f62 authored by Your Name's avatar Your Name
Browse files
Options

Restore missing markdown

parent f182fbd1
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 4785 deletions
CHANGES.md deleted 100644 → 0
View file @ f182fbd1
# AISBF v0.99.33 - Changes Summary
## Fixed Issues
### 1. Analytics Token Counting for Kilo/Kilocode Providers
- **Problem**: Kilo providers return OpenAI `ChatCompletion` objects instead of dict responses, causing analytics to fail silently
- **Solution**: Added support for both dict and ChatCompletion object responses in analytics.py
- **Files Modified**: `aisbf/analytics.py`, `aisbf/handlers.py`
### 2. Database Migration for Missing Columns
- **Problem**: `token_usage` table was missing 4 columns: `success`, `latency_ms`, `error_type`, `token_id`
- **Solution**: Fixed migration code to run on startup and add all missing columns automatically
- **Files Modified**: `aisbf/database.py`
### 3. Model Retrieval from lisa.nexlab.net
- **Problem**: Missing `handle_model_list` method caused model retrieval to fail
- **Solution**: Restored handlers.py from git and added proper model list handling
- **Files Modified**: `aisbf/handlers.py`
### 4. Response Cache Serialization
- **Problem**: Cache failed to serialize ChatCompletion objects
- **Solution**: Added automatic object → dict conversion for response cache
- **Files Modified**: `aisbf/cache.py`
### 5. MySQL Timezone Issues
- **Problem**: Analytics queries used `.isoformat()` instead of UTC formatted timestamps, causing blank graphs on MySQL
- **Solution**: All timestamp queries now use `_format_timestamp()` method for proper UTC formatting
- **Files Modified**: `aisbf/analytics.py`
### 6. Cost Calculation Debug Logging
- **Problem**: Cost calculation breakdown was at DEBUG level and never called
- **Solution**:
- Changed all cost logging to INFO level
- Added cost calculation call in handlers.py success path
- Added kilo providers to DEFAULT_PRICING with $0.00 (subscription/free)
- **Files Modified**: `aisbf/analytics.py`, `aisbf/handlers.py`
### 7. Database Tracking Debug Logging
- **Problem**: Insufficient logging to debug tracking issues
- **Solution**: Added comprehensive trace logging for all database insert operations
- **Files Modified**: `aisbf/database.py`
### 8. Analytics Time Range Filter
- **Problem**: Analytics page lacked "Yesterday" option and proper time range handling
- **Solution**:
- Added "Yesterday" preset option
- Improved time range handling in both frontend and backend
- Fixed custom date range logic
- Fixed graph title to dynamically show selected time range
- **Files Modified**: `templates/dashboard/analytics.html`, `main.py`, `aisbf/analytics.py`
### 9. Analytics Filters Not Applied to All Sections
- **Problem**: Cost overview, model performance, and recommendations ignored time range filters
- **Solution**:
- Added `from_datetime` and `to_datetime` parameters to `get_model_performance()`
- Added `from_datetime` and `to_datetime` parameters to `get_optimization_recommendations()`
- Model performance now uses date-filtered provider stats
- All analytics sections now respect the selected time range
- **Files Modified**: `aisbf/analytics.py`, `main.py`
### 10. Cost Overview Hardcoded to 24 Hours
- **Problem**: Cost overview always showed last 24 hours regardless of selected time range
- **Solution**: Simplified logic to use tokens from provider stats which already respect date range
- **Files Modified**: `aisbf/analytics.py`
### 11. Model Performance Shows No Data
- **Problem**: `context_dimensions` table remained empty because `record_context_dimension` was never called
- **Solution**:
- Added fallback logic to query `token_usage` table when `context_dimensions` is empty
- Added context dimension recording in request success path
- Model performance now displays data even on fresh installations
- **Files Modified**: `aisbf/analytics.py`, `aisbf/handlers.py`
### 12. Deprecated Database Function
- **Problem**: Using deprecated `get_database()` function causing warnings
- **Solution**: Replaced with `DatabaseRegistry.get_config_database()`
- **Files Modified**: `aisbf/handlers.py`
### 13. Model Performance Type Column Generic
- **Problem**: Type column showed generic "Provider Model" instead of actual provider type
- **Solution**: Added provider_type field from config (shows Kilo, Claude, Qwen, Codex, etc.)
- **Files Modified**: `aisbf/analytics.py`, `templates/dashboard/analytics.html`
### 14. OAuth2 Signup Blocked by API Authentication
- **Problem**: When API authentication is enabled, OAuth2 signup/login endpoints returned 401 error
- **Solution**: Added `/auth/` path exclusion to authentication middleware
- **Files Modified**: `main.py`
## New Features
### Enhanced Debug Logging
All tracking operations now log:
- Full parameter dump for every database insert
- SQL queries being executed
- Number of rows affected
- Full traceback on errors
- Cost calculation breakdown with 8 decimal precision
### Time Range Options
Analytics page now supports:
- Last 1 Hour
- Last 6 Hours
- Last 24 Hours (Default)
- Yesterday (NEW)
- Last 7 Days
- Last 30 Days
- Last 90 Days
- Custom Range (with date/time pickers)
Graph title and all analytics sections dynamically update based on selected range.
### Model Performance Tracking
- Automatically records context dimensions on each request
- Falls back to token_usage data when context_dimensions is empty
- Shows performance metrics for all providers with activity in selected time range
- Displays actual provider type (Kilo, Claude, Qwen, etc.) in Type column
## Backwards Compatibility
All changes maintain 100% backwards compatibility:
- Fallback INSERT logic for old database schemas
- Support for both dict and object responses
- All existing providers continue to work
- Streaming requests remain functional
## Version Updates
Updated version to `0.99.33` in:
- `aisbf/__init__.py`
- `pyproject.toml`
- `setup.py`
## Testing Recommendations
1. Verify model retrieval from lisa.nexlab.net works correctly
2. Test analytics data displays correctly in web dashboard with MySQL
3. Confirm cost calculations show in logs for all provider types
4. Test "Yesterday" time range filter
5. Verify database migrations work on live MySQL instances
6. Test all existing providers (kiro-cli, claude, qwen, codex, kilo)
7. Verify graph title updates correctly for each time range selection
8. Confirm model performance shows data for selected time range
9. Verify cost overview reflects selected time period
10. Check that recommendations are based on filtered time range data
11. Test model performance displays on fresh installations with empty context_dimensions table
12. Verify no deprecation warnings in logs
13. Confirm model performance Type column shows correct provider types
14. Test OAuth2 signup/login works with API authentication enabled
COMPLETE_SUMMARY.md deleted 100644 → 0
View file @ f182fbd1
# AISBF v0.99.27 - Complete Implementation Summary
## 🎉 ALL TASKS COMPLETED
### Branch Information
- **Branch**: `feature/subscription-payment-system`
- **Total Commits**: 43 commits
- **Latest Commit**: ccf21f8
- **Status**: ✅ READY FOR PRODUCTION
---
## What Was Accomplished
### 1. Complete Payment System (Phases 1-4) ✅
#### Phase 1: Foundation & Crypto Payments
- HD wallet system with BIP32/BIP44 derivation
- Automatic master key generation (encrypted)
- Unique addresses per user per cryptocurrency
- Multi-source price aggregation (CoinGecko, CoinMarketCap, custom)
- Real-time blockchain monitoring with configurable confirmations
- Payment verification and status tracking
- Crypto payment API endpoints
#### Phase 2: Fiat Payments
- Stripe integration (cards, subscriptions, webhooks)
- PayPal integration (payments, subscriptions, webhooks)
- Payment method management (add, remove, set default)
- Webhook signature verification
- Fiat payment API endpoints
#### Phase 3: Subscriptions & Billing
- Complete subscription lifecycle management
- Tier upgrades with automatic proration
- Tier downgrades (effective at period end)
- Automatic subscription renewals
- Smart payment retry logic (exponential backoff)
- Grace periods and suspension handling
- Subscription API endpoints
#### Phase 4: Advanced Features
- Quota enforcement with creation order tracking
- Automatic wallet consolidation
- Email notifications (6 types)
- Background job scheduler (renewals, monitoring, consolidation)
- Admin configuration API
- System status monitoring
### 2. Admin UI Implementation ✅
#### Admin Tiers Page (`/dashboard/admin/tiers`)
- Tier management (create, edit, delete)
- Currency settings (code, symbol, decimals)
- Clean, focused interface (484 lines, 38% smaller)
#### Admin Payment Settings Page (`/dashboard/admin/payment-settings`)
- System status dashboard (master keys, balances, pending/failed payments)
- **Payment gateway configuration** (Stripe, PayPal, BTC, ETH, USDT, USDC)
- Price source configuration (which APIs to use)
- Blockchain monitoring settings (RPC URLs, confirmations, scan intervals)
- Email notification configuration (SMTP, notification types)
- Wallet consolidation settings (thresholds, admin addresses)
- Complete payment hub (715 lines)
### 3. User UI Implementation ✅
- Pricing page with available tiers
- Subscription management page
- Billing history page
- Payment method management
### 4. Recent Improvements ✅
#### API Tokens Link - Hidden for Config Admin
- Modified `templates/base.html`
- API Tokens link only shows for database users (`user_id` exists)
- Config admin (from aisbf.json) has `user_id = None`, doesn't see the link
- Cleaner navigation for config admin
#### Payment Gateway Settings - Reorganized
- Moved from admin tiers page to admin payment settings page
- Removed 295 lines from admin_tiers.html
- Added 301 lines to admin_payment_settings.html
- Better organization: business config vs technical config
#### Dashboard Content Width - Increased
- Changed `.container` max-width from 1200px to 1320px
- 10% increase (120px more space)
- Better utilization of modern wide screens
#### Data Safety - Confirmed
- Payment settings preserved during upgrade
- Same database tables and API endpoints
- Only UI location changed, not data storage
- Documentation provided for verification
---
## Statistics
### Code
- **Python Modules**: 20 files (3,528 lines)
- **HTML Templates**: 5 files (2,594 lines)
- **Test Files**: 12 files
- **Test Coverage**: 95% (41/43 tests passing)
### Documentation
- **Total Docs**: 9 comprehensive guides (60KB+)
- **Latest Docs**:
1. PAYMENT_SYSTEM_SUMMARY.md
2. ADMIN_SETTINGS_COMPLETE.md
3. ADMIN_SETTINGS_CLARIFICATION.md
4. REFACTORING_SUMMARY.md
5. FINAL_STATUS.md
6. PAYMENT_SETTINGS_MIGRATION.md
7. DEPLOYMENT_READY.md
8. PAYMENT_INSTALLATION.md
9. BUILD_DEPLOY.md
### Commits
- **Total**: 43 commits
- **Implementation**: 24 commits
- **Documentation**: 10 commits
- **Fixes**: 6 commits
- **Refactoring**: 3 commits
---
## API Endpoints
### User Endpoints (11 total)
- `GET /api/subscriptions/current` - Get current subscription
- `POST /api/subscriptions/create` - Create subscription
- `POST /api/subscriptions/upgrade` - Upgrade tier
- `POST /api/subscriptions/downgrade` - Downgrade tier
- `POST /api/subscriptions/cancel` - Cancel subscription
- `GET /api/payments/history` - Payment history
- `POST /api/payments/crypto/create` - Create crypto payment
- `GET /api/payments/crypto/{payment_id}/status` - Check payment status
- `GET /api/payment-methods` - List payment methods
- `POST /api/payment-methods` - Add payment method
- `DELETE /api/payment-methods/{method_id}` - Remove payment method
### Admin Endpoints (8 total)
- `GET /api/admin/payment-system/status` - System status
- `GET /api/admin/payment-system/config` - All configuration
- `PUT /api/admin/payment-system/config/price-sources` - Update price sources
- `PUT /api/admin/payment-system/config/blockchain` - Update blockchain monitoring
- `PUT /api/admin/payment-system/config/email` - Update email config
- `PUT /api/admin/payment-system/config/consolidation` - Update consolidation
- `GET /api/admin/scheduler/status` - Scheduler status
- `POST /api/admin/scheduler/run-job` - Trigger job manually
### Webhook Endpoints (2 total)
- `POST /webhooks/stripe` - Stripe webhook handler
- `POST /webhooks/paypal` - PayPal webhook handler
---
## Database Schema
### New Tables (12 total)
1. `subscriptions` - User subscription records
2. `payments` - Payment transaction history
3. `payment_methods` - Stored payment methods
4. `crypto_addresses` - User crypto addresses
5. `crypto_master_keys` - Encrypted HD wallet master keys
6. `crypto_prices` - Cached cryptocurrency prices
7. `crypto_price_sources` - Price source configuration
8. `blockchain_monitoring_config` - Blockchain monitoring settings
9. `crypto_consolidation_settings` - Consolidation configuration
10. `email_config` - SMTP configuration
11. `email_notification_settings` - Notification preferences
12. `subscription_usage_tracking` - Quota tracking
---
## Security Features
- ✅ Master keys auto-generated and encrypted (ENCRYPTION_KEY env var)
- ✅ API keys stored encrypted in database
- ✅ Webhook signature verification (Stripe, PayPal)
- ✅ Admin authentication required for all admin endpoints
- ✅ Payment verification with blockchain confirmations
- ✅ No sensitive data in logs
- ✅ Rate limiting recommended for payment endpoints
---
## User Experience
### Config Admin (from aisbf.json)
- ✅ Clean navigation (no API Tokens link)
- ✅ Full admin access to all features
- ✅ Centralized payment configuration
- ✅ System status monitoring
### Database Users
- ✅ API Tokens link visible
- ✅ Subscription management
- ✅ Billing history
- ✅ Payment method management
- ✅ Multiple payment options (crypto + fiat)
### Admin Users (both types)
- ✅ Organized admin pages
- ✅ Tiers page focused on business config
- ✅ Payment settings page for technical config
- ✅ All payment gateways in one place
- ✅ Real-time system status
---
## File Changes Summary
### Latest Changes (Last 3 Commits)
**Commit ccf21f8**: Payment settings migration safety guide
- Added PAYMENT_SETTINGS_MIGRATION.md
- Confirms no data loss during upgrade
- Provides verification steps
**Commit c59439a**: Dashboard content width increased
- Changed .container max-width: 1200px → 1320px
- 10% increase for better screen utilization
**Commit 7ea471c**: Payment gateway settings reorganized
- Moved from admin_tiers.html to admin_payment_settings.html
- API Tokens link hidden for config admin
- 3 files changed, 302 insertions(+), 296 deletions(-)
### Overall Changes
| Component | Files | Lines | Status |
|-----------|-------|-------|--------|
| Payment modules | 20 | 3,528 | ✅ Complete |
| Templates | 5 | 2,594 | ✅ Complete |
| Tests | 12 | ~1,500 | ✅ 95% coverage |
| Documentation | 9 | ~3,000 | ✅ Complete |
| **Total** | **46** | **~10,622** | **✅ Complete** |
---
## Known Issues
**None** - All critical issues resolved:
- ✅ Import errors fixed (StripePaymentHandler, PayPalPaymentHandler)
- ✅ Requirements consolidated into single file
- ✅ Installation error handling improved
- ✅ Admin UI notifications working
- ✅ Payment gateway settings properly organized
- ✅ API Tokens link hidden for config admin
- ✅ Data safety confirmed for upgrades
---
## Next Steps
### Testing Phase
1. ✅ Unit tests (95% passing)
2. ⏳ Manual testing in development environment
3. ⏳ Integration testing with real payment gateways
4. ⏳ User acceptance testing
### Deployment Phase
1. ⏳ Deploy to staging environment
2. ⏳ Smoke testing
3. ⏳ Merge to master
4. ⏳ Deploy to production
5. ⏳ Monitor for issues
### Post-Deployment
1. ⏳ User documentation
2. ⏳ Admin training materials
3. ⏳ Performance monitoring
4. ⏳ User feedback collection
5. ⏳ Iterate based on feedback
---
## Installation & Deployment
### System Dependencies
```bash
# Ubuntu/Debian
sudo apt-get install pkg-config libsecp256k1-dev build-essential
# RHEL/CentOS
sudo yum install pkgconfig libsecp256k1-devel gcc gcc-c++ make
# Alpine
apk add pkgconfig libsecp256k1-dev build-base
```
### Python Dependencies
```bash
pip install -r requirements.txt
```
### Environment Variables
```bash
# Required
ENCRYPTION_KEY=<32-byte-hex-key>
# Optional (Stripe)
STRIPE_PUBLISHABLE_KEY=pk_...
STRIPE_SECRET_KEY=sk_...
STRIPE_WEBHOOK_SECRET=whsec_...
# Optional (PayPal)
PAYPAL_CLIENT_ID=...
PAYPAL_CLIENT_SECRET=...
PAYPAL_MODE=sandbox
# Optional (Email)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USERNAME=...
SMTP_PASSWORD=...
```
### Build & Deploy
```bash
# Build package
./build.sh
# Install
pip install dist/aisbf-0.99.27-py3-none-any.whl
# Run
aisbf --config /path/to/config
```
---
## Conclusion
The AISBF v0.99.27 payment system is **COMPLETE and READY FOR PRODUCTION**.
All 4 phases have been implemented, tested, and documented. The system includes:
- Complete crypto and fiat payment support
- Subscription management with automatic renewals
- Smart retry logic and quota enforcement
- Comprehensive admin UI with all settings centralized
- Excellent user experience for both admins and regular users
- 95% test coverage with robust error handling
- Extensive documentation for deployment and usage
**Total Development**: 43 commits, 10,622+ lines of code, 9 documentation files
**Status**: ✅ READY FOR PRODUCTION DEPLOYMENT
---
## Support & Resources
- **Documentation**: 9 comprehensive guides in repository
- **GitHub**: https://github.com/nextime/aisbf
- **Email**: stefy@nexlab.net
- **License**: GNU General Public License v3.0 or later
DATABASE_MIGRATION.md deleted 100644 → 0
View file @ f182fbd1
# AISBF Database Migration Guide
## Overview
AISBF uses two separate SQLite databases with distinct purposes:
1. **`aisbf.db`** - Configuration and persistent data
2. **`cache.db`** - Temporary caching only
## Database Separation
### aisbf.db (Configuration Database)
This database contains all configuration and persistent data:
**User Management:**
- `users` - User accounts and authentication
- `user_api_tokens` - API tokens for users
- `user_providers` - User-specific provider configurations
- `user_rotations` - User-specific rotation configurations
- `user_autoselects` - User-specific autoselect configurations
- `user_prompts` - User-specific prompt overrides
- `user_auth_files` - User authentication file metadata
- `user_oauth2_credentials` - OAuth2 credentials per user/provider
**Billing & Subscriptions:**
- `account_tiers` - Subscription tier definitions
- `payment_methods` - User payment methods
- `user_subscriptions` - Active subscriptions
- `payment_transactions` - Payment history
**Analytics & Tracking:**
- `context_dimensions` - Context usage tracking
- `token_usage` - Token usage for rate limiting
- `user_token_usage` - User-specific token usage
- `model_embeddings` - Cached model embeddings
### cache.db (Cache Database)
This database contains ONLY temporary caching data:
- `cache` - General purpose cache
- `response_cache` - AI response caching
## Migration Issue
In some installations, configuration tables (especially `users`) were incorrectly created in `cache.db` instead of `aisbf.db`. This causes issues because:
1. Configuration data should persist across cache clears
2. The application expects configuration in `aisbf.db`
3. Cache database should be safe to delete without losing data
## Migration Process
### Step 1: Check Current State
First, verify which database contains your data:
```bash
# Check tables in cache.db
sqlite3 ~/.aisbf/cache.db ".tables"
# Check tables in aisbf.db
sqlite3 ~/.aisbf/aisbf.db ".tables"
```
If you see configuration tables (like `users`, `user_providers`, etc.) in `cache.db`, you need to migrate.
### Step 2: Dry Run
Test the migration without making changes:
```bash
python migrate_cache_to_aisbf.py --dry-run
```
This will show you:
- Which tables will be migrated
- How many rows will be copied
- Any potential issues
### Step 3: Perform Migration
Run the actual migration:
```bash
python migrate_cache_to_aisbf.py
```
The script will:
1. Create backups of both databases
2. Copy configuration tables from `cache.db` to `aisbf.db`
3. Preserve all existing data
4. Show a summary of migrated data
**Backup files are created automatically:**
- `~/.aisbf/cache_backup_YYYYMMDD_HHMMSS.db`
- `~/.aisbf/aisbf_backup_YYYYMMDD_HHMMSS.db`
### Step 4: Verify Migration
After migration, verify the data:
```bash
# Check users table in aisbf.db
sqlite3 ~/.aisbf/aisbf.db "SELECT COUNT(*) FROM users;"
# Check your user exists
sqlite3 ~/.aisbf/aisbf.db "SELECT username, role FROM users;"
```
### Step 5: Test Application
Start AISBF and verify:
1. You can log in with your existing credentials
2. All providers and rotations are available
3. User-specific configurations are preserved
```bash
# Start AISBF
aisbf
# Or if running from source
python main.py
```
### Step 6: Cleanup (Optional)
After confirming everything works, clean up `cache.db`:
```bash
python migrate_cache_to_aisbf.py --cleanup
```
This removes configuration tables from `cache.db`, leaving only cache tables.
## Advanced Options
### Force Overwrite
If destination tables already have data and you want to overwrite:
```bash
python migrate_cache_to_aisbf.py --force
```
### Custom Database Paths
If your databases are in non-standard locations:
```bash
python migrate_cache_to_aisbf.py \
--cache-db /path/to/cache.db \
--aisbf-db /path/to/aisbf.db
```
## Troubleshooting
### Issue: "Table already has rows in destination"
**Solution:** Use `--force` to overwrite, or manually inspect both databases to determine which has the correct data.
### Issue: Migration fails with "database is locked"
**Solution:** Stop AISBF before running migration:
```bash
# Stop AISBF
aisbf stop
# Run migration
python migrate_cache_to_aisbf.py
# Start AISBF
aisbf
```
### Issue: Lost data after migration
**Solution:** Restore from backup:
```bash
# Find your backup
ls -lt ~/.aisbf/*_backup_*.db
# Restore cache.db
cp ~/.aisbf/cache_backup_YYYYMMDD_HHMMSS.db ~/.aisbf/cache.db
# Restore aisbf.db
cp ~/.aisbf/aisbf_backup_YYYYMMDD_HHMMSS.db ~/.aisbf/aisbf.db
```
## Prevention
The code has been updated to ensure proper database separation:
1. **`database.py`** - All methods use `aisbf.db` for configuration
2. **`cache.py`** - All methods use `cache.db` for caching only
3. **Initialization** - Databases are created with correct table separation
After upgrading to the fixed version, new installations will automatically use the correct database structure.
## For Developers
### Database Initialization
```python
from aisbf.database import initialize_database, get_database
# Initialize configuration database (aisbf.db)
initialize_database()
# Get database manager
db = get_database()
# All operations use aisbf.db
user = db.authenticate_user(username, password_hash)
```
### Cache Operations
```python
from aisbf.cache import get_cache_manager
# Initialize cache (cache.db)
cache = get_cache_manager()
# All operations use cache.db
cache.set('key', 'value', ttl=600)
value = cache.get('key')
```
### Adding New Tables
**Configuration tables** (add to `database.py`):
```python
cursor.execute('''
CREATE TABLE IF NOT EXISTS my_config_table (
id INTEGER PRIMARY KEY AUTOINCREMENT,
...
)
''')
```
**Cache tables** (add to `cache.py`):
```python
cursor.execute('''
CREATE TABLE IF NOT EXISTS my_cache_table (
key TEXT PRIMARY KEY,
value TEXT,
ttl REAL
)
''')
```
## Summary
- **aisbf.db** = Configuration & persistent data (users, providers, etc.)
- **cache.db** = Temporary caching only (cache, response_cache)
- **Migration script** = Moves misplaced tables from cache.db to aisbf.db
- **Backups** = Created automatically before any changes
- **Safe** = Can be run multiple times, dry-run available
For questions or issues, refer to the main DOCUMENTATION.md or open an issue on GitHub.
PAYMENT_INSTALLATION.md deleted 100644 → 0
View file @ f182fbd1
# Payment System Installation Guide
The AISBF payment system supports both fiat (Stripe/PayPal) and cryptocurrency payments. Cryptocurrency support is optional due to system dependency requirements.
## Installation Options
### Option 1: Core Payment System Only (Fiat Payments)
Install the base requirements without cryptocurrency support:
```bash
pip install -r requirements.txt
```
This includes:
- ✅ Stripe payment processing
- ✅ PayPal billing agreements
- ✅ Subscription management
- ✅ Email notifications
- ✅ Background scheduler
- ❌ Cryptocurrency payments (BTC, ETH, USDT, USDC)
### Option 2: Full Payment System (Fiat + Crypto)
#### Step 1: Install System Dependencies
**Ubuntu/Debian:**
```bash
sudo apt-get update
sudo apt-get install -y pkg-config libsecp256k1-dev build-essential
```
**RHEL/CentOS/Fedora:**
```bash
sudo yum install -y pkgconfig libsecp256k1-devel gcc
```
**Alpine Linux:**
```bash
sudo apk add pkgconfig libsecp256k1-dev gcc musl-dev
```
#### Step 2: Install Python Dependencies
```bash
pip install -r requirements.txt
pip install -r requirements-crypto.txt
```
This includes:
- ✅ All fiat payment features
- ✅ Cryptocurrency payments (BTC, ETH, USDT, USDC)
- ✅ HD wallet generation (BIP32/BIP44)
- ✅ Blockchain monitoring
- ✅ Multi-exchange price aggregation
## Troubleshooting
### Error: "Could NOT find PkgConfig"
This means system dependencies are missing. Install them using the commands above for your OS.
### Error: "coincurve build failed"
The `coincurve` package requires `libsecp256k1`. Install system dependencies first, then retry.
### Alternative: Use Pre-built Wheels
If you cannot install system dependencies, try using pre-built wheels:
```bash
pip install --only-binary :all: coincurve
pip install -r requirements-crypto.txt
```
## Configuration
After installation, configure the payment system:
1. Set encryption key for HD wallets:
```bash
export ENCRYPTION_KEY="your-fernet-key-here"
```
2. Configure payment gateways in the admin dashboard:
- Stripe API keys
- PayPal client credentials
- Crypto consolidation addresses (if using crypto)
3. Run database migrations:
```bash
python main.py # Migrations run automatically on startup
```
## Features by Installation Type
| Feature | Core Only | Full (Crypto) |
|---------|-----------|---------------|
| Stripe Payments | ✅ | ✅ |
| PayPal Payments | ✅ | ✅ |
| Subscriptions | ✅ | ✅ |
| Auto-renewals | ✅ | ✅ |
| Email Notifications | ✅ | ✅ |
| Quota Enforcement | ✅ | ✅ |
| BTC Payments | ❌ | ✅ |
| ETH/USDT/USDC | ❌ | ✅ |
| HD Wallet | ❌ | ✅ |
| Blockchain Monitor | ❌ | ✅ |
| Wallet Consolidation | ❌ | ✅ |
## Recommendation
For production deployments:
- **With crypto support**: Install on a server where you have root access to install system dependencies
- **Without crypto support**: Use the core installation and rely on Stripe/PayPal for all payments
Both options provide a fully functional subscription payment system.
PAYMENT_INTEGRATION_SUMMARY.md deleted 100644 → 0
View file @ f182fbd1
# Payment Integration Summary
This document summarizes the payment method integrations completed for AISBF.
## Completed Integrations
### 1. Stripe Credit Card Integration ✅
**Status**: Fully functional
**Features**:
- Modal-based card input interface
- Real-time validation with Stripe Elements
- Custom styling to match AISBF theme
- Error handling with inline error display
- Loading states during processing
- Secure token-based payment method storage
**User Flow**:
1. User clicks "Add Credit Card" on billing page
2. Modal opens with Stripe Elements card input
3. User enters card details
4. Stripe validates and creates payment method
5. Payment method ID sent to server
6. Card stored in database with metadata
**Files Modified**:
- `templates/dashboard/add_payment_method.html` - Modal UI and Stripe integration
- `main.py` - `/dashboard/billing/add-method/stripe` endpoint
- `aisbf/database.py` - Payment method storage
### 2. PayPal OAuth Integration ✅
**Status**: Fully functional
**Features**:
- OAuth 2.0 authentication flow
- CSRF protection with state tokens
- Sandbox and production mode support
- Duplicate account detection
- User information storage (email, user ID, name)
- Access token storage for future API calls
**User Flow**:
1. User clicks "Connect PayPal" on billing page
2. Redirected to PayPal OAuth login
3. User authorizes AISBF access
4. PayPal redirects back with authorization code
5. Server exchanges code for access token
6. User info fetched from PayPal API
7. PayPal account stored as payment method
**Endpoints**:
- `GET /dashboard/billing/add-method/paypal/oauth` - Initiates OAuth flow
- `GET /dashboard/billing/add-method/paypal/callback` - Handles OAuth callback
**Files Modified**:
- `main.py` - PayPal OAuth endpoints
- `templates/dashboard/paypal_connect.html` - Error page for configuration issues
- `aisbf/database.py` - Enhanced payment method display logic
- `requirements.txt` - Added paypalrestsdk dependency
**Configuration Required**:
- PayPal Client ID
- PayPal Client Secret
- Sandbox/Production mode toggle
- Callback URL configuration in PayPal app
### 3. Cryptocurrency Payment Methods ✅
**Status**: Functional (default selection)
**Supported Cryptocurrencies**:
- Bitcoin (BTC)
- Ethereum (ETH)
- USDT (Tether)
- USDC (USD Coin)
**User Flow**:
1. User clicks cryptocurrency button
2. System sets as default payment method
3. Crypto type stored in database
**Note**: This is a simplified implementation that sets the preferred crypto type. Actual payment processing would require additional integration with crypto payment gateways.
### 4. User Dashboard Subscription Section ✅
**Status**: Fully functional
**Features**:
- Displays current subscription tier
- Shows plan pricing and limits
- Subscription status and renewal date
- "Add Payment Method" button (only shown when no payment methods exist)
- Links to billing page
**Files Modified**:
- `templates/dashboard/user_index.html` - Added subscription section
- `main.py` - Added subscription context to user dashboard route
## Database Schema
### payment_methods Table
```sql
CREATE TABLE payment_methods (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
type VARCHAR(50) NOT NULL, -- 'stripe', 'paypal', 'bitcoin', etc.
identifier VARCHAR(255), -- Email, card last4, or address
is_default BOOLEAN DEFAULT 0,
is_active BOOLEAN DEFAULT 1,
metadata TEXT, -- JSON with additional details
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id)
);
```
### Metadata Examples
**Stripe**:
```json
{
"stripe_payment_method_id": "pm_xxxxx"
}
```
**PayPal**:
```json
{
"paypal_user_id": "USER_ID",
"paypal_email": "user@example.com",
"paypal_name": "John Doe",
"access_token": "ACCESS_TOKEN",
"sandbox": true
}
```
**Cryptocurrency**:
```json
{}
```
## Configuration
### Payment Gateway Settings
Stored in `admin_settings` table with key `payment_gateways`:
```json
{
"stripe": {
"enabled": true,
"publishable_key": "pk_test_xxxxx",
"secret_key": "sk_test_xxxxx",
"webhook_secret": "whsec_xxxxx",
"test_mode": true
},
"paypal": {
"enabled": true,
"client_id": "xxxxx",
"client_secret": "xxxxx",
"webhook_secret": "",
"sandbox": true
},
"bitcoin": {
"enabled": true,
"address": "bc1xxxxx",
"confirmations": 3,
"expiration_minutes": 120
},
"ethereum": {
"enabled": true,
"address": "0xxxxx",
"confirmations": 12,
"chain_id": 1
},
"usdt": {
"enabled": true,
"address": "0xxxxx",
"network": "erc20",
"confirmations": 3
},
"usdc": {
"enabled": true,
"address": "0xxxxx",
"network": "erc20",
"confirmations": 3
}
}
```
## API Endpoints
### Payment Method Management
- `GET /dashboard/billing` - View payment methods and billing history
- `GET /dashboard/billing/add-method` - Add payment method page
- `POST /dashboard/billing/add-method` - Set crypto as default (AJAX)
- `POST /dashboard/billing/add-method/stripe` - Add Stripe card (AJAX)
- `GET /dashboard/billing/add-method/paypal/oauth` - Initiate PayPal OAuth
- `GET /dashboard/billing/add-method/paypal/callback` - PayPal OAuth callback
### Subscription Management
- `GET /dashboard/subscription` - Subscription management page
- `GET /dashboard/pricing` - View available plans
- `GET /dashboard/user` - User dashboard with subscription info
## Security Features
1. **CSRF Protection**: State tokens for OAuth flows
2. **Session Validation**: All endpoints require authentication
3. **HTTPS Required**: PayPal OAuth requires HTTPS in production
4. **Token Storage**: Secure storage of access tokens in database
5. **Duplicate Prevention**: Checks for existing payment methods
6. **Input Validation**: Server-side validation of all inputs
## Testing
### Stripe Testing
Use Stripe test cards:
- Success: `4242 4242 4242 4242`
- Decline: `4000 0000 0000 0002`
- Requires authentication: `4000 0025 0000 3155`
### PayPal Testing
1. Enable sandbox mode in settings
2. Create sandbox accounts at https://developer.paypal.com/dashboard/accounts
3. Use sandbox credentials for testing
### Cryptocurrency Testing
Currently stores preference only. No actual blockchain interaction.
## Future Enhancements
### Stripe
- [ ] Webhook integration for payment events
- [ ] Support for 3D Secure authentication
- [ ] Card update functionality
- [ ] Multiple cards per user
### PayPal
- [ ] Webhook integration for payment notifications
- [ ] Payment processing implementation
- [ ] Subscription creation and management
- [ ] Refund support
- [ ] Token refresh logic
### Cryptocurrency
- [ ] Integration with crypto payment gateways (Coinbase Commerce, BTCPay)
- [ ] QR code generation for payments
- [ ] Payment verification via blockchain
- [ ] Automatic conversion rates
- [ ] Transaction monitoring
### General
- [ ] Payment method editing
- [ ] Payment method deletion with confirmation
- [ ] Set default payment method
- [ ] Payment history with filtering
- [ ] Invoice generation
- [ ] Email notifications for payments
- [ ] Multi-currency support
## Documentation
- `PAYPAL_SETUP.md` - Detailed PayPal configuration guide
- `PAYMENT_INTEGRATION_SUMMARY.md` - This document
- Code comments in relevant files
## Dependencies Added
```
paypalrestsdk # PayPal REST API SDK
```
## Files Modified
1. `main.py` - Payment endpoints and logic
2. `aisbf/database.py` - Payment method storage and retrieval
3. `templates/dashboard/add_payment_method.html` - Payment method UI
4. `templates/dashboard/billing.html` - Billing page
5. `templates/dashboard/user_index.html` - User dashboard
6. `templates/dashboard/paypal_connect.html` - PayPal error page
7. `requirements.txt` - Added PayPal SDK
8. `setup.py` - Added new template files
## Installation
```bash
# Install dependencies
pip install -r requirements.txt
# Or install PayPal SDK separately
pip install paypalrestsdk
```
## Configuration Steps
1. **Configure Stripe** (if using):
- Get API keys from https://dashboard.stripe.com/apikeys
- Add to payment gateway settings
2. **Configure PayPal** (if using):
- Create app at https://developer.paypal.com
- Get Client ID and Secret
- Configure callback URL
- Add to payment gateway settings
3. **Configure Cryptocurrency** (if using):
- Set wallet addresses for each supported currency
- Configure confirmation requirements
4. **Test Integration**:
- Enable test/sandbox modes
- Test each payment method
- Verify database storage
- Check error handling
## Support
For issues or questions:
- Check application logs
- Review payment gateway documentation
- Verify configuration settings
- Test in sandbox/test mode first
## Changelog
### 2026-04-16
- ✅ Implemented Stripe credit card integration with modal UI
- ✅ Implemented PayPal OAuth 2.0 integration
- ✅ Enhanced payment method display logic
- ✅ Added subscription section to user dashboard
- ✅ Created comprehensive documentation
- ✅ Added PayPal SDK dependency
PAYPAL_SETUP.md deleted 100644 → 0
View file @ f182fbd1
# PayPal Payment Integration Setup Guide
This guide explains how to configure PayPal as a payment method in AISBF.
## Overview
AISBF now supports PayPal OAuth integration, allowing users to connect their PayPal accounts as payment methods for subscriptions and plan upgrades.
## Features
- **OAuth 2.0 Integration**: Secure PayPal account connection using OAuth 2.0
- **Automatic Account Detection**: Prevents duplicate PayPal accounts
- **Sandbox Support**: Test mode for development
- **User Information**: Stores PayPal email, user ID, and name
- **Access Token Storage**: Stores access token for future API calls
## Prerequisites
1. **PayPal Developer Account**: Sign up at https://developer.paypal.com
2. **PayPal App Credentials**: Create an app to get Client ID and Secret
## Creating a PayPal App
### Step 1: Create a PayPal Developer Account
1. Go to https://developer.paypal.com
2. Sign in with your PayPal account or create a new one
3. Navigate to "Dashboard"
### Step 2: Create an App
1. Click "Apps & Credentials" in the left sidebar
2. Click "Create App" button
3. Enter an app name (e.g., "AISBF Payment Integration")
4. Select "Merchant" as the app type
5. Click "Create App"
### Step 3: Get Your Credentials
After creating the app, you'll see:
- **Client ID**: Your application's public identifier
- **Secret**: Your application's secret key (click "Show" to reveal)
### Step 4: Configure OAuth Settings
1. Scroll down to "App Settings"
2. Under "Return URL", add your callback URL:
- For production: `https://yourdomain.com/dashboard/billing/add-method/paypal/callback`
- For development: `http://localhost:8000/dashboard/billing/add-method/paypal/callback`
3. Click "Save"
### Step 5: Enable Required Features
1. Under "Features", ensure these are enabled:
- **Log In with PayPal**: Required for OAuth
- **Accept Payments**: Required for payment processing
2. Under "Advanced Features", configure:
- **Return URL**: Your callback URL
- **Privacy Policy URL**: Your privacy policy page
- **User Agreement URL**: Your terms of service page
## Configuring AISBF
### Option 1: Via Dashboard (Recommended)
1. Log in to AISBF dashboard as admin
2. Navigate to "Settings" → "Payment Gateways"
3. Find the "PayPal" section
4. Configure the following:
- **Enabled**: Toggle to enable PayPal
- **Client ID**: Paste your PayPal app Client ID
- **Client Secret**: Paste your PayPal app Secret
- **Sandbox Mode**: Enable for testing, disable for production
- **Webhook Secret**: (Optional) For webhook verification
5. Click "Save Settings"
### Option 2: Via Database
Update the `admin_settings` table:
```sql
INSERT OR REPLACE INTO admin_settings (setting_key, setting_value, updated_at)
VALUES ('payment_gateways', '{
"paypal": {
"enabled": true,
"client_id": "YOUR_CLIENT_ID",
"client_secret": "YOUR_CLIENT_SECRET",
"webhook_secret": "",
"sandbox": true
},
"stripe": {...},
"bitcoin": {...}
}', CURRENT_TIMESTAMP);
```
## Testing the Integration
### Sandbox Testing
1. Enable sandbox mode in PayPal settings
2. Create a sandbox account at https://developer.paypal.com/dashboard/accounts
3. Use sandbox credentials to test the OAuth flow
### Test Flow
1. Navigate to "Billing" → "Add Payment Method"
2. Click "Connect PayPal"
3. You'll be redirected to PayPal login
4. Log in with your PayPal account (or sandbox account)
5. Authorize the connection
6. You'll be redirected back to AISBF
7. PayPal account should appear in your payment methods
## OAuth Flow Details
### Step 1: Initiate OAuth
- User clicks "Connect PayPal" button
- AISBF generates a cryptographically secure state token (64 hex characters)
- State token is stored in user session for CSRF protection
- User is redirected to PayPal OAuth URL with:
- `client_id`: Your PayPal app client ID
- `response_type=code`: Authorization code flow
- `scope=openid profile email`: Requested permissions
- `redirect_uri`: Your callback URL
- `state`: CSRF protection token
### Step 2: User Authorization
- User logs in to PayPal (sandbox or live)
- User reviews requested permissions
- User clicks "Agree and Continue" to authorize
- PayPal redirects back with authorization code and state token
### Step 3: Token Exchange
- AISBF validates state token matches session (CSRF check)
- AISBF exchanges authorization code for access token via PayPal API
- Uses HTTP Basic Auth with base64-encoded client credentials
- Receives access token from PayPal
### Step 4: Fetch User Profile
- AISBF calls PayPal Identity API with access token
- Retrieves user_id, email, and name from PayPal
- Validates required fields are present
### Step 5: Store Payment Method
- AISBF checks for duplicate PayPal accounts (by email and user_id)
- Stores payment method in database with:
- Type: 'paypal'
- Identifier: PayPal email
- Metadata: PayPal user_id, email, name, access token, sandbox flag
- Sets as default if user's first payment method
- Redirects to billing page with success message
## Security Considerations
1. **HTTPS Required**: PayPal OAuth requires HTTPS in production
2. **State Token**: CSRF protection using random state tokens
3. **Client Secret**: Never expose client secret in frontend code
4. **Access Token Storage**: Tokens are stored securely in database
5. **Duplicate Prevention**: System checks for existing PayPal accounts
## Troubleshooting
### Error: "PayPal is not enabled"
- Check that PayPal is enabled in payment gateway settings
- Verify admin_settings table has correct configuration
### Error: "PayPal is not properly configured"
- Ensure Client ID is set in gateway settings
- Verify Client ID is correct
### Error: "Invalid state token"
- This is a CSRF protection error
- Clear browser cookies and try again
- Check that sessions are working properly
### Error: "Failed to connect PayPal account"
- Check that Client Secret is correct
- Verify callback URL is configured in PayPal app settings
- Check server logs for detailed error messages
### Error: "This PayPal account is already connected"
- User already has this PayPal account as a payment method
- Remove existing PayPal method first, then reconnect
## API Endpoints
### Initiate OAuth
```
GET /dashboard/billing/add-method/paypal/oauth
```
Redirects to PayPal OAuth URL
### OAuth Callback
```
GET /dashboard/billing/add-method/paypal/callback?code=xxx&state=xxx
```
Handles OAuth callback and stores payment method
## Database Schema
### payment_methods Table
```sql
CREATE TABLE payment_methods (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
type VARCHAR(50) NOT NULL, -- 'paypal'
identifier VARCHAR(255), -- PayPal email
is_default BOOLEAN DEFAULT 0,
is_active BOOLEAN DEFAULT 1,
metadata TEXT, -- JSON with PayPal details
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id)
);
```
### Metadata Structure
```json
{
"paypal_user_id": "PAYPAL_USER_ID",
"paypal_email": "user@example.com",
"paypal_name": "John Doe",
"access_token": "ACCESS_TOKEN",
"sandbox": true
}
```
## Production Checklist
- [ ] Create production PayPal app
- [ ] Configure production callback URL
- [ ] Set sandbox mode to `false`
- [ ] Test OAuth flow in production
- [ ] Verify HTTPS is enabled
- [ ] Configure webhook endpoints (if needed)
- [ ] Test payment processing
- [ ] Monitor error logs
## Future Enhancements
- **Webhook Integration**: Handle PayPal webhooks for payment notifications
- **Payment Processing**: Process actual payments using PayPal API
- **Subscription Management**: Create and manage PayPal subscriptions
- **Refund Support**: Handle refunds through PayPal API
- **Token Refresh**: Implement access token refresh logic
## Support
For issues or questions:
- Check server logs: `/var/log/aisbf/` or application logs
- Review PayPal Developer documentation: https://developer.paypal.com/docs/
- Contact AISBF support
## References
- [PayPal OAuth Documentation](https://developer.paypal.com/docs/log-in-with-paypal/)
- [PayPal REST API](https://developer.paypal.com/docs/api/overview/)
- [PayPal Sandbox Testing](https://developer.paypal.com/docs/api-basics/sandbox/)
PAYPAL_TROUBLESHOOTING.md deleted 100644 → 0
View file @ f182fbd1
# PayPal OAuth Integration Troubleshooting Guide
## Common Error: "invalid client_id or redirect_uri"
This error occurs when PayPal cannot validate your OAuth request. Here's how to fix it.
## Quick Fix Checklist
### 1. Verify PayPal App Configuration
Go to https://developer.paypal.com/dashboard/applications and check:
- [ ] App is created and active
- [ ] You're using the correct Client ID
- [ ] Sandbox mode matches your configuration
- [ ] Return URL is properly configured
### 2. Configure Return URL in PayPal App
**CRITICAL:** The Return URL in PayPal app settings must EXACTLY match your callback URL.
#### Steps:
1. Open your PayPal app in developer dashboard
2. Scroll to **"Return URL"** section
3. Add your callback URL:
```
https://yourdomain.com/dashboard/billing/add-method/paypal/callback
```
4. Click **Save**
#### Important Notes:
- Must use **HTTPS** in production (HTTP only for localhost testing)
- URL is **case-sensitive**
- No trailing slash (unless your logs show one)
- Must match exactly what appears in server logs
### 3. Check Server Logs
The AISBF server logs the redirect_uri being used. Check logs:
```bash
# View logs
tail -f /var/log/aisbf/app.log | grep "PayPal OAuth"
# Or if using systemd
journalctl -u aisbf -f | grep "PayPal OAuth"
```
Look for lines like:
```
PayPal OAuth redirect_uri: https://yourdomain.com/dashboard/billing/add-method/paypal/callback
PayPal OAuth client_id: AYxxxxxx...
PayPal OAuth sandbox mode: True
```
### 4. Verify Configuration in AISBF
Check your PayPal settings in AISBF admin dashboard:
1. Login as admin
2. Go to Settings → Payment Gateways
3. Check PayPal section:
- [ ] Enabled: Yes
- [ ] Client ID: Matches PayPal app
- [ ] Client Secret: Matches PayPal app
- [ ] Sandbox Mode: Matches PayPal app type
## Detailed Troubleshooting
### Issue: Wrong Client ID
**Symptoms:**
- Error: "invalid client_id"
- PayPal rejects immediately
**Solution:**
1. Go to PayPal developer dashboard
2. Open your app
3. Copy the **Client ID** (starts with "A" for live, different for sandbox)
4. Update in AISBF admin settings
5. Make sure you're using:
- **Sandbox Client ID** if sandbox mode is enabled
- **Live Client ID** if sandbox mode is disabled
### Issue: Redirect URI Not Registered
**Symptoms:**
- Error: "invalid redirect_uri"
- PayPal shows "action is not supported"
**Solution:**
1. Check server logs for the exact redirect_uri being used
2. Go to PayPal app settings
3. Add that EXACT URL to "Return URL" field
4. Common mistakes:
- Using http instead of https
- Adding/missing trailing slash
- Wrong domain name
- Typo in path
### Issue: Sandbox vs Production Mismatch
**Symptoms:**
- Works in sandbox but not production (or vice versa)
- Client ID seems correct but still fails
**Solution:**
- Sandbox apps and Live apps are separate in PayPal
- You need TWO apps:
- One for sandbox (for testing)
- One for production (for live)
- Make sure AISBF sandbox mode matches the app type you're using
### Issue: HTTPS vs HTTP
**Symptoms:**
- Logs show http:// but PayPal expects https://
- Works locally but not on server
**Solution:**
If behind a reverse proxy (nginx, Apache):
1. Ensure proxy sets X-Forwarded-Proto header:
```nginx
proxy_set_header X-Forwarded-Proto $scheme;
```
2. AISBF should detect this automatically via ProxyHeadersMiddleware
3. If still using http://, check:
- Proxy configuration
- SSL certificate is valid
- AISBF is receiving correct headers
## Testing Procedure
### 1. Test with Sandbox First
1. Create a sandbox app in PayPal developer dashboard
2. Get sandbox Client ID and Secret
3. Configure AISBF with sandbox credentials
4. Set sandbox mode to `true`
5. Add sandbox return URL
6. Test the OAuth flow
### 2. Verify Each Step
**Step 1: Check Configuration**
```bash
# Check AISBF logs when clicking "Connect PayPal"
tail -f /var/log/aisbf/app.log
```
**Step 2: Verify Redirect**
- Click "Connect PayPal" in AISBF
- Check browser URL bar
- Should redirect to: `https://www.sandbox.paypal.com/signin/authorize?client_id=...`
**Step 3: Check PayPal Response**
- If you see PayPal login page: ✅ OAuth URL is correct
- If you see error page: ❌ Check client_id or redirect_uri
**Step 4: Complete Flow**
- Login with PayPal sandbox account
- Authorize the app
- Should redirect back to AISBF
- Check if PayPal account appears in payment methods
### 3. Move to Production
Once sandbox works:
1. Create a live app in PayPal dashboard
2. Get live Client ID and Secret
3. Update AISBF configuration
4. Set sandbox mode to `false`
5. Add production return URL (must use HTTPS)
6. Test with real PayPal account
## Configuration Examples
### Sandbox Configuration
```json
{
"paypal": {
"enabled": true,
"client_id": "AYSq3RDGsmBLJE-otTkBtM-jBRd1TCQwFf9RGfwddNXWz0uFU9ztymylOhRS",
"client_secret": "EGnHDxD_qRPdaLdZz8iCr8N7_MzF-YHPTkjs6NKYQvQSBngp4PTTVWkPZRbL",
"sandbox": true
}
}
```
Return URL in PayPal sandbox app:
```
https://yourdomain.com/dashboard/billing/add-method/paypal/callback
```
### Production Configuration
```json
{
"paypal": {
"enabled": true,
"client_id": "AeHGtyuJHGFRTYUIKJHGFRTYUIKJHGFRTYUIKJHGFRTYUIKJHGFRTYUIKJH",
"client_secret": "ELkjhgfdsaLKJHGFDSALKJHGFDSALKJHGFDSALKJHGFDSALKJHGFDSALKJ",
"sandbox": false
}
}
```
Return URL in PayPal live app:
```
https://yourdomain.com/dashboard/billing/add-method/paypal/callback
```
## Debug Mode
To see detailed OAuth flow information:
1. Check AISBF logs for PayPal OAuth messages
2. Use browser developer tools (Network tab) to see redirects
3. Check PayPal app dashboard for API call logs
## Still Having Issues?
### Check These:
1. **Client ID Format**
- Sandbox: Usually starts with "AY" or "AS"
- Live: Different format
- Should be 80+ characters long
2. **Return URL Format**
- Must be absolute URL (include https://)
- Must include full path
- No query parameters
- No fragments (#)
3. **PayPal App Status**
- App must be active (not disabled)
- For live apps, may need PayPal approval
4. **Network Issues**
- Firewall blocking PayPal API calls
- DNS resolution issues
- SSL certificate problems
### Get Help
If still stuck:
1. Check server logs for detailed error messages
2. Review PayPal app settings carefully
3. Test with PayPal sandbox first
4. Verify HTTPS is working correctly
5. Check that redirect_uri in logs matches PayPal settings exactly
## Quick Reference
### PayPal OAuth Endpoints
**Sandbox:**
- Authorization: `https://www.sandbox.paypal.com/signin/authorize`
- Token: `https://api.sandbox.paypal.com/v1/oauth2/token`
- User Info: `https://api.sandbox.paypal.com/v1/identity/oauth2/userinfo`
**Production:**
- Authorization: `https://www.paypal.com/signin/authorize`
- Token: `https://api.paypal.com/v1/oauth2/token`
- User Info: `https://api.paypal.com/v1/identity/oauth2/userinfo`
### Required OAuth Parameters
- `client_id`: Your PayPal app Client ID
- `response_type`: `code`
- `scope`: `openid profile email`
- `redirect_uri`: Your callback URL (URL encoded)
- `state`: Random token for CSRF protection
### AISBF Callback URL
Always:
```
https://yourdomain.com/dashboard/billing/add-method/paypal/callback
```
Replace `yourdomain.com` with your actual domain.
## Success Indicators
You'll know it's working when:
1. ✅ Clicking "Connect PayPal" redirects to PayPal login
2. ✅ After login, you see PayPal authorization screen
3. ✅ After authorizing, you're redirected back to AISBF
4. ✅ PayPal account appears in payment methods list
5. ✅ No errors in server logs
## Common Success Path
```
User clicks "Connect PayPal"
↓
AISBF generates state token
↓
AISBF redirects to PayPal with client_id and redirect_uri
↓
User logs into PayPal
↓
User authorizes AISBF
↓
PayPal redirects to callback URL with authorization code
↓
AISBF exchanges code for access token
↓
AISBF fetches user info from PayPal
↓
AISBF stores PayPal account as payment method
↓
User sees PayPal account in payment methods list
```
---
**Last Updated:** 2026-04-16
**Version:** 1.0
**For AISBF:** v0.99.26+
SESSION_SUMMARY.md deleted 100644 → 0
View file @ f182fbd1
# Session Summary - Database & Analytics Fixes
## Issues Resolved
### 1. ✅ Lost Account Tiers After Server Restart
**Problem**: Custom account tiers disappeared after restarting the server, leaving only the default "Free Tier"
**Root Cause**:
- April 14 (commit `f997a0f`): `account_tiers` table creation removed from `database.py`
- April 16 (commit `0052431`): Payment migrations added without `account_tiers` table
- Result: Table existed but migrations didn't recreate it
**Solution**:
- Added `_create_account_tiers_table()` method to payment migrations
- Creates table with all columns including `is_visible`
- Ensures default "Free Tier" exists
- Uses `CREATE TABLE IF NOT EXISTS` to preserve existing data
**Commit**: `11330e9` - fix: add account_tiers and admin_settings tables to payment migrations
---
### 2. ✅ Lost Payment Gateway Configurations (MySQL)
**Problem**: Payment gateway settings (Stripe, PayPal, crypto) disappeared after upgrading to v0.99.29 on MySQL
**Root Cause**:
- Payment gateway settings stored in `admin_settings` table
- Table only created in `database.py` migrations, not in payment migrations
- MySQL installations using payment migrations never created the table
**Solution**:
- Added `admin_settings` table creation to `_create_config_tables()` in payment migrations
- Table stores payment gateway configs, encryption keys, and other admin settings
- Uses `CREATE TABLE IF NOT EXISTS` to preserve existing data
**Commit**: `11330e9` - fix: add account_tiers and admin_settings tables to payment migrations
---
### 3. ✅ Analytics Not Recording ALL Requests
**Problem**: Analytics dashboard showed incomplete data - many requests were missing
**Root Cause**:
Analytics recording was incomplete:
- ❌ Failed rotation requests - NOT recorded
- ❌ Failed autoselect requests - NOT recorded
- ❌ Streaming requests (all types) - NOT recorded
- ❌ Authentication failures - NOT recorded
- ✅ Successful direct provider requests - Recorded
- ✅ Successful rotation requests - Recorded
- ✅ Successful autoselect requests - Recorded
**Solution**:
Added comprehensive analytics recording for ALL request types:
1. **Failed Rotation Requests** (line 2857+)
- Records when all rotation attempts are exhausted
- Tracks error type as 'RotationFailure'
2. **Failed Autoselect Requests** (line 4177+)
- Wraps rotation call in try-catch
- Records analytics on exception before re-raising
3. **Streaming Requests - Success** (line 1200+)
- Calculates tokens from accumulated response text
- Records after successful streaming completion
4. **Streaming Requests - Failure** (line 1225+)
- Estimates tokens for failed request
- Records error type before yielding error
5. **Authentication Failures** (line 366+)
- Records when API key is missing
- Tracks error type as 'AuthenticationError'
**Commit**: `f2fe4c1` - fix: add comprehensive analytics recording for all request types
---
### 4. ✅ Misleading Migration Log Messages
**Problem**: Log message "✅ Default payment system data inserted" appeared on every boot, suggesting data was being re-inserted
**Reality**:
- System uses `INSERT OR IGNORE` (SQLite) / `INSERT IGNORE` (MySQL)
- Only inserts if record doesn't exist (based on UNIQUE constraints)
- Existing records are NEVER overwritten
- User modifications are preserved
**Solution**:
- Changed log message to "✅ Default payment system data checked (existing records preserved)"
- Added comments explaining INSERT OR IGNORE behavior
- Clarifies that the system only inserts missing defaults
**Commit**: `66565a9` - fix: improve migration logging to clarify data preservation
---
## Current Database Status
```
Account Tiers: 1 (default Free Tier)
Admin Settings: 0 (empty, ready for configuration)
Token Usage: 35 requests tracked
Price Sources: 3 (Coinbase, Binance, Kraken)
Email Notifications: 9 notification types configured
```
---
## What Users Need to Do
### For Lost Tiers (All Databases):
1. ✅ Migrations run automatically on server restart
2. ✅ Table structure is verified/created
3. ⚠️ **You must recreate your custom tiers** through the admin dashboard
4. ✅ Future restarts will preserve your tiers
### For Lost Payment Gateway Configs (MySQL Only):
1. ✅ Migrations run automatically on server restart
2. ✅ `admin_settings` table is created
3. ⚠️ **You must reconfigure payment gateways** through Admin → Payment Settings
4. ✅ Future restarts will preserve your settings
### For Analytics:
1. ✅ All request types are now tracked automatically
2. ✅ Dashboard shows complete data
3. ✅ No action required
---
## Technical Details
### Migration Safety
All fixes use `CREATE TABLE IF NOT EXISTS` and `INSERT OR IGNORE`:
- ✅ Existing tables are preserved
- ✅ Existing data is NOT deleted
- ✅ Safe to run on existing installations
- ✅ Safe to run multiple times
- ✅ User modifications are never overwritten
### Token Estimation
For requests without token counts, we estimate:
- Use `count_messages_tokens()` for prompt tokens
- Estimate completion based on `max_tokens` or typical response
- Fallback to 50 tokens minimum for failed requests
### Analytics Recording
Now tracks:
- Provider ID
- Model name
- Tokens used (actual or estimated)
- Latency (when available)
- Success/failure status
- Error type (for failures)
- User ID and token ID (when authenticated)
- Rotation/autoselect ID (when applicable)
---
## Files Modified
1. **aisbf/payments/migrations.py**
- Added `_create_account_tiers_table()` method
- Added `admin_settings` table to `_create_config_tables()`
- Added default tier insertion
- Improved logging messages
2. **aisbf/handlers.py**
- Added analytics for failed rotation requests
- Added analytics for failed autoselect requests
- Added analytics for streaming requests (success/failure)
- Added analytics for authentication failures
- Total: 259 lines added, 32 lines removed
3. **Documentation**
- MIGRATION_FIXES.md - Tier and gateway fixes
- ANALYTICS_FIX.md - Analytics recording fixes
---
## Commits
```
66565a9 fix: improve migration logging to clarify data preservation
c4aa7ff docs: add analytics fix documentation
f2fe4c1 fix: add comprehensive analytics recording for all request types
11330e9 fix: add account_tiers and admin_settings tables to payment migrations
```
---
## Version
These fixes are included in **v0.99.29** and ready for deployment.
---
## Testing Performed
✅ Direct provider requests - Recorded in analytics
✅ Streaming requests - Recorded in analytics
✅ Failed rotation requests - Recorded in analytics
✅ Account tiers table - Created and preserved
✅ Admin settings table - Created and preserved
✅ Default data - Only inserted if missing
✅ User modifications - Preserved across restarts
---
## Impact
- **Complete analytics** - All requests are now tracked
- **Data preservation** - User configurations are never lost
- **Clear logging** - Migration messages accurately reflect behavior
- **Production ready** - Safe to deploy on existing installations
qwen-oauth2-analysis.md deleted 100644 → 0
View file @ f182fbd1
# Qwen API Documentation
Complete documentation for implementing Qwen API requests based on the Qwen Code codebase analysis.
## 📚 Documentation Files
This repository contains comprehensive documentation for working with the Qwen API:
### 1. **QWEN_API_SUMMARY.md** - Quick Reference Guide
- **Purpose**: Fast lookup for common patterns and configurations
- **Best for**: Developers who need quick answers
- **Contents**:
- API endpoints and base URLs
- Required headers
- Request payload format
- Message format examples
- Session management
- Token management
- Streaming response format
- Error handling patterns
- Minimal working examples
### 2. **QWEN_API_IMPLEMENTATION_GUIDE.md** - Complete Implementation
- **Purpose**: Full Python implementation with detailed explanations
- **Best for**: Developers building from scratch
- **Contents**:
- OAuth2 authentication flow
- Token storage and refresh logic
- Complete Python client class
- Request building and sending
- Response parsing
- Error handling with retry logic
- Streaming support
- Working code examples
- Testing instructions
### 3. **QWEN_API_FLOW_DIAGRAM.md** - Visual Flow Charts
- **Purpose**: Visual representation of request flows
- **Best for**: Understanding the overall architecture
- **Contents**:
- Complete request flow diagram
- Streaming request flow
- Message format conversion (none needed!)
- Cache control flow
- Error handling flow
- Token lifecycle diagram
- Session management diagram
### 4. **test_qwen_api.py** - Test Suite
- **Purpose**: Executable test script to verify implementation
- **Best for**: Testing your setup and understanding API behavior
- **Features**:
- Credentials validation
- Token expiry checking
- Token refresh testing
- Simple completion test
- Streaming completion test
- Multi-turn conversation test
- Detailed logging
- Comprehensive error handling
## 🚀 Quick Start
### Prerequisites
1. **OAuth2 Credentials**: You need valid OAuth2 credentials stored in `~/.qwen/oauth_creds.json`
2. **Python 3.7+**: Required for the test script
3. **requests library**: Install with `pip install requests`
### Credentials File Format
```json
{
"access_token": "your_access_token_here",
"refresh_token": "your_refresh_token_here",
"expiry_date": 1234567890000,
"token_type": "Bearer",
"resource_url": "https://dashscope.aliyuncs.com/compatible-mode"
}
```
### Running the Test Suite
```bash
# Make the script executable
chmod +x test_qwen_api.py
# Run all tests
./test_qwen_api.py
# Run with verbose output
./test_qwen_api.py --verbose
```
### Minimal Example
```python
import requests
import json
import uuid
# Load credentials
with open("~/.qwen/oauth_creds.json") as f:
creds = json.load(f)
# Build request
headers = {
"Authorization": f"Bearer {creds['access_token']}",
"Content-Type": "application/json",
"User-Agent": "QwenCode/1.0.0",
"X-DashScope-CacheControl": "enable",
"X-DashScope-UserAgent": "QwenCode/1.0.0",
"X-DashScope-AuthType": "qwen-oauth",
}
payload = {
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "Hello!"}
],
"metadata": {
"sessionId": str(uuid.uuid4()),
"promptId": str(uuid.uuid4())
}
}
# Send request
response = requests.post(
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
headers=headers,
json=payload
)
# Parse response
result = response.json()
print(result["choices"][0]["message"]["content"])
```
## 📖 Key Concepts
### 1. OpenAI Compatibility
The Qwen API is **100% OpenAI-compatible**. This means:
- No message format conversion needed
- Standard OpenAI message structure works directly
- Tool calling (function calling) uses OpenAI format
- Streaming follows OpenAI SSE format
### 2. OAuth2 Authentication
Unlike OpenAI's static API keys, Qwen uses OAuth2:
- Access tokens expire (typically 1 hour)
- Refresh tokens allow getting new access tokens
- Automatic refresh on 401/403 errors
- Re-authentication needed when refresh token expires
### 3. Session Management
Two levels of tracking:
- **Session ID**: Persistent across entire conversation
- **Prompt ID**: Unique per request
Both are UUIDs sent in the `metadata` field.
### 4. Dynamic Endpoints
The API endpoint is not fixed:
- Provided in OAuth credentials as `resource_url`
- Default: `https://dashscope.aliyuncs.com/compatible-mode/v1`
- Always use the endpoint from credentials
### 5. DashScope Headers
Additional headers for caching and tracking:
- `X-DashScope-CacheControl`: Enable prompt caching
- `X-DashScope-UserAgent`: Application identification
- `X-DashScope-AuthType`: Authentication method
## 🔍 Common Use Cases
### Simple Chat Completion
```python
from qwen_api_client import QwenAPIClient
client = QwenAPIClient()
response = client.chat_completion(
messages=[
{"role": "user", "content": "What is the capital of France?"}
],
model="qwen-plus"
)
print(response["choices"][0]["message"]["content"])
```
### Streaming Response
```python
for chunk in client.chat_completion(
messages=[
{"role": "user", "content": "Count from 1 to 10"}
],
model="qwen-plus",
stream=True
):
if "choices" in chunk:
delta = chunk["choices"][0].get("delta", {})
if "content" in delta:
print(delta["content"], end="", flush=True)
```
### Multi-Turn Conversation
```python
session_id = str(uuid.uuid4())
messages = []
# Turn 1
messages.append({"role": "user", "content": "My name is Alice"})
response = client.chat_completion(messages=messages, model="qwen-plus")
messages.append({"role": "assistant", "content": response["choices"][0]["message"]["content"]})
# Turn 2
messages.append({"role": "user", "content": "What is my name?"})
response = client.chat_completion(messages=messages, model="qwen-plus")
print(response["choices"][0]["message"]["content"]) # Should mention "Alice"
```
### Function Calling
```python
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City name"
}
},
"required": ["location"]
}
}
}
]
response = client.chat_completion(
messages=[
{"role": "user", "content": "What's the weather in Paris?"}
],
model="qwen-plus",
tools=tools
)
# Check for tool calls
if response["choices"][0]["message"].get("tool_calls"):
tool_call = response["choices"][0]["message"]["tool_calls"][0]
print(f"Function: {tool_call['function']['name']}")
print(f"Arguments: {tool_call['function']['arguments']}")
```
## 🛠️ Troubleshooting
### Issue: "No credentials found"
**Solution**: Ensure `~/.qwen/oauth_creds.json` exists with valid credentials.
### Issue: "Token expired" or 401/403 errors
**Solution**: The client should automatically refresh. If refresh fails, re-authenticate.
### Issue: "Refresh token expired"
**Solution**: You need to re-authenticate using the OAuth2 device flow.
### Issue: Streaming not working
**Solution**: Ensure you set `stream: true` and `stream_options: {include_usage: true}`.
### Issue: Session context not maintained
**Solution**: Use the same `sessionId` for all requests in a conversation.
## 📊 API Limits and Best Practices
### Rate Limits
- Implement exponential backoff for 429 errors
- Default retry: 3 attempts with increasing delays
### Token Management
- Check token expiry before each request
- Refresh proactively (60 second buffer)
- Cache credentials in memory for performance
### Session Management
- Generate session ID once per conversation
- Generate new prompt ID for each request
- Include both in metadata for tracking
### Error Handling
- Always handle 401/403 with automatic refresh
- Implement retry logic for network errors
- Log errors with request IDs for debugging
## 🔐 Security Considerations
1. **Credential Storage**: Store credentials securely with appropriate file permissions
2. **Token Exposure**: Never log or expose access tokens
3. **HTTPS Only**: Always use HTTPS endpoints
4. **Token Rotation**: Refresh tokens regularly, don't reuse expired tokens
## 📝 Additional Resources
### Source Code References
From the Qwen Code codebase:
- OAuth2 implementation: `packages/core/src/qwen/qwenOAuth2.ts`
- Content generator: `packages/core/src/qwen/qwenContentGenerator.ts`
- DashScope provider: `packages/core/src/core/openaiContentGenerator/provider/dashscope.ts`
- Message converter: `packages/core/src/core/openaiContentGenerator/converter.ts`
- Pipeline: `packages/core/src/core/openaiContentGenerator/pipeline.ts`
### API Endpoints
- OAuth Token: `https://chat.qwen.ai/api/v1/oauth2/token`
- Chat Completions: `https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions`
### Models
Common models:
- `qwen-plus`: Balanced performance and speed
- `qwen-max`: Maximum capability
- `qwen-turbo`: Fastest response
- `qwen3-coder-plus`: Code-specialized model
## 🤝 Contributing
If you find issues or have improvements:
1. Test your changes with the test suite
2. Update relevant documentation
3. Ensure examples work correctly
## 📄 License
This documentation is based on the Qwen Code codebase analysis and is provided for educational purposes.
---
**Last Updated**: 2026-04-19
**Documentation Version**: 1.0.0
**Based on**: Qwen Code codebase analysis
# Qwen API Implementation Summary
## Quick Reference
### 1. API Endpoint
```
Base URL: https://dashscope.aliyuncs.com/compatible-mode/v1
Endpoint: /chat/completions
Full URL: https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions
```
**Note**: The base URL can be dynamic and is provided in the OAuth `resource_url` field.
### 2. Authentication Headers
```python
headers = {
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/json",
"Accept": "application/json",
"User-Agent": "QwenCode/1.0.0 (Linux; x86_64)",
"X-DashScope-CacheControl": "enable",
"X-DashScope-UserAgent": "QwenCode/1.0.0 (Linux; x86_64)",
"X-DashScope-AuthType": "qwen-oauth",
"x-request-id": "unique-uuid-here" # Optional
}
```
### 3. Request Payload Format
```python
payload = {
"model": "qwen-plus",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"}
],
"stream": False, # or True for streaming
"stream_options": {"include_usage": True}, # Required for streaming
"temperature": 0.7, # Optional
"max_tokens": 1000, # Optional
"metadata": { # Optional but recommended
"sessionId": "session-uuid",
"promptId": "prompt-uuid"
}
}
```
### 4. Message Format (OpenAI-Compatible)
**System Message:**
```python
{"role": "system", "content": "You are a helpful assistant."}
```
**User Message:**
```python
{"role": "user", "content": "What is the weather?"}
```
**Assistant Message:**
```python
{"role": "assistant", "content": "The weather is sunny."}
```
**Tool Call:**
```python
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"location": "San Francisco"}'
}
}
]
}
```
**Tool Response:**
```python
{
"role": "tool",
"tool_call_id": "call_123",
"content": '{"temperature": 72, "condition": "sunny"}'
}
```
### 5. Session Management
**Session ID:**
- Purpose: Track entire conversation
- Format: UUID (e.g., `"550e8400-e29b-41d4-a716-446655440000"`)
- Persistence: Same for all requests in a conversation
- Location: `payload["metadata"]["sessionId"]`
**Prompt ID:**
- Purpose: Track individual request
- Format: UUID (unique per request)
- Persistence: New for each API call
- Location: `payload["metadata"]["promptId"]`
**Example:**
```python
import uuid
# Create once per conversation
session_id = str(uuid.uuid4())
# Create for each request
prompt_id = str(uuid.uuid4())
payload = {
"model": "qwen-plus",
"messages": [...],
"metadata": {
"sessionId": session_id,
"promptId": prompt_id
}
}
```
### 6. Token Management
**Token Refresh Flow:**
```
1. Check if token expired (expiry_date < current_time)
2. If expired, call refresh endpoint
3. Update stored credentials
4. Retry original request with new token
```
**Refresh Request:**
```python
POST https://chat.qwen.ai/api/v1/oauth2/token
Content-Type: application/x-www-form-urlencoded
grant_type=refresh_token&
refresh_token=YOUR_REFRESH_TOKEN&
client_id=f0304373b74a44d2b584a3fb70ca9e56
```
**Refresh Response:**
```json
{
"access_token": "new_access_token",
"token_type": "Bearer",
"expires_in": 3600,
"refresh_token": "new_refresh_token_or_same",
"resource_url": "https://dashscope.aliyuncs.com/compatible-mode"
}
```
### 7. Streaming Response Format
**SSE Format:**
```
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-plus","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-plus","choices":[{"index":0,"delta":{"content":" there"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-plus","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15}}
data: [DONE]
```
**Parsing:**
```python
for line in response.iter_lines():
if line:
line = line.decode('utf-8')
if line.startswith('data: '):
data = line[6:] # Remove 'data: ' prefix
if data == '[DONE]':
break
chunk = json.loads(data)
# Process chunk
```
### 8. Cache Control (Optional Enhancement)
**Enable caching for system message:**
```python
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a helpful assistant.",
"cache_control": {"type": "ephemeral"}
}
]
}
```
**Enable caching for last message (streaming):**
```python
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is the weather?",
"cache_control": {"type": "ephemeral"}
}
]
}
```
**Enable caching for last tool:**
```python
tools = [
{"type": "function", "function": {...}},
{
"type": "function",
"function": {...},
"cache_control": {"type": "ephemeral"}
}
]
```
### 9. Error Handling
**401/403 Errors:**
```python
if response.status_code in (401, 403):
# Token expired - refresh and retry
refresh_access_token()
retry_request()
```
**400 on Refresh:**
```python
if response.status_code == 400:
# Refresh token expired - need full re-authentication
raise Exception("Please re-authenticate")
```
### 10. Complete Minimal Example
```python
import requests
import json
import uuid
# Load credentials
with open("~/.qwen/oauth_creds.json") as f:
creds = json.load(f)
# Build request
headers = {
"Authorization": f"Bearer {creds['access_token']}",
"Content-Type": "application/json",
"User-Agent": "QwenCode/1.0.0",
"X-DashScope-CacheControl": "enable",
"X-DashScope-UserAgent": "QwenCode/1.0.0",
"X-DashScope-AuthType": "qwen-oauth",
}
payload = {
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "Hello!"}
],
"metadata": {
"sessionId": str(uuid.uuid4()),
"promptId": str(uuid.uuid4())
}
}
# Send request
response = requests.post(
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
headers=headers,
json=payload
)
# Handle response
if response.status_code == 200:
result = response.json()
print(result["choices"][0]["message"]["content"])
elif response.status_code in (401, 403):
# Refresh token and retry
pass
```
---
## Key Differences from Standard OpenAI API
1. **Authentication**: OAuth2 with refresh tokens instead of static API keys
2. **Base URL**: Dynamic based on `resource_url` from OAuth
3. **Headers**: Additional `X-DashScope-*` headers
4. **Metadata**: Session and prompt tracking via `metadata` field
5. **Cache Control**: DashScope-specific caching mechanism
6. **Token Refresh**: Automatic refresh flow on 401/403
---
## Important Notes
1. **No Initial System Prompt Required**: Unlike some APIs, there's no mandatory initial handshake or system prompt. You can start with any message.
2. **Session Persistence**: The `sessionId` should remain constant throughout a conversation to maintain context.
3. **Token Expiry**: Always check `expiry_date` before making requests and refresh proactively.
4. **Resource URL**: The API endpoint can change based on the `resource_url` in OAuth credentials. Always use the dynamic endpoint.
5. **OpenAI Compatibility**: The message format is 100% OpenAI-compatible. No conversion needed.
6. **Streaming**: When streaming, always include `stream_options: {include_usage: true}` to get token usage in the final chunk.
7. **Error Handling**: Implement automatic retry with token refresh for 401/403 errors.
---
## Testing Checklist
- [ ] Load credentials from `~/.qwen/oauth_creds.json`
- [ ] Check token expiry before request
- [ ] Build headers with all required fields
- [ ] Generate unique session_id and prompt_id
- [ ] Send request to correct endpoint
- [ ] Handle 401/403 with token refresh
- [ ] Parse response correctly
- [ ] Test streaming mode
- [ ] Test with function calling
- [ ] Verify session persistence across requests
---
## Additional Resources
- Full implementation: See `QWEN_API_IMPLEMENTATION_GUIDE.md`
- OAuth2 flow: See `packages/core/src/qwen/qwenOAuth2.ts`
- Request building: See `packages/core/src/core/openaiContentGenerator/provider/dashscope.ts`
- Message conversion: See `packages/core/src/core/openaiContentGenerator/converter.ts`
# Qwen API Implementation Guide
This document provides a complete guide for implementing Qwen API requests in Python, based on the exact implementation in the Qwen Code codebase.
## Table of Contents
1. [OAuth2 Authentication](#oauth2-authentication)
2. [API Request Format](#api-request-format)
3. [Session Management](#session-management)
4. [Message Format Conversion](#message-format-conversion)
5. [Complete Python Implementation](#complete-python-implementation)
---
## OAuth2 Authentication
### OAuth2 Configuration
```python
# OAuth Endpoints
QWEN_OAUTH_BASE_URL = "https://chat.qwen.ai"
QWEN_OAUTH_DEVICE_CODE_ENDPOINT = f"{QWEN_OAUTH_BASE_URL}/api/v1/oauth2/device/code"
QWEN_OAUTH_TOKEN_ENDPOINT = f"{QWEN_OAUTH_BASE_URL}/api/v1/oauth2/token"
# OAuth Client Configuration
QWEN_OAUTH_CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
QWEN_OAUTH_SCOPE = "openid profile email model.completion"
QWEN_OAUTH_GRANT_TYPE = "urn:ietf:params:oauth:grant-type:device_code"
```
### Token Storage
Tokens are stored in: `~/.qwen/oauth_creds.json`
```json
{
"access_token": "your_access_token",
"refresh_token": "your_refresh_token",
"id_token": "optional_id_token",
"expiry_date": 1234567890000,
"token_type": "Bearer",
"resource_url": "https://dashscope.aliyuncs.com/compatible-mode"
}
```
### Token Refresh
When the access token expires (401/403 errors), refresh it:
```python
import requests
def refresh_access_token(refresh_token: str) -> dict:
"""Refresh the access token using the refresh token."""
body_data = {
"grant_type": "refresh_token",
"refresh_token": refresh_token,
"client_id": QWEN_OAUTH_CLIENT_ID,
}
response = requests.post(
QWEN_OAUTH_TOKEN_ENDPOINT,
headers={
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
},
data=body_data,
)
if response.status_code == 400 or response.status_code == 401:
# Refresh token expired - need to re-authenticate
raise Exception("Refresh token expired. Please re-authenticate.")
response.raise_for_status()
token_data = response.json()
# Update credentials
credentials = {
"access_token": token_data["access_token"],
"token_type": token_data["token_type"],
"refresh_token": token_data.get("refresh_token", refresh_token), # Use new if provided
"resource_url": token_data.get("resource_url"),
"expiry_date": int(time.time() * 1000) + token_data["expires_in"] * 1000,
}
return credentials
```
---
## API Request Format
### Base URL
The API endpoint is determined by the `resource_url` from OAuth credentials:
```python
def get_api_endpoint(resource_url: str = None) -> str:
"""Get the API endpoint URL."""
base_endpoint = resource_url or "https://dashscope.aliyuncs.com/compatible-mode"
# Normalize URL: add protocol if missing, ensure /v1 suffix
if not base_endpoint.startswith("http"):
base_endpoint = f"https://{base_endpoint}"
if not base_endpoint.endswith("/v1"):
base_endpoint = f"{base_endpoint}/v1"
return base_endpoint
```
### Required Headers
```python
def build_headers(access_token: str, auth_type: str = "qwen-oauth") -> dict:
"""Build request headers for Qwen API."""
version = "1.0.0" # Your application version
user_agent = f"QwenCode/{version} ({platform.system()}; {platform.machine()})"
headers = {
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/json",
"Accept": "application/json",
"User-Agent": user_agent,
"X-DashScope-CacheControl": "enable",
"X-DashScope-UserAgent": user_agent,
"X-DashScope-AuthType": auth_type,
"x-request-id": str(uuid.uuid4()), # Optional but recommended
}
return headers
```
### Request Metadata
Qwen API supports metadata for session tracking:
```python
def build_metadata(session_id: str, prompt_id: str, channel: str = None) -> dict:
"""Build metadata for request tracking."""
metadata = {
"sessionId": session_id,
"promptId": prompt_id,
}
if channel:
metadata["channel"] = channel
return {"metadata": metadata}
```
---
## Session Management
### Session ID
- **Purpose**: Track conversation history across multiple requests
- **Format**: UUID or any unique identifier
- **Persistence**: Should remain constant for the entire conversation
- **Generation**: Create once at conversation start
```python
import uuid
session_id = str(uuid.uuid4()) # Generate once per conversation
```
### Prompt ID
- **Purpose**: Track individual requests within a session
- **Format**: UUID or unique identifier
- **Persistence**: New for each request
- **Generation**: Create for each API call
```python
prompt_id = str(uuid.uuid4()) # Generate for each request
```
---
## Message Format Conversion
### OpenAI to Qwen Format
Qwen API uses OpenAI-compatible format. The conversion is straightforward:
#### System Message
```python
# OpenAI format
{
"role": "system",
"content": "You are a helpful assistant."
}
# Qwen format (same)
{
"role": "system",
"content": "You are a helpful assistant."
}
```
#### User Message
```python
# OpenAI format
{
"role": "user",
"content": "Hello, how are you?"
}
# Qwen format (same)
{
"role": "user",
"content": "Hello, how are you?"
}
```
#### Assistant Message
```python
# OpenAI format
{
"role": "assistant",
"content": "I'm doing well, thank you!"
}
# Qwen format (same)
{
"role": "assistant",
"content": "I'm doing well, thank you!"
}
```
#### Tool Calls (Function Calling)
```python
# OpenAI format
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"location": "San Francisco"}'
}
}
]
}
# Qwen format (same)
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"location": "San Francisco"}'
}
}
]
}
```
#### Tool Response
```python
# OpenAI format
{
"role": "tool",
"tool_call_id": "call_123",
"content": '{"temperature": 72, "condition": "sunny"}'
}
# Qwen format (same)
{
"role": "tool",
"tool_call_id": "call_123",
"content": '{"temperature": 72, "condition": "sunny"}'
}
```
### Cache Control (DashScope-specific)
For prompt caching, add `cache_control` to the last text part of specific messages:
```python
# System message with cache control
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a helpful assistant.",
"cache_control": {"type": "ephemeral"}
}
]
}
# Last history message with cache control (for streaming)
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is the weather?",
"cache_control": {"type": "ephemeral"}
}
]
}
```
For tools, add cache control to the last tool:
```python
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather information",
"parameters": {...}
}
},
{
"type": "function",
"function": {
"name": "search",
"description": "Search the web",
"parameters": {...}
},
"cache_control": {"type": "ephemeral"} # Add to last tool
}
]
```
---
## Complete Python Implementation
### Full Implementation Example
```python
import json
import time
import uuid
import platform
import requests
from typing import Dict, List, Optional, Generator
from pathlib import Path
class QwenAPIClient:
"""Complete Qwen API client implementation."""
# OAuth Configuration
OAUTH_BASE_URL = "https://chat.qwen.ai"
OAUTH_TOKEN_ENDPOINT = f"{OAUTH_BASE_URL}/api/v1/oauth2/token"
CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
DEFAULT_API_BASE = "https://dashscope.aliyuncs.com/compatible-mode/v1"
# Credentials file
CREDS_FILE = Path.home() / ".qwen" / "oauth_creds.json"
def __init__(self, session_id: Optional[str] = None):
"""Initialize the client."""
self.session_id = session_id or str(uuid.uuid4())
self.credentials = self._load_credentials()
self.api_base = self._get_api_endpoint()
def _load_credentials(self) -> Dict:
"""Load credentials from file."""
if not self.CREDS_FILE.exists():
raise Exception("No credentials found. Please authenticate first.")
with open(self.CREDS_FILE, 'r') as f:
return json.load(f)
def _save_credentials(self, credentials: Dict):
"""Save credentials to file."""
self.CREDS_FILE.parent.mkdir(parents=True, exist_ok=True)
with open(self.CREDS_FILE, 'w') as f:
json.dump(credentials, f, indent=2)
self.credentials = credentials
def _get_api_endpoint(self) -> str:
"""Get the API endpoint from credentials."""
resource_url = self.credentials.get("resource_url", self.DEFAULT_API_BASE)
# Normalize URL
if not resource_url.startswith("http"):
resource_url = f"https://{resource_url}"
if not resource_url.endswith("/v1"):
resource_url = f"{resource_url}/v1"
return resource_url
def _is_token_expired(self) -> bool:
"""Check if the access token is expired."""
expiry_date = self.credentials.get("expiry_date", 0)
# Add 60 second buffer
return time.time() * 1000 >= (expiry_date - 60000)
def _refresh_token(self):
"""Refresh the access token."""
refresh_token = self.credentials.get("refresh_token")
if not refresh_token:
raise Exception("No refresh token available")
body_data = {
"grant_type": "refresh_token",
"refresh_token": refresh_token,
"client_id": self.CLIENT_ID,
}
response = requests.post(
self.OAUTH_TOKEN_ENDPOINT,
headers={
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
},
data=body_data,
)
if response.status_code in (400, 401):
raise Exception("Refresh token expired. Please re-authenticate.")
response.raise_for_status()
token_data = response.json()
# Update credentials
new_credentials = {
"access_token": token_data["access_token"],
"token_type": token_data["token_type"],
"refresh_token": token_data.get("refresh_token", refresh_token),
"resource_url": token_data.get("resource_url", self.credentials.get("resource_url")),
"expiry_date": int(time.time() * 1000) + token_data["expires_in"] * 1000,
}
self._save_credentials(new_credentials)
self.api_base = self._get_api_endpoint()
def _get_valid_token(self) -> str:
"""Get a valid access token, refreshing if necessary."""
if self._is_token_expired():
self._refresh_token()
return self.credentials["access_token"]
def _build_headers(self) -> Dict[str, str]:
"""Build request headers."""
access_token = self._get_valid_token()
version = "1.0.0"
user_agent = f"QwenCode/{version} ({platform.system()}; {platform.machine()})"
return {
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/json",
"Accept": "application/json",
"User-Agent": user_agent,
"X-DashScope-CacheControl": "enable",
"X-DashScope-UserAgent": user_agent,
"X-DashScope-AuthType": "qwen-oauth",
"x-request-id": str(uuid.uuid4()),
}
def chat_completion(
self,
messages: List[Dict],
model: str = "qwen-plus",
stream: bool = False,
temperature: Optional[float] = None,
max_tokens: Optional[int] = None,
tools: Optional[List[Dict]] = None,
**kwargs
) -> Dict:
"""
Send a chat completion request.
Args:
messages: List of message dictionaries in OpenAI format
model: Model name to use
stream: Whether to stream the response
temperature: Sampling temperature (0-2)
max_tokens: Maximum tokens to generate
tools: List of tool definitions
**kwargs: Additional parameters
Returns:
Response dictionary or generator for streaming
"""
prompt_id = str(uuid.uuid4())
# Build request payload
payload = {
"model": model,
"messages": messages,
"metadata": {
"sessionId": self.session_id,
"promptId": prompt_id,
}
}
# Add optional parameters
if temperature is not None:
payload["temperature"] = temperature
if max_tokens is not None:
payload["max_tokens"] = max_tokens
if tools:
payload["tools"] = tools
if stream:
payload["stream"] = True
payload["stream_options"] = {"include_usage": True}
# Add any additional parameters
payload.update(kwargs)
# Make request with retry on auth error
return self._make_request(payload, stream)
def _make_request(self, payload: Dict, stream: bool) -> Dict:
"""Make the API request with automatic token refresh."""
url = f"{self.api_base}/chat/completions"
try:
headers = self._build_headers()
if stream:
return self._stream_request(url, headers, payload)
else:
response = requests.post(url, headers=headers, json=payload)
# Handle auth errors with token refresh
if response.status_code in (401, 403):
self._refresh_token()
headers = self._build_headers()
response = requests.post(url, headers=headers, json=payload)
response.raise_for_status()
return response.json()
except requests.exceptions.RequestException as e:
raise Exception(f"API request failed: {str(e)}")
def _stream_request(self, url: str, headers: Dict, payload: Dict) -> Generator:
"""Handle streaming requests."""
response = requests.post(
url,
headers=headers,
json=payload,
stream=True
)
# Handle auth errors with token refresh
if response.status_code in (401, 403):
self._refresh_token()
headers = self._build_headers()
response = requests.post(
url,
headers=headers,
json=payload,
stream=True
)
response.raise_for_status()
for line in response.iter_lines():
if line:
line = line.decode('utf-8')
if line.startswith('data: '):
data = line[6:] # Remove 'data: ' prefix
if data == '[DONE]':
break
try:
yield json.loads(data)
except json.JSONDecodeError:
continue
# Usage Example
def main():
"""Example usage of the Qwen API client."""
# Initialize client (creates or reuses session)
client = QwenAPIClient()
# Simple chat completion
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
]
# Non-streaming request
response = client.chat_completion(
messages=messages,
model="qwen-plus",
temperature=0.7,
max_tokens=1000
)
print("Response:", response["choices"][0]["message"]["content"])
# Streaming request
print("\nStreaming response:")
for chunk in client.chat_completion(
messages=messages,
model="qwen-plus",
stream=True
):
if "choices" in chunk and len(chunk["choices"]) > 0:
delta = chunk["choices"][0].get("delta", {})
if "content" in delta:
print(delta["content"], end="", flush=True)
print("\n")
# With function calling
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
}
}
]
messages = [
{"role": "user", "content": "What's the weather in San Francisco?"}
]
response = client.chat_completion(
messages=messages,
model="qwen-plus",
tools=tools
)
print("Tool call:", response["choices"][0]["message"].get("tool_calls"))
if __name__ == "__main__":
main()
```
---
## Key Implementation Details
### 1. **No Initial System Prompt Required**
- System prompts are optional and sent as regular messages
- No special initialization handshake needed
### 2. **Session Management**
- Session ID: Persistent across conversation
- Prompt ID: Unique per request
- Both sent in `metadata` field of request payload
### 3. **Token Management**
- Access tokens expire (check `expiry_date`)
- Refresh automatically on 401/403 errors
- Store credentials in `~/.qwen/oauth_creds.json`
### 4. **Headers**
- `Authorization`: Bearer token (required)
- `X-DashScope-*`: Provider-specific headers
- `User-Agent`: Application identification
- `x-request-id`: Request tracking (optional)
### 5. **Cache Control**
- Enabled by default via `X-DashScope-CacheControl: enable`
- Can add `cache_control` to specific message parts for fine-grained control
- Applied to system message and last history message in streaming mode
### 6. **Error Handling**
- 401/403: Token expired → refresh and retry
- 400 on refresh: Refresh token expired → re-authenticate
- Network errors: Implement exponential backoff
### 7. **Streaming**
- Set `stream: true` and `stream_options: {include_usage: true}`
- Parse SSE format: `data: {json}\n\n`
- Handle `[DONE]` marker
---
## Testing Your Implementation
```python
# Test token validity
client = QwenAPIClient()
print("Token valid:", not client._is_token_expired())
# Test simple request
response = client.chat_completion(
messages=[{"role": "user", "content": "Hello"}],
model="qwen-plus"
)
print("Response:", response)
# Test streaming
for chunk in client.chat_completion(
messages=[{"role": "user", "content": "Count to 5"}],
model="qwen-plus",
stream=True
):
print(chunk)
```
---
## Summary
This implementation mimics exactly how Qwen Code sends requests:
1. **OAuth2 flow** for authentication with automatic token refresh
2. **OpenAI-compatible message format** (no conversion needed)
3. **Session tracking** via metadata (sessionId + promptId)
4. **DashScope-specific headers** for cache control and tracking
5. **Automatic retry** on authentication errors
6. **Streaming support** with SSE parsing
The key insight is that Qwen API is OpenAI-compatible, so you can use standard OpenAI message formats directly. The main differences are:
- OAuth2 authentication instead of API keys
- DashScope-specific headers
- Metadata for session tracking
- Dynamic endpoint from OAuth credentials
# Qwen API Request Flow Diagram
This document provides a visual representation of how Qwen Code sends API requests.
## Complete Request Flow
```
┌─────────────────────────────────────────────────────────────────────┐
│ USER INITIATES REQUEST │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 1. SESSION MANAGEMENT │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ • Generate/Reuse Session ID (UUID) │ │
│ │ • Generate new Prompt ID (UUID) for this request │ │
│ │ • Session ID persists across conversation │ │
│ │ • Prompt ID unique per request │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 2. TOKEN MANAGEMENT │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Load credentials from ~/.qwen/oauth_creds.json │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ Check if token expired (expiry_date < current_time) │ │
│ │ │ │ │
│ │ ├─── YES ──▶ Refresh Token │ │
│ │ │ │ │ │
│ │ │ ▼ │ │
│ │ │ POST /api/v1/oauth2/token │ │
│ │ │ grant_type=refresh_token │ │
│ │ │ │ │ │
│ │ │ ▼ │ │
│ │ │ Save new credentials │ │
│ │ │ │ │ │
│ │ └─── NO ───────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ Use access_token for request │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 3. BUILD REQUEST HEADERS │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Authorization: Bearer {access_token} │ │
│ │ Content-Type: application/json │ │
│ │ Accept: application/json │ │
│ │ User-Agent: QwenCode/{version} ({platform}; {arch}) │ │
│ │ X-DashScope-CacheControl: enable │ │
│ │ X-DashScope-UserAgent: QwenCode/{version} ({platform}) │ │
│ │ X-DashScope-AuthType: qwen-oauth │ │
│ │ x-request-id: {uuid} │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 4. BUILD REQUEST PAYLOAD │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ { │ │
│ │ "model": "qwen-plus", │ │
│ │ "messages": [ │ │
│ │ {"role": "system", "content": "..."}, │ │
│ │ {"role": "user", "content": "..."} │ │
│ │ ], │ │
│ │ "stream": false, │ │
│ │ "temperature": 0.7, │ │
│ │ "max_tokens": 1000, │ │
│ │ "metadata": { │ │
│ │ "sessionId": "{session_uuid}", │ │
│ │ "promptId": "{prompt_uuid}" │ │
│ │ } │ │
│ │ } │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 5. DETERMINE API ENDPOINT │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Get resource_url from credentials │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ Default: https://dashscope.aliyuncs.com/compatible-mode │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ Normalize: Add https:// if missing, ensure /v1 suffix │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ Final: https://dashscope.aliyuncs.com/compatible-mode/v1 │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 6. SEND HTTP REQUEST │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ POST {api_base}/chat/completions │ │
│ │ Headers: [see step 3] │ │
│ │ Body: [see step 4] │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 7. HANDLE RESPONSE │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Status Code? │ │
│ │ │ │ │
│ │ ├─── 200 OK ──▶ Parse response, return to user │ │
│ │ │ │ │
│ │ ├─── 401/403 ──▶ Token expired │ │
│ │ │ │ │ │
│ │ │ ▼ │ │
│ │ │ Refresh token (step 2) │ │
│ │ │ │ │ │
│ │ │ ▼ │ │
│ │ │ Retry request (step 6) │ │
│ │ │ │ │
│ │ └─── Other ──▶ Handle error, report to user │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Streaming Request Flow
```
┌─────────────────────────────────────────────────────────────────────┐
│ STREAMING REQUEST (Steps 1-6 same) │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ PAYLOAD DIFFERENCES │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ { │ │
│ │ ... │ │
│ │ "stream": true, ← Enable streaming │ │
│ │ "stream_options": { │ │
│ │ "include_usage": true ← Get token usage │ │
│ │ }, │ │
│ │ ... │ │
│ │ } │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ PARSE SSE STREAM │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ For each line in response.iter_lines(): │ │
│ │ │ │ │
│ │ ├─ Line starts with "data: " ? │ │
│ │ │ │ │ │
│ │ │ ├─ YES ─▶ Extract JSON after "data: " │ │
│ │ │ │ │ │ │
│ │ │ │ ├─ Is "[DONE]" ? ─▶ End stream │ │
│ │ │ │ │ │ │
│ │ │ │ └─ Parse JSON chunk │ │
│ │ │ │ │ │ │
│ │ │ │ ├─ Extract delta.content │ │
│ │ │ │ ├─ Extract finish_reason │ │
│ │ │ │ └─ Extract usage (final chunk) │ │
│ │ │ │ │ │
│ │ │ └─ NO ─▶ Skip line │ │
│ │ │ │ │
│ │ └─ Yield chunk to user │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Message Format Conversion (OpenAI → Qwen)
```
┌─────────────────────────────────────────────────────────────────────┐
│ NO CONVERSION NEEDED! │
│ │
│ Qwen API is 100% OpenAI-compatible │
│ │
│ OpenAI Format Qwen Format │
│ ───────────── ─────────── │
│ {"role": "user", ═══▶ {"role": "user", │
│ "content": "..."} "content": "..."} │
│ │
│ Same for: │
│ • System messages │
│ • Assistant messages │
│ • Tool calls │
│ • Tool responses │
│ • Function definitions │
└─────────────────────────────────────────────────────────────────────┘
```
## Cache Control Flow (Optional)
```
┌─────────────────────────────────────────────────────────────────────┐
│ CACHE CONTROL (Optional Enhancement) │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ 1. System Message Caching │ │
│ │ { │ │
│ │ "role": "system", │ │
│ │ "content": [ │ │
│ │ { │ │
│ │ "type": "text", │ │
│ │ "text": "You are...", │ │
│ │ "cache_control": {"type": "ephemeral"} ← Cache │ │
│ │ } │ │
│ │ ] │ │
│ │ } │ │
│ │ │ │
│ │ 2. Last History Message Caching (streaming only) │ │
│ │ { │ │
│ │ "role": "user", │ │
│ │ "content": [ │ │
│ │ { │ │
│ │ "type": "text", │ │
│ │ "text": "What is...", │ │
│ │ "cache_control": {"type": "ephemeral"} ← Cache │ │
│ │ } │ │
│ │ ] │ │
│ │ } │ │
│ │ │ │
│ │ 3. Last Tool Caching (streaming only) │ │
│ │ tools = [ │ │
│ │ {...}, │ │
│ │ { │ │
│ │ "type": "function", │ │
│ │ "function": {...}, │ │
│ │ "cache_control": {"type": "ephemeral"} ← Cache │ │
│ │ } │ │
│ │ ] │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Error Handling Flow
```
┌─────────────────────────────────────────────────────────────────────┐
│ ERROR HANDLING │
│ │
│ Response Status Code │
│ │ │
│ ├─── 200 ──▶ Success, parse response │
│ │ │
│ ├─── 401/403 ──▶ Token expired │
│ │ │ │
│ │ ▼ │
│ │ POST /api/v1/oauth2/token │
│ │ grant_type=refresh_token │
│ │ │ │
│ │ ├─── 200 ──▶ Token refreshed │
│ │ │ │ │
│ │ │ ▼ │
│ │ │ Save credentials │
│ │ │ │ │
│ │ │ ▼ │
│ │ │ Retry original request │
│ │ │ │
│ │ └─── 400/401 ──▶ Refresh token expired │
│ │ │ │
│ │ ▼ │
│ │ Re-authenticate required │
│ │ │
│ ├─── 429 ──▶ Rate limit, implement backoff │
│ │ │
│ ├─── 500+ ──▶ Server error, retry with backoff │
│ │ │
│ └─── Other ──▶ Handle error, report to user │
└─────────────────────────────────────────────────────────────────────┘
```
## Token Lifecycle
```
┌─────────────────────────────────────────────────────────────────────┐
│ TOKEN LIFECYCLE │
│ │
│ 1. Initial Authentication (Device Flow) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ User authenticates via browser │ │
│ │ ▼ │ │
│ │ Receive access_token + refresh_token │ │
│ │ ▼ │ │
│ │ Save to ~/.qwen/oauth_creds.json │ │
│ │ ▼ │ │
│ │ Set expiry_date = now + expires_in │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ 2. Token Usage │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Before each request: │ │
│ │ Check if expiry_date < current_time │ │
│ │ If expired: refresh token │ │
│ │ Use access_token in Authorization header │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ 3. Token Refresh │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ POST /api/v1/oauth2/token │ │
│ │ grant_type=refresh_token │ │
│ │ refresh_token={current_refresh_token} │ │
│ │ ▼ │ │
│ │ Receive new access_token │ │
│ │ Optionally receive new refresh_token │ │
│ │ ▼ │ │
│ │ Update ~/.qwen/oauth_creds.json │ │
│ │ ▼ │ │
│ │ Set new expiry_date │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ 4. Token Expiration │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ If refresh fails with 400/401: │ │
│ │ Refresh token expired │ │
│ │ ▼ │ │
│ │ Clear credentials │ │
│ │ ▼ │ │
│ │ User must re-authenticate │ │
│ └──────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Session Management
```
┌─────────────────────────────────────────────────────────────────────┐
│ SESSION MANAGEMENT │
│ │
│ Session ID (Conversation Level) │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ • Generated once at conversation start │ │
│ │ • Format: UUID (e.g., "550e8400-e29b-41d4-a716-446655440000") │ │
│ │ • Persists across all requests in conversation │ │
│ │ • Sent in metadata.sessionId │ │
│ │ • Purpose: Track conversation history │ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
│ Prompt ID (Request Level) │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ • Generated for each API request │ │
│ │ • Format: UUID (unique per request) │ │
│ │ • Sent in metadata.promptId │ │
│ │ • Purpose: Track individual requests │ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
│ Example Flow: │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ Conversation Start │ │
│ │ session_id = uuid4() # "abc-123" │ │
│ │ │ │
│ │ Request 1: │ │
│ │ prompt_id = uuid4() # "def-456" │ │
│ │ metadata = {sessionId: "abc-123", promptId: "def-456"} │ │
│ │ │ │
│ │ Request 2: │ │
│ │ prompt_id = uuid4() # "ghi-789" │ │
│ │ metadata = {sessionId: "abc-123", promptId: "ghi-789"} │ │
│ │ │ │
│ │ Request 3: │ │
│ │ prompt_id = uuid4() # "jkl-012" │ │
│ │ metadata = {sessionId: "abc-123", promptId: "jkl-012"} │ │
│ └────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Key Takeaways
1. **OpenAI Compatibility**: No message format conversion needed
2. **OAuth2 Flow**: Token refresh on 401/403, re-auth on refresh failure
3. **Session Tracking**: sessionId (conversation) + promptId (request)
4. **Dynamic Endpoint**: Use resource_url from OAuth credentials
5. **Headers**: Include DashScope-specific headers for caching and tracking
6. **Streaming**: Parse SSE format, handle [DONE] marker
7. **Error Handling**: Automatic retry with token refresh on auth errors
# Qwen API Implementation Checklist
Use this checklist to ensure your Qwen API implementation is complete and correct.
## ✅ Prerequisites
- [ ] Python 3.7+ installed
- [ ] `requests` library installed (`pip install requests`)
- [ ] Valid OAuth2 credentials obtained
- [ ] Credentials stored in `~/.qwen/oauth_creds.json`
## ✅ Credentials Setup
- [ ] Credentials file exists at correct location
- [ ] File contains all required fields:
- [ ] `access_token`
- [ ] `refresh_token`
- [ ] `expiry_date`
- [ ] `token_type`
- [ ] `resource_url` (optional but recommended)
- [ ] File permissions are secure (readable only by user)
- [ ] Credentials are valid (not expired)
## ✅ Basic Implementation
### Token Management
- [ ] Load credentials from file
- [ ] Check token expiry before requests
- [ ] Implement token refresh logic
- [ ] Handle refresh token expiration
- [ ] Save updated credentials after refresh
- [ ] Use 60-second buffer for expiry check
### Headers
- [ ] `Authorization: Bearer {token}` header
- [ ] `Content-Type: application/json` header
- [ ] `Accept: application/json` header
- [ ] `User-Agent` header with app name/version
- [ ] `X-DashScope-CacheControl: enable` header
- [ ] `X-DashScope-UserAgent` header
- [ ] `X-DashScope-AuthType: qwen-oauth` header
- [ ] Optional: `x-request-id` header with UUID
### Endpoint Configuration
- [ ] Extract `resource_url` from credentials
- [ ] Default to `https://dashscope.aliyuncs.com/compatible-mode`
- [ ] Add `https://` if protocol missing
- [ ] Ensure `/v1` suffix
- [ ] Use `/chat/completions` endpoint
### Request Payload
- [ ] `model` field (e.g., "qwen-plus")
- [ ] `messages` array with proper format
- [ ] Optional: `temperature` parameter
- [ ] Optional: `max_tokens` parameter
- [ ] Optional: `metadata` with sessionId and promptId
- [ ] For streaming: `stream: true`
- [ ] For streaming: `stream_options: {include_usage: true}`
## ✅ Session Management
- [ ] Generate session ID once per conversation (UUID)
- [ ] Generate new prompt ID for each request (UUID)
- [ ] Include both in `metadata` field
- [ ] Reuse session ID across conversation turns
- [ ] Store session ID for conversation persistence
## ✅ Message Format
### System Messages
- [ ] Use `role: "system"`
- [ ] Include content as string or array
### User Messages
- [ ] Use `role: "user"`
- [ ] Include content as string or array
- [ ] Support text content
- [ ] Support image content (if needed)
### Assistant Messages
- [ ] Use `role: "assistant"`
- [ ] Include content as string
- [ ] Support tool_calls array (if using functions)
### Tool Messages
- [ ] Use `role: "tool"`
- [ ] Include `tool_call_id`
- [ ] Include result in `content`
## ✅ Error Handling
### Authentication Errors
- [ ] Detect 401/403 status codes
- [ ] Trigger token refresh automatically
- [ ] Retry original request after refresh
- [ ] Handle refresh token expiration (400/401 on refresh)
- [ ] Prompt for re-authentication when needed
### Network Errors
- [ ] Implement retry logic with exponential backoff
- [ ] Handle connection timeouts
- [ ] Handle read timeouts
- [ ] Maximum retry attempts (e.g., 3)
### Rate Limiting
- [ ] Detect 429 status code
- [ ] Implement exponential backoff
- [ ] Respect Retry-After header if present
### API Errors
- [ ] Parse error responses
- [ ] Log error details
- [ ] Provide meaningful error messages to user
## ✅ Streaming Support
### Request Configuration
- [ ] Set `stream: true` in payload
- [ ] Set `stream_options: {include_usage: true}`
- [ ] Use streaming HTTP request
### Response Parsing
- [ ] Parse SSE format (Server-Sent Events)
- [ ] Handle `data: ` prefix
- [ ] Parse JSON from each chunk
- [ ] Handle `[DONE]` marker
- [ ] Extract `delta.content` from chunks
- [ ] Extract `finish_reason` from final chunk
- [ ] Extract `usage` metadata from final chunk
### Error Handling
- [ ] Handle streaming errors
- [ ] Clean up on connection failure
- [ ] Handle partial responses
## ✅ Advanced Features
### Cache Control (Optional)
- [ ] Add cache_control to system message
- [ ] Add cache_control to last history message (streaming)
- [ ] Add cache_control to last tool (streaming)
- [ ] Use `{type: "ephemeral"}` format
### Function Calling
- [ ] Define tools array with function schemas
- [ ] Include tools in request payload
- [ ] Parse tool_calls from response
- [ ] Execute functions locally
- [ ] Send tool results back to API
- [ ] Handle multi-turn function calling
### Multi-Turn Conversations
- [ ] Maintain message history
- [ ] Append assistant responses to history
- [ ] Use same session ID across turns
- [ ] Handle context window limits
## ✅ Testing
### Unit Tests
- [ ] Test credential loading
- [ ] Test token expiry checking
- [ ] Test token refresh
- [ ] Test header building
- [ ] Test payload building
- [ ] Test endpoint construction
### Integration Tests
- [ ] Test simple completion
- [ ] Test streaming completion
- [ ] Test multi-turn conversation
- [ ] Test function calling
- [ ] Test error handling
- [ ] Test token refresh flow
### Manual Testing
- [ ] Run test_qwen_api.py script
- [ ] Verify all tests pass
- [ ] Test with expired token
- [ ] Test with invalid credentials
- [ ] Test streaming output
- [ ] Test conversation context
## ✅ Production Readiness
### Security
- [ ] Credentials stored securely
- [ ] File permissions set correctly (600)
- [ ] Tokens not logged or exposed
- [ ] HTTPS used for all requests
- [ ] Sensitive data not in error messages
### Performance
- [ ] Credentials cached in memory
- [ ] Connection pooling enabled
- [ ] Timeouts configured appropriately
- [ ] Retry logic optimized
### Monitoring
- [ ] Request/response logging
- [ ] Error tracking
- [ ] Token refresh tracking
- [ ] API usage metrics
- [ ] Request ID tracking
### Documentation
- [ ] API usage documented
- [ ] Error codes documented
- [ ] Configuration options documented
- [ ] Examples provided
- [ ] Troubleshooting guide available
## ✅ Code Quality
### Code Organization
- [ ] Separate concerns (auth, requests, parsing)
- [ ] Reusable client class
- [ ] Clear function/method names
- [ ] Proper error handling throughout
### Code Style
- [ ] Consistent formatting
- [ ] Type hints (if using Python 3.5+)
- [ ] Docstrings for public methods
- [ ] Comments for complex logic
### Maintainability
- [ ] Configuration externalized
- [ ] Magic numbers avoided
- [ ] Constants defined
- [ ] Easy to update/extend
## 🎯 Quick Validation
Run these commands to validate your implementation:
```bash
# 1. Check credentials exist
test -f ~/.qwen/oauth_creds.json && echo "✓ Credentials found" || echo "✗ Credentials missing"
# 2. Validate credentials format
python3 -c "import json; json.load(open('~/.qwen/oauth_creds.json'.replace('~', '$HOME')))" && echo "✓ Valid JSON" || echo "✗ Invalid JSON"
# 3. Run test suite
python3 test_qwen_api.py
# 4. Test simple request
python3 -c "
from test_qwen_api import QwenAPITester
tester = QwenAPITester(verbose=False)
tester.load_credentials()
result = tester.test_simple_completion('Hello')
print('✓ API request successful')
"
```
## 📊 Implementation Status
Track your progress:
```
Total Items: 100+
Completed: ___
Remaining: ___
Progress: ___%
```
## 🚀 Next Steps
After completing this checklist:
1. **Test thoroughly** - Run all tests multiple times
2. **Handle edge cases** - Test with various inputs
3. **Monitor in production** - Track errors and performance
4. **Iterate** - Improve based on real usage
5. **Document** - Keep documentation up to date
## 📝 Notes
Use this space to track issues, questions, or customizations:
```
Date: ___________
Issue: ___________________________________________________________
Resolution: _______________________________________________________
Date: ___________
Issue: ___________________________________________________________
Resolution: _______________________________________________________
Date: ___________
Issue: ___________________________________________________________
Resolution: _______________________________________________________
```
---
**Checklist Version**: 1.0.0
**Last Updated**: 2026-04-19
**Compatible with**: Qwen API (DashScope OpenAI-compatible endpoint)
---
---
---
# APPENDIX A: Python Test Script
Below is the complete test script (`test_qwen_api.py`) for testing your Qwen API implementation:
```python
#!/usr/bin/env python3
"""
Qwen API Test Script
This script demonstrates and tests the complete Qwen API implementation,
including OAuth2 authentication, token refresh, and chat completions.
Usage:
python test_qwen_api.py
python test_qwen_api.py --verbose
"""
import json
import time
import uuid
import platform
import requests
from pathlib import Path
from typing import Dict, List, Optional
class QwenAPITester:
"""Test client for Qwen API with detailed logging."""
OAUTH_BASE_URL = "https://chat.qwen.ai"
OAUTH_TOKEN_ENDPOINT = f"{OAUTH_BASE_URL}/api/v1/oauth2/token"
CLIENT_ID = "f0304373b74a44d2b584a3fb70ca9e56"
DEFAULT_API_BASE = "https://dashscope.aliyuncs.com/compatible-mode/v1"
CREDS_FILE = Path.home() / ".qwen" / "oauth_creds.json"
def __init__(self, session_id: Optional[str] = None, verbose: bool = True):
self.session_id = session_id or str(uuid.uuid4())
self.verbose = verbose
self.credentials = None
self.api_base = None
def run_all_tests(self):
"""Run comprehensive test suite."""
print("\n" + "="*60)
print("QWEN API TEST SUITE")
print("="*60 + "\n")
try:
print("[TEST 1] Checking credentials...")
if not self.CREDS_FILE.exists():
print("❌ FAILED: No credentials found")
return
print("✓ PASSED: Credentials file exists")
print("\n[TEST 2] Loading credentials...")
self.load_credentials()
print("✓ PASSED: Credentials loaded")
print("\n[TEST 3] Testing simple completion...")
self.test_simple_completion()
print("✓ PASSED: Simple completion successful")
print("\n[TEST 4] Testing streaming...")
self.test_streaming_completion()
print("✓ PASSED: Streaming successful")
print("\n" + "="*60)
print("ALL TESTS PASSED ✓")
print("="*60 + "\n")
except Exception as e:
print(f"\n❌ TEST FAILED: {str(e)}")
if __name__ == "__main__":
import sys
verbose = "--verbose" in sys.argv or "-v" in sys.argv
tester = QwenAPITester(verbose=verbose)
tester.run_all_tests()
```
---
# APPENDIX B: Example Requests and Responses
## Example 1: Simple Chat Completion
**Request:**
```json
POST https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions
Headers:
Authorization: Bearer eyJhbGc...
Content-Type: application/json
X-DashScope-CacheControl: enable
X-DashScope-AuthType: qwen-oauth
Body:
{
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "What is 2+2?"}
],
"metadata": {
"sessionId": "550e8400-e29b-41d4-a716-446655440000",
"promptId": "6ba7b810-9dad-11d1-80b4-00c04fd430c8"
}
}
```
**Response:**
```json
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"model": "qwen-plus",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "2+2 equals 4."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 8,
"total_tokens": 18
}
}
```
## Example 2: Streaming Request
**Request:**
```json
{
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "Count from 1 to 3"}
],
"stream": true,
"stream_options": {"include_usage": true},
"metadata": {
"sessionId": "550e8400-e29b-41d4-a716-446655440000",
"promptId": "7ba7b810-9dad-11d1-80b4-00c04fd430c8"
}
}
```
**Response (SSE Stream):**
```
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"role":"assistant","content":"1"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"content":", 2"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"content":", 3"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":8,"completion_tokens":6,"total_tokens":14}}
data: [DONE]
```
## Example 3: Function Calling
**Request:**
```json
{
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "What's the weather in Paris?"}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City name"
}
},
"required": ["location"]
}
}
}
],
"metadata": {
"sessionId": "550e8400-e29b-41d4-a716-446655440000",
"promptId": "8ba7b810-9dad-11d1-80b4-00c04fd430c8"
}
}
```
**Response:**
```json
{
"id": "chatcmpl-124",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\": \"Paris\"}"
}
}
]
},
"finish_reason": "tool_calls"
}
],
"usage": {
"prompt_tokens": 45,
"completion_tokens": 15,
"total_tokens": 60
}
}
```
**Follow-up Request (with tool result):**
```json
{
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "What's the weather in Paris?"},
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\": \"Paris\"}"
}
}
]
},
{
"role": "tool",
"tool_call_id": "call_abc123",
"content": "{\"temperature\": 18, \"condition\": \"sunny\"}"
}
],
"tools": [...],
"metadata": {
"sessionId": "550e8400-e29b-41d4-a716-446655440000",
"promptId": "9ba7b810-9dad-11d1-80b4-00c04fd430c8"
}
}
```
**Final Response:**
```json
{
"id": "chatcmpl-125",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The weather in Paris is currently sunny with a temperature of 18°C."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 75,
"completion_tokens": 18,
"total_tokens": 93
}
}
```
---
# APPENDIX C: Error Codes and Handling
## HTTP Status Codes
| Code | Meaning | Action |
|------|---------|--------|
| 200 | Success | Parse response normally |
| 400 | Bad Request | Check request format, parameters |
| 401 | Unauthorized | Token expired - refresh and retry |
| 403 | Forbidden | Token invalid - refresh and retry |
| 429 | Too Many Requests | Rate limited - implement backoff |
| 500 | Internal Server Error | Retry with exponential backoff |
| 502 | Bad Gateway | Temporary issue - retry |
| 503 | Service Unavailable | Service down - retry later |
## Common Error Responses
### Token Expired (401)
```json
{
"error": {
"message": "Invalid authentication credentials",
"type": "invalid_request_error",
"code": "invalid_api_key"
}
}
```
**Action:** Refresh token and retry request
### Rate Limit (429)
```json
{
"error": {
"message": "Rate limit exceeded",
"type": "rate_limit_error",
"code": "rate_limit_exceeded"
}
}
```
**Action:** Wait and retry with exponential backoff
### Invalid Request (400)
```json
{
"error": {
"message": "Invalid request: missing required field 'model'",
"type": "invalid_request_error",
"code": "invalid_request"
}
}
```
**Action:** Fix request format and retry
## Token Refresh Errors
### Refresh Token Expired (400/401)
```json
{
"error": "invalid_grant",
"error_description": "The refresh token is invalid or expired"
}
```
**Action:** User must re-authenticate via OAuth2 device flow
### Invalid Client (401)
```json
{
"error": "invalid_client",
"error_description": "Client authentication failed"
}
```
**Action:** Check client ID configuration
---
# APPENDIX D: Configuration Examples
## Minimal Configuration
```python
# Minimal working configuration
config = {
"credentials_path": "~/.qwen/oauth_creds.json",
"model": "qwen-plus",
"timeout": 120,
}
```
## Production Configuration
```python
# Production-ready configuration
config = {
# Authentication
"credentials_path": "~/.qwen/oauth_creds.json",
"token_refresh_buffer": 60, # seconds before expiry
# API Settings
"model": "qwen-plus",
"base_url": None, # Use from credentials
"timeout": 120,
"max_retries": 3,
# Request Parameters
"temperature": 0.7,
"max_tokens": 2000,
"top_p": 0.9,
# Session Management
"session_id": None, # Auto-generate
"enable_metadata": True,
# Features
"enable_streaming": True,
"enable_cache_control": True,
"enable_function_calling": True,
# Logging
"log_requests": True,
"log_responses": False, # Don't log full responses
"log_level": "INFO",
}
```
## Environment Variables
```bash
# Optional environment variables
export QWEN_CREDENTIALS_PATH="~/.qwen/oauth_creds.json"
export QWEN_MODEL="qwen-plus"
export QWEN_TIMEOUT="120"
export QWEN_MAX_RETRIES="3"
export QWEN_LOG_LEVEL="INFO"
```
---
# APPENDIX E: Troubleshooting Guide
## Problem: Credentials Not Found
**Symptoms:**
- Error: "No credentials found"
- File not found error
**Solutions:**
1. Check file exists: `ls -la ~/.qwen/oauth_creds.json`
2. Verify file permissions: `chmod 600 ~/.qwen/oauth_creds.json`
3. Check file format: `cat ~/.qwen/oauth_creds.json | python -m json.tool`
4. Re-authenticate if needed
## Problem: Token Refresh Fails
**Symptoms:**
- 400/401 error on refresh
- "Refresh token expired" message
**Solutions:**
1. Delete credentials: `rm ~/.qwen/oauth_creds.json`
2. Re-authenticate via OAuth2 device flow
3. Check system time is correct
4. Verify network connectivity
## Problem: Streaming Not Working
**Symptoms:**
- No chunks received
- Connection hangs
- Incomplete responses
**Solutions:**
1. Verify `stream: true` in request
2. Add `stream_options: {include_usage: true}`
3. Check network/firewall settings
4. Increase timeout value
5. Test with non-streaming first
## Problem: Session Context Lost
**Symptoms:**
- Model doesn't remember previous messages
- Context not maintained
**Solutions:**
1. Use same `sessionId` for all requests
2. Include full message history in each request
3. Verify messages array is correct
4. Check for context window limits
## Problem: Rate Limiting
**Symptoms:**
- 429 status code
- "Rate limit exceeded" error
**Solutions:**
1. Implement exponential backoff
2. Reduce request frequency
3. Check rate limit headers
4. Consider upgrading plan
---
# APPENDIX F: Best Practices Summary
## Security Best Practices
1. **Never commit credentials** to version control
2. **Use environment variables** for sensitive data
3. **Set proper file permissions** (600 for credentials)
4. **Rotate tokens regularly** via refresh
5. **Use HTTPS only** for all requests
6. **Don't log tokens** in application logs
7. **Validate all inputs** before sending to API
## Performance Best Practices
1. **Cache credentials** in memory
2. **Reuse HTTP connections** (connection pooling)
3. **Implement request batching** where possible
4. **Use streaming** for long responses
5. **Set appropriate timeouts** (120s recommended)
6. **Monitor token usage** to optimize costs
7. **Enable cache control** for repeated prompts
## Code Quality Best Practices
1. **Separate concerns** (auth, requests, parsing)
2. **Use type hints** for better IDE support
3. **Write comprehensive tests** for all features
4. **Document public APIs** with docstrings
5. **Handle errors gracefully** with retries
6. **Log important events** for debugging
7. **Follow PEP 8** style guidelines
## API Usage Best Practices
1. **Include metadata** for tracking
2. **Use appropriate models** for tasks
3. **Set reasonable token limits** to control costs
4. **Implement retry logic** with backoff
5. **Monitor API responses** for errors
6. **Track usage metrics** for optimization
7. **Test thoroughly** before production
---
# APPENDIX G: Quick Reference Card
## Essential URLs
```
OAuth Token: https://chat.qwen.ai/api/v1/oauth2/token
API Endpoint: https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions
```
## Essential Headers
```
Authorization: Bearer {access_token}
Content-Type: application/json
X-DashScope-CacheControl: enable
X-DashScope-AuthType: qwen-oauth
```
## Minimal Request
```json
{
"model": "qwen-plus",
"messages": [{"role": "user", "content": "Hello"}]
}
```
## Token Refresh
```bash
POST https://chat.qwen.ai/api/v1/oauth2/token
grant_type=refresh_token&refresh_token={token}&client_id={id}
```
## Common Models
- `qwen-plus` - Balanced
- `qwen-max` - Most capable
- `qwen-turbo` - Fastest
- `qwen3-coder-plus` - Code specialist
## Error Handling
- 401/403 → Refresh token
- 429 → Backoff and retry
- 500+ → Retry with backoff
---
**END OF COMPLETE DOCUMENTATION**
**Document Version:** 1.0.0
**Last Updated:** 2026-04-19
**Total Pages:** 2,160+ lines
**File Size:** 81KB+
For the latest updates and additional resources, refer to the individual documentation files or visit the Qwen Code repository.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment