WorkingTitle Operational Handbook¶
Welcome to the comprehensive operational documentation for the WorkingTitle system. This handbook contains all the essential information for managing, monitoring, and maintaining the production infrastructure.
📊 System Overview¶
Production Infrastructure¶
- Server: 195.24.67.210 (Debian/Ubuntu)
- Domain: tu-du-dum.com
- SSL: Let's Encrypt (Auto-renewal)
- Database: PostgreSQL with automated backups
- Monitoring: Real-time health checks and alerting
Key Features¶
- ✅ Automated Database Backups - Daily backups with 2-tier rotation
- ✅ Production Stems Backup - Weekly audio file backups
- ✅ Health Monitoring - 24/7 system monitoring with email alerts
- ✅ Automated Cleanup - Log rotation and space management
- ✅ SSL/TLS Security - HTTPS with automatic certificate renewal
- ✅ Professional Documentation - This searchable handbook
🚨 Emergency Procedures¶
Database Recovery¶
# Quick database restore from backup
ssh root@195.24.67.210
cd /var/www/workingtitle/scripts
./database-restore-production.sh
System Health Check¶
# Check system status
ssh root@195.24.67.210
systemctl status workingtitle-monitor
systemctl status workingtitle-backup
Contact Information¶
- Primary Contact: admin@tu-du-dum.com
- Alert Email: text@workingtitle.ru
- Server Access: ssh root@195.24.67.210
📈 Recent Updates¶
Latest Improvements (2025-01-27)¶
- ✅ Centralized server health system in
server_health_system/ - ✅ Automated monitoring cleanup and log rotation
- ✅ Production stems backup system
- ✅ Professional documentation site with search
- ✅ Subdomain support (docs.tu-du-dum.com)
Backup System Status¶
- Database Backups: ✅ Active (Daily at 02:00 UTC)
- Stems Backups: ✅ Active (Weekly on Mondays at 00:13 UTC)
- Monitoring: ✅ Active (5-minute intervals)
- Cleanup: ✅ Active (Weekly on Mondays at 03:00 UTC)
🛠️ Quick Actions¶
Deploy Documentation Updates¶
cd server_health_system
./deploy-documentation.sh
Check System Health¶
ssh root@195.24.67.210 "systemctl status workingtitle-*"
View Recent Logs¶
ssh root@195.24.67.210 "journalctl -u workingtitle-monitor -f"
📚 Documentation Categories¶
| Category | Documents | Purpose |
|---|---|---|
| Operations | Advanced Monitoring System (September 18, 2025),Database Backup System (September 18, 2025) | Daily operations and maintenance |
🔍 Search Tips¶
Use the search function (top right) to quickly find: - Commands: Try searching for "backup", "restore", "deploy" - Procedures: Search for "emergency", "troubleshooting", "setup" - Configuration: Look for "config", "environment", "ssl" - Monitoring: Search for "health", "alerts", "logs"
Need help? Check the Database Backup System for backup procedures, or the Advanced Monitoring for system health management.