Built for scale, designed for performance. Explore our modern microservices architecture powering millions of videos.
A distributed, microservices-based architecture designed for high availability, scalability, and fault tolerance
graph TB
subgraph "Client Layer"
Web[Web App
React/Next.js]
Mobile[Mobile App
React Native/Expo]
end
subgraph "API Layer"
Gateway[API Gateway
:3000
Express/TypeScript]
end
subgraph "Service Layer"
Auth[Auth Service
:3001
PostgreSQL]
User[User Service
:3002
PostgreSQL]
Content[Content Service
:3003
MongoDB]
Family[Family Service
:3004
PostgreSQL]
end
subgraph "AI/ML Layer"
ML[ML Service
:8000
Python/FastAPI]
end
subgraph "Data Layer"
MongoDB[(MongoDB
Replica Set)]
Postgres1[(PostgreSQL
Auth DB)]
Postgres2[(PostgreSQL
User DB)]
Postgres3[(PostgreSQL
Family DB)]
Redis[(Redis
Cache)]
end
subgraph "Storage Layer"
MinIO[MinIO
S3 Storage
:9000]
end
subgraph "Queue Layer"
Kafka[Kafka
:9092]
Bull[Bull Queue
Redis-backed]
end
Web --> Gateway
Mobile --> Gateway
Gateway --> Auth
Gateway --> User
Gateway --> Content
Gateway --> Family
Content --> MongoDB
Content --> MinIO
Content --> Bull
Content --> ML
Auth --> Postgres1
User --> Postgres2
Family --> Postgres3
Auth --> Redis
User --> Redis
Content --> Redis
Bull --> Redis
ML --> MinIO
ML --> MongoDB
Content --> Kafka
ML --> Kafka
style Gateway fill:#4CAF50
style Content fill:#2196F3
style ML fill:#FF9800
style MongoDB fill:#47A248
style MinIO fill:#C72E49
Independent services for auth, users, content, and family controls enabling independent scaling and deployment
Bull queues and Kafka streams for handling video transcoding and AI moderation without blocking user requests
Redis caching layer reduces database load and provides sub-millisecond response times
Python-based ML service with TensorFlow and PyTorch for real-time content safety and categorization
From upload to delivery: How we process, transcode, and moderate millions of videos
sequenceDiagram
participant User
participant API as API Gateway
participant Content as Content Service
participant Queue as Bull Queue
participant Worker as Video Worker
participant FFmpeg
participant MinIO as MinIO Storage
participant ML as ML Service
participant DB as MongoDB
User->>API: Upload Video
API->>Content: POST /videos/upload
Content->>MinIO: Store Original
Content->>DB: Create Video Record
(status: QUEUED)
Content->>Queue: Add to Processing Queue
Content-->>User: 201 Created
Queue->>Worker: Pick Job
Worker->>DB: Update status: PROCESSING
Worker->>FFmpeg: Extract Metadata
FFmpeg-->>Worker: {duration, resolution}
Worker->>FFmpeg: Transcode Resolutions
(360p-8K)
FFmpeg-->>Worker: Transcoded Files
Worker->>FFmpeg: Generate Thumbnail
FFmpeg-->>Worker: thumbnail.jpg
Worker->>MinIO: Upload All Files
MinIO-->>Worker: URLs
Worker->>ML: Moderate Video
ML->>ML: Run AI Models
ML-->>Worker: Moderation Result
alt Video Approved
Worker->>DB: Update status: READY
Worker->>ML: Trigger Tagging
ML->>DB: Update Tags & Rating
else Video Flagged
Worker->>DB: Update status: FLAGGED
end
Worker->>Queue: Job Complete
User->>API: GET /videos/{id}
API->>Content: Request Video
Content->>DB: Fetch Video
DB-->>Content: Video Data
Content-->>User: Video URLs
Multi-model AI pipeline ensuring safe, family-friendly content
graph TB
Start([Video Processing]) --> ML[ML Service Receives Request]
ML --> Download[Download from MinIO]
Download --> Models{Run AI Models}
Models --> NSFW[NSFW Detection
TensorFlow/MobileNet]
Models --> Violence[Violence Detection
PyTorch]
Models --> Text[Text Moderation
Transformers]
Models --> Audio[Audio Analysis
Whisper]
NSFW --> Score1[NSFW Score]
Violence --> Score2[Violence Score]
Text --> Score3[Toxicity Score]
Audio --> Score4[Audio Safety Score]
Score1 --> Aggregate[Aggregate Scores]
Score2 --> Aggregate
Score3 --> Aggregate
Score4 --> Aggregate
Aggregate --> Decision{Make Decision}
Decision -->|Score > 0.9| Reject[REJECT
Status: BLOCKED]
Decision -->|0.6 < Score < 0.9| Review[FLAG FOR REVIEW
Status: FLAGGED]
Decision -->|Score < 0.6| Approve[APPROVE
Status: READY]
Approve --> Tag[Background Tagging]
Tag --> Category[Categorization]
Tag --> Rating[Content Rating]
Category --> Update[Update Database]
Rating --> Update
Reject --> Update
Review --> Update
Update --> End([Complete])
style NSFW fill:#FF5722
style Violence fill:#F44336
style Text fill:#E91E63
style Audio fill:#9C27B0
style Approve fill:#4CAF50
style Review fill:#FF9800
style Reject fill:#F44336
TensorFlow-based visual content analysis identifying inappropriate imagery
PyTorch models trained to identify violent or aggressive content
Transformer models analyzing captions, titles, and on-screen text for toxicity
Whisper-powered speech recognition checking audio content safety
Modern, battle-tested technologies powering Thiqa
Modern web and mobile applications built with cutting-edge frameworks
Scalable microservices architecture with Node.js and TypeScript
Python-based machine learning for content moderation and tagging
Multi-database strategy optimized for different data patterns
Distributed object storage for video content delivery
Reliable message processing and event streaming
Containerized deployment with monitoring and observability
Enterprise-grade security and authentication
Production-ready infrastructure with high availability and fault tolerance
graph TB
subgraph "Load Balancer"
LB[Nginx/HAProxy
SSL Termination]
end
subgraph "Application Cluster"
GW1[API Gateway 1]
GW2[API Gateway 2]
GW3[API Gateway N]
CS1[Content Service 1]
CS2[Content Service 2]
CS3[Content Service N]
ML1[ML Service 1]
ML2[ML Service 2]
end
subgraph "Database Cluster"
MONGO_PRIMARY[(MongoDB
Primary)]
MONGO_SECONDARY1[(MongoDB
Secondary)]
MONGO_SECONDARY2[(MongoDB
Secondary)]
PG_PRIMARY[(PostgreSQL
Primary)]
PG_REPLICA[(PostgreSQL
Replica)]
end
subgraph "Cache Cluster"
REDIS_MASTER[(Redis
Master)]
REDIS_REPLICA1[(Redis
Replica)]
REDIS_REPLICA2[(Redis
Replica)]
end
subgraph "Storage Cluster"
MINIO1[MinIO Node 1]
MINIO2[MinIO Node 2]
MINIO3[MinIO Node 3]
MINIO4[MinIO Node N]
end
LB --> GW1
LB --> GW2
LB --> GW3
GW1 --> CS1
GW1 --> CS2
GW2 --> CS1
GW2 --> CS3
CS1 --> MONGO_PRIMARY
CS2 --> MONGO_PRIMARY
CS3 --> MONGO_PRIMARY
MONGO_PRIMARY --> MONGO_SECONDARY1
MONGO_PRIMARY --> MONGO_SECONDARY2
CS1 --> REDIS_MASTER
CS2 --> REDIS_MASTER
REDIS_MASTER --> REDIS_REPLICA1
REDIS_MASTER --> REDIS_REPLICA2
CS1 --> MINIO1
CS2 --> MINIO2
PG_PRIMARY --> PG_REPLICA
style LB fill:#4CAF50
style MONGO_PRIMARY fill:#47A248
style REDIS_MASTER fill:#DC382D
Horizontal scaling of services based on load and traffic patterns
Database replication and service redundancy for 99.9% uptime
Prometheus and Grafana for real-time metrics and alerting
Intelligent traffic distribution across multiple service instances
Designed to handle millions of concurrent users and billions of video views
graph LR
LB[Load Balancer] --> A1[Instance 1]
LB --> A2[Instance 2]
LB --> A3[Instance 3]
LB --> AN[Instance N]
A1 --> DB[(Database)]
A2 --> DB
A3 --> DB
AN --> DB
A1 --> CACHE[(Cache)]
A2 --> CACHE
A3 --> CACHE
AN --> CACHE
style LB fill:#4CAF50
style DB fill:#2196F3
Add more service instances to handle increased load
graph LR
PROD[Producers] --> Q[Message Queue]
Q --> W1[Worker 1]
Q --> W2[Worker 2]
Q --> W3[Worker 3]
Q --> WN[Worker N]
W1 --> STORE[(Storage)]
W2 --> STORE
W3 --> STORE
WN --> STORE
style Q fill:#FF9800
style STORE fill:#2196F3
Process videos asynchronously with multiple workers