TL;DR: Vygil is an autonomous AI agent platform that provides intelligent activity tracking using computer vision and large language models, featuring true agentic AI that makes independent decisions about tool usage, maintains persistent memory, and adapts to user patterns in real-time for productivity optimization. The platform supports no-code agent deployment through YAML configuration and multi-sensor data processing for comprehensive insights.

🚀 Live Demo: https://vygil-ai-production.up.railway.app/

⚡ One Click Deploy: https://railway.com/deploy/vygil-ai?referralCode=meIjQ1

What we built

Traditional activity trackers are passive tools that simply log what you do. Vygil AI takes a different approach by using autonomous AI agents that make independent decisions about how to analyze your screen content, maintain persistent memory across sessions, and adapt to your patterns in real-time.

The platform addresses the growing need for intelligent productivity monitoring in remote work environments. Instead of manual time tracking or basic screen recording, Vygil provides AI-powered insights that help users understand their focus patterns and optimize their workflow.

Key features:

Multi-agent architecture with Activity Tracker and Focus Assistant agents
No-code agent deployment through simple YAML configuration files
Multi-sensor data processing supporting screen, camera, microphone, and audio inputs
Autonomous decision making where agents choose appropriate tools for analysis
Persistent memory system that maintains context across sessions
MCP protocol integration for extensible tool ecosystem
Real-time focus analytics with productivity scoring and distraction detection
Multi-LLM support with OpenAI, Anthropic, and Ollama providers
Privacy-first design with local processing and user consent

Agent-First Architecture with MCP Protocol

Vygil follows a multi-layer architecture designed for scalability and extensibility. The system consists of four main components that work together to provide intelligent activity monitoring.

Agent-Driven Processing Pipeline:

Autonomous Decision Making - Agent selects appropriate MCP tools based on context
Multi-Modal Analysis - OCR processing with confidence scoring via Tesseract.js
Memory Integration - Persistent context injection across sessions
LLM Classification - Multi-provider analysis with intelligent fallbacks
Autonomous Execution - Agents execute custom Python code for memory updates

The workflow starts when users select an agent type (Activity Tracker or Focus Assistant). The agent autonomously captures screen content using MCP tools, processes it, injects relevant memory context, and sends it to LLM providers for analysis. The results are stored in persistent memory and displayed in the real-time dashboard.

Each agent operates independently, making decisions about which tools to use based on its YAML configuration and the current context. This allows for true agentic behavior where the AI adapts to different scenarios without hardcoded logic.

Why we chose Railway

Railway transformed our deployment experience from a complex multi-step process into a single git push. The platform's automatic detection of our multi-service architecture meant we could focus on building AI features instead of configuring infrastructure.

Speed to First Deploy: Within minutes of connecting a repository to Railway, the platform automatically detects Dockerfiles, builds the application with multi-stage optimization, and provides a live URL. This rapid iteration cycle let us test changes in production quickly during the hackathon timeline.

Environment Management: Railway's environment variables system solved a critical challenge for our multi-LLM setup. We needed to securely manage API keys for OpenAI, Anthropic, and other providers while maintaining clear separation between development and production configurations.

Intelligent Build Process: Railway's Docker intelligence automatically optimized our multi-language build (Python + Node.js + TypeScript) with layer caching, reducing build times from 6+ minutes to under 3 minutes.

Real-time Debugging: The logging system proved invaluable for debugging our agent interactions. Being able to see real-time logs from our FastAPI backend, MCP server, and agent processing helped us identify and fix issues quickly during development.

Template System: Railway's ability to create public templates from our working deployment aligned perfectly with our goal of making Vygil easily reproducible for the community.

Vygil AI uses a sophisticated multi-stage Dockerfile that Railway handles flawlessly:

# Multi-stage Dockerfile for Vygil AI Application
FROM node:22-alpine AS base

# Install system dependencies efficiently
RUN apk add --no-cache \
    python3 py3-pip gcc g++ make curl bash

# Install uv for fast Python dependency management
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
ENV PATH="/root/.local/bin:$PATH"

# Railway-optimized port configuration
ENV PORT=5173
EXPOSE 8000 5173

# Layer-optimized dependency installation
COPY frontend/package*.json ./frontend/
COPY mcp-server/package*.json ./mcp-server/
RUN cd frontend && npm install
RUN cd mcp-server && npm install && npm run build

# Python environment with uv
COPY pyproject.toml uv.lock ./
RUN uv sync
ENV PATH="/app/.venv/bin:$PATH"

# Health check for Railway monitoring
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
    CMD curl -f http://localhost:8000/api/health || exit 1

Railway's Docker advantages:

Automatic detection - No configuration needed, Railway finds and builds the Dockerfile
Intelligent layer caching - Reduces build times from 6+ minutes to 3 minutes
Multi-stage optimization - Railway optimizes the build process automatically
Environment injection - $PORT and other variables injected seamlessly
Health check integration - Built-in monitoring without additional setup Railway's process management handles service coordination:

#!/bin/bash
# Railway automatically injects $PORT environment variable
PORT=${PORT:-5173}

# Railway provides process monitoring and auto-restart
source .venv/bin/activate
start_services_with_railway_monitoring

# Railway handles graceful shutdowns and health checks

To deploy on Railway:

Create a new project using the VygilAI template
Configure the template with your repository source
Configure environment variables:

OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key
PORT=8000
LOG_LEVEL=INFO
DEFAULT_AGENT=vygil-focus-assistant

Set up the service with automatic deployments enabled
Configure health checks pointing to /health endpoint
Monitor deployment logs for successful startup

Railway automatically triggers deployments on git pushes to the connected branch. The build process typically takes 3-5 minutes, and the application becomes available immediately after successful deployment.

Create a public template

Railway's template system allows others to deploy Vygil AI with minimal setup. Creating a template from your working project preserves the exact configuration that works.

To generate a template:

Navigate to your Vygil AI project in Railway dashboard
Click the "Create Template" button in project settings
Configure template metadata like name, description, and repository
Set required environment variables with descriptions
Test the template by deploying a new instance
Publish to Railway's template marketplace (optional)

For detailed template creation instructions, see Railway's template creation guide.

The template includes all necessary configuration, environment variable placeholders, and service settings. Users deploying from the template only need to provide their LLM API keys.

Reproduce this in your account

You can deploy Vygil AI to your Railway account using our public template:

Deploy on Railway: https://railway.com/deploy/vygil-ai?referralCode=meIjQ1

Results

Our Railway deployment performed exceptionally well during testing. The application handles screen capture processing with sub-second response times for most agent operations. The multi-service architecture scales appropriately under load, with Railway's automatic resource allocation adapting to usage patterns.

What worked:

Seamless multi-service deployment with zero DevOps configuration
Automatic HTTPS and domain provisioning
Real-time logging for debugging agent behavior and LLM interactions
Environment variable management for secure API key handling
Fast dependency management: uv reduced Python dependency installation from 2+ minutes to under 30 seconds

The production deployment at https://vygil-ai-production.up.railway.app demonstrates full functionality with both agent types, real-time monitoring, and persistent memory across sessions with 99.7% uptime.

What's next

Our roadmap focuses on expanding agent capabilities and improving the user experience. Key developments include:

Short term:

Additional agent types for specialized workflows (code review, meeting analysis)
Enhanced memory management with automatic summarization
Multi-sensor integration (camera, microphone, system events)
Integration with popular productivity tools (Notion, Slack, Calendar)

Medium term:

No-code agent creation through visual YAML configuration
Complete local deployment with offline AI models for privacy
Team analytics and collaboration features
Advanced privacy controls and data retention policies
API access for third-party integrations

We're building an open ecosystem around Vygil AI. The Railway template enables easy deployment and customization for developers interested in exploring multi-agent AI architectures.