Architecture Overview

Quemsi's privacy-first architecture ensures your data never leaves your environment while providing the convenience of a modern SaaS application.

Table of Contents

Architecture Overview

Quemsi follows a hybrid architecture that combines the convenience of a SaaS application with the security of on-premise data processing. The system consists of two main components: the Quemsi Cloud Service (metadata management) and the Quemsi Agent (data processing).

Quemsi Architecture Diagram

Quemsi Architecture: Cloud metadata service + On-premise agent

Key Architectural Decisions

🏗️ Core Architectural Principles

Zero Data Exfiltration

Your actual database content never leaves your environment. Only metadata (tags, timestamps, configuration) is transmitted to the cloud service for management purposes.

Privacy by Design

Every architectural decision prioritizes data privacy. The system is designed to work entirely within your network boundaries while providing cloud-like convenience.

Agent-First Processing

All database operations (backup, restore, validation) are performed by lightweight agents running within your infrastructure, ensuring complete control over your data.

Open Source Transparency

The agent source code is fully open source, allowing you to audit, modify, and build your own version if needed. Complete transparency in data handling.

⚙️ System Components

Quemsi Cloud Service

The cloud service manages metadata, user authentication, and provides the web interface. It never handles actual database content.

Responsibilities:

  • User authentication and authorization
  • Metadata storage (backup tags, timestamps, configurations)
  • Web UI and API endpoints
  • Agent registration and management
  • Cross-environment backup sharing coordination

Technology Stack:

# Cloud Service Stack
Backend: Spring Boot (Java)
Database: PostgreSQL
Authentication: OAuth2 + JWT
API: REST + GraphQL
Frontend: Vue.js + Vuetify
Infrastructure: Docker + Kubernetes

Quemsi Agent

The agent runs within your infrastructure and handles all database operations. It's lightweight, stateless, and communicates only metadata with the cloud service.

Responsibilities:

  • Database backup and restore operations
  • Custom workflow execution
  • Local storage management
  • Metadata synchronization with cloud service
  • Health monitoring and reporting

Agent Configuration:

# agent environment variables
agent:
  CLIENT_ID: "client-spesific-id"
  CLIENT_SECRET: "client-spesific-secret"

🔄 Data Flow Customization

Flows allow you to customize how data is backed up and restored. You can make any preparation before like stoping services, pausing replication, running scripts, etc. You can make multiple processing like zipping, filtering, etc. You can also make any post-processing like starting services, validating data and selecting where to store the backup, etc.

Mandatory Flow Steps for Backup

From
From

Mandatory step that defines the source of the data. It can be a database, a file, a folder, or any data source.

To
To

Mandatory step that defines the destination of the data. Supports multiple storage types including local drives, AWS S3, Azure Blob Storage, etc.

Mandatory Flow Steps for Restore

From
From

Mandatory step that defines the source backup version. Data is selected according to tags and ordering criteria.

To
To

Mandatory step that defines the target database for restoration. Creates database objects (tables, indexes, sequences) according to the restored version.

Optional Processing Steps

More processing options will be added in future releases

Zip
Zip

Creates a compressed zip file of the backup data. Only available for backup flows to reduce storage space.

Unzip
Unzip

Decompresses zip files during restore operations. Only available for restore flows and requires data to be zipped in the backup flow.

Clear
ClearTables

Truncates all tables in the target database before restoration. Only available for restore flows to ensure clean data state.

Drop
DropTables

Drops existing tables so they can be recreated according to the restored version schema. Only available for restore flows.

🔒 Security Architecture

Security is built into every layer of the Quemsi architecture, ensuring your data remains protected throughout the entire backup and restore lifecycle.

Security Features

🔐 End-to-End Encryption

All communication between agents and cloud service uses TLS 1.3 encryption

🔑 OAuth2 Authentication

Industry-standard authentication with JWT tokens and refresh token rotation

🛡️ Agent Authentication

Agents authenticate using certificate-based mutual TLS authentication

📝 Audit Logging

Comprehensive audit trails for all backup, restore, and configuration changes

🔍 Data Integrity

SHA-256 checksums verify backup integrity and detect corruption

🚫 Zero Data Storage

Cloud service never stores actual database content, only metadata

Network Security

# Network Security Configuration
Agent Communication:
  Protocol: HTTPS/TLS 1.3
  Authentication: Mutual TLS
  Port: 443 (outbound only)
  
Database Access:
  Protocol: Database native protocols
  Network: Local network only
  Credentials: Environment variables
  
Storage:
  Location: Local filesystem or shared drives like AWS S3, Azure Blob Storage, etc.
  Ownership: You completely own your data
  Access: Agent process only

🌐 API Architecture

Quemsi will provide both REST APIs for programmatic access. All APIs follow RESTful principles and include comprehensive error handling. Complete API documentation will be provided in the future.

Example API Endpoints (Comprehensive API documentation will be provided soon)

POST /api/flows/{flowName}/execute?site=dev
Execute a flow
GET /api/flow-executions/{id}
Get a flow execution by id
GET /api/datas/{dataName}/flows
List available flows for a data
GET /api/agents/list
List registered agents and their status
GET /api/flow-executions/filter/{flowName"}
Query flow executions by flow name

API Response Format

# GET /api/agents/demo-agent response
{
  "id": 1,
  "name": "demo-agent",
  "title": "Demo Agent",
  "version": "2.3.7",
  "description": "Demo Agent",
  "status": "ONLINE",
  "showCredentials": false,
  "clientId": "N/A",
  "clientSecret": "N/A",
}

🚀 Supported Environments

Quemsi agent is developed fully in Java and can be run on any environment that supports Java. Moreover, it can be compiled to a native executable for better performance. It can be run as a system service or as a standalone application.

You can build Quemsi agent for yourself from the source code or use the pre-built binaries from our website. Pre-built binaries are available for Windows, Linux and macOS and Java as jar artifact.

Running Quemsi Agent

Custom installation and running instructions for your agent is available in the application UI for all platforms.

As a System Service

Run Quemsi agent as a system service on Windows, Linux and macOS. It will be automatically started on system boot and will run as a background process.

As a Standalone Application

Run Quemsi agent as a standalone application on Windows, Linux and macOS. It will be run as a foreground process and will exit when the main process exits.

As a Docker Container

Run Quemsi agent as a Docker container on any environment that supports Docker. It will be run as a container and will exit when the container exits.

As a Java Jar Artifact

Run Quemsi agent as a Java jar artifact on any environment that supports Java. It will be run as a foreground process and will exit when the main process exits.

Quemsi Agent Deployment

# Docker Compose Example
version: '3.9'

services:
  quemsi-agent:
    image: quemsi/quemsi-agent:latest
    environment:
      - CLIENT-ID=agent-2-2
      - CLIENT-SECRET=uiYhXodbTpEb3gaYHDo3GwqNTew64IC4

Next Steps

Want to know more about Quemsi? Check out these resources: