> ## Documentation Index
> Fetch the complete documentation index at: https://acem-52171079.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Architecture

> Technical design, data flow, and technology stack

The DataDot server is designed as a modern, asynchronous application. It leverages key technologies to ensure high performance for real-time AI interactions.

## Tech Stack

<CardGroup cols={3}>
  <Card title="FastAPI" icon="bolt" href="https://fastapi.tiangolo.com/">
    A modern, fast (high-performance) web framework for building APIs with Python 3.10+.
  </Card>

  <Card title="SQLAlchemy" icon="database" href="https://www.sqlalchemy.org/">
    The Python SQL toolkit and Object Relational Mapper, used for all relational database interactions.
  </Card>

  <Card title="PydanticSettings" icon="sliders" href="https://docs.pydantic.dev/latest/concepts/pydantic_settings/">
    Roboust configuration management using environment variables.
  </Card>

  <Card title="LanceDB / Chroma" icon="vector-square">
    Pluggable support for various vector databases to power the RAG pipeline.
  </Card>

  <Card title="APScheduler" icon="clock">
    Advanced Python Scheduler for handling background jobs and periodic tasks.
  </Card>

  <Card title="Passlib & JWT" icon="key">
    Secure password hashing and token-based authentication (JSON Web Tokens).
  </Card>
</CardGroup>

## System Diagrams

### High-Level Architecture

The below diagram illustrates the primary components and how data flows between the client, the API server, and the various backing services.

<Frame caption="System Architecture">
  ```mermaid theme={null}
  graph TD
      Client[Frontend Client/MCP] -->|HTTP/WS| Server[FastAPI Server]
      
      subgraph Server Internals
        Auth[Auth Middleware]
        Router[API Router]
        Service[Service Layer]
        Task[Background Tasks]
      end
      
      Server --> Auth
      Auth --> Router
      Router --> Service
      
      Service -->|Read/Write| SQL[(SQL Database)]
      Service -->|Vector Search| VectorDB[(Vector DB)]
      Service -->|Generate| LLM[LLM Provider]
      
      Router -.->|Async| Task
  ```
</Frame>

### Chat & RAG Flow

When a user sends a message, the system orchestrates a complex flow to provide a context-aware response.

1. **Receive**: The `chat` endpoint receives the user message.
2. **Retrieve**: The `embedding_service` searches the Vector DB for relevant documents.
3. **Construct**: The `chat_service` builds a prompt containing the user message, conversation history, and retrieved context.
4. **Generate**: The prompt is sent to the configured LLM provider (e.g., OpenAI).
5. **Stream**: The response is streamed back to the client in real-time.

```mermaid theme={null}
sequenceDiagram
    participant User
    participant API as API Server
    participant VDB as Vector DB
    participant LLM
    
    User->>API: Send Message
    API->>VDB: Similarity Search (Query)
    VDB-->>API: Relevant Chunks
    API->>LLM: Prompt (History + Context + Query)
    LLM-->>API: Stream tokens
    API-->>User: Stream response
```

## Database Layer

The application uses **SQLAlchemy** (AsyncIO) for database persistence.

* **Session Management**: Handled via `app.core.database.get_db` dependency.
* **Initialization**: `init_db()` in `app.core.database` ensures tables are created on startup.
* **Migrations**: Currently, the system uses `Base.metadata.create_all()` for schema creation.

### Key Models (`app/models/`)

* **User**: Stores user credentials and profile info.
* **Workspace**: Groups resources (chats, docs) for isolation.
* **WorkspaceDocument**: Metadata for uploaded files.
* **ChatSession**: Represents a conversation thread.
* **ChatMessage**: Individual messages within a session.

## Background Jobs

If `ENABLE_BACKGROUND_JOBS` is set to `True`, the server initializes an **APScheduler** instance.

* **Location**: `app/tasks/scheduler.py`
* **Lifecycle**: Started/Shutdown via `app.main.lifespan`.
* **Use Cases**: Periodic vector store cleanup, email notifications, system maintenance.