markbaseengine/INTEGRATION_GUIDE.md

MarkBaseEngine Integration Guide

For momentry_core (Rust Backend) & momentry_studio (Frontend)

---

Overview

MarkBaseEngine provides a high-performance inference engine for multimodal AI models (Text, Vision, Audio) on Apple Silicon. This guide explains how to integrate MarkBaseServer with your Rust backend and frontend.

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      momentry_studio (Frontend)                  │
│                   TypeScript/React/Svelte/etc.                  │
└────────────────────────┬────────────────────────────────────────┘
                         │ HTTP/WebSocket
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                     momentry_core (Rust Backend)                 │
│              API Gateway, Auth, Business Logic                   │
└────────────────────────┬────────────────────────────────────────┘
                         │ HTTP REST API
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                    MarkBaseServer (Swift)                        │
│         OpenAI-Compatible API: Text/Vision/Audio                │
│         Port: 8080 (or 8083-8097 for dev)                       │
└────────────────────────┬────────────────────────────────────────┘
                         │ Metal GPU
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                    MarkBaseEngine (Core)                         │
│        Model Loading, Inference, KV Cache, Multimodal           │
│        Models: E4B-MarkBase, 12B, 26B-Standard, 31B             │
└─────────────────────────────────────────────────────────────────┘

---

MarkBaseServer API Endpoints

Base URL

• Local: http://127.0.0.1:8080/v1
• Production: http://10.10.10.201:8080/v1

Endpoints

Endpoint	Method	Description
/health	GET	Server health check
/v1/models	GET	List available models
/v1/chat/completions	POST	Text generation
/v1/multimodal/chat/completions	POST	Vision+Audio+Text generation

---

1. Text Model Integration

Rust Backend (momentry_core)

use reqwest::Client;
use serde::{Deserialize, Serialize};

#[derive(Serialize)]
struct ChatRequest {
    model: String,
    messages: Vec<Message>,
    max_tokens: Option<u32>,
    temperature: Option<f32>,
    stream: Option<bool>,
}

#[derive(Serialize, Deserialize)]
struct Message {
    role: String,
    content: String,
}

#[derive(Deserialize)]
struct ChatResponse {
    id: String,
    choices: Vec<Choice>,
    usage: Usage,
}

#[derive(Deserialize)]
struct Choice {
    message: Message,
    finish_reason: String,
}

#[derive(Deserialize)]
struct Usage {
    prompt_tokens: u32,
    completion_tokens: u32,
    total_tokens: u32,
}

// Call MarkBaseServer for text generation
async fn generate_text(prompt: &str, model: &str) -> Result<String, Box<dyn std::error::Error>> {
    let client = Client::new();
    
    let request = ChatRequest {
        model: model.to_string(),
        messages: vec![
            Message { role: "user".to_string(), content: prompt.to_string() }
        ],
        max_tokens: Some(100),
        temperature: Some(0.7),
        stream: Some(false),
    };
    
    let response = client
        .post("http://10.10.10.201:8080/v1/chat/completions")
        .json(&request)
        .send()
        .await?
        .json::<ChatResponse>()
        .await?;
    
    Ok(response.choices[0].message.content)
}

// Available models
const MODELS: &[&str] = &[
    "gemma-4-e4b-markbase",   // 4B, optimized for speed
    "gemma-4-12b-it-4bit",    // 12B, balanced
    "gemma-4-26b-standard",   // 26B, high quality
    "gemma-4-31b",            // 31B, highest quality
];

Frontend (momentry_studio)

interface ChatRequest {
  model: string;
  messages: Array<{role: string, content: string}>;
  max_tokens?: number;
  temperature?: number;
  stream?: boolean;
}

interface ChatResponse {
  id: string;
  choices: Array<{
    message: {role: string, content: string};
    finish_reason: string;
  }>;
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
  };
}

// Call via momentry_core backend proxy
async function generateText(prompt: string, model: string = 'gemma-4-e4b-markbase'): Promise<string> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model,
      messages: [{ role: 'user', content: prompt }],
      max_tokens: 100,
      temperature: 0.7,
    }),
  });
  
  const data: ChatResponse = await response.json();
  return data.choices[0].message.content;
}

---

2. Vision Model Integration

Input Format

Vision models accept images encoded as base64 or URLs.

Rust Backend

#[derive(Serialize)]
struct MultimodalChatRequest {
    model: String,
    messages: Vec<MultimodalMessage>,
    max_tokens: Option<u32>,
}

#[derive(Serialize)]
struct MultimodalMessage {
    role: String,
    content: Vec<ContentPart>,
}

#[derive(Serialize)]
#[serde(tag = "type")]
enum ContentPart {
    #[serde(rename = "text")]
    Text { text: String },
    #[serde(rename = "image_url")]
    ImageUrl { image_url: ImageUrl },
}

#[derive(Serialize)]
struct ImageUrl {
    url: String,  // base64 data URI or HTTP URL
}

// Vision inference
async fn analyze_image(image_path: &str, prompt: &str) -> Result<String, Box<dyn std::error::Error>> {
    let client = Client::new();
    
    // Read and encode image as base64
    let image_data = std::fs::read(image_path)?;
    let base64 = base64::encode(&image_data);
    let data_uri = format!("data:image/jpeg;base64,{}", base64);
    
    let request = MultimodalChatRequest {
        model: "gemma-4-12b-it-4bit".to_string(),
        messages: vec![
            MultimodalMessage {
                role: "user".to_string(),
                content: vec![
                    ContentPart::ImageUrl { 
                        image_url: ImageUrl { url: data_uri } 
                    },
                    ContentPart::Text { text: prompt.to_string() },
                ],
            },
        ],
        max_tokens: Some(200),
    };
    
    let response = client
        .post("http://10.10.10.201:8080/v1/multimodal/chat/completions")
        .json(&request)
        .send()
        .await?
        .json::<ChatResponse>()
        .await?;
    
    Ok(response.choices[0].message.content)
}

Frontend

interface MultimodalMessage {
  role: string;
  content: Array<{type: 'text', text: string} | {type: 'image_url', image_url: {url: string}}>;
}

async function analyzeImage(imageFile: File, prompt: string): Promise<string> {
  // Convert image to base64
  const base64 = await new Promise<string>((resolve) => {
    const reader = new FileReader();
    reader.onload = () => resolve(reader.result as string);
    reader.readAsDataURL(imageFile);
  });
  
  const response = await fetch('/api/multimodal/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'gemma-4-12b-it-4bit',
      messages: [{
        role: 'user',
        content: [
          { type: 'image_url', image_url: { url: base64 } },
          { type: 'text', text: prompt },
        ],
      }],
      max_tokens: 200,
    }),
  });
  
  const data = await response.json();
  return data.choices[0].message.content;
}

---

3. Audio Model Integration

Audio Input Format

Audio models accept audio files (WAV, MP3, AAC) encoded as base64.

Rust Backend

#[derive(Serialize)]
struct AudioChatRequest {
    model: String,
    messages: Vec<AudioMessage>,
    max_tokens: Option<u32>,
}

#[derive(Serialize)]
struct AudioMessage {
    role: String,
    content: Vec<AudioContentPart>,
}

#[derive(Serialize)]
#[serde(tag = "type")]
enum AudioContentPart {
    #[serde(rename = "text")]
    Text { text: String },
    #[serde(rename = "audio_url")]
    AudioUrl { audio_url: AudioUrl },
}

#[derive(Serialize)]
struct AudioUrl {
    url: String,  // base64 data URI
}

// Audio transcription/analysis
async fn process_audio(audio_path: &str, prompt: &str) -> Result<String, Box<dyn std::error::Error>> {
    let client = Client::new();
    
    let audio_data = std::fs::read(audio_path)?;
    let base64 = base64::encode(&audio_data);
    let data_uri = format!("data:audio/wav;base64,{}", base64);
    
    let request = AudioChatRequest {
        model: "gemma-4-12b-it-4bit".to_string(),
        messages: vec![
            AudioMessage {
                role: "user".to_string(),
                content: vec![
                    AudioContentPart::AudioUrl { 
                        audio_url: AudioUrl { url: data_uri } 
                    },
                    AudioContentPart::Text { text: prompt.to_string() },
                ],
            },
        ],
        max_tokens: Some(100),
    };
    
    let response = client
        .post("http://10.10.10.201:8080/v1/multimodal/chat/completions")
        .json(&request)
        .send()
        .await?
        .json::<ChatResponse>()
        .await?;
    
    Ok(response.choices[0].message.content)
}

Frontend

async function processAudio(audioFile: File, prompt: string): Promise<string> {
  const base64 = await new Promise<string>((resolve) => {
    const reader = new FileReader();
    reader.onload = () => resolve(reader.result as string);
    reader.readAsDataURL(audioFile);
  });
  
  const response = await fetch('/api/multimodal/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'gemma-4-12b-it-4bit',
      messages: [{
        role: 'user',
        content: [
          { type: 'audio_url', audio_url: { url: base64 } },
          { type: 'text', text: prompt },
        ],
      }],
      max_tokens: 100,
    }),
  });
  
  const data = await response.json();
  return data.choices[0].message.content;
}

---

4. Streaming Responses

Server-Sent Events (SSE)

MarkBaseServer supports streaming via SSE when stream: true is set.

Rust Backend

use futures::StreamExt;

async fn stream_text(prompt: &str, model: &str) -> Result<String, Box<dyn std::error::Error>> {
    let client = Client::new();
    
    let request = ChatRequest {
        model: model.to_string(),
        messages: vec![Message { role: "user".to_string(), content: prompt.to_string() }],
        max_tokens: Some(100),
        stream: Some(true),
    };
    
    let mut stream = client
        .post("http://10.10.10.201:8080/v1/chat/completions")
        .json(&request)
        .send()
        .await?
        .bytes_stream();
    
    let mut full_text = String::new();
    
    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        let text = String::from_utf8_lossy(&chunk);
        
        // Parse SSE format: "data: {...}\n\n"
        for line in text.lines() {
            if line.starts_with("data: ") {
                let json_str = &line[6..];
                if json_str == "[DONE]" { break; }
                
                let chunk_data: serde_json::Value = serde_json::from_str(json_str)?;
                if let Some(content) = chunk_data["choices"][0]["delta"]["content"].as_str() {
                    full_text.push_str(content);
                    // Send to frontend via WebSocket
                }
            }
        }
    }
    
    Ok(full_text)
}

Frontend

async function streamText(prompt: string, onChunk: (text: string) => void): Promise<void> {
  const response = await fetch('/api/chat/stream', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'gemma-4-e4b-markbase',
      messages: [{ role: 'user', content: prompt }],
      stream: true,
    }),
  });
  
  const reader = response.body?.getReader();
  const decoder = new TextDecoder();
  
  while (reader) {
    const { done, value } = await reader.read();
    if (done) break;
    
    const text = decoder.decode(value);
    for (const line of text.split('\n')) {
      if (line.startsWith('data: ')) {
        const json = line.slice(6);
        if (json === '[DONE]') break;
        
        const data = JSON.parse(json);
        const content = data.choices[0]?.delta?.content || '';
        onChunk(content);
      }
    }
  }
}

---

5. Model Selection Guide

Model	Size	Speed	Quality	Use Case
E4B-MarkBase	4.4GB	49ms/token	Good	Real-time chat, quick responses
12B	6.3GB	6ms/token (158 tok/s)	Better	Balanced speed/quality
26B-Standard	15GB	30ms/token	High	Complex reasoning, code generation
31B	17GB	38ms/token	Highest	Deep analysis, expert tasks

Recommendation Matrix

Scenario	Recommended Model
Chat UI autocomplete	E4B-MarkBase
Document summarization	12B or 26B-Standard
Code generation	26B-Standard
Vision analysis	12B (has VisionTower12B)
Audio transcription	12B (has AudioTower12B)
Expert reasoning	31B

---

6. Performance Optimization

KV Cache Management

MarkBaseServer automatically manages KV cache. For long conversations:

// Clear context for new conversation
async fn reset_context(session_id: &str) {
    // MarkBaseServer handles this internally
    // Just start a new messages array
}

Concurrent Requests

MarkBaseServer handles concurrent requests efficiently:

• Text: Up to 10 concurrent streams
• Vision: 2-3 concurrent (GPU intensive)
• Audio: 2-3 concurrent (GPU intensive)

Memory Limits

• M5Max48 (48GB): Max 3 models loaded concurrently
• M5 (128GB): All 4 models can be loaded

---

7. Deployment Configuration

MarkBaseServer Startup

# Local development (M5 128GB)
cd ~/MarkBaseEngine
./start_server.sh

# Production (M5Max48 via TBT5)
# Deploy models first:
rsync -avP ~/MarkBaseEngine/models/ 10.10.10.201:/Volumes/TBT5/models/

# Start server on M5Max48:
ssh 10.10.10.201
cd /Volumes/TBT5/MarkBaseEngine
./build/release/MarkBaseServer ./models/E4B-MarkBase 8080 gemma-4-e4b-markbase

Rust Backend Configuration

// config.rs
pub struct MarkBaseConfig {
    pub base_url: String,
    pub default_model: String,
    pub timeout_ms: u64,
}

impl Default for MarkBaseConfig {
    fn default() -> Self {
        Self {
            base_url: "http://10.10.10.201:8080/v1".to_string(),
            default_model: "gemma-4-e4b-markbase".to_string(),
            timeout_ms: 30000,
        }
    }
}

---

8. Error Handling

Common Errors

Error	Cause	Solution
Connection refused	Server not running	Check ./start_server.sh
Model not found	Wrong model name	Check /v1/models endpoint
Timeout	Large input/slow model	Increase timeout, use faster model
GPU memory limit	Too many concurrent	Reduce concurrent requests
NaN output	Forward pass bug	Report to MarkBaseEngine team

Rust Error Handling

use thiserror::Error;

#[derive(Error, Debug)]
pub enum MarkBaseError {
    #[error("Connection failed: {0}")]
    ConnectionFailed(String),
    #[error("Model not found: {0}")]
    ModelNotFound(String),
    #[error("Timeout after {0}ms")]
    Timeout(u64),
    #[error("Invalid response: {0}")]
    InvalidResponse(String),
}

impl From<reqwest::Error> for MarkBaseError {
    fn from(e: reqwest::Error) -> Self {
        if e.is_timeout() {
            MarkBaseError::Timeout(30000)
        } else if e.is_connect() {
            MarkBaseError::ConnectionFailed(e.to_string())
        } else {
            MarkBaseError::InvalidResponse(e.to_string())
        }
    }
}

---

9. Testing & Validation

Health Check

async fn check_health() -> bool {
    let client = Client::new();
    let response = client
        .get("http://10.10.10.201:8080/health")
        .send()
        .await;
    
    response.is_ok()
}

Model List

async fn list_models() -> Result<Vec<String>, Box<dyn std::error::Error>> {
    let client = Client::new();
    let response = client
        .get("http://10.10.10.201:8080/v1/models")
        .send()
        .await?
        .json::<serde_json::Value>()
        .await?;
    
    let models = response["data"]
        .as_array()
        .unwrap_or(&vec![])
        .iter()
        .filter_map(|m| m["id"].as_str().map(|s| s.to_string()))
        .collect();
    
    Ok(models)
}

---

10. Security Considerations

API Gateway (momentry_core)

// Add authentication layer
use actix_web::{web, HttpRequest, HttpResponse};

async fn chat_proxy(
    req: HttpRequest,
    body: web::Json<ChatRequest>,
) -> HttpResponse {
    // Validate auth token
    let auth = req.headers().get("Authorization");
    if !validate_auth(auth) {
        return HttpResponse::Unauthorized().finish();
    }
    
    // Rate limiting
    if !check_rate_limit(&req) {
        return HttpResponse::TooManyRequests().finish();
    }
    
    // Forward to MarkBaseServer
    let response = forward_to_markbase(body.into_inner());
    
    HttpResponse::Ok().json(response)
}

Input Validation

fn validate_chat_request(req: &ChatRequest) -> Result<(), String> {
    if req.messages.is_empty() {
        return Err("Messages array cannot be empty".to_string());
    }
    
    if req.max_tokens.unwrap_or(100) > 2048 {
        return Err("max_tokens cannot exceed 2048".to_string());
    }
    
    Ok(())
}

---

11. Complete Example: momentry_core Integration

// src/markbase_client.rs
use reqwest::Client;
use serde::{Deserialize, Serialize};
use std::time::Duration;

pub struct MarkBaseClient {
    client: Client,
    base_url: String,
    default_model: String,
}

impl MarkBaseClient {
    pub fn new(base_url: &str, default_model: &str) -> Self {
        let client = Client::builder()
            .timeout(Duration::from_secs(30))
            .build()
            .unwrap();
        
        Self {
            client,
            base_url: base_url.to_string(),
            default_model: default_model.to_string(),
        }
    }
    
    pub async fn chat(&self, prompt: &str) -> Result<String, MarkBaseError> {
        self.chat_with_model(prompt, &self.default_model).await
    }
    
    pub async fn chat_with_model(&self, prompt: &str, model: &str) -> Result<String, MarkBaseError> {
        let request = ChatRequest {
            model: model.to_string(),
            messages: vec![Message { role: "user".to_string(), content: prompt.to_string() }],
            max_tokens: Some(100),
            temperature: Some(0.7),
            stream: Some(false),
        };
        
        let url = format!("{}{}", self.base_url, "/chat/completions");
        let response = self.client
            .post(&url)
            .json(&request)
            .send()
            .await?
            .json::<ChatResponse>()
            .await?;
        
        Ok(response.choices[0].message.content)
    }
    
    pub async fn vision(&self, image_base64: &str, prompt: &str) -> Result<String, MarkBaseError> {
        let request = MultimodalChatRequest {
            model: self.default_model.clone(),
            messages: vec![
                MultimodalMessage {
                    role: "user".to_string(),
                    content: vec![
                        ContentPart::ImageUrl { 
                            image_url: ImageUrl { url: format!("data:image/jpeg;base64,{}", image_base64) } 
                        },
                        ContentPart::Text { text: prompt.to_string() },
                    ],
                },
            ],
            max_tokens: Some(200),
        };
        
        let url = format!("{}{}", self.base_url, "/multimodal/chat/completions");
        let response = self.client
            .post(&url)
            .json(&request)
            .send()
            .await?
            .json::<ChatResponse>()
            .await?;
        
        Ok(response.choices[0].message.content)
    }
    
    pub async fn audio(&self, audio_base64: &str, prompt: &str) -> Result<String, MarkBaseError> {
        let request = AudioChatRequest {
            model: self.default_model.clone(),
            messages: vec![
                AudioMessage {
                    role: "user".to_string(),
                    content: vec![
                        AudioContentPart::AudioUrl { 
                            audio_url: AudioUrl { url: format!("data:audio/wav;base64,{}", audio_base64) } 
                        },
                        AudioContentPart::Text { text: prompt.to_string() },
                    ],
                },
            ],
            max_tokens: Some(100),
        };
        
        let url = format!("{}{}", self.base_url, "/multimodal/chat/completions");
        let response = self.client
            .post(&url)
            .json(&request)
            .send()
            .await?
            .json::<ChatResponse>()
            .await?;
        
        Ok(response.choices[0].message.content)
    }
    
    pub async fn health_check(&self) -> bool {
        let url = format!("{}{}", self.base_url.replace("/v1", ""), "/health");
        self.client.get(&url).send().await.is_ok()
    }
}

// Usage in main.rs
#[actix_web::main]
async fn main() -> std::io::Result<()> {
    let markbase = MarkBaseClient::new(
        "http://10.10.10.201:8080/v1",
        "gemma-4-e4b-markbase",
    );
    
    // Test connection
    if !markbase.health_check().await {
        eprintln!("MarkBaseServer not responding!");
    }
    
    // Use in routes
    HttpServer::new(|| {
        App::new()
            .app_data(web::Data::new(markbase.clone()))
            .route("/api/chat", web::post().to(chat_handler))
            .route("/api/vision", web::post().to(vision_handler))
            .route("/api/audio", web::post().to(audio_handler))
    })
    .bind("127.0.0.1:3000")?
    .run()
    .await
}

---

12. Monitoring & Logging

Performance Metrics

use std::time::Instant;

async fn monitored_chat(client: &MarkBaseClient, prompt: &str) -> Result<(String, u64), MarkBaseError> {
    let start = Instant::now();
    let response = client.chat(prompt).await?;
    let latency_ms = start.elapsed().as_millis() as u64;
    
    // Log to monitoring system
    log::info!("Chat latency: {}ms, tokens: {}", latency_ms, response.len());
    
    Ok((response, latency_ms))
}

Structured Logging

use serde_json::json;

fn log_request(model: &str, prompt_len: usize, latency_ms: u64) {
    let log_entry = json!({
        "timestamp": chrono::Utc::now().to_rfc3339(),
        "model": model,
        "prompt_length": prompt_len,
        "latency_ms": latency_ms,
        "server": "MarkBaseServer",
    });
    
    println!("{}", log_entry);
}

---

Summary

This guide provides complete integration patterns for:

1. Text Models: Simple chat completion via /v1/chat/completions
2. Vision Models: Image analysis via /v1/multimodal/chat/completions with base64 images
3. Audio Models: Audio processing via /v1/multimodal/chat/completions with base64 audio
4. Streaming: SSE support for real-time UI updates
5. Model Selection: Choose based on speed/quality tradeoff
6. Performance: Optimized for Apple Silicon Metal GPU

Next Steps

1. Set up MarkBaseServer on production server (M5Max48)
2. Integrate Rust client into momentry_core
3. Build frontend UI with streaming support
4. Add authentication and rate limiting
5. Deploy and monitor performance

---

Document Version: 1.0 Last Updated: 2026-06-23 Author: MarkBaseEngine Team