apiary byo llm implementation

Introduction

Apiary's Bring Your Own Large Language Model (BYO-LLM) implementation allows founders to integrate their preferred LLM into the platform, while maintaining user key security and control. This approach empowers users to manage their own keys in-browser, ensuring that sensitive information remains encrypted and out of reach from the server.

Key Components

1. Client-Side Storage: User keys are stored securely in browser localStorage.

// Set user key in localStorage
localStorage.setItem('userKey', 'encrypted_user_key');

1. LLM Client: A client-side library dispatches LLM calls, utilizing the stored user key for authentication.

// Import LLM client library
import { LLMClient } from '@apiary/byo-llm-client';

// Initialize LLM client with user key
const llmClient = new LLMClient('encrypted_user_key');

1. Server Integration: The server acts as a proxy, forwarding requests to the LLM while remaining agnostic to user keys.

// Server-side code (Node.js example)
import express from 'express';
import { LLMProxy } from '@apiary/byo-llm-proxy';

const app = express();
const llmProxy = new LLMProxy();

app.post('/llm', (req, res) => {
  const llmRequest = req.body;
  const response = await llmProxy.forward(llmRequest);
  res.json(response);
});

Security Considerations

By storing user keys in-browser and using a client-side library for LLM calls, Apiary's BYO-LLM implementation ensures that sensitive information remains encrypted and secure.

• Key Management: Users retain control over their encryption keys, reducing the risk of unauthorized access.
• Server Agnosticism: The server never sees or stores user keys, minimizing exposure to potential security vulnerabilities.

Code Snippets

Client-Side LLM Call

// Dispatch LLM call using client-side library
const llmResponse = await llmClient.call({
  prompt: 'What is the meaning of life?',
});
console.log(llmResponse);

Server-Side Proxy Integration

// Forward LLM request to client-side library
const llmRequest = req.body;
const response = await llmProxy.forward(llmRequest);
res.json(response);

Related/Sources

• Apiary BYO-LLM Documentation
• LLM Client Library
• LLM Proxy Library