MCP Gateway

The Problem

Every MCP server exposes its own set of tools. Connect five servers and you have 50+ tools cluttering your agent's context. Connect twenty and you're drowning. LLMs don't scale well when you dump hundreds of tool definitions into their context window.

The Idea

What if every MCP server in the world was accessible through just two commands?

getBestTool(prompt)  → finds the right tool for what you need
executeTool(toolInfo) → runs it and returns results

That's MCP Gateway. A single unified layer that sits in front of any number of MCP servers and makes them all accessible through semantic search and a standard execution interface.

How It Works

Tool Discovery — `getBestTool(prompt)`

When you connect an MCP server, the gateway:

Ingests all tool definitions from the server via the MCP tools/list protocol
Indexes them in MeiliSearch with metadata (provider, tags, descriptions)
Generates embeddings via a RAG API for semantic search

When your agent needs a tool, it calls the search endpoint with a natural language query. The gateway returns the best-matching tools ranked by semantic relevance — not keyword matching, but understanding what you actually need.

Tool Execution — `executeTool(toolInfo)`

Once you have the right tool, execution is a single MCP tools/call:

Gateway resolves the tool to its source connector
Looks up stored credentials (OAuth tokens, API keys)
Proxies the call to the backend MCP server over HTTP
Returns the result with retry logic and error handling

Architecture

Your Agent (Claude, GPT, etc.)
    ↓ MCP JSON-RPC
MCP Gateway (FastAPI)
    ├── Auth (JWT/OIDC)
    ├── Session Management (Redis)
    ├── Tool Search (MeiliSearch + RAG embeddings)
    ├── Credential Vault (encrypted OAuth/API keys)
    └── HTTP Proxy → MCP Server A
                   → MCP Server B
                   → MCP Server C
                   → ...

Key Features

Semantic tool search — Find the right tool from hundreds using natural language
OAuth 2.0 + API key management — Secure credential storage with per-user scoping
Session-based execution — Stateful MCP sessions with tool binding caching
Observability — OpenTelemetry traces, Prometheus metrics, Grafana dashboards
Multi-tenant — Per-user connector isolation with OIDC authentication

Current Stack

Component	Technology
API	FastAPI (Python)
Search	MeiliSearch + RAG embeddings
Database	MongoDB
Sessions	Redis
Auth	JWT / OIDC (Keycloak)
Monitoring	OpenTelemetry + Prometheus + Grafana

Scaling Strategy

The gateway is designed to handle thousands of tools through:

Usage-driven caching — Hot tools surfaced first
Dynamic top-k selection — Agents search before each call instead of loading all tools
Meta-tools pattern — Expose only findRelevantTool and executeTool, keeping agent context minimal

MCP Gateway is in active development. The core search + execute loop is working. OAuth flows, multi-tenant isolation, and the observability stack are built. Connector ecosystem is growing.