A unified gateway server for managing interactions with multiple LLM providers (OpenAI, Anthropic, DeepSeek, Gemini) with built-in cost optimization, caching, and monitoring. Features include automatic model selection based on task requirements, semantic caching to reduce redundant API calls, detailed usage analytics, and a CLI for direct interaction. The implementation focuses on reliability with comprehensive error handling, request retries, and rate limiting, making it particularly valuable for production deployments that need to balance performance and costs across multiple LLM providers.
No reviews yet. Be the first to review!
Sign in to join the conversation
Highlight AIDivides a document into smaller segments or chunks. Parameters: document (string), chunk_size (integer), overlap (integer), method (string)
Generates a concise summary of a given document. Parameters: document (string), format (string), max_length (integer)
Extracts structured JSON data from a document according to a specified schema. Parameters: document (string), json_schema (JSON object)
Navigates to a specified URL using a browser automation tool. Parameters: url (string), wait_until (string)
Extracts text content from the page based on a given CSS selector. Parameters: selector (string)
Processes an image file to extract text using OCR. Parameters: image_path (string), preprocessing_options (JSON object), ocr_language (string)
Extracts text from a PDF file using OCR and LLM correction. Parameters: file_path (string), extraction_method (string), max_pages (integer), reformat_as_markdown (boolean)
Executes a jq command on provided JSON input data. Parameters: args_str (string), input_data (string)
Registers an external API using its OpenAPI specification. Parameters: api_name (string), openapi_url (string)
Calls a dynamically registered tool corresponding to an API endpoint. Parameters: tool_name (string), inputs (JSON object)
Lists all currently registered dynamic APIs.
Unregisters a previously registered API. Parameters: api_name (string)
Runs a tournament among different models for a specified task. Parameters: task_type (string), prompt (string), competitors (list of JSON objects), evaluation_criteria (list of strings)
Lists all available tools along with their descriptions and schemas. Parameters: include_schemas (boolean), tool_names (optional list of strings)
Provides tool recommendations based on a specified task description. Parameters: task (string), constraints (JSON object)
