llama-index-llms-openai-like

llama-index-llms-openai-like is a LlamaIndex integration package that lets you use any OpenAI-API-compatible endpoint as an LLM within the LlamaIndex framework. It provides the OpenAILike class, a thin wrapper around LlamaIndex’s OpenAI client that points at a custom api_base and lets you declare model capabilities explicitly.

This is the standard way to connect self-hosted or third-party inference servers, such as local runtimes (LM Studio, vLLM, llama.cpp servers), and OpenAI-compatible providers, into LlamaIndex pipelines. You supply the model name, API base URL, context window, and flags for whether the model is a chat model and supports function calling, and it plugs into the rest of LlamaIndex’s querying, agent, and RAG machinery.

What You Get

• An OpenAILike LLM class that targets any OpenAI-compatible api_base endpoint.
• Explicit configuration of context window, chat vs. completion mode, and function-calling support.
• Drop-in compatibility with LlamaIndex query engines, agents, and RAG pipelines.
• A thin, well-scoped wrapper over the LlamaIndex OpenAI client for non-OpenAI backends.

Common Use Cases

• Using a locally hosted model server (vLLM, LM Studio, llama.cpp) as the LLM in LlamaIndex.
• Connecting an OpenAI-compatible third-party inference provider to LlamaIndex pipelines.
• Swapping the underlying model behind a LlamaIndex RAG or agent workflow without changing app logic.

Under The Hood

Architecture - The package is a small integration under llama-index-integrations/llms/llama-index-llms-openai-like/llama_index/llms/openai_like/, defining OpenAILike (and OpenAILikeResponses) as subclasses of the LlamaIndex OpenAI LLM that relax OpenAI-specific assumptions: they accept an arbitrary api_base, take explicit context_window, is_chat_model, and is_function_calling_model flags, and otherwise reuse the parent’s request/response handling. Tech Stack - Python built on llama-index-llms-openai and llama-index-core, packaged with hatch as part of the run-llama/llama_index monorepo; it targets the same Python versions as core LlamaIndex. Code Quality - As an official monorepo integration it inherits the project’s tooling (tests, mypy, codespell, shared CI) and is intentionally minimal, delegating almost all behavior to the well-tested OpenAI integration it extends. API Design - Instantiation is a single constructor call mirroring the OpenAI LLM plus a few capability flags, so developers already familiar with LlamaIndex LLMs can adopt it with almost no new concepts.