/ Home

GenAI Frameworks

Note: genai framework collection

1: Agno

Agno is a platform for building and managing AI agents.

2: Autogen

AutoGen (GitHub) is an open-source framework developed by Microsoft for building LLM applications, including agents capable of complex reasoning and interactions. AutoGen simplifies the creation of conversational agents that can collaborate or compete to solve tasks.

3: BeeAI

BeeAI Framework is a comprehensive toolkit, developed by IBM Research, for building intelligent, autonomous agents and multi-agent systems. It provides everything you need to create agents that can reason, take actions, and collaborate to solve complex problems in both Python and TypeScript.

4: CrewAI

CrewAI (GitHub) is a framework for orchestrating autonomous AI agents. CrewAI enables you to create AI teams where each agent has specific roles, tools, and goals, working together to accomplish complex tasks. Each member (agent) brings unique skills and expertise, collaborating seamlessly to achieve your objectives.

5: DSPy

DSPy is a framework that systematically optimizes language model prompts and weights, making it easier to build and refine complex systems with LMs by automating the tuning process and improving reliability. For further information on DSPy, please visit the documentation.

6: ADK

Google’s Agent Development Kit streamlines building, orchestrating, and tracing generative-AI agents out of the box, letting you move from prototype to production far faster than wiring everything yourself.

7: Haystack

Haystack is the open-source Python framework developed by deepset. Its modular design allows users to implement custom pipelines to build production-ready LLM applications, like retrieval-augmented generative pipelines and state-of-the-art search systems. It integrates with Hugging Face Transformers, Elasticsearch, OpenSearch, OpenAI, Cohere, Anthropic and others, making it an extremely popular framework for teams of all sizes.

8: Instructor

Instructor is a popular library to get structured LLM outputs.Instructor makes it easy to reliably get structured data like JSON from Large Language Models (LLMs) like GPT-3.5, GPT-4, GPT-4-Vision, including open source models like Mistral/Mixtral from Together, Anyscale, Ollama, and llama-cpp-python. By leveraging various modes like Function Calling, Tool Calling and even constrained sampling modes like JSON mode, JSON Schema; Instructor stands out for its simplicity, transparency, and user-centric design. Under the hood, Instructor leverages Pydantic to do the heavy lifting, and provides a simple, easy-to-use API on top of it by helping you manage validation context, retries with Tenacity, and streaming Lists and Partial responses.

https://python.useinstructor.com/

9: LangChain

LangChain is an open-source framework for building applications powered by large language models (LLMs), available in both Python and JavaScript. It provides modular components like chains, agents, memory, and retrieval to connect LLMs with data, tools, and workflows. Its ecosystem also includes LangSmith (observability), LangGraph (agent orchestration), and LangServe (deployment).

10: LiteLLM

Use any LLM as a drop in replacement for GPT. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs). The LiteLLM SDK is a Python library that allows you to use any LLM as a drop in replacement for the OpenAI SDK.

11: LiveKit

LiveKit Agents (repo) is an open-source Python and Node.js framework for building production-grade multimodal and voice AI agents. It provides a complete set of tools and abstractions for feeding realtime media through AI pipelines, supporting both high-performance STT-LLM-TTS voice pipelines and speech-to-speech models with any AI provider.

https://github.com/livekit/agents

12: LlamaIndex

LlamaIndex (GitHub) is an advanced “data framework” tailored for augmenting LLMs with private data. It streamlines the integration of diverse data sources and formats (APIs, PDFs, docs, SQL, etc.) through versatile data connectors and structures data into indices and graphs for LLM compatibility. The platform offers a sophisticated retrieval/query interface for enriching LLM inputs with context-specific outputs.

https://github.com/run-llama/llama_index

13: LlamaIndex Workflows

LlamaIndex Workflows is a flexible, event-driven framework designed to build robust AI agents. In LlamaIndex, workflows are created by chaining together multiple steps—each defined and validated using the @step decorator. Every step processes specific event types, allowing you to orchestrate complex processes such as AI agent collaboration, RAG flows, data extraction, and more.

14: Mastra

Mastra is the TypeScript agent framework designed to provide the essential primitives for building AI applications. It enables developers to create AI agents with memory and tool-calling capabilities, implement deterministic LLM workflows, and leverage RAG for knowledge integration.

https://mastra.ai/

15: Mirascope

Developing LLM-applications with Mirascope feels just like writing the Python code you’re already used to. Python Toolkit for LLMs: Mirascope simplifies the development of applications using Large Language Models (LLMs) by providing an intuitive interface similar to standard Python coding practices.

https://github.com/mirascope/mirascope

16: OpenAI Agents SDK

The OpenAI Agents SDK is a lightweight, open-source framework that lets developers build AI agents and orchestrate multi-agent workflows. It provides building blocks—such as tools, handoffs, and guardrails to configure large language models with custom instructions and integrated tools. Its Python-first design supports dynamic instructions and function tools for rapid prototyping and integration with external systems.

https://openai.github.io/openai-agents-python/

17: Pipecat

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents. Developed by Daily, it enables fully programmable AI voice agents and supports multimodal interactions, positioning itself as a flexible solution for developers looking to build conversational AI systems.

https://github.com/pipecat-ai/pipecat

18: Pydantic AI

PydanticAI is a Python agent framework designed to simplify the development of production-grade generative AI applications. It brings the same type-safety, ergonomic API design, and developer experience found in FastAPI to the world of GenAI app development.

19: Quarkus LangChain4j

A Quarkus based integration of LangChain4j for AI development with built-in OTel tracing for AI calls.

20: Ragas

Ragas is an open-source tool designed for model-based evaluation of RAG pipelines. It performs reference-free evaluations, meaning you don’t need ground-truth data to assess your system’s performance. Ragas can evaluate various aspects of your RAG pipeline, such as faithfulness, answer relevancy, and context precision.

Common use cases for Ragas include:

• Evaluate RAG Pipelines: Generate synthetic test sets and assess pipeline performance using metrics like faithfulness and answer relevance.
• Custom Prompt Adaptation: Write and optimize custom prompts with automatic adaptation for improved retrieval and generation.
• CI Pipeline Integration: Integrate Ragas with CI pipelines using Pytest for automated evaluation and monitoring.

21: Semantic Kernel

Semantic Kernel is a powerful open-source SDK from Microsoft. It facilitates the combination of LLMs with popular programming languages like C#, Python, and Java. Semantic Kernel empowers developers to build sophisticated AI applications by seamlessly integrating AI services, data sources, and custom logic, accelerating the delivery of enterprise-grade AI solutions.

https://learn.microsoft.com/en-us/semantic-kernel/overview/

22: Smolagents

Smolagents is a minimalist and open-source AI agent framework developed by Hugging Face, designed to simplify the creation and deployment of powerful agents with just a few lines of code. It focuses on simplicity and efficiency, making it easy for developers to leverage LLMs for various applications.

23: Spring AI

Spring-based framework for AI development with built-in OTel tracing for AI calls.

24: Strands Agents SDK

The Strands Agents SDK, developed by AWS, is a toolkit for building AI agents that can interact with various tools and services, including AWS Bedrock.

https://strandsagents.com/latest/

25: Vercel AI SDK

The Vercel AI SDK is the TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js, and more.

https://ai-sdk.dev/docs/introduction

26: VoltAgent

VoltAgent is an open-source TypeScript framework that acts as this essential toolkit. It simplifies the development of AI agent applications by providing modular building blocks, standardized patterns, and abstractions. Whether you’re creating chatbots, virtual assistants, automated workflows, or complex multi-agent systems, VoltAgent handles the underlying complexity, allowing you to focus on defining your agents’ capabilities and logic.