Skip to content

Introduction

Frameworks are the blueprints to solve a particular problem with a Large Language Model(LLM).

Frameworks: - determine the interaction with supported LLMs. - have preprocessing for input prompts and post processing steps for LLM response, depending on the type of problem to solve. - can also interact with datasources (please refer to Datasource for further details.) to provide LLM with additional context to provide the best answer. - are developed by NEXTgpt contributors.

Types of Frameworks

Please check the sub sections for different types of Frameworks and how they can be used.

  1. AzureOpenAI Framework

  2. Prompt File Template (PFT) Framework

  3. RAG Framework

  4. SSRAG Framework

  5. Doc Score Framework

  6. Tabular Query Framework

  7. REPL Framework

  8. DSP Framework

  9. Meta Framework

Overview

Framework Supports upload Filetype supported Datasource required
AzureOpenAI No N.A. No
PFT Yes txt, pdf No
RAG No N.A. Yes
SSRAG No N.A. Yes
DocScore Yes txt, pdf No
Tabular Yes csv No
REPL Yes txt No
DSP No N.A. Yes
Meta * * *

* Meta Framework depends on the types of Frameworks used to create the usecase.

Comparison among RAG, SSRAG, and DSP Frameworks

RAG, SSRAG and DSP Frameworks are often be used for the similar type of usecases, which require user input question to retrieve relevant content from a datasource and send the retrieved content as context to the LLM along with the original user input question.

  • RAG framework takes input user question and send to LLM to generate a search query. The generated search query is sent to the datasoruce to retrieve the relevant content. Finally original input user question is fused with retrieved content and sent to LLM again to find the best answer.

    For example, RAG Framework could search a datasource of recipes to suggest a delicious dessert based on the ingredients list you provide as user input.

  • SSRAG (Single Shot RAG) is similar to RAG, except the fact that the input question is sent as it is to the datasource to retreve the relevant content.

  • DSP (Demonstrate, Search, and Predict) is a three-step approach that focuses on learning from labelled training data (e.g. Q&A pairs), search the datasource, and predicting the final answer. DSP is useful for finding answers for complex questions, which require to generate sub-questions to reach to the final answer.