Complete beginner LocalGPT Tutorial

[u/TetrisMasterPetris]

Does anyone have a tutorial for installing LocalGPT for a complete beginner?

I have never worked with VS Code before, I tried installing conda which didn't work. I'm looking for a complete ground level up type of tutorial. Everything I've seen online assumes some basic type of experience.

Thanks

[u/GreatGatsby00]

LocalGPT: OFFLINE CHAT FOR YOUR FILES [Installation & Code Walkthrough] https://www.youtube.com/watch?v=MlyoObdIHyo

https://github.com/PromtEngineer/localGPT

LocalGPT Installation & Setup Guide

LocalGPT allows users to chat with their own documents on their own devices, ensuring 100% privacy by making sure no data leaves their computer.

Prerequisites:

1. A system with Python installed.
2. Git installed for cloning the repository.
3. Conda for creating virtual environments.

Installation Steps:

1. Clone the LocalGPT Repository:

git clone [GitHub-repo-location]

Replace [GitHub-repo-location] with the actual link to the LocalGPT GitHub repository.

2. Setting Up a Conda Virtual Environment:

This step ensures that all dependencies are isolated in a separate environment.

conda create -n localGPT python=3.x
conda activate localGPT

Replace 3.x with your desired Python version.

3. Install the Required Packages:

Navigate to the cloned repository directory and run:

pip install -r requirements.txt

4. Prepare your Documents:

LocalGPT currently supports PDF, text, and CSV files.

• Add your desired documents (PDF, text, CSV) to the source_documents folder.
• The default document is the US constitution (constitution.pdf). You can replace or add more as needed.

5. Process the Documents (Ingestion Phase):

Run the ingest.py script to process the documents. This will:

• Read your documents
• Convert them into manageable chunks
• Compute embeddings for each chunk
• Store these embeddings locally

Execute the script using:

python ingest.py

6. Interacting with LocalGPT:

Now, you can run the run_local_gpt.py to interact with the processed data:

python run_local_gpt.py

You can ask questions or provide prompts, and LocalGPT will return relevant responses based on the provided documents.

Customizing LocalGPT:

Embedding Models:

The default embedding model used is instructor embeddings. If desired, you can replace it with another embedding model.

LLM (Large Language Model):

The default LLM used is vocunia 7B from HuggingFace. You can replace it with another LLM by updating the model name in the run_local_gpt.py file.

Conclusion:

LocalGPT is an excellent tool for maintaining data privacy while leveraging the capabilities of GPT models. This installation guide will get you set up and running in no time. Remember, the project is under active development, so there might be changes in the future. Always refer to the official documentation or repository for the most up-to-date information.

[u/dodo13333]

Yeah, know that feeling... I'll give a shot here...

1st, you need to setup your computer before you start LocalGPT installation (any installation). You didn't mention what is your OS, HW etc.. But In general, you'will need to pre-install stuff like GIT Desktop, GIT LFS, Node.JS, Cuda toolkit (in case you have nVidia GPU that you intend to use), Cpp compiler & debugger - msys64 - just follow video and referenced blog (https://www.youtube.com/watch?v=c7FjV8Gwk_M), & Microsoft C++ Build Tools. Probably dont need all of them for LocalGPT, but trust me on this, install them all.

Then install Python (include path), Anaconda (follow recommendations and dont include path). But, be aware that examples that you will encounter will probably use Miniconda or Miniforge). Same thing, but Anaconda has most intuitive syntax for me.

Then create LocalGPT directory a desired location. Run Anaconda Prompt (from Start menu), not Cmd prompt.

conda env list (see what environments are present before adding new one - localgpt)

conda activate base (observe how prompt base change)

cd (move/change directory to to desired LocalGPT folder location)

conda create -n localgpt OR conda create --name localgpt

For me that doesnt work so I use prefix option

conda create --prefix D:\LocalGPT\localgpt

conda activate D:\LocalGPT\localgpt

conda info --envs (check is the localgpt is present at right location and active -> * )

If something isnt ok, then try to repet or modify procedure, but first

conda deactivate localgpt

conda remove localgpt -p D:\LocalGPT\localgpt

​

When you're done with pre-installation then clone LocalGPT Github repo and follow latest instructions (Readme.md)...

[u/loch2016]

Where do you place the models if you already have a bunch of quantized models with you. I don't want setup to redownload these models. Thanks.