DeepWiki-Open: Automatically Generating AI Documentation for GitHub, GitLab Repositories

General Introduction

DeepWiki-Open is an open source project designed to automatically generate structured documentation for the code repositories of GitHub, GitLab and Bitbucket. It uses AI technology to analyze the code structure, file content and logical relationships, and quickly generate a knowledge base page similar to Wikipedia. Users only need to enter the repository URL, you can get a clear description of the code architecture, functional modules and implementation details. The project supports local deployment with data stored locally for privacy and security. It also provides Retrieval Augmented Generation (RAG) functionality, allowing users to engage in contextual Q&A with the codebase and delve deeper into complex problems with DeepResearch functionality.Developed by AsyncFuncAI, DeepWiki-Open is free and open source and is suitable for developers, teams, and open-source communities to help quickly understand complex code.

Function List

• Automatic Documentation Generation: Analyze GitHub, GitLab, or Bitbucket repositories to generate structured wiki-style documentation.
• Code Intelligence Analysis: Identifies code structure, key modules and file relationships, providing clear descriptions.
• Contextual Q&A: Adoption RAG technology to support users in asking warehouse-related questions and getting accurate answers.
• DeepResearch Function: Supports multiple rounds of research, analyzes complex issues in depth, and provides detailed conclusions.
• Local Deployment Support: Data is stored locally, no cloud service is required, data privacy is guaranteed.
• Interactive diagrams: Generate Mermaid-formatted architecture diagrams and flowcharts to visualize code logic.
• Multi-platform support: Compatible with GitHub, GitLab and Bitbucket, supporting both public and private repositories.
• Streaming AI Responses: via Google Gemini The model provides real-time document generation and Q&A.
• Open source free: completely open source, users are free to modify and extend the function.

Using Help

Installation process

DeepWiki-Open supports Docker and manual installation for Ubuntu, macOS or Windows (via WSL2). Here are the detailed steps.

1. Cloning of warehouses

Clone DeepWiki-Open's code base locally:

git clone https://github.com/AsyncFuncAI/deepwiki-open.git
cd deepwiki-open

2. Configuring environment variables

DeepWiki-Open requires Google Gemini and OpenAI API keys for document generation and code embedding. In the project root directory, create the .env Documentation:

echo "GOOGLE_API_KEY=your_google_api_key" > .env
echo "OPENAI_API_KEY=your_openai_api_key" >> .env

• GOOGLE_API_KEY: from Google AI Studio Acquisition, for use in Google Gemini models.
• OPENAI_API_KEY: from OpenAI Platform Fetch for generating code embedding.
• Optional Variables:
  • PORT: API server port, default 8001The
  • NEXT_PUBLIC_SERVER_BASE_URL: API server address, default http://localhost:8001The

3. Installation with Docker (recommended)

Docker is easy to install and suitable for rapid deployment.

• Ensure that Docker and Docker Compose are installed.
• Run the following command to start the service:

docker-compose up

• Or pull the pre-built image and run it:

docker pull ghcr.io/asyncfuncai/deepwiki-open:latest
docker run -p 8001:8001 -p 3000:3000 \
-e GOOGLE_API_KEY=your_google_api_key \
-e OPENAI_API_KEY=your_openai_api_key \
-v ~/.adalflow:/root/.adalflow \
ghcr.io/asyncfuncai/deepwiki-open:latest

• After the service starts, the API runs on the http://localhost:8001The front-end page is in the http://localhost:3000The
• It is also possible to mount the .env Documentation:

docker run -p 8001:8001 -p 3000:3000 \
-v $(pwd)/.env:/app/.env \
-v ~/.adalflow:/root/.adalflow \
ghcr.io/asyncfuncai/deepwiki-open:latest

4. Manual installation

If you are not using Docker, you can install it manually.

• Install Python dependencies (backend):

pip install -r api/requirements.txt

• Start the API service:

python -m api.main

• Install JavaScript dependencies (front-end):

npm install

• Start the front-end development server:

npm run dev

• The front-end page in the http://localhost:3000The

5. Verification of installation

• Open your browser and visit http://localhost:3000The
• Enter the repository URL (e.g. https://github.com/facebook/react maybe https://bitbucket.org/redradish/atlassian_app_versions).
• Click "Generate Wiki" to view the generated document.

Main Functions

Automatic Document Generation

1. On the front-end page, enter the repository URL (e.g. https://gitlab.com/gitlab-org/gitlab).
2. For private repositories, click "+ Add access tokens" and enter your personal access token for GitHub, GitLab, or Bitbucket.
3. Click "Generate Wiki", the system clones the repository and analyzes the code to generate documentation.
4. The generated documentation includes:
   • Code Architecture Overview.
   • Functional description of key modules.
   • Detailed descriptions of files and directories.
   • Interactive Mermaid Architecture Diagram and Flowchart.

Contextualized questions and answers (Ask function)

1. Find the "Ask" area on the Documentation page.
2. Enter a question such as "How is the database connection for this repository implemented?" .
3. The system retrieves the code snippets through RAG technique and generates contextually relevant answers.
4. Answers are displayed in a streaming fashion, supporting real-time interaction.

DeepResearch Features

1. Enable the "Deep Research" switch in the "Ask" screen.
2. Enter complex questions such as "How can the authentication process be optimized for this warehouse?" .
3. Systematic execution of multiple rounds of research:
   • research program : Generate the initial analysis framework.
   • Research Updates : Iterative analysis to add details.
   • final conclusion : Synthesize all information and provide detailed answers.
4. The research process was iterated up to 5 times and the responses were displayed in a streaming format.

Featured Function Operation

Streaming AI Response

• Document generation and Q&A responses are displayed in real time using Google Gemini models.
• Example: After entering a question, the answer is loaded paragraph by paragraph to reduce waiting time.

Interactive Mermaid Charts

• Architecture diagram and flowchart of the system-generated warehouse, based on Mermaid technology.
• Charts support zooming, dragging and clicking interactions.
• Example: After generating a TensorFlow repository document, you can view the model training flowchart.

Support for private repositories

• A personal access token for GitHub, GitLab, or Bitbucket is required.
• The token is only used for cloning the repository and is not stored on the server.
• Example: Enter the private repository URL and token to generate the document.

Local data storage

• Documentation and analysis data are stored in the ~/.adalflow directory (Docker can customize the mount).
• No cloud dependency for privacy-sensitive scenarios.

Project structure

The code structure of DeepWiki-Open is as follows:

deepwiki/
├── api/                  # 后端 API 服务器
│   ├── main.py           # API 入口
│   ├── api.py            # FastAPI 实现
│   ├── rag.py            # RAG 功能实现
│   ├── data_pipeline.py  # 数据处理工具
│   └── requirements.txt  # Python 依赖
├── src/                  # 前端 Next.js 应用
│   ├── app/              # Next.js 页面
│   │   └── page.tsx      # 主页面
│   └── components/       # React 组件
│       └── Mermaid.tsx   # Mermaid 图表渲染
├── public/               # 静态资源
├── package.json          # JavaScript 依赖
└── .env                  # 环境变量

caveat

• Ensure that the API key is valid; an invalid key will cause the generation to fail.
• Docker deployments are recommended to allocate more than 4GB of memory.
• Manual installation requires Python 3.8+ and Node.js 16+.
• Larger repositories may take longer to generate documentation, and it is recommended to start testing with smaller repositories.
• If you encounter CORS errors, ensure that the front and back ends are running on the same machine, or adjust the NEXT_PUBLIC_SERVER_BASE_URLThe

application scenario

1. Quick start for new developers
   New developers can generate project documentation to quickly understand code structure and module functionality, shortening the learning curve.
2. Open Source Project Contribution
   Contributors quickly master repository logic and improve code submission efficiency through documentation and Q&A features.
3. Internal document maintenance
   Enterprises deploy DeepWiki-Open to generate documentation for private repositories, reducing manual maintenance costs.
4. Technical interview preparation
   Candidates analyze the target company's open source repository, generate documentation, and then dive into the technical details with DeepResearch.
5. Education and learning
   Students use DeepWiki-Open to analyze well-known projects (e.g. React, Django) and learn about code design through diagrams and quizzes.

QA

1. Is DeepWiki-Open free?
   DeepWiki-Open is completely free and open source, but users are required to provide API keys for Google and OpenAI, and API usage may incur costs.
2. What code hosting platforms are supported?
   Support for GitHub, GitLab, and Bitbucket, covering both public and private repositories.
3. How to export the generated document?
   Documents are stored locally ~/.adalflowThe file can be copied manually into Markdown or other formats.
4. What is the difference between DeepResearch and Ask features?
   Ask provides a single contextual quiz, and DeepResearch dives deeper into complex problems with multiple rounds of iterations.
5. What if the document generation fails?
   Check the API key, repository URL format, or try a small repository; check the terminal logs for error details.