XWDoc — API Documentation Browser

Note: The project is completely generated by AI（Github Copilot） and is still in the process of conversion. There may be some errors and incomplete parts.

XWDoc is a Rust implementation of DevDocs, an API documentation browser that combines multiple developer documents in a clean and well-organized web interface, featuring instant search, offline support, mobile version, dark theme, keyboard shortcuts, etc.

XWDoc是DevDocs的Rust实现，一个API文档浏览器，它将多个开发者文档结合在一个干净、组织良好的Web界面中，具有即时搜索、离线支持、移动版本、深色主题、键盘快捷键等功能。

Start

服务器命令

# 启动服务器（默认端口3000）
cargo run server

# 指定主机和端口
cargo run server --host 0.0.0.0 --port 8080

文档命令

# 列出可用的文档
cargo run docs list

# 下载文档
cargo run docs download babel html css

# 下载所有可用文档
cargo run docs download --all

# 下载默认文档集
cargo run docs download --default

# 更新已安装的文档
cargo run docs download --installed

# 生成文档
cargo run docs generate babel

# 生成特定版本的文档
cargo run docs generate babel --version 7.14

# 生成单个页面
cargo run docs page babel usage

# 打包文档
cargo run docs package babel

# 清理文档包
cargo run docs clean

# 生成文档清单
cargo run docs manifest

项目结构

项目基于Rust的模块化设计，分为以下主要组件：

1. 核心组件 (core): 提供配置、错误处理和基本类型定义。

2. 文档组件 (docs): 管理文档元数据和索引信息。

3. 抓取器 (scrapers): 从不同来源（网络或本地文件）抓取文档内容。

   • URL抓取器：从网络下载文档
   • 文件抓取器：从本地文件系统读取文档
   • 特定文档类型抓取器：适用于HTML、CSS、JavaScript、Rust、TypeScript、Babel等

4. 过滤器 (filters): 处理和清理HTML内容，包括：

   • HTML过滤器：清理HTML结构，标准化URL等
   • 文本过滤器：处理和提取内容
   • URL过滤器：处理URL规范化和内部链接处理

5. 存储 (storage): 存储和检索文档文件。

6. Web服务器 (web): 提供Web界面，包括路由、处理程序和静态文件服务。

7. 命令行工具 (cli): 提供命令行交互。

开发

先决条件

• Rust 1.54.0或更高版本
• Cargo包管理器

构建

# 构建项目
cargo build

# 发布构建
cargo build --release

测试

# 运行所有测试
cargo test

# 运行特定测试
cargo test babel_scraper

贡献

欢迎贡献代码！请遵循以下步骤：

1. Fork仓库
2. 创建功能分支 (git checkout -b feature/amazing-feature)
3. 提交更改 (git commit -m 'Add some amazing feature')
4. 推送到分支 (git push origin feature/amazing-feature)
5. 打开Pull Request

Supported Documentation Types

The project supports several documentation types, with specialized scrapers for each:

• HTML
• CSS
• JavaScript
• TypeScript
• Rust
• Babel (JavaScript compiler)

Recent Improvements

Improved Babel Documentation Scraper

We've recently made significant improvements to the Babel documentation scraper:

1. Simplified Code Structure: Reimplemented the scraper with more concise, maintainable code that better matches the original Ruby implementation.

2. Fixed URL Handling: Resolved issues with malformed URLs (such as https://babeljs.io/docs/https://github.com) that were being generated in the original implementation.

3. Enhanced Entry Classification: Improved the logic for categorizing documentation entries into their proper types.

For more details about these improvements, see the Babel Scraper Improvements documentation.

To test the improved Babel scraper:

# Run the main program with improved Babel scraper
cargo run

# Run the integration test to compare original and improved implementations
cargo run --bin babel_integration_test

Testing

The project includes both unit tests and integration tests:

# Run all tests
cargo test

# Run specific tests
cargo test url_scraper

Extension Development

If you wish to extend the project, here are some common development tasks:

Adding a New Document Scraper

1. Create a new scraper module in the src/scrapers/docs directory
2. Implement the Scraper trait
3. Add the new document type in the scrape function in lib.rs

Adding a New Filter

1. Create a new filter in the src/filters directory
2. Implement the Filter trait
3. Use the filter in the Pipeline

Improving the Web Interface

1. Update routes in src/web/routes.rs
2. Add new request handlers in src/web/handlers.rs

Contributing

Contributions are welcome! Please follow these steps:

1. Fork the project
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push the branch (git push origin feature/amazing-feature)
5. Create a Pull Request

Acknowledgements

• DevDocs - The original project and source of inspiration
• All contributors and developers