base on PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/Docker <div align="center"> English | [简体中文](docs/README_zh-CN.md) | [日本語](docs/README_ja-JP.md) <img src="./docs/images/banner.png" width="320px" alt="PDF2ZH"/> <h2 id="title">PDFMathTranslate</h2> <p> <!-- PyPI --> <a href="https://pypi.org/project/pdf2zh/"> <img src="https://img.shields.io/pypi/v/pdf2zh"></a> <a href="https://pepy.tech/projects/pdf2zh"> <img src="https://static.pepy.tech/badge/pdf2zh"></a> <a href="https://hub.docker.com/repository/docker/byaidu/pdf2zh"> <img src="https://img.shields.io/docker/pulls/byaidu/pdf2zh"></a> <a href="https://gitcode.com/Byaidu/PDFMathTranslate/overview"> <img src="https://gitcode.com/Byaidu/PDFMathTranslate/star/badge.svg"></a> <a href="https://huggingface.co/spaces/reycn/PDFMathTranslate-Docker"> <img src="https://img.shields.io/badge/%F0%9F%A4%97-Online%20Demo-FF9E0D"></a> <a href="https://www.modelscope.cn/studios/AI-ModelScope/PDFMathTranslate"> <img src="https://img.shields.io/badge/ModelScope-Demo-blue"></a> <a href="https://github.com/Byaidu/PDFMathTranslate/pulls"> <img src="https://img.shields.io/badge/contributions-welcome-green"></a> <a href="https://t.me/+Z9_SgnxmsmA5NzBl"> <img src="https://img.shields.io/badge/Telegram-2CA5E0?style=flat-squeare&logo=telegram&logoColor=white"></a> <!-- License --> <a href="./LICENSE"> <img src="https://img.shields.io/github/license/Byaidu/PDFMathTranslate"></a> </p> <a href="https://trendshift.io/repositories/12424" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12424" alt="Byaidu%2FPDFMathTranslate | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> </div> PDF scientific paper translation and bilingual comparison. - 📊 Preserve formulas, charts, table of contents, and annotations _([preview](#preview))_. - 🌐 Support [multiple languages](#language), and diverse [translation services](#services). - 🤖 Provides [commandline tool](#usage), [interactive user interface](#gui), and [Docker](#docker) Feel free to provide feedback in [GitHub Issues](https://github.com/Byaidu/PDFMathTranslate/issues), [Telegram Group](https://t.me/+Z9_SgnxmsmA5NzBl) or [QQ Group](https://qm.qq.com/q/DixZCxQej0). <h2 id="updates">Updates</h2> - [Dec. 19 2024] Non-PDF/A documents are now supported using `-cp` _(by [@reycn](https://github.com/reycn))_ - [Dec. 13 2024] Additional support for backend by _(by [@YadominJinta](https://github.com/YadominJinta))_ - [Dec. 10 2024] The translator now supports OpenAI models on Azure _(by [@yidasanqian](https://github.com/yidasanqian))_ <h2 id="preview">Preview</h2> <div align="center"> <img src="./docs/images/preview.gif" width="80%"/> </div> <h2 id="demo">Online Service 🌟</h2> You can try our application out using either of the following demos: - [Public free service](https://pdf2zh.com/) online without installation _(recommended)_. - [Demo hosted on HuggingFace](https://huggingface.co/spaces/reycn/PDFMathTranslate-Docker) - [Demo hosted on ModelScope](https://www.modelscope.cn/studios/AI-ModelScope/PDFMathTranslate) without installation. Note that the computing resources of the demo are limited, so please avoid abusing them. <h2 id="install">Installation and Usage</h2> ### Methods For different use cases, we provide four distinct methods to use our program: <details open> <summary>1. Commandline</summary> 1. Python installed (3.8 <= version <= 3.12) 2. Install our package: ```bash pip install pdf2zh ``` 3. Execute translation, files generated in [current working directory](https://chatgpt.com/share/6745ed36-9acc-800e-8a90-59204bd13444): ```bash pdf2zh document.pdf ``` </details> <details> <summary>2. Portable (w/o Python installed)</summary> 1. Download [setup.bat](https://raw.githubusercontent.com/Byaidu/PDFMathTranslate/refs/heads/main/script/setup.bat) 2. Double-click to run. </details> <details> <summary>3. Graphic user interface</summary> 1. Python installed (3.8 <= version <= 3.12) 2. Install our package: ```bash pip install pdf2zh ``` 3. Start using in browser: ```bash pdf2zh -i ``` 4. If your browswer has not been started automatically, goto ```bash http://localhost:7860/ ``` <img src="./docs/images/gui.gif" width="500"/> See [documentation for GUI](./docs/README_GUI.md) for more details. </details> <details> <summary>4. Docker</summary> 1. Pull and run: ```bash docker pull byaidu/pdf2zh docker run -d -p 7860:7860 byaidu/pdf2zh ``` 2. Open in browser: ``` http://localhost:7860/ ``` For docker deployment on cloud service: <div> <a href="https://www.heroku.com/deploy?template=https://github.com/Byaidu/PDFMathTranslate"> <img src="https://www.herokucdn.com/deploy/button.svg" alt="Deploy" height="26"></a> <a href="https://render.com/deploy"> <img src="https://render.com/images/deploy-to-render-button.svg" alt="Deploy to Koyeb" height="26"></a> <a href="https://zeabur.com/templates/5FQIGX?referralCode=reycn"> <img src="https://zeabur.com/button.svg" alt="Deploy on Zeabur" height="26"></a> <a href="https://app.koyeb.com/deploy?type=git&builder=buildpack&repository=github.com/Byaidu/PDFMathTranslate&branch=main&name=pdf-math-translate"> <img src="https://www.koyeb.com/static/images/deploy/button.svg" alt="Deploy to Koyeb" height="26"></a> </div> </details> ### Unable to install? The present program needs an AI model(`wybxc/DocLayout-YOLO-DocStructBench-onnx`) before working and some users are not able to download due to network issues. If you have a problem with downloading this model, we provide a workaround using the following environment variable: ```shell set HF_ENDPOINT=https://hf-mirror.com ``` If the solution does not work to you / you encountered other issues, please refer to [frequently asked questions](https://github.com/Byaidu/PDFMathTranslate/wiki#-faq--%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98). <h2 id="usage">Advanced Options</h2> Execute the translation command in the command line to generate the translated document `example-mono.pdf` and the bilingual document `example-dual.pdf` in the current working directory. Use Google as the default translation service. <img src="./docs/images/cmd.explained.png" width="580px" alt="cmd"/> In the following table, we list all advanced options for reference: | Option | Function | Example | | -------------- | ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | | files | Local files | `pdf2zh ~/local.pdf` | | links | Online files | `pdf2zh http://arxiv.org/paper.pdf` | | `-i` | [Enter GUI](#gui) | `pdf2zh -i` | | `-p` | [Partial document translation](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#partial) | `pdf2zh example.pdf -p 1` | | `-li` | [Source language](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#languages) | `pdf2zh example.pdf -li en` | | `-lo` | [Target language](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#languages) | `pdf2zh example.pdf -lo zh` | | `-s` | [Translation service](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#services) | `pdf2zh example.pdf -s deepl` | | `-t` | [Multi-threads](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#threads) | `pdf2zh example.pdf -t 1` | | `-o` | Output dir | `pdf2zh example.pdf -o output` | | `-f`, `-c` | [Exceptions](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#exceptions) | `pdf2zh example.pdf -f "(MS.*)"` | | `-cp` | Compatibility Mode | `pdf2zh example.pdf --compatible` | | `--share` | Public link | `pdf2zh -i --share` | | `--authorized` | Authorization | `pdf2zh -i --authorized users.txt [auth.html]` | | `--prompt` | [Custom Prompt](https://github.com/Byaidu/PDFMathTranslate/blob/main/docs/ADVANCED.md#prompt) | `pdf2zh --prompt [prompt.txt]` | For detailed explanations, please refer to our document about [Advanced Usage](./docs/ADVANCED.md) for a full list of each option. <h2 id="downstream">Secondary Development (APIs)</h2> For downstream applications, please refer to our document about [API Details](./docs/APIS.md) for futher information about: - [Python API](./docs/APIS.md#api-python), how to use the program in other Python programs - [HTTP API](./docs/APIS.md#api-http), how to communicate with a server with the program installed <h2 id="todo">TODOs</h2> - [ ] Parse layout with DocLayNet based models, [PaddleX](https://github.com/PaddlePaddle/PaddleX/blob/17cc27ac3842e7880ca4aad92358d3ef8555429a/paddlex/repo_apis/PaddleDetection_api/object_det/official_categories.py#L81), [PaperMage](https://github.com/allenai/papermage/blob/9cd4bb48cbedab45d0f7a455711438f1632abebe/README.md?plain=1#L102), [SAM2](https://github.com/facebookresearch/sam2) - [ ] Fix page rotation, table of contents, format of lists - [ ] Fix pixel formula in old papers - [ ] Async retry except KeyboardInterrupt - [ ] Knuth–Plass algorithm for western languages - [ ] Support non-PDF/A files - [ ] Plugins of [Zotero](https://github.com/zotero/zotero) and [Obsidian](https://github.com/obsidianmd/obsidian-releases) <h2 id="acknowledgement">Acknowledgements</h2> - Document merging: [PyMuPDF](https://github.com/pymupdf/PyMuPDF) - Document parsing: [Pdfminer.six](https://github.com/pdfminer/pdfminer.six) - Document extraction: [MinerU](https://github.com/opendatalab/MinerU) - Document Preview: [Gradio PDF](https://github.com/freddyaboulton/gradio-pdf) - Multi-threaded translation: [MathTranslate](https://github.com/SUSYUSTC/MathTranslate) - Layout parsing: [DocLayout-YOLO](https://github.com/opendatalab/DocLayout-YOLO) - Document standard: [PDF Explained](https://zxyle.github.io/PDF-Explained/), [PDF Cheat Sheets](https://pdfa.org/resource/pdf-cheat-sheets/) - Multilingual Font: [Go Noto Universal](https://github.com/satbyy/go-noto-universal) <h2 id="contrib">Contributors</h2> <a href="https://github.com/Byaidu/PDFMathTranslate/graphs/contributors"> <img src="https://opencollective.com/PDFMathTranslate/contributors.svg?width=890&button=false" /> </a>  <h2 id="star_hist">Star History</h2> <a href="https://star-history.com/#Byaidu/PDFMathTranslate&Date"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Byaidu/PDFMathTranslate&type=Date&theme=dark" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Byaidu/PDFMathTranslate&type=Date" /> <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Byaidu/PDFMathTranslate&type=Date"/> </picture> </a> ", Assign "at most 3 tags" to the expected json: {"id":"12424","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"