base on Let's write an OS which can run on RISC-V in Rust from scratch! # rCore-Tutorial-v3 rCore-Tutorial version 3.6. See the [Documentation in Chinese](https://rcore-os.github.io/rCore-Tutorial-Book-v3/). rCore-Tutorial API Docs. See the [API Docs of Ten OSes ](#OS-API-DOCS) If you don't know Rust Language and try to learn it, please visit [Rust Learning Resources](https://github.com/rcore-os/rCore/wiki/study-resource-of-system-programming-in-RUST) Official QQ group number: 735045051 ## news - 23/06/2022: Version 3.6.0 is on the way! Now we directly update the code on chX branches, please periodically check if there are any updates. ## Overview This project aims to show how to write an **Unix-like OS** running on **RISC-V** platforms **from scratch** in **[Rust](https://www.rust-lang.org/)** for **beginners** without any background knowledge about **computer architectures, assembly languages or operating systems**. ## Features * Platform supported: `qemu-system-riscv64` simulator or dev boards based on [Kendryte K210 SoC](https://canaan.io/product/kendryteai) such as [Maix Dock](https://www.seeedstudio.com/Sipeed-MAIX-Dock-p-4815.html) * OS * concurrency of multiple processes each of which contains mutiple native threads * preemptive scheduling(Round-Robin algorithm) * dynamic memory management in kernel * virtual memory * a simple file system with a block cache * an interactive shell in the userspace * **only 4K+ LoC** * [A detailed documentation in Chinese](https://rcore-os.github.io/rCore-Tutorial-Book-v3/) in spite of the lack of comments in the code(English version is not available at present) ## Prerequisites ### Install Rust See [official guide](https://www.rust-lang.org/tools/install). Install some tools: ```sh $ rustup target add riscv64gc-unknown-none-elf $ cargo install cargo-binutils --vers =0.3.3 $ rustup component add llvm-tools-preview $ rustup component add rust-src ``` ### Install Qemu Here we manually compile and install Qemu 7.0.0. For example, on Ubuntu 18.04: ```sh # install dependency packages $ sudo apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev \ gawk build-essential bison flex texinfo gperf libtool patchutils bc \ zlib1g-dev libexpat-dev pkg-config libglib2.0-dev libpixman-1-dev git tmux python3 python3-pip ninja-build # download Qemu source code $ wget https://download.qemu.org/qemu-7.0.0.tar.xz # extract to qemu-7.0.0/ $ tar xvJf qemu-7.0.0.tar.xz $ cd qemu-7.0.0 # build $ ./configure --target-list=riscv64-softmmu,riscv64-linux-user $ make -j$(nproc) ``` Then, add following contents to `~/.bashrc`(please adjust these paths according to your environment): ``` export PATH=$PATH:/path/to/qemu-7.0.0/build ``` Finally, update the current shell: ```sh $ source ~/.bashrc ``` Now we can check the version of Qemu: ```sh $ qemu-system-riscv64 --version QEMU emulator version 7.0.0 Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers ``` ### Install RISC-V GNU Embedded Toolchain(including GDB) Download the compressed file according to your platform From [Sifive website](https://www.sifive.com/software)(Ctrl+F 'toolchain'). Extract it and append the location of the 'bin' directory under its root directory to `$PATH`. For example, we can check the version of GDB: ```sh $ riscv64-unknown-elf-gdb --version GNU gdb (SiFive GDB-Metal 10.1.0-2020.12.7) 10.1 Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ``` ### Install serial tools(Optional, if you want to run on K210) ```sh $ pip3 install pyserial $ sudo apt install python3-serial ``` ## Run our project ### Qemu ```sh $ git clone https://github.com/rcore-os/rCore-Tutorial-v3.git $ cd rCore-Tutorial-v3/os $ make run ``` After outputing some debug messages, the kernel lists all the applications available and enter the user shell: ``` /**** APPS **** mpsc_sem usertests pipetest forktest2 cat initproc race_adder_loop threads_arg race_adder_mutex_spin race_adder_mutex_blocking forktree user_shell huge_write race_adder race_adder_atomic threads stack_overflow filetest_simple forktest_simple cmdline_args run_pipe_test forktest matrix exit fantastic_text sleep_simple yield hello_world pipe_large_test sleep phil_din_mutex **************/ Rust user shell >> ``` You can run any application except for `initproc` and `user_shell` itself. To run an application, just input its filename and hit enter. `usertests` can run a bunch of applications, thus it is recommended. Type `Ctrl+a` then `x` to exit Qemu. ### K210 Before chapter 6, you do not need a SD card: ```sh $ git clone https://github.com/rcore-os/rCore-Tutorial-v3.git $ cd rCore-Tutorial-v3/os $ make run BOARD=k210 ``` From chapter 6, before running the kernel, we should insert a SD card into PC and manually write the filesystem image to it: ```sh $ cd rCore-Tutorial-v3/os $ make sdcard ``` By default it will overwrite the device `/dev/sdb` which is the SD card, but you can provide another location. For example, `make sdcard SDCARD=/dev/sdc`. After that, remove the SD card from PC and insert it to the slot of K210. Connect the K210 to PC and then: ```sh $ git clone https://github.com/rcore-os/rCore-Tutorial-v3.git $ cd rCore-Tutorial-v3/os $ make run BOARD=k210 ``` Type `Ctrl+]` to disconnect from K210. ## Show runtime debug info of OS kernel version The branch of ch9-log contains a lot of debug info. You could try to run rcore tutorial for understand the internal behavior of os kernel. ```sh $ git clone https://github.com/rcore-os/rCore-Tutorial-v3.git $ cd rCore-Tutorial-v3/os $ git checkout ch9-log $ make run ...... [rustsbi] RustSBI version 0.2.0-alpha.10, adapting to RISC-V SBI v0.3 .______ __ __ _______.___________. _______..______ __ | _ \ | | | | / | | / || _ \ | | | |_) | | | | | | (----`---| |----`| (----`| |_) || | | / | | | | \ \ | | \ \ | _ < | | | |\ \----.| `--' |.----) | | | .----) | | |_) || | | _| `._____| \______/ |_______/ |__| |_______/ |______/ |__| [rustsbi] Implementation: RustSBI-QEMU Version 0.0.2 [rustsbi-dtb] Hart count: cluster0 with 1 cores [rustsbi] misa: RV64ACDFIMSU [rustsbi] mideleg: ssoft, stimer, sext (0x222) [rustsbi] medeleg: ima, ia, bkpt, la, sa, uecall, ipage, lpage, spage (0xb1ab) [rustsbi] pmp0: 0x10000000 ..= 0x10001fff (rw-) [rustsbi] pmp1: 0x2000000 ..= 0x200ffff (rw-) [rustsbi] pmp2: 0xc000000 ..= 0xc3fffff (rw-) [rustsbi] pmp3: 0x80000000 ..= 0x8fffffff (rwx) [rustsbi] enter supervisor 0x80200000 [KERN] rust_main() begin [KERN] clear_bss() begin [KERN] clear_bss() end [KERN] mm::init() begin [KERN] mm::init_heap() begin [KERN] mm::init_heap() end [KERN] mm::init_frame_allocator() begin [KERN] mm::frame_allocator::lazy_static!FRAME_ALLOCATOR begin ...... ``` ## Rustdoc Currently it can only help you view the code since only a tiny part of the code has been documented. You can open a doc html of `os` using `cargo doc --no-deps --open` under `os` directory. ### OS-API-DOCS The API Docs for Ten OS 1. [Lib-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch1/os/index.html) 1. [Batch-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch2/os/index.html) 1. [MultiProg-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch3-coop/os/index.html) 1. [TimeSharing-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch3/os/index.html) 1. [AddrSpace-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch4/os/index.html) 1. [Process-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch5/os/index.html) 1. [FileSystem-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch6/os/index.html) 1. [IPC-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch7/os/index.html) 1. [SyncMutex-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch8/os/index.html) 1. [IODevice-OS API doc](https://learningos.github.io/rCore-Tutorial-v3/ch9/os/index.html) ## Working in progress Our first release 3.6.0 (chapter 1-9) has been published, and we are still working on it. * chapter 9: need more descripts about different I/O devices Here are the updates since 3.5.0: ### Completed * [x] automatically clean up and rebuild before running our project on a different platform * [x] fix `power` series application in early chapters, now you can find modulus in the output * [x] use `UPSafeCell` instead of `RefCell` or `spin::Mutex` in order to access static data structures and adjust its API so that it cannot be borrowed twice at a time(mention `& .exclusive_access().task[0]` in `run_first_task`) * [x] move `TaskContext` into `TaskControlBlock` instead of restoring it in place on kernel stack(since ch3), eliminating annoying `task_cx_ptr2` * [x] replace `llvm_asm!` with `asm!` * [x] expand the fs image size generated by `rcore-fs-fuse` to 128MiB * [x] add a new test named `huge_write` which evaluates the fs performance(qemu\~500KiB/s k210\~50KiB/s) * [x] flush all block cache to disk after a fs transaction which involves write operation * [x] replace `spin::Mutex` with `UPSafeCell` before SMP chapter * [x] add codes for a new chapter about synchronization & mutual exclusion(uniprocessor only) * [x] bug fix: we should call `find_pte` rather than `find_pte_create` in `PageTable::unmap` * [x] clarify: "check validity of level-3 pte in `find_pte` instead of checking it outside this function" should not be a bug * [x] code of chapter 8: synchronization on a uniprocessor * [x] switch the code of chapter 6 and chapter 7 * [x] support signal mechanism in chapter 7/8(only works for apps with a single thread) * [x] Add boards/ directory and support rustdoc, for example you can use `cargo doc --no-deps --open` to view the documentation of a crate * [x] code of chapter 9: device drivers based on interrupts, including UART, block, keyboard, mouse, gpu devices * [x] add CI autotest and doc in github ### Todo(High priority) * [ ] review documentation, current progress: 8/9 * [ ] use old fs image optionally, do not always rebuild the image * [ ] shell functionality improvement(to be continued...) * [ ] give every non-zero process exit code an unique and clear error type * [ ] effective error handling of mm module * [ ] add more os functions for understanding os conecpts and principles ### Todo(Low priority) * [ ] rewrite practice doc and remove some inproper questions * [ ] provide smooth debug experience at a Rust source code level * [ ] format the code using official tools ### Crates We will add them later. ", Assign "at most 3 tags" to the expected json: {"id":"6606","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"