base on A fast type checker and language server for Python # Pyrefly: A fast type checker and language server for Python with powerful IDE features [](https://pypi.python.org/pypi/pyrefly) [](https://discord.gg/Cf7mFQtW7W) [](https://opensource.org/licenses/MIT) Pyrefly is a type checker and language server for Python, which provides lightning-fast type checking along with IDE features such as code navigation, semantic highlighting, and code completion. It is available as a [command-line tool](https://pyrefly.org/en/docs/installation/) and a [VSCode extension](https://marketplace.visualstudio.com/items?itemName=meta.pyrefly). See the [Pyrefly website](https://pyrefly.org) for full documentation and how to add Pyrefly to your editor of choice. Currently under active development with known issues. Please open an issue if you find bugs. ### Getting Started - Try out pyrefly in your browser: [Sandbox](https://pyrefly.org/sandbox/) - Get the command-line tool: `pip install pyrefly` - Get the VSCode extension: [Link](https://marketplace.visualstudio.com/items?itemName=meta.pyrefly) ### Key Features: - Type Inference: Pyrefly infers types in most locations, apart from function parameters. It can infer types of variables and return types. - Flow Types: Pyrefly can understand your program's control flow to refine static types. - Incrementality: Pyrefly aims for large-scale incrementality at the module level, with optimized checking and parallelism. ## Getting Involved If you have questions or would like to report a bug, please [create an issue](https://github.com/facebook/pyrefly/issues). See our [contributing guide](https://github.com/facebook/pyrefly/blob/main/CONTRIBUTING.md) for information on how to contribute to Pyrefly. Join our [Discord](https://discord.com/invite/Cf7mFQtW7W) to chat about Pyrefly and types. This is also where we hold biweekly office hours. ## Choices There are a number of choices when writing a Python type checker. We are taking inspiration from [Pyre1](https://pyre-check.org/), [Pyright](https://github.com/microsoft/pyright) and [MyPy](https://mypy.readthedocs.io/en/stable/). Some notable choices: - We infer types in most locations, apart from parameters to functions. We do infer types of variables and return types. As an example, `def foo(x): return True` would result in something equivalent to had you written `def foo(x: Any) -> bool: ...`. - We attempt to infer the type of `[]` to however it is used first, then fix it after. For example `xs = []; xs.append(1); xs.append("")` will infer that `xs: List[int]` and then error on the final statement. - We use flow types which refine static types, e.g. `x: int = 4` will both know that `x` has type `int`, but also that the immediately next usage of `x` will be aware the type is `Literal[4]`. - We aim for large-scale incrementality (at the module level) and optimized checking with parallelism, aiming to use the advantages of Rust to keep the code a bit simpler. - We expect large strongly connected components of modules, and do not attempt to take advantage of a DAG-shape in the source code. ## Code layout Pyrefly is split into a number of crates (mostly under `crates/`): - `pyrefly_util` are general purpose utilities, which have nothing to do with Python or type checking. Examples include IO wrappers, locking, command line helpers etc. - `pyrefly_derive` are proc-macros for deriving traits such as `TypeEq` and `Visit`. - `pyrefly_python` are Python utilities with no type-checking aspects, such as modelling modules or `sys.info`. - `pyrefly_bundled` are the third-party [typeshed stubs](https://github.com/python/typeshed). - `pyrefly_config` defines the Pyrefly configuration, along with support for reading Mypy/Pyright configuration. - `pyrefly_types` defines the Pyrefly type along with operations on it. - `pyrefly_wasm` defines the sandbox code that compiles to WASM. - `pyrefly` itself is the type checker and everything else. ## Design There are many nuances of design that change on a regular basis. But the basic substrate on which the checker is built involves three steps: 1. Figure out what each module exports. That requires solving all `import *` statements transitively. 2. For each module in isolation, convert it to bindings, dealing with all statements and scope information (both static and flow). 3. Solve those bindings, which may require the solutions of bindings in other modules. If we encounter unknowable information (e.g. recursion) we use `Type::Var` to insert placeholders which are filled in later. For each module, we solve the steps sequentially and completely. In particular, we do not try and solve a specific identifier first (like [Roslyn](https://github.com/dotnet/roslyn) or [TypeScript](https://www.typescriptlang.org/)), and do not use fine-grained incrementality (like [Rust Analyzer](https://github.com/rust-lang/rust-analyzer) using [Salsa](https://github.com/salsa-rs/salsa)). Instead, we aim for raw performance and a simpler module-centric design - there's no need to solve a single binding in isolation if solving all bindings in a module is fast enough. ### Example of bindings Given the program: ```python 1: x: int = 4 2: print(x) ``` We might produce the bindings: - `define int@0` = `from builtins import int` - `define x@1` = `4: int@0` - `use x@2` = `x@1` - `anon @2` = `print(x@2)` - `export x` = `x@2` Of note: - The keys are things like `define` (the definition of something), `use` (a usage of a thing) and `anon` (a statement we need to type check, but don't care about the result of). - In many cases the value of a key refers to other keys. - Some keys are imported from other modules, via `export` keys and `import` values. - In order to disambiguate identifiers we use the textual position at which they occur (in the example we've used `@line`, but in reality it's the byte offset in the file). ### Example of `Var` Given the program: ```python 1: x = 1 2: while test(): 3: x = x 4: print(x) ``` We end up with the bindings: - `x@1` = `1` - `x@3` = `phi(x@1, x@3)` - `x@4` = `phi(x@1, x@3)` The expression `phi` is the join point of the two values, e.g. `phi(int, str)` would be `int | str`. We skip the distinction between `define` and `use`, since it is not necessary for this example. When solving `x@3` we encounter recursion. Operationally: - We start solving `x@3`. - That requires us to solve `x@1`. - We solve `x@1` to be `Literal[1]` - We start solving `x@3`. But we are currently solving `x@3`, so we invent a fresh `Var` (let's call it `?1`) and return that. - We conclude that `x@3` must be `Literal[1] | ?1`. - Since `?1` was introduced by `x@3` we record that `?1 = Literal[1] | ?1`. We can take the upper reachable bound of that and conclude that `?1 = Literal[1]`. - We simplify `x@3` to just `Literal[1]`. ", Assign "at most 3 tags" to the expected json: {"id":"13811","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"