For this release we've focused on improving the IDE review experience. You can now easily choose different branches to compare and review, as well as seeing the review comments inline in your editor and applying suggested fixes directly.

Note that there is an issue with the upload of the 1.26.0 release for JetBrains IDEs - we are working on it and will post an update when it is resolved.

New IDE Features

• You can now choose different branches to compare and review. Easily review either the current file, your uncommitted changes or against any selected branch.
• You can now see Sourcery review comments inline in your editor and apply or resolve them directly through a code action.
• Comments that no longer apply to the code are now shown as outdated in the Sourcery sidebar.
• You can now apply suggested fixes directly from the sidebar, which auto-resolves and collapses the comment.
• We have improved the model used for chat - it now uses Claude 3.5 Haiku.
• All review and chat requests in the IDE are now routed to Anthropic models, with the benefit of a zero retention policy for your data.

Bug Fixes

• Various optimisations and fixes to improve stability and performance of reviews in IDE and GitHub/GitLab.

Features

• Improved the review experience in the IDE:
  • We now use Mermaid diagrams to better visualize code changes
  • You can now review the current file as well as your recent changes
  • We now use Claude 3.5 Haiku for IDE reviews, giving more accurate and helpful results as well as faster response times
  • We now persist the UI state between sessions, meaning that we save your selected options
• Enhanced GitHub bot interactions with new features:
  • Support for mention commands and free-text requests to re-review PRs
  • We now review bot PRs when triggered by mention commands

Bug Fixes

• Addressed various GitHub bot issues:
  • Better handling of review comment replies
  • Improved PR comment fetching
• Fixed validation review errors in IDE reviews
• Resolved issue with Anthropic API key handling in IDE reviews

Features

• We are now using the latest version of Sonnet 3.5 for many of our queries - giving more accurate and helpful results.
• Add config to allow you to switch on and off all aspects of the AI and rules-based reviews
• Refined and improved our documentation review
• We now automatically find issues in GitHub that might be related to your PR. Check these out to get more context when reviewing.
• Our review guide now includes Mermaid diagrams! These are a great way to visualise your code changes.
• You can now route all requests in both the IDE and the bot to Anthropic - this will let you take advantage of our zero-retention agreement with Anthropic. Just let us know if you'd like to make the switch here.
• You can now insert a PR summary into your review by mentioning @sourcery-ai summary in your review. You can also add this in to your summary template and we'll insert it in-place.
• You can now request that Sourcery generate a title for your PR. Just mention @sourcery-ai in your title.
• If you're working in GitHub we now look for linked issues in the PR body and check whether they're implemented in the PR. This is a great way to ensure that nothing's been missed.
• We've reworked and improved our review instructions - we are now much better at strictly following the instructions, heavily reducing the false positive rate.
• We can now help you fix CI failures! If there's a CI failure on your PR we will post a comment with an explanation, the logs and suggested fixes.

Bug Fixes

• Various fixes to our file filtering logic
• Refined our security review to reduce false positives
• Improved error handling in the bot
• Lots of work under the hood to reduce the number of noisy comments
• Address errors in the bot replying to users
• Fix validation review error in IDE reviews

We've made significant improvements to our code reviews, IDE plugins, and GitHub integration. These include UX enhancements, more customization options, and new features for better code management and collaboration.

New Features

• We've added the ability to add specific repository-level instructions for code reviews. This allows you to customize the review process for specific repositories, providing more control over the code review process. You can access this via the dashboard.
• We've improved the default behaviour and messaging for running code reviews in the IDE.
• You can now ask the sourcery-ai bot to generate a GitHub issue from a conversation.
• We've improved the login experience by making sure that we use the Pro features from any Sourcery account linked to your email address.
• Added Mermaid diagrams to our review guide.
• On install the Sourcery bot will now review the most recently changed PR for each org member.
• Various improvements to our reviews, including lowering the false positive rate for the security review and improving how we generate complexity comments.

Bug Fixes

• Add fallback for tiktoken to improve behaviour when running behind firewalls.
• Fix token calculation issues that could cause reviews to fail.
• Exclude files with "generated" in path to ensure we aren't reviewing generated code.
• Various stability improvements to GitHub bot, dashboard and IDE

You can now customize your reviews with your own style guides, conventions, migration rules, and review rules for specific files or directories.

You can also chat with the Sourcery bot in the review comments to continue a conversation. Understand the issue better, get advice, and generate code in real time.

New Features

Reviews

• Add custom style guides, conventions, migration rules, and review rules for specific files or directories
• Reply to Sourcery review comments to continue a conversation with the Sourcery bot
• Translate the PR summary and review guide to the user-configured language

IDE plugins:

• Improve login failure experience to show connection errors and improve instructions

Release 1.19.0

We've now brought our instant AI code review functionality into your IDE. The new 'Review' tab in the Sourcery sidebar lets you get a review straight away, keeping you in flow and letting you address issues without context switching.

On GitHub we've revamped our dashboard to make it easier to add and remove team members, and we've added configuration - you can now get your reviews in any language (within reason). Install it here. It is free for open source repos, and has a 2-week trial otherwise.

We'd love to hear your feedback! Reach out at Reviews@sourcery.ai. For general inquiries, contact us at hello@sourcery.ai. Ask for help or suggest improvements on GitHub.

Full Changelog

Code Review

Features

• Added a PR summary - this gets appended to the PR description as part of the review.
• Added a review guide to help reviewers understand your PR.
• Reworked our dashboard on the website to improve team signup and seat selection
• You can now configure your reviews to be in a wide variety of languages

Bug Fixes

• We now filter out common directories that might contain generated code
• We now filter out testing comments on non test-related files
• We've improved our comment validation to filter out more of the noisy comments.
• Ensured that we don't flag hardcoded secrets on lines which haven't changed in the PR
• Prevented the Sourcery Rules from reviewing files in the Python standard library.

Release 1.18.0

Our Code Review functionality is generally available for GitHub! Install it here. It is free for open source repos, and has a 2-week trial otherwise.

We'd love to hear your feedback! Reach out at Reviews@sourcery.ai. For general inquiries, contact us at hello@sourcery.ai. Ask for help or suggest improvements on GitHub.

Full Changelog

Code Review

Features

• Expanded the list of supported languages during reviews to include most programming languages supported by GitHub.
• Added new review areas to improve the quality of your code, including:
  • a complexity review that spots overly complex code and suggests improvements;
  • a testing review for improving test suites;
  • a security review area that spots and reports hard-coded secrets (like passwords) in your code.
• Sourcery will now include suggested changes in review comments of the suggestion type (where applicable) to help you improve your code.
• Reviews can now be triggered by posting @sourcery-ai review in a pull request comment.
  • Posting @sourcery-ai review can be used both to request a first review and to re-request a review after making changes;
  • The contents of the comment must be exactly @sourcery-ai review for the review to be triggered - any additional text will cause Sourcery to ignore the comment;
  • The review can be triggered by open-source projects even if Sourcery is not installed on the repository.

Bug Fixes

• Added a fix to prevent review comments from spanning too many lines, making them easier to read on GitHub.
• Fixed a bug that prevented some Sourcery in-line suggestions from being displayed in reviews.
• Fixed a bug where Sourcery would post duplicate comments in reviews.
• Fixed a bug where Sourcery would review the same pull request multiple times without being triggered.
• Fixed an issue where Sourcery would post an unreasonable number of comments on some pull requests.
• Fixed an issue where Sourcery would post praise-like comments in reviews, which are not actionable feedback.
• Fixed an issue where Sourcery would question changes that were clearly the intent of the pull request.
• Fixed internal issues that caused Sourcery to crash when reviewing certain pull requests.
• Sourcery will now skip reviewing pull requests with too many changes to prevent crashes.

Other Improvements

• Added a details section to the review summary showing the focus areas of the review and the number of issues found in each area.
• Sourcery will now skip reviewing the following types of files:
  • Lock files;
  • Minified files;
  • Packaging files.
• Sourcery will now mention the author of the pull request in the review summary for better feedback in reviews. The mention is in the form of @author.
• Improved the structure of review comments to make them more readable and actionable.
• Improved the messaging when reviews are skipped.
• Sourcery will now skip reviewing pull requests created by other bots, such as automated dependency updates by Dependabot.
• Introduced many other small improvements to make reviews more accurate and actionable.

Other Improvements

• Added docs for the new code review functionality.
• Added a new troubleshooting section to help understand why Sourcery might be disabled for your account.
• (Python) Fixed a bug in use-fstring-for-formatting that caused Sourcery to suggest using f-strings with starred expressions.
• (Python) Fixed the parsing of tuples in pattern matching.
• Improved the overall performance of Sourcery's rules and in-line suggestions, making them faster and reducing the memory footprint.
• Updated the docs for logging in on PyCharm.

Release 1.17.0

We made a mistake during our release process and v1.17.0 is not available. Please upgrade to v1.18.0 instead.

Our Code Review functionality is now generally available for GitHub! Install it here. It is free for open source repos, and has a 2 week trial otherwise. As part of this we have moved our legacy GitHub bot functionality over to the new system - instead of opening a new pull request with the Sourcery rules, these are shown as inline comments on your PR.

Full Changelog - Release 1.16.0

Features

• Added the ability to log in with your token in our Jetbrains plugin.
• Improved log in options and wording in the VS Code extension.

Code Review Features

• Code review is now free for open source users.
• Numerous small tweaks to improve the quality of our review comments.
• Improved the look, feel and tone of review comments to make them more concise and useful.
• Improved look and feel of the review summary, showing blocking comments where we find them.
• Introduced a security review checking for hardcoded secrets.
• We now review test files to suggest improvements to the tests.
• Our code reviews in GitHub now show existing Sourcery rules as well, and the legacy GitHub bot was removed in favour of the code review.
• Added ability to show code suggestions to review comments.
• Added a new complexity review to suggest improvements where code has become too complex.
• Improved the list of included and excluded files that we analyse.
• Improved bot detection to avoid adding code reviews to PRs created by bots.
• Removed comments that just looked at code formatting.
• Added full documentation of our AI Code Review.

Bug Fixes

• Improved stability of code review bot with several fixes to how we post reviews to GitHub.
• Fixed a timeout issue in the code review bot that sometimes prevented reviews from posting.
• Improved stability of coding assistant through creation and deployment of a new backend server.
• We no longer show the 'Generate Tests' code lens in test files.
• Improved error handling and messaging in the Coding Assistant.
• Removed use of temp directory to fix issue when running in remote SSH session.
• Correctly handle non-utf8 dependency files, fixing a coding assistant crash.
• Fix issue where functions after classes often got mistaken for class methods.

This release introduces a number of fixes and stability improvements mainly focused on the Sourcery Coding Assistant.

Our Code Review functionality is in active development in a closed beta. To request access, please email tim@sourcery.ai.

Full Changelog - Release 1.15.0

Features

• Added a settings cog to the coding assistant to make it easier to access settings like switching off code lenses, plus docs and feedback
• Improve error messaging for server errors in the chat

Code Review (closed beta) Features

• Improve filtering of AI code review comments
• Added Sourcery rules to the code review
• Add ability for code review to approve small typo-fix PRs
• Temporarily remove individual comments on test files, since these were often incorrect
• Ensure we only get one comment per line in the code review
• Provide PR body and title as contextual information to GitHub reviews
• Add experimental GitLab support for code review
• Add ability for code review to handle large diffs

Bug Fixes

• Fix issue with multiple refactorings not clearing 'scanning' messages in VS Code
• Don't attempt to show code lenses in IDEs other than VS Code and Jetbrains
• Treat WSL paths as UNIX paths

Release 1.14.0

We made a mistake during our release process and v1.14.0 is not available. Please upgrade to v1.15.0 instead.

This release significantly improves the code review assistant. Several stability improvements and bug fixes are also included.

Our Code Review functionality is still in a closed Alpha. To request access, please email tim@sourcery.ai.

Full Changelog - Release 1.13.0

Changes

• (Coding Assistant - Review) Improve code reviews in various aspects.
• (Coding Assistant - Review) Make the review summary more concise.
• (IDE) Update all billing-related links to point to the Sourcery dashboard.
• (IDE) Instead of showing the welcome file every time the IDE is opened, show it only when the IDE is opened for the first time.

Fixes

• (Coding Assistant) Ensure the configuration file is reloaded after opting into the Coding Assistant.
• (Coding Assistant) Improve stability of code lenses.
• (IDE) Correctly show refactorings immediately after logging in.

This release introduces a number of fixes and stability improvements mainly focused on the Sourcery Coding Assistant.

Our Code Review functionality is still in a closed Alpha. To request access, please email tim@sourcery.ai.

Full Changelog - Release 1.12.0

Changes

• (Coding Assistant - Review) Improve the look and feel of review comments.
• (Coding Assistant - Review) Show diffs in the review comments.

Fixes

• (Coding Assistant) Fix appearance of recipe buttons in the Dark High Contrast theme.
• (Coding Assistant) Improve the Coding Assistant server architecture to enhance stability.
• (Coding Assistant) Reduce amount of rate-limiting errors.
• (Coding Assistant) Reduce amount of timeout errors.
• (Docs) Fix typo in the docs for bin-op-identity.
• (IDE) Fix a bug where code lenses and diagnostics would not update on changes to documents on Windows.
• (Rules - Python) Fix a bug where functions defined after classes sometimes got mistaken for class methods - #328.

This release introduces a number of fixes and small features to improve the stability of the IntelliJ plugin, especially on Windows, as well as improvements to our model prompts.

Full Changelog - Release 1.11.0

Features

• (Docs) Code blocks now have a copy-to-clipboard feature.
• (Coding Assistant) Context indicators in the coding assistant will now navigate directly to the relevant lines.
• (Coding Assistant) Created a new server to improve network stability.
• (Coding Assistant) Added a detailed feedback interaction.

Changes

• (Coding Assistant) "Add Docstrings" now detects if a docstring already exists, and if so will attempt to improve, rather than regenerate it.
• (Coding Assistant) "Simplify Code" and "Optimize Performance" have had changes to avoid introducing breaking changes.
• (Coding Assistant) "Generate Docstrings" will no longer show as a code lens for Python test functions.
• (Coding Assistant) Improvements to the layout and style of the Access Settings screen.
• (Coding Assistant) Improvements to LLMs used behind the scenes.
• (Coding Assistant) Improvements to the style for high-contrast colour profiles.

Fixes

• (VSCode) Fix so Sourcery works in a VSCode SSH session.
• (IntelliJ) Prevent a messaging deadlock on Windows which caused Sourcery to stop working.
• (Plugin) Fix a permissions issue on macOS.
• (Coding Assistant) Words which match file names are no longer treated as file links.
• (Coding Assistant) Fixed a crash when an unrecognized language was used.
• (Coding Assistant) Fixed an issue where moving the Sourcery panel would make the chat disappear.
• (Coding Assistant) Chat loads with previous errors.
• (Coding Assistant) Fixed an issue where the chat loaded without previous errors, resulting in a persistent "pending" state.
• (Coding Assistant) Fixed persistent "Token Limit Exceeded" issues.
• (Coding Assistant) Fixed a failure to launch when requirements files were non-utf8 encoded.

The Sourcery Coding Assistant is now available in all JetBrains IDEs including Intellij and PyCharm!

Get it here:

Full Changelog - Release 1.10.1

Features

• Intellij now fully supports the Coding Assistant
• The generate docstrings recipe has improved support for JavaScript, TypeScript and Rust
• The context Sourcery uses for Coding Assistant messages is now shown with the messages
• Added coding assistant docs

Bug Fixes

• Resolved AccessDeniedException in the Intellij plugin along with other Windows issues
• Ensured long URLs display correctly
• Improved truncation logic for long code blocks
• Ensured logging in through the coding assistant correctly instantiates assistant handlers (when authorized)
• Fixed Ask Sourcery Command Palette error
• Improved how we handle recipes without text selection
• Fixed issue where we would show [MISSING LINK] whenever chat word contains a with a /

Release 1.10 fixed a number of bugs in our coding assistant from version 1.9.0 and added in support for the coding assistant for JetBrains IDEs. It's currently available in a closed beta for JetBrains users, but will be more broadly available shortly.

Full Changelog - Release 1.10.0

Features

• Add code lens support for JavaScript and TypeScript
• Add like/dislike feedback to coding assistant chat
• Code review branch choice is simplified and easier to use
• Added the coding assistant to JetBrains IDEs

Bug Fixes

• Fixed an issue with parenthesis precedence in starred expressions
• Assistant code lenses now open the coding assistant sidebar when clicked
• Assistant code lenses now focus the coding assistant chat when clicked
• Changed dict-assign-update-to-union refactoring to suggestion
• Fixed a few styles in the coding assistant app
• Chat cancellation no longer shows as an error
• Make useless-else-on-loop not trigger if any break is detected
• Use correctly-resolved coding assistant sub-features for opt-in screen purposes
• Fixed loading of scripts and styles on Windows

We've reworked our Jetbrains plugin, making lots of under-the-hood changes. These will enable us to deliver a more consistent experience between Jetbrains IDEs and VS Code. In particular we will soon be making the Sourcery Coding Assistant available for JetBrains IDEs.

This means quite a few changes will be apparent in the new version:

• We now support all JetBrains IDEs
• We now support class-level extract method refactorings in JetBrains IDEs
• We have removed the Sourcery tool window (you can get equivalent functionality through the Problems tool window)
• We have removed the Detect Clones feature (we may revisit this in future)
• We have removed batch code inspections (you can use the Sourcery CLI to make batch changes)

We have also continued to make improvements for the Sourcery Coding Assistant in VS Code. We improved both the output for some recipes and the code search used in the background. Also, the chat history is now stored in your local sqlite database.

The assistant is currently only available in closed beta. If you want to get early access to it, please sign up to the waitlist.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community. Ask for help or suggest improvements on GitHub.

Full Changelog - Release 1.9.0

Features

• Rework of JetBrains plugin to use the same Language Server Protocol framework that our VS Code plugin uses.
• Improved handling of errors from triggering recipes without selected text
• Improve Generate Tests recipe for Python by determining the scope of the selected text
• Sourcery now supports all JetBrains IDEs!
• Generate Tests: improvements to the prompt to only show relevant imports and add IDs to parametrized test cases
• Improved system message for review mode
• More intelligently trim PRs that are too large for review mode
• Require all users to login when using Sourcery
• Greatly improved the Sourcery extension sidebar in VS Code, so that it is lear what actions you need to take to log in and enable the coding assistant.
• Add styling for lists and paragraphs in markdown to improve UI of chat responses

Bug Fixes

• Improved handling of certificate errors in the coding assistant
• Fixed issue with team billing dashboard when returning from Stripe
• Changed use-getitem-for-re-match-groups to a suggestion
• Fixed issue where embeddings take up excessive space on disk
• Corrected overflow of code blocks in markdown chat responses
• Handle different base directories when parsing gitignores for generating embeddings
• Remove some logging noise when loading rules
• Updater only removes deletes previous binaries older than a week - should Address certificate errors for nightly builds

Some more improvements for the Sourcery Coding Assistant. We improved both the output for some recipes and the code search used in the background. Also, the chat history is now stored in your local sqlite database.

The assistant is currently only available in closed alpha. If you want to get early access to it, please sign up to the waitlist.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community. Ask for help or suggest improvements on GitHub.

Full Changelog - Release 1.8.0

Features

• Explain Code recipe: Make it more concise and use listings.
• Generate Tests recipe: Use more relevant example tests.
• Remove recipes with respective assistants: Review, Troubleshooting, Rename.
• Retain Coding Assistant chat history on restart.
• Improve Coding Assistant code search for non-Python languages.
• Incrementally update the Coding Assistant semantic search index.

Bug Fixes

• Only show opt-in when user has Coding Assistant feature enabled.
• Once a user opts in, immediately make Coding Assistant work.
• When enabling the Coding Assistant recipes are immediately shown.
• Prevent multiple of the same email from being sent to a user.
• Fix issues with the local sqlite database.
• Use lock when indexing semantic search.
• lib/gitignore: support non-relative paths when checking ignore rules.

This release considerably improves the Sourcery coding assistant recipes and chat. Basic support for non-Python languages has been added. We are also introducing two new modes: Troubleshooting and Code Review.

The assistant is currently only available in closed alpha. If you want to get early access to it, please sign up to the waitlist.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community. Ask for help or suggest improvements on GitHub.

Full Changelog - Release 1.7.0

Features

This new version of Sourcery includes more features and improvements to our coding assistant:

• Added a new coding_assistant.project_description field to the Sourcery configuration file (.sourcery.yaml). This field is used to inform the Sourcery coding assistant about the project and improve the quality of the suggestions. For instance, an appropriate .sourcery.yaml file for the sourcery-analytics project would contain

  coding_assistant:
    project_description: |-
      Sourcery Analytics is a command line tool and library for statically analyzing
      Python code quality.

• Added basic support for non-Python languages:
  • C
  • C++
  • C#
  • Go
  • Java
  • JavaScript
  • Shell scripts
  • Ruby
  • Rust
  • TypeScript
• Split the "Logic" section of the Explain Code recipe into "What?" and "How?" sections
• Made the chat system message more informative, thus improving model responses
• Improved the Optimize Performance recipe
• Added a new Troubleshooting mode, which will be gradually rolled out to users
• Added a new Code Review mode, which will also be gradually rolled out to users
• Users with access to the coding assistant first need to opt in to use it

Bug Fixes

• Prevent TypeErrors when retrieving the currently active file
• Fix exception raised when generating code lens where a Python function has no body

This release adds more features to our coding assistant, including two new recipes: Simplify Code and Generate Docstring. We also improved the way you can interact with the assistant by using code lens. The Debug Code recipe has been renamed as Troubleshoot and is now more powerful.

The assistant is currently only available in closed alpha. If you want to get early access to it, please sign up to the waitlist.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community. Ask for help or suggest improvements on GitHub.

Full Changelog - Release 1.6.0

Features

• Coding Assistant:
  • Add the Simplify Code recipe
  • Add the Generate Docstring recipe
  • Support the cancellation of in-progress messages
  • Add path validation to chat messages
  • Make code lens show direct links to appropriate recipes
  • Rename the debugging recipe as Troubleshoot and improve its functionality
• Add config option to toggle off Sourcery code lens

Bug Fixes

• Fix code lens position where there are decorators in Python
• Correctly handle ignored files in built-in rules
• Fix support for tsx and jsx files via language ID
• Don't send initial Coding Assistant chat message if the conversation has already started
• Fix issue with sourcery hub when trying to enable/disable rules

This release continued to add features to our pair programming coding assistant, in particular a new Ask Sourcery command that lets you easily request assistance from within the code editor. This can be triggered from the command palette, selecting some code and right-clicking, or from a code lens above Python classes and functions.

The assistant is currently only available in closed alpha. If you want to get early access to it, please sign up to the waitlist.

We also addressed the slow activation times that users were seeing with our VS Code extension.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community. Ask for help or suggest improvements on GitHub.

Full Changelog - Release 1.5.0

Features

• Show an initial message in the coding assistant
• Add a new Ask Sourcery command to let you trigger recipes or ask questions
• Added a code lens to trigger this command from Python classes and functions
• Extended the scope of the generate tests recipe
• Introduced a new Review Code recipe - allowing you to get code reviews of the selected code
• Improved the error messages in the assistant
• Enhanced the renaming recipe to handle special names in Python
• Upgrade to longer context model so we can send more context through to the assistant

Bug Fixes

• Limit total history sent to openai to avoid token limits
• Fix internal exception for code with syntax errors
• Fix exception that could be thrown with coding assistant where no file was open
• Fixed slow activation time in the VS Code extension (https://github.com/sourcery-ai/sourcery-vscode/issues/171)

This release includes the first version of our pair programming coding assistant that allows you to:

• create unit tests
• get an explanation of your code
• debug your code
• get better names for functions
• optimize your code for performance
• generate and add docstrings
• chat with the coding assistant to get explanations, ideas or answers to your questions

The assistant is currently only available in closed alpha. If you want to get early access to it, please sign up to the waitlist.

This release also re-adds the ability to log in by inserting your Sourcery token, fixed a few bugs and improved some error messages.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community. Ask for help or suggest improvements on GitHub.

Full Changelog - Release 1.4.0

Features

• Implemented the first version of our closed-alpha coding assistant

• Re-added the ability to log in by inserting your Sourcery token

• Improved cross-file type inference for several Sourcery rules

• We now show the rule ID to be skipped in the skip commands

• New Sourcery rules:

  • use iloc for indexing in Pandas (use_iloc)
  • remove redundant Path.exists checks (remove-redundant-path-exists)

Bug Fixes

• Improved path matching in the paths configuration of custom rules
• Improved error messaging when loading rule configuration files fails
• Fixed a bug where qualified names in rule replacements caused Sourcery to crash
• Stop assuming the return type of getattr based on its default argument. Fixes #349
• Ensure that bytes and strs compare differently in Sourcery rule conditions
• Stop searching for version control systems upon read permission errors. Fixes #348

For this release we've added some new rules, improved validation when creating custom rules, and squished some pesky bugs.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community.

Full Changelog - Release 1.3.0

Features

• Add method-chaining pandas rule

• enable rule saving in interactive scan

• no complex tests rules: dont-import-test-modules, no-loop-in-tests, no-conditionals-in-tests

• rule config validation errors in PyCharm

• show url on hub command in CLI

• surface rule config validation errors in the language server

• validate node kinds in the rules config

Bug Fixes

• Rule encoding issue for rules rendering on Windows
• Change Docker images used for Linux build
• Correctly handle indents for multiline substitutions
• get typescript hypergraph rules working again
• improve pycharm startup times

Documentation

• update js docs for public beta

We've now moved our multi-language support into public beta. This means that everyone will now start seeing Sourcery suggestions for JavaScript and TypeScript files - you no longer need to request a sign-up.

We'd love to hear your feedback! Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community.

Full Changelog - Release 1.2.0

Features

• Following user feedback we've removed the Code Quality Report from the GitHub bot
• improve type recognition for stdlib modules datetime, csv, json, os, pathlib.Path, collections.Counter
• pandas rule use-isna
• We've also been working behind the scenes on porting our Python rules into our new Hypergraph engine which is much more powerful and performant - expect to hear more over the next few months

Bug Fixes

• address crash in the logs when opening .sourcery.yaml file in VS Code
• possible fix for timeout issue on loading Sourcery in PyCharm

Documentation

• Reworked documentation to split out rules for Python and JavaScript

On the Python side, we focused on data science in this release. Now, you'll get suggestions related to some Pandas best practices, like the apply function or the inplace argument.

We've continued working on multi-language support. Now, you'll get suggestions for JavaScript code in PyCharm as well. We've also introduced several tweaks to the multi-language engine and the rule syntax.

Full Changelog

Features

Python

• dataframe-append-to-concat Pandas rule
• pandas-avoid-inplace rule
• Improve Pandas support and recognize more DataFrame objects

JavaScript

• Enable multi-language support in the PyCharm plugin.

Documentation

• Add explanations to all JavaScript rules

There are fewer updates in this alpha release as we've been focused on research and development of our core engine.

As well as a couple of new JavaScript rules and a fix for an edge case in the Python rule simplify-fstring-formatting, we've released reference documentation for every JavaScript rule that ships with Sourcery - see for instance documentation for the new rule dont-reassign-caught-exceptions at https://docs.sourcery.ai/Reference/Default-Rules/javascript-rules/dont-reassign-caught-exceptions/

Also new in this release is a command in VSCode for Sourcery called "Search and Replace across workspace" - this will open a panel where you can use our pattern matching and replacing syntax to apply a one-off rule. This feature is in early stages of development - please reach out at hello@sourcery.ai if you are interested in learning more.

Full Changelog

Added

• dont-reassign-caught-exceptions javascript rule
• dont-reassign-foreach-variables javascript rule
• dont-use-wrappers-for-builtins javascript rule
• Reference docs for GPSG rule sets
• Built-in Javascript rule documentation on the website
• (VSCode only) Sourcery Search-And-Replace

Fixed

• simplify-fstring-formatting will no longer trigger where constant "{" and "}" are being used, to avoid creating invalid code.

This release adds many new JavaScript and TypeScript rules using Sourcery's (alpha-stage) multi-language functionality.

If you are interested in getting access to the Beta for Sourcery for JavaScript and Typescript, please sign up here.

Full Changelog

Added

• Docs: How To Use Sourcery For Code Reviews?
• Handle skip comments in JavaScript
• no-eval JavaScript rule
• remove-redundant-boolean JavaScript rule
• use-braces JavaScript rule
• dont-negate-is-instanceof-operands JavaScript rule
• remove-unreachable-code JavaScript rule
• misplaced-break-or-continue JavaScript rule
• dont-shadow-arguments JavaScript rule
• remove-redundant-if-statement JavaScript rule
• dont-reassign-parameters JavaScript rule
• inline-immediately-returned-variable JavaScript rule
• avoid-function-declarations-in-blocks JavaScript rule
• Some refactorings like simplify-boolean-comparison will now trigger in match-case guard clauses
• possible-incorrect-bitwise-operator JavaScript rule
• dont-self-assign-variables JavaScript rule
• generators-should-yield JavaScript rule
• throw-new-errors JavaScript rule
• remove-redundant-slice-index JavaScript rule
• only-delete-object-properties JavaScript rule
• dont-use-with JavaScript rule
• dont-concatenate-string-literals JavaScript rule
• no-new-symbol JavaScript rule
• return-outside-function JavaScript rule
• yield-outside-generator JavaScript rule
• avoid-infinite-loops JavaScript rule

Changed

• Only show a maximum of 3 lines of code for comments in the CLI for Javascript rules
• Extended use-array-literal to handle arguments
• use-object-destructuring and combine-object-destructure extended to also work for let statements
• Added hover for JavaScript diagnostics

Fixed

• issue with replacement of subsequent indented new lines in JavaScript
• issue where JavaScript comments showed code actions that did nothing
• issue with partially-applied use-braces refactoring
• remove-dict-keys no longer proposed when the target has multiple values
• Correctly report number of scanned files even if they have no issues
• merge_assign_and_aug_assign can no longer incorrectly lead to access to a variable not assigned yet
• Prevent dict-literal, list-literal and remove-unnecessary-cast from incorrectly triggering in match-case patterns

This release adds support for Sourcery's (alpha-stage) multi-language functionality in its Language Server implementation. Alpha testers may begin to see refactorings suggested for JavaScript and TypeScript files in their Language-Server compatible IDEs, such as VSCode.

If you are interested in getting access to the Beta for Sourcery for Javascript and Typescript, please sign up here.

Full changelog

Added

• assignment-operator javascript rule
• avoid-using-var javascript rule
• binary-operator-identity javascript rule
• de-morgan javascript rule
• flatten-nested-try javascript rule
• flip-comparison javascript rule
• invert-ternary javascript rule
• max-min javascript rule
• merge-assign-operators javascript rule
• merge-else-if javascript rule
• merge_nested_ifs javascript rule
• no-new-function javascript rule
• prefer-arrow-callback javascript rule
• simplify-ternary javascript rule
• use-array-literal javascript rule
• use-object-destructuring javascript rule
• use-ternary-operator javascript rule
• while-guard-to-condition javascript rule
• wrap-immediately-invoked-function-expressions javascript rule

Changed

• swap-if-expression will no longer trigger where there are nested if expressions.
• Split docstrings-for-packages out from the docstrings-for-modules GPSG rule.

Fixed

• Hex conversion issue in multi-language support mode
• Issue with returning overlapping matches in blocks in multi-language support mode
• Incorrect conversion of 0/0 in js binary-operator-identity rule.

Full changelog

Added

• You can now access the Sourcery Hub via the sourcery hub command in the CLI and a status bar icon in PyCharm. This lets you easily enable and disable rules for your projects.
• On the website dashboard, team members can now see their team information and admins can add new team members via unique invite links
• Expanded Command Line reference docs
• Docs restructuring: default, optional, and custom rules

New Billing Dashboard for Team Administrators

This Sourcery release includes a brand-new billing dashboard for team administrators to view and manage their payment settings.

Full changelog

Added

• Google Python Style Guide rules: tags based on the Google Python Style Guide sections
• (Website) Billing dashboard for team administrators.

Fixed

• Migration of include setting from old version of config now treats an empty list as including the "default" rule set, so that Sourcery's default rules still run.
• A problem where Sourcery would introduce newlines beneath long refactored lines.
• False positives on instance-method-first-arg-name by not triggering if there are any decorators present.
• Issue with replace-interpolation-with-fstring failing for strings with quotes.
• Issue where clone diagnostics persisted after clones had been manually resolved in VS Code.
• Issue with formatting path names with double underscores in the GitHub commenter.
• Incongruous mention of "file(s)" in CLI phrasing when code passed in directly.

Sourcery VS Code Hub

You can now configure Sourcery directly from VS Code. Just click the Sourcery button in the VS Code status bar at the bottom.

Then you can login, read the Getting Started instructions or select Sourcery rules to enable and disable

Python 3.10 support

Sourcery now parses and understands Python 3.10 code including the all important match syntax.

Google Python Style Guide

You can now easily run the Google Python Style Guide.

Enable it directly from config with:

rule_settings:
  enable:
    - default  # Continue to run the default rules
    - google-python-style-guide  # Add in the additional

Or run it directly from CLI with sourcery review --enable google-python-style-guide PATH

Full changelog

Added

• Enable Google Python Style Guide directly from config with:

  rule_settings:
    enable:
      - default  # Continue to run the default rules
      - google-python-style-guide  # Add in the additional

• Enable Google Python Style Guide directly from CLI with sourcery review --enable google-python-style-guide PATH
• Sourcery will now display quick fixes on VS Code even if other tools show diagnostics for the same piece of code.
• Support Python 3.10
  • Support parsing and analysing match statements.
  • Support parsing of with statements containing multiple bracketed context managers.

Changed

• Rename .sourcery.yaml keys:
  • refactor -> rule_settings
  • rule_settings.include -> rule_settings.enable
  • rule_settings.skip -> rule_settings.disable
• Redesign of the hub pages.
• Rule ids and tags must now be less than 88 characters and match this regex: ^\[A-Za-z\]\[A-Za-z0-9-\_/:\]\*$

Fixed

• Automatically renames refactor -> rule_settings and other config keys when adding disable rule ids from IDEs
• Handles config error on IDE startup and shows better user errors
• Issue with capturing and replacing aliases in custom rules.

Improved IDE Sign-In

In this update, Sourcery takes the tedium out of signing in in your IDE. Previously, we asked you to enter a token from your Sourcery dashboard into your IDE settings. Now, we are able to handle the full login flow automatically. Use the Sourcery: Login command from VSCode, or the Login button in the Sourcery window in Pycharm, then follow the instructions in your browser.

Full changelog

Added

• CLI options --summary and --no-summary
• Configuration file field rule_tags to define additional rule tags
• The Sourcery Hub can now be used to manage per-project rule skips.

Changed

• pre-commit recommended setup: use the --no-summary option
• Added google-python-style-guide tag to the Google Python Style Guide rules.
• Login to VS Code with the Sourcery: Login command from the command palette
• Login to PyCharm by clicking the login button in the top left of the Sourcery window

Fixed

• Documentation for convert-any-to-in
• Ensured inline-immediately-returned-variable doesn't trigger where the variable has a return annotation.
• The Sourcery Hub no longer causes the rest of Sourcery to hang
• The JSON schema will no longer report a validation error for python_version specified as a string

Rule tags

Rules can now be tagged by adding a tags key. Then you can include these tags from the CLI to choose a subset of rules to run.

This makes it easy to specify which rules you want to run in CI. First add a tag to the rules you want:

rules:
  - id: important-rule
    tags:
      - ci
    pattern: '...'

Then run the ci tag with sourcery review --include ci .

Set up Sourcery in pre-commit and CI

It is now much easier to configure Sourcery to run in pre-commit or CI. Run:

• sourcery init pre-commit
• sourcery init ci

Full changelog

Added

• Docs: custom rule recipe: Enforce Naming Conventions Rules: You can now add tags to rules, and include or exclude them with the include and exclude/skip config and CLI options. Sourcery's built-in rules carry the built-in tag.
• IDE: LSP command to login through browser

Changed

• CLI: make rule ids clickable links in non-verbose mode
• CLI: Enhanced sourcery init command in CLI - you can now use it to add pre-commit config, as well as view instructions to add Sourcery to CI.
• CLI: review: do not print a summary in a pre-commit environment

We continued improving the Command Line Interface: With the new --diff option, you can now run Sourcery only for the code in your current PR.

sourcery review --diff "git diff main" .

We had an issue with the release 0.12.10 This changelog contains the changes of both 0.12.10 and 0.12.11

Added

• A JSON schema in the docs to enable IDE autocompletion of sourcery config files
• CLI: Review only changed code with sourcery review --diff="git diff HEAD" .
• CLI: Load config paths or URLs using --config
• Introduction tutorial in the documentation intended for first-timers
• Name condition contains
• Tip on next steps after running sourcery login from CLI.

Fixed

• Error shown when starting VS Code without a .sourcery.yaml file present.
• File-level skip comments are now respected for custom rules
• Prevent invert-any-all from inverting chains of comparisons like 0 <= value <= 255. Fixes Issue # 286

Changed

• Improved some of the descriptions in the Google Python Style Guide rules.
• GPSG global variables rule: allow logger
• GPSG non-test rules exclude test_*.py and *_test.py instead of the test
• In the CLI, Sourcery will search for a .sourcery.yaml file in the current directory or any of its ancestors, rather than just the current directory.
• Make .sourcery.yaml match paths relative to the config file. This applies to ignore, rules.paths.include and rules.paths.exclude

This time around we've been hard at work in the background - making the Command Line Interface quicker and more responsive, and fixing some pesky issues.

Added

• CLI: show a status message during file parsing and improve startup time.
• docs: added a recipe to Flag Hard-Coded Values

Changed

• Make .sourcery.yaml match paths relative to the config file. This applies to ignore, rules.paths.include and rules.paths.exclude

Fixed

• Fix faulty case in for-append-to-extend. Fixes Issue # 279
• Links to the individual refactorings in the plugins and our docs are now correct
• Issue where editors were displaying refactorings in the wrong location
• CLI: show progress during review always on stderr
• use-itertools-product will now only trigger when we can infer that the types being iterated over are finite sequences. This addresses Issue #281
• GPSG snake-case-arguments rule now correctly excludes argument separators and "_"

We've been hard at work on documenting all of the awesome things that you can do with our custom rules. We've reworked our documentation and written up how you can start enforcing the Google Python Style Guide in your team right now.

Our other notable improvement this time round is a new summary view when you run the review command using our Command Line Interface.

Added

• CLI review command / overview: summary view.
• Google Python Style Guide rules added to reference documentation

Changed

• CLI review default output: 1 line per issue, with standard format links to issue locations. You can use the --verbose flag to get a detailed output with each issue's description and diff.
• merge-isinstance refactoring will now apply to chains of or operators.
• Public documentation has been restructured and improved.
• The CLI review command now validates that include and exclude rules exist in either the core ruleset or the config file being used.

Fixed

• Always unparse Ellipsis as ..., fixing Issue #273
• Issue where incorrect code could be removed where a False in an if test was followed by elif branches
• Issue where whitespace could be incorrectly removed when applying refactorings after saving a file.
• Correctly identify attribute usage in set-comprehension. Fixes Issue #271.
• Made performance improvements to speed up Sourcery analysis on some files.
• Recommendations will now highlight on the correct lines when there is whitespace at the beginning of their block.
• Sped up calculation of code actions in VS Code, removing lag.
• Fixed incorrect description of ignore directories in the docs
• VS Code multiple-root workspaces no longer "share" Sourcery configuration
• Example config created by sourcery init: Use new syntax in the no-print-statements rule

We've continued to make our custom rules more powerful!

This includes:

1. A new ... syntax for matching anything in a position. Say you want to write a rule that matches any print statement. You can now write pattern: print(...) and this will find all print statements with any number of positional or keyword arguments.

2. Fully qualified name recognition. Say you want to transform typing.List into list in line with PEP 585. You can define the following rule:

   rules:
     - id: convert-typing-list
       pattern: typing.List
       replacement: list
       description: Use `list` rather than `typing.List`

   This will find all occurrences of typing.List and replace them with list, even if they have been imported using from typing import List.

Added

• Conditions for filtering out captures
• Match anything ... or nothing !!! in patterns
• is_identifier public condition to match variables that are valid identifiers
• statement_count custom rule condition - this lets you count how many statements you matched.

Changed

• Custom rule patterns will now leniently match for type annotations in assignments, parameters and function definitions. This means that the pattern ${target} = ${value} will now match assignments such as i: int = 0.
• Recognize fully qualified names in custom rule patterns. This means that the pattern os.path.join will match join if it is imported as from os.path import join (and all of the possible variations).
• Run custom rule conditions on the entire match using the pattern name.
• Disallow pattern from being used as a regular capture name.
• Docstring values can now be captured by using any existing syntax inside the string

Fixed

• Don't propose default-mutable-arg if the arg name is shadowed in the default argument. e.g. def f(arg=[arg])
• Don't propose dict-assign-update-to-union when the update accesses the dict variable
• Don't propose remove-redundant-except-handler for non-built-in exceptions
• Escape HTML in PyCharm diffs

Last time we introduced custom rules into Sourcery.

This time they're becoming even more powerful as you can filter pattern matches with a condition.

Let's say we want to stop global variables being declared. First let's write down what a global variable is:

1. It's declared at the top level of the module
2. It's not a constant using uppercase like DAYS_IN_WEEK = 7
3. Let's allow private globals starting with _

Here's a rule to identify global variables:

rules:
  - id: no-global-variables
    pattern: ${var} = ${value}
    condition: |
      var.in_module_scope()
        and not var.is_upper_case()
        and not var.starts_with("_")
    description: Don't declare `${var}` as a global variable

Check out the conditions reference to see what conditions are available.

Added

• Custom rules can contain a condition field that means the rule only matches when the condition is satisfied
• CLI: display a progress bar showing how many files have been reviewed
• Docs: Flag Dependencies to a Library with Sourcery custom rules
• Docs: Establish Rules for Dependencies Between Your Packages with Sourcery custom rules
• use-file-iterator refactoring

Changed

• reintroduce-else will now trigger more often
• Refactorings list-literal and dict-literal will now trigger in every instance of list() and dict(), instead of only in assignments
• CLI: display rule descriptions and explanations as Markdown
• CLI docs: present the review command instead of refactor

Fixed

• dict-assign-update-to-union doesn't trigger for global variables
• Files using tab indenting will be always be refactored with tabs
• Identical clone detection will now only trigger for functions if they have the same name
• Recover from PermissionErrors when rotating log files
• Sourcery now run refactorings on functions containing nested functions

We've focused on improving our documentation for custom rules. Check out these how-to guides:

• How to use Sourcery custom rules to get rid of deprecated code
• How to use Sourcery custom rules to remove debugging statements from your code

Added

• Create Sourcery Rule command to VS Code. This creates a new stub rule in your project-level Sourcery configuration (creating that config if it doesn't exist). You can access this command from the Command Palette, or by selecting the code that you want to be the base pattern for your rule, right clicking and choosing Sourcery: Create Rule.
• Support for captures in custom rules descriptions
• CLI option --exclude to exclude some rules for the commands review and refactor
• Documentation: How to use Sourcery custom rules to get rid of deprecated code
• Documentation: How to use Sourcery custom rules to remove debugging statements from your code from your code?
• Reference Documentation about Sourcery custom rules

Changed

• hoist-similar-statement-from-if has been split out from hoist-statement-from-if. This may mean some skip comments need to be applied.
• The --include option has now been sped up, as it now runs only the proposers necessary for the included proposals.

Fixed

• Class-level refactorings will now appear again.
• Issue with hoist-repeated-if-condition that caused if-statements to appear with empty tests if the hoisted expression was a boolop
• Improve responsiveness of CLI to keyboard interrupts
• Fall back to locally saved authentication if Sourcery auth server throws an error
• Show explanation in VSCode, PyCharm and the CLI for Sourcery custom rules

We've added a walkthrough to our VS Code extension to make it easier to get started with Sourcery. Go to the Help > Get Started menu to check it out.

We've added two new refactorings for pandas! These aim to improve performance by streamlining apply operations. If you have any feedback or more ideas for rules you'd like to see in pandas please let us know!

Custom rules are now even more powerful!

• You can specify paths that rules should be included or excluded from running on.
• You can now add tests for rules in-line in the config file

Added

• sourcery init command to produce a default .sourcery.yaml configuration file
• replace-apply-with-method-call refactoring for Pandas Series and DataFrame
• replace-apply-with-numpy-operation refactoring for Pandas Series and DataFrame
• Rules can specify paths that they include/exclude from running on
• Support for tests in custom rule configuration
• Walkthrough for VS Code
• The .sourcery.yaml file is now validated when it is opened in PyCharm

Changed

• Added version to config file and removed recommendation_level from output
• Allow pattern/replacement for custom rules to contain any combination of statement or expression

Fixed

• Issue where for-index-replacement could remove indices that were accessed later.
• Tuples should now be given the correct parentheses everywhere (fixes issue )
• Ensure consistency between similar fstring examples
• sum-comprehension: Do not apply if the name sum has already been used.
• remove-redundant-fstring won't remove the content of multi-line fstrings.

You can now build your own rules into Sourcery!

Are there issues that keep popping up in your projects, nuances around libraries you use, or feedback you want to make sure your whole team gets? Just create a Sourcery rule and we'll automatically find and fix those issues for you.

To get started, we've put together a tutorial for you to create your first rule.

And we've added a section to our docs with more details about what types of rules you can currently create.

Anyone can create up to 3 rules for free - or you can sign up for a Team plan to create unlimited rules.

Added

• Validate config including rule syntax and show appropriate errors to users
• Write up to 3 rules with the free plan
• Added number of occurrences to the description of each improvement that Sourcery recommends
• Enable recommendations that add imports in PyCharm

Changed

• The metrics hover on functions in the IDE is now defined as a type of Sourcery comment. This means the option to switch it off via IDE configuration has been removed - it can now be switched off if required by skipping the function-quality proposal in the Sourcery configuration file.
• Update Quality Report comments instead of recreating them in the GitHub bot.

Fixed

• avoid-builtin-shadow no longer triggers for class variables
• Fix hoist-similar-statement-from-if to consider if-statements containing elifs as different from each other (fixes issue #241)
• Improved stability of VS Code and PyCharm extensions when errors are encountered
• Retain .sourcery.yaml formatting and comments when updated from IDEs
• Rules that make no code change will now convert to comments instead of causing a crash.

We've been busy working under the hood on a new feature which we'll be unveiling soon! In the meantime we have:

Changed

• The low code quality warning has been changed into a standard Sourcery comment. This means that it can now be turned off in the standard way with a # sourcery skip: low-code-quality comment. It can now also be completely switched off by adding low-code-quality to the skip section in the Sourcery configuration file. This does mean that existing # sourcery no-metrics comments must be changed into the skip format as above.

Fixed

• Don't propose convert-to-enumerate in async for loops
• merge-duplicate-blocks will no longer duplicate conditions that may have side-effects
• Some changes have been made to reduce the memory footprint of the Sourcery binary.

Added

• remove-assert-true refactoring
• You can now set which rule types you wish to see in the PyCharm Sourcery settings.

Fixed

• LSP gracefully handles empty token and missing ruleType settings which resolves issues in Vim
• Prevent equality-identity from triggering with float or complex numbers since NaN == NaN is false
• Prevent hoist-statement-from-loop from triggering when the assignment value writes variables
• use-fstring-for-concatenation now keeps the correct string quote for both sides of the concatenation
• Prevent instance-method-first-arg-name from firing for built-in class methods __new__, __init_subclass__, and __class_getitem__, which instead will trigger class-method-first-arg-name.
• Allow use-fstring-for-concatenation to trigger on long lines if this results in shorter lines (fixes issue #222)
• GitHub config request_review: Fix so that a single value can be used again.

Added

• The Sourcery CLI now includes a review command. Use this to get Sourcery to give suggestions and comments in addition to standard refactorings. Commented lines will be highlighted in red.

• break-or-continue-outside-loop refactoring

• class-method-first-arg-name refactoring

• collection-to-bool refactoring

• compare-via-equals suggestion

• dict-assign-update-to-union refactoring

• do-no-use-bare-except suggestion

• hoist-repeated-if-condition refactoring

• simplify-single-exception-tuple refactoring

• remove-redundant-boolean refactoring

• remove-redundant-constructor-in-dict-union refactoring

• remove-redundant-exception refactoring

• return-or-yield-outside-function refactoring

• use-getitem-for-re-match-groups refactoring

• use-dictionary-union refactoring

• while-guard-to-condition refactoring

Changed

• Add removal of bools to remove-unnecessary-cast

• The remove-duplicate-key refactoring was split into a refactoring remove-duplicate-set-key and a suggestion remove-duplicate-dict-key

• remove-duplicate-dict-key was enhanced to handle unpacked dictionaries

• remove-duplicate-dict-key and remove-duplicate-set-key were enhanced to refactor dicts and sets in any context (previously they were only applied in assignments).

• simplify-len-comparison will now also simplify len(s) < 0 as False and len(s) >= 0 as True

• raise-specific-error will now also cover raising BaseException and raising from other exceptions

Fixed

• Comments will now show on the correct line where there are functions with docstrings. Addresses issue #211

• F-strings containing escaped characters will not be refactored into constant strings. Addresses issue #213.

• str-prefix-suffix now triggers in the correct way for applying endswith

• use-assigned-var: Do not apply if the variable is overwritten in a loop

• use-itertools-product will now combine nested products. Fixes issue #214.

• We now assume any and all write to their arguments, since they exhaust iterators. This fixes issue #210

Added

• Comments: Sourcery will now spot issues in your code that don't have an automated code fix. Instead we will give a detailed explanation of the issue along with a guide on how to fix them. Just hover for more info. The first comment that we have introduced is raise-specific-error - here we can identify that a base Exception is being raised (and explain how it is better to raise a specific exception), but we can't automatically detect which exception should be raised in this case.

• There is now a new config option to specify which rule types to display in the plugins - you can choose to any combination of refactorings, suggestions and comments.

• Sourcery refactorings can now add imports where necessary. This means that we can suggest using standard library functions such as contextlib.suppress.

• aware-datetime-for-utc suggestion

• remove-unnecessary-cast refactoring

• replace-interpolation-with-fstring refactoring

• swap-if-expression refactoring

• use-contextlib-suppress refactoring

• use-fstring-for-formatting refactoring

• use-itertools-product refactoring

• use-or-for-fallback refactoring

• avoid-builtin-shadow comment

• unwrap-iterable-construction refactoring

Changed

• path-read will now suggest importing pathlib if it is not already imported.
• The refactoring level toggle option has been removed from VS Code and PyCharm. Which types of Sourcery rule to show can now be configured in the config file. In VS Code this can also be configured in the Sourcery settings. More configuration options for PyCharm coming soon!

Fixed

• Fix issue with string concatenation in pandas
• Fix issue where qualified names for functions were incorrect in the presence of certain keywords.
• Fix issue in raise-from-previous-error where raise e from e was suggested.
• PosixPath issue causing Sourcery to crash in PyCharm
• Bug in path-read and aware-datetime-for-utc that caused them to never be triggered
• remove-redundant-fstring: Do not trigger if the f-string contains escaped curly braces

[0.10.3] - 2022-02-24

Added

• use-datetime-now-not-today refactoring
• remove-str-from-fstring refactoring (split out from simplify-fstring-formatting)
• New --include option to command-line interface and config files. This allows you to specify Sourcery to only apply certain refactoring rules.

Changed

• simplify-fstring-formatting will no longer remove calls to str in formatted values. This is not valid for classes which override __repr__.
• use-named-expression will now only trigger where the variable being assigned to is used in the body of the if
• chain-compares will now trigger when comparing to constants, and will avoid mixing different types of comparison
• Extend remove-str-from-print refactoring to deal with multiple arguments

Fixed

• Disallow use-next refactoring when statements have side-effects
• Side effect check for for-append-to-extend refactoring with tuple loop variables
• Fix issue with the use-any refactoring
• Fix issue where a non integer size metric was being display
• Don't propose useless-else-on-loop when there is a break in an inner loop's else
• Ensure we keep two lines between functions when extracting new ones at module level.
• Ensure default-get doesnt trigger where the default has side effects
• Don't inline assignments of strings into f-strings (Fixes issue)
• Keep original quotes in fstring refactorings
• Don't convert unicode CLDR in literal unicode values within fstrings - fixes this issue.

Added

• chain-compares refactoring
• max-min-default refactoring
• remove-dict-items refactoring
• remove-redundant-condition refactoring
• use-string-remove-affix refactoring
• use-fstring-for-concatenation refactoring
• skip-sorted-list-construction refactoring
• missing-dict-items suggestion
• use-next refactoring
• for-append-to-extend refactoring
• merge-list-appends-into-extend refactoring
• raise-from-previous-error suggestion

Changed

• Enhanced use-named-expression refactoring so that it also triggers on appropriate while loops.
• remove-redundant-slice-index extended to cover steps and removal of end index

Fixed

• Addressed issue where VS Code plugin was showing empty hovers
• Disallow conversion of async list comprehensions to generators in comprehension-to-generator
• Fixed broken links for suggestion recommendation docs
• Disallow merging strings on different lines in use-fstring-for-concatenation

Added

• Sourcery will now show suggestions which are not strict refactorings, but which will improve the code quality or avoid common coding mistakes. In VS Code refactorings will now show as yellow warnings, while suggestions will be at the blue information level. In PyCharm refactorings will stay as warnings, while suggestions will show as weak warnings. These suggestions can be toggled on or off with a new command in VS Code - Toggle Sourcery Recommendation Level, and through a toggle button in the Sourcery panel in PyCharm. The toggle can also be made directly in the user configuration file.
• default-mutable-arg suggestion - this is the first suggestion, and triggers when you use a dictionary, set or list as a default argument to a function.
• simplify-empty-collection-comparison proposal
• non-equal-comparison proposal
• use-named-expression proposal
• path-read proposal

Changed

• Sourcery now suggests some refactorings that require Python 3.9 by default. In these cases a new code action is shown allowing the user to configure which version of Python Sourcery should assume is being used (this is configured in the user or project level sourcery yaml configuration files).
• merge-nested-ifs will no longer trigger if either condition contains a named expression.
• for-index-underscore will now also trigger on comprehensions and generators.
• Enhanced remove-unused-enumerate to handle comprehensions

Fixed

• use-any and comprehension proposers will no longer incorrectly suggest incorporating code containing named expressions.
• Don't refactor constants in square-identity
• Fix timeout issue in PyCharm plugin.

Added

• introduce-default-else proposal
• move-assign-in-block proposal

Fixed

• Fix issue with switch refactoring
• Fix replace-dict-items-with-values suggestions for unstructured keys - https://github.com/sourcery-ai/sourcery/issues/181

Changed

• use-any, dict-literal and list-literal proposals will now trigger where there is an annotation present.
• Removed return-identity proposal as now superceded.

Try Sourcery without an account!

You can now use Sourcery straight away on installation, with no need to create an account or enter a token. This gives you access to our Free tier. To try out our Pro features you can create an account to get a 30 day free trial.

New refactorings

Sourcery refactoring id: equality-identity

Description:

Simplify equality comparisons that are always True or False

Before:

if 1 == 1:
    always_do_this()

if 1 != 1:
    never_do_this()

After:

if True:
    always_do_this()

if False:
    never_do_this()

Explanation:

When comparing a value to itself, the outcome will always be True, as long as the equality operator has not been overridden, and this also holds for the opposite comparison and False. It is more readable to use a direct comparison to the boolean value.

Sourcery refactoring id: identity-comprehension

Description

Convert list/set/tuple comprehensions that do not change the input elements into

Before

# List comprehensions
[item for item in coll]
[item for item in friends.names()]

# Dict comprehensions
{k: v for k, v in coll}
{k: v for k, v in coll.items()}  # Only if we know coll is a `dict`

# Unneeded call to `.items()`
dict(coll.items())  # Only if we know coll is a `dict`

# Set comprehensions
{item for item in coll}

After

# List comprehensions
list(iter(coll))
list(iter(friends.names()))

# Dict comprehensions
dict(coll)
dict(coll)

# Unneeded call to `.items()`
dict(coll)

# Set comprehensions
set(coll)

Explanation

All these comprehensions are just creating a copy of the original collection. They can all be simplified by simply constructing a new collection directly. The resulting code is easier to read and shows the intent more clearly.

Sourcery refactoring id: merge-assign-and-aug-assign

Description:

Replaces an assignment and an augmented assignment with a single assignment.

Before:

other_value = 33
number = 42
number += other_value

After:

other_value = 33
number = 42 + other_value

Explanation:

When we mutate a variable multiple times without reading or writing its value in between, it's more readable and more efficient to change its value only once. This way, it's clearer which values this variable can have at various points.

This refactoring works with all 4 augmented assignment operators:

• +=
• -=
• *=
• /=

Sourcery refactoring id: remove-str-from-print

Description:

Removes unnecessary calls to str() from within print()

Before:

print(str(1))

After:

print(1)

Explanation:

Objects passed into calls to the print() function already have str() called on them, so it is not required to do so yourself.

Sourcery refactoring id: simplify-division

Description:

Use Python's built-in feature for succinct division syntax.

Before:

result = int(42 / 10)

After:

result = 42 // 10

Before:

result = 42 // 10
remainder = 42 % 10

After:

result, remainder = divmod(42, 10)

Explanation:

Python has some great features to simplify division expressions. If you're interested only in the whole number component of the quotient, you can use the // integer division operator. If you want to have the whole number component and the remainder in separate variables, the built-in divmod function comes handy.

Sourcery refactoring id: simplify-substring-search

Description:

Simplify finding if substrings are present in strings by using in

Before:

my_str = "Hello world"
if my_str.find("ello") == -1:
    print("Not Found!")

After:

my_str = "Hello world"
if "ello" not in my_str:
    print("Not Found!")

Before:

my_str = "Hello world"
if my_str.count("ello") > 0:
    print("Found!")

After:

my_str = "Hello world"
if "ello" in my_str:
    print("Found!")

Explanation:

Making use of Python's in operator for detecting if a substring is present in a string is more readable than using the find or count methods, and is also suggested in the documentation. These methods are more suitable for cases where you need more information about the substring's location or the number of times it appears.

Fixed

• Fix issue where merging nested if conditions with named expresssions could change behaviour due to missing brackets.
• Fix issue when refactoring strings containing different types of quotes
• Stop hoist-if-from-if proposal from hoisting ifs that would then be incorrectly entered

New refactorings

Remove Redundant Slice Index

Sourcery refactoring id: remove-redundant-slice-index

Description:

Removes unnecessary explicit 0 when slicing the beginning of a sequence.

Before:

With an end index:

numbers[0:5]

Without an end index:

numbers[0:]

After:

With an end index:

numbers[:5]

Without an end index:

numbers[:]

Explanation:

The default starting value for a slice is 0, so it is unnecessary to explicitly define it. This refactoring removes such zeros, slightly shortening the code.

Remove Unused Enumerate

Sourcery refactoring id: remove-unused-enumerate

Description

Remove unnecessary calls to enumerate when the index variable is not used.

Before

for index, hat in enumerate(hats):
    print("I like this hat: ", hat)

for i, (key, value) in enumerate(my_dictionary.items(), start=100):
    do_something(key)
    do_something_else(value)

After

for hat in hats:
    print("I like this hat: ", hat)

for key, value in my_dictionary.items():
    do_something(key)
    do_something_else(value)

Explanation

Enumerating iterables with enumerate is a good practice for having access to both the values and their respective indices. However, when the indices are not necessary, it is cleaner to simply iterate over the original iterable and remove the call to enumerate.

Simplify String Length Comparison

Sourcery refactoring id: simplify-string-len-comparison

Description:

Changes an indirect comparison of a string's length to 0 into a direct comparison of the string to the empty string.

Before:

if len(s) == 0:
    ...

if len(r) > 0:
    ...

After:

if s == "":
    ...

if r != "":
    ...

Explanation:

This refactoring avoids an unnecessary calculation and keeps the logic in the domain of string types. It tends to unlock further improvements to string comparisons.

Useless Else on Loop

Sourcery refactoring id: useless-else-on-loop

Description:

Moves code in else blocks that is always executed to the main body

Before:

evens = []
for n in numbers:
    if n % 2:
        evens.append(n)
else:
    print("Done!")

After:

evens = []
for n in numbers:
    if n % 2:
        evens.append(n)
print("Done!")

Explanation:

Loops should only have an else clause if they can exit early with a break statement. If there is no break then the code in the else is always executed. In this case the else statements can be moved to the same scope as the loop itself, making the code slightly easier to understand (no need to look up what the else does).

Enhancements

• If the Sourcery plugins are unable to contact our authentication server they will now fallback to our Free plan - this means that you can still use Sourcery in this case.
• The simplify-len-comparison proposal has been extended to cover cases such as len(seq) != 0
• The various comprehension proposals will now be suggested in more cases where there is a type annotation on the variable being

Bug fixes

• Fixed issue where Sourcery would throw an exception if run with no internet connection.
• Fix issue where remove-redundant-if was incorrectly suggested in situations with nested changes to condition variables
• Fixed issue where use-any could be incorrectly suggested in cases where the for loop being replaced unpacked and used multiple variables

New Refactorings

Remove empty nested block

Sourcery refactoring id: remove-empty-nested-block

Description:

Remove nested block which has no effect

Before:

for i in range(3):
    pass

After:

Here the code has been removed:

Explanation:

An if or for which is empty and has no effect should not be present in the code. It clutters things up and makes it harder to see what the code is really doing.

Simplify dictionary update

Sourcery refactoring id: simplify-dictionary-update

Description:

Add single value to dictionary directly rather than using update()

Before:

def foo(d: dict) -> None:
    d.update({"request": HttpRequest()})
    frobnicate(d)

After:

def foo(d: dict) -> None:
    d["request"] = HttpRequest()
    frobnicate(d)

Explanation:

When we are just adding one entry to a dictionary it is simpler to do it directly rather than via the update() method. This also eliminates the overhead of constructing another dictionary and calling a method, so should slightly improve performance.

Enhancements

• We no longer suggest conversion to a generator on comprehensions inside str.join

Bug fixes

• Fixed issue where metrics hover would show on the decorator line in VS Code
• Fixed issue where we could incorrectly suggest remove-redundant-if if global state had changed.
• Limited de-morgan inequality refactoring to numerical types
• Fix issue where the scan for refactorings and detect clones commands sometimes failed to work on VS Code in Windows
• Limit use-count refactoring to only apply on lists and strings

Beta build sign-up

You can now sign up on the website to get access to Sourcery's beta updates. Just log in, go to your profile page and check the box under the 'Nightly builds' heading.

This will let you try out our latest refactorings and fixes the day they become available, rather than waiting for our plugin releases. Each morning when a build is available it will get automatically downloaded, then will prompt you to restart your IDE to get the new features.

New refactorings

Remove redundant f-string

Sourcery refactoring id: remove-redundant-fstring

Description:

Replace f-string with no interpolated values with string.

Before:

description = f"This is a totally normal string"

After:

description = "This is a totally normal string"

Explanation:

This refactoring spots cases where use of an f-string is unnecessary since there are no interpolated values. It's easy to forget to remove the f if you have such a value and then remove it later.

Simplify f-string formatting

Sourcery refactoring id: simplify-fstring-formatting

Description:

Simplify unnecessary nesting, casting and constant values in f-strings

Before:

description = f'This {1} thing is {f"{str(adjective)}"}'

After:

description = f"This 1 thing is {adjective}"

Explanation:

When writing an f-string it is not necessary to interpolate the values of constants or cast variables to strings - these will be done automatically. It is also not necessary to nest f-strings when adding the contents of the nested string directly achieves the same result.

Ternary to If Expression

Sourcery refactoring id: ternary-to-if-expression

Description:

Replace boolean ternary with inline if expression

Before:

protocol_type = uses_ssl() and "https" or "http"

After:

protocol_type = "https" if uses_ssl() else "http"

Explanation:

In some cases, especially in old codebases that were written in Python 2, boolean expressions are used as ternary operators. However, replacing boolean ternary expressions with inline ifs where appropriate better portrays the code's intention and therefore makes it easier for others to read it.

Enhancements

• Sourcery can now provide refactorings inside f-string format values
• Extracted methods will now be prefixed with a TODO comment reminding you to rename them in each place they are used: # TODO Rename this here and in [first_usage] and [second_usage]
• Extended assign-if-exp to cover the case where the default value is assigned first

Before:

x = 2
if condition:
    x = 1

After:

x = 1 if condition else 2

• default-get refactorings now work for if expressions
• Sourcery will now show a notification when you first successfully log in.

Bug fixes

• Sourcery will now provide metrics and refactorings on functions containing named expressions
• use-any will no longer trigger if it introduces a short-circuit where global state is written
• merge-repeated-if will no longer trigger if the first if writes global state that could affect the second
• use-any will no longer trigger if the for loop target is used later on

Note for PyCharm users - version 2020.2 or later is required for Sourcery 0.9.3 and above.

Class level extract method (VS Code only)

Sourcery can now perform class-level refactorings! It scans your classes for duplicate code and will suggest extracting it into methods.

Normally Sourcery scans for refactorings on every code change. Class-level scans need to do more analysis, so this will only run on file open and save to improve performance.

Class-level extract method

Sourcery refactoring id: class-extract-method

Description:

Extract duplicate code from different methods in a class into a new one.

Available to all users while in alpha, then will move to Pro/Team

Before:

class TestClass:
    def extraction_example(self):
        self.speed_slider = Scale(
            self.parent, from_=1, to=10, orient=HORIZONTAL, label="Speed"
        )
        self.speed_slider.pack()
        self.speed_slider.set(DEFAULT_SPEED)
        self.speed_slider.configure(background="white")

    def intervening_function(self):
        return 1

    def next_example(self):
        self.force_slider = Scale(
            self.parent, from_=1, to=10, orient=HORIZONTAL, label="Force"
        )
        self.force_slider.pack()
        self.force_slider.set(DEFAULT_FORCE)
        self.force_slider.configure(background="white")

After:

class TestClass:
    def extraction_example(self):
        self.speed_slider = self._extracted_from_next_example_2("Speed", DEFAULT_SPEED)

    def intervening_function(self):
        return 1

    def next_example(self):
        self.force_slider = self._extracted_from_next_example_2("Force", DEFAULT_FORCE)

    def _extracted_from_next_example_2(self, label, arg1):
        result = Scale(self.parent, from_=1, to=10, orient=HORIZONTAL, label=label)
        result.pack()
        result.set(arg1)
        result.configure(background="white")
        return result

Explanation:

Do not Repeat Yourself (DRY) is an important tenet of writing clean, maintainable code. Duplicated code bloats the code base, making it harder to read and understand. It often also leads to bugs. Where changes are made in only some of the duplicated areas unintended behaviour will often arise.

One of the main ways to remove duplication is to extract the common areas into another method and call that. Sourcery can detect areas of duplicate code that are in different methods in the same class and extract them. It is recommended that you then rename the extracted function and any arguments that have not been automatically named. In the above example a suitable method name would be create_slider, and arg1 would be default_value.

In VS Code each location to be extracted from is shown as a sub-heading in the Problems pane so you can keep track of what is happening.

Class-level refactorings such as this one run on file open and file save, rather than on every change to the code.

New option to permanently turn off a type of refactoring for a user

When a refactoring only uses one proposal, there will now be a menu option in VS Code and PyCharm to 'Never show me this refactoring', or 'Turn off this refactoring permanently'. This will permanently disable this proposal, and it will not be displayed to the user again on that machine. This configuration is stored in a sourcery.yaml file in the user's config location. Other configuration can also be set up in this file, for example to configure the clone detection parameters.

When this option is taken a notification is displayed to the user asking for feedback and giving them the location of this config file if they wish to edit it and undo the action.

As part of these changes the 'Skip all' menu option has been removed, to avoid having too many Sourcery options. Adding # sourcery: skip to a function definition will still work as before.

New refactorings

Merge Except Handler

Sourcery refactoring id: merge-except-handler

Description:

Merge except handlers with identical code blocks

Before:

try:
    connect(credentials)
except ConnectionError as e:
    log_error(e)
except RuntimeError as e:
    log_error(e)

After:

try:
    connect(credentials)
except (ConnectionError, RuntimeError) as e:
    log_error(e)

Explanation:

This reduces repetition across equivalent exception handlers by combining them into one.

Enhancements

• The min-max-identity refactoring will now trigger on appropriate if expressions

Bug fixes

• No longer suggest comprehensions containing an await or yield
• Scan for refactoring and detect clones will now work correctly in VS Code on Windows
• Fix issue where Sourcery could incorrectly suggest removal of a variable used in a pandas query
• Fix spike in CPU/memory on VS Code shutdown

Detect clones configuration

It is now possible to configure the detect clones functionality for finding duplicate code.

This configuration must be added at the project level in the .sourcery.yaml file, under the clone_detection heading. See here for more details on the Sourcery configuration file.

• min_lines: The minimum number of lines each code section must have to be picked up. The minimum value for this is 3.
• min_duplicates: The minimum number of duplicate code sections there must be. The minimum value for this is 2.
• identical_clones_only: When this is set only exactly identical sections of duplicate code will be picked up.

clone_detection:
  min_lines: 3
  min_duplicates: 2
  identical_clones_only: false

Working memory metric improvements

The working memory metric will now only consider the most complex element within collections or call arguments, rather than incrementing for each element.

This means that when declaring a list like this one:

hats = ["BOWLER", "TOP" + "HAT", "TRILBY"]

The working memory for this statement will now be 3 (incrementing for hats, TOP and HAT) whereas before it would have been 5. This will prevent the phenomenon where creating dictionaries, lists or data classes would give functions unreasonably high working memory scores. For a full discussion of the working memory metric see our blog post here.

New refactorings

Swap Variable

Sourcery refactoring id: swap-variable

Description:

Swap variable values with tuple assignment

Before:

temp = a
a = b
b = temp

After:

a, b = b, a

Explanation:

When trying to swap the values held by two variables, we may avoid using an extra temporary variable unnecessarily by employing Python's tuple assignment. As a consequence of this refactoring, the code is not only more concise but also reflects its intention more clearly.

Merge Else If Into Elif

Sourcery refactoring id: merge-else-if-into-elif

Description:

Merge else clause's nested if statement into elif

Before:

def interpret_response(response):
    if response.status == "200":
        return response.data
    else:
        if response.status == "404":
            return "Not Found"
        else:
            return "Error"

After:

def interpret_response(response):
    if response.status == "200":
        return response.data
    elif response.status == "404":
        return "Not Found"
    else:
        return "Error"

Explanation:

Flattening if statements nested within else clauses generates code that is easier to read and expand upon.

Inline variable

Sourcery refactoring id: inline-variable

Description:

Inline variable that is only used once

Before:

thelist = []
for i in range(10):
    k = i**2
    thelist.append(k)

After:

thelist = []
for i in range(10):
    thelist.append(i**2)

Explanation:

Inlining variable can help to streamline the code, but can also make it less readable. Sourcery will only inline variables where doing so allows further readability changes to be made, such as converting a loop into a list comprehension.

Remove redundant continue

Sourcery refactoring id: remove-redundant-continue

Description:

Remove redundant continue statement

Before:

mylist2 = []
for i in mylist:
    if i != 2:
        mylist2.append(i)
    else:
        continue

After:

mylist2 = []
for i in mylist:
    if i != 2:
        mylist2.append(i)

Explanation:

If a continue is not followed by any other statements in a for or while loop then it is not necessary, so can be removed. Removing unnecessary lines declutters the code and makes it easier to understand. This refactoring will only be triggered if it unlocks further improvements.

Reintroduce else

Sourcery refactoring id: reintroduce-else

Description:

Lift code into else after break in control flow

Before:

summer_hats = []
for hat in hats:
    if hat in WINTER_HATS:
        continue
    summer_hats.append(hat)

After:

summer_hats = []
for hat in hats:
    if hat in WINTER_HATS:
        continue
    else:
        summer_hats.append(hat)

Explanation:

Where the body of an if statement ends with a break in the control flow, such as a continue, return or raise, the subsequent statements can be lifted into the else clause. On its own this change does not improve the code, so Sourcery will only suggest it where it unlocks furher improvements. In the example above once the code has been lifted into the else the conditional can be inverted and the continue removed, which then lets the for loop be converted into a list comprehension.

Remove Duplicate Key

Sourcery refactoring id: remove-duplicate-key

Description:

Remove duplicate keys in declaration of sets and dicts

Before:

addresses = {*address_list1, *address_list2, *address_list1}

After:

addresses = {*address_list2, *address_list1}

Explanation:

Keys of Sets and Dictionaries must be unique. Hence, repeated keys are redundant and can be removed from the declaration to increase the conciseness and clarity of the code.

For dictionaries that have repeated keys with differing values (i.e. {key1: 'some_value', key1: 'another_value'}), Sourcery will only remove the key-value pairs that would be removed at run time.

Flatten Nested Try

Sourcery refactoring id: flatten-nested-try

Description:

Merge nested try-statement into a single try

Before:

def testConnection(db, credentials):
    try:
        try:
            db.connect(credentials)
        except InvalidCredentials:
            return "Check your credentials"
        except ConnectionError:
            return "Error while trying to connect"
    finally:
        print("Connection attempt finished")
    return "Connection Successful"

After:

def testConnection(db, credentials):
    try:
        db.connect(credentials)
    except InvalidCredentials:
        return "Check your credentials"
    except ConnectionError:
        return "Error while trying to connect"
    finally:
        print("Connection attempt finished")
    return "Connection Successful"

Explanation:

Flattening try-except statements nested within a try-finally generates equivalent code that is easier to read and expand upon.

Collection into Set

Sourcery refactoring id: collection-into-set

Description:

Use set when checking membership of a collection of literals

Before:

if currency in ["EUR", "USD"]:
    take_payment()

After:

if currency in {"EUR", "USD"}:
    take_payment()

Explanation:

When checking if a variable is one of a collection of literals it is both more natural and more performant to use a set rather than a list or tuple to define that collection. Note that this can only be done where Sourcery can ascertain that the value being checked is of a hashable type.

Enhancements

• Sourcery will now suggest refactorings which involve the removal of comments. This allows us to make more suggestions and be more impactful on your code. These suggestions are flagged and the comments that are removed are clearly shown when hovering over.
• Simplify generators contained in calls to bytes
• Adjust return-identity proposal so that it does not propose in situations where a series of checks are being made.
• Flip Compare now works for all constant types
• Merge comparison refactoring extended to work with sets and tuples
• Ensure switch proposal is not suggested where the if conditions end with breaks of control flow
• Refinements to where the else-after-guard proposal is suggested

Bug fixes

• Fixed issue with Sourcery incorrectly removing brackets after a splat operator

New Refactorings

Replace dict.get(x, None) with dict.get(x)

Before:

hats = {"bowler": Bowler(), "sombrero": Sombrero()}
fedora = hats.get("fedora", None)

After:

hats = {"bowler": Bowler(), "sombrero": Sombrero()}
fedora = hats.get("fedora")

Explanation:

When using a dictionary's get method you can specify a default to return if the key is not found. This defaults to None, so it is unnecessary to specify None if this is the required behaviour. Removing the unnecessary argument makes the code slightly shorter and clearer.

Remove unit step from range

Sourcery refactoring id: remove-unit-step-from-range

Description:

Replace range(x, y, 1) with range(x, y)

Before:

for i in range(y, len(x), 1):
    do_t()

After:

for i in range(y, len(x)):
    do_t()

Explanation:

The default step value for a call to range() is 1, so it is unnecessary to explicitly define it. This refactoring removes this argument, slightly shortening the code.

Small fixes and enhancements

• Enhance removal of redundant exception handlers to support tuples
• Improved installation instructions for VS Code
• Fix for issue where conditionals could be removed incorrectly
• Fix issues with internal tracking metrics

Duplicate code detection in PyCharm

Our new duplicate code detection feature is now available for PyCharm!

You can now select multiple files or folders and choose 'Sourcery - Detect Clones' to scan for duplicated code within them. This will find sections of code that are 3 lines long or greater that are duplicated in multiple places. It will also find near-duplicate code where variable names have been changed but the code structure remains the same. The duplicate code found will appear in the Sourcery tool window.

This feature is available in our Pro subscription. It is in active development so we would love your feedback.

Disabling code quality warnings

Have a piece of code that you don't want to touch that is giving a low quality warning?

You can now disable these warnings in VS Code and PyCharm. Add # sourcery no-metrics at the end of the line with the function definition to do so. There's also a new quick-fix action on the warnings to disable them.

Minor fixes

• Fixed issue where Sourcery scan commands were incorrectly showing in VS Code command palette
• Running a scan for refactorings in VS Code now shows a link to the problems pane in the notification that the scan is complete.

Enhancements

• Added links to documentation from refactoring hover
• Added local logging to plugins to improve diagnosis of issues going forward

Bug fixes

• Fixed issue where plugins were eating too much memory
• Improved performance of CLI
• Fixed issue with incorrect comprehension proposal for async for loops.

Sourcery Pro launched

You can now purchase a Sourcery Pro subscription.

With Sourcery Pro you can:

• Remove duplicate code with our new automated method extraction
• Refactor your whole project in the IDE
• Use our GitHub bot on private repos
• Find duplicate code across your project (VS Code only at present)

Get more details here.

Duplicate code detection

You can now select multiple files or folders and scan for duplicated code within them. This will find sections of code that are 3 lines long or greater that are duplicated in multiple places. It will also find near-duplicate code where variable names have been changed but the code structure remains the same.

This feature is available in our Pro subscription, for VS Code users only initially. It is in active development so we would love your feedback.

Sublime support

Sourcery is now available in Sublime.

Get more details and setup instructions here.

New refactorings

Use str.join() instead of for loop: use-join

This will convert code like this:

result = ""
for text in long_list_of_text():
    result = result + text
return result

into:

result = "".join(long_list_of_text())
return result

Simplify Constant Sum: simplify-constant-sum

Changes this:

results = sum(1 for token in source.split() if token == 1)

to this:

results = sum(token == 1 for token in source.split())

Minor fixes

• Make simplify-boolean-comparison suggest more often
• use-count now triggers on the result of simplify-constant-sum rather than directly
• remove-redundant-if now slightly more powerful
• Improved handling of if..elif chains with repeated code
• Don't remove-redundant-if when invalidated by a context manager
• Fix issue where switch refactoring could produce incorrect result
• Fix issue with hoisting statements from loops which could produce incorrect results
• Fixed issue in PyCharm where Sourcery could lose connection to binary when it was scanning a very large function.

Improved low code quality warning message

This message now displays the name of the function where we have detected low code quality. This lets you easily see at a glance where the problems in your code are.

Bug fixes

• Improved PyCharm stability when viewing files with large numbers of functions
• Fix indentation issue when refactoring functions inside classes in PyCharm
• Fix error in GitHub bot quality report where section of code has zero weighting

Extract Method Refactoring

Sourcery can now propose refactorings to extract blocks of code into new functions. This currently proposes in two circumstances:

• When finding duplicate and near duplicate blocks of code within a single function
• When finding a large block of code in the function that would be better off in it's own function

This feature will be available in the upcoming Pro subscription.

Full project scan for refactorings

Select the project or a folder within a project in the IDE plugins, and then have Sourcery scan every file within it. All refactoring suggestions will be displayed as they are found in the Problems window.

This makes it easy to refactor a large portion of the codebase quickly without manually having to visit every file to see the Sourcery suggestions.

This feature will be available in the upcoming Pro subscription.

Improve project and module code metrics in Quality Report

Previously when aggregating metrics we simply calculated metrics for each function and then averaged them. This did not account for the differing size of functions - a 100 line function has more impact on quality than a 10 line one.

We now weight each metric (apart from method length) by the number of lines in each method when averaging them. This gives a better view of the actual file and project-level quality.

New Refactorings

Hoist code from else after guard clause: remove-else-after-guard

Hoist code from an else after a guard clause. This will happen where the main body of an if contains only a raise or return statement, and there are multiple statements in the else.

def f(a=None):
    if a is None:
        return 42
    else:
        # some long statement
        var = (i**2 for i in range(a))
        return sum(var)

is converted into:

def f(a=None):
    if a is None:
        return 42

    # some long statement
    var = (i**2 for i in range(a))
    return sum(var)

Inline variable that is immediately returned after if

Where a value is set on each branch of an if and then immediately returned, instead return it directly from each branch.

def f():
    if condition:
        val = 42
    else:
        val = 0
    return val

is converted into:

def f():
    if condition:
        return 42
    else:
        return 0

Add Command Line Interface --backup option

Use with sourcery refactor --in-place to backup changed files with given suffix:

sourcery refactor --in-place --backup .bak main.py

If any refactorings are found in main.py a copy will be made at main.py.bak before it is updated in place.

Bug Fixes and other adjustments

• Only propose for-index-replacement for lists, as it's possible __len__ and __getitem__ are implemented but __iter__ is not.
• Minor changes to the swap-if-else refactoring. This will now also be triggered where it can enable the above remove-else-after-guard refactoring. The description has also changed to reflect this.
• Fix issue where block comments above a changed line stopped refactorings being suggested
• We now don't convert an if/else to an if expression where the test contains a boolean operation. This prevents creation of hard-to-read if expression lines.
• Quality warnings in PyCharm Problems pane no longer show as raw HTML
• IDE Code metrics now have link to our documentation. We have added a title to the code metrics in the IDE with a link to our documentation. This will let you easily see how the metrics are calculated and how to improve your code.
• Fix formatting issue in Vim where a blank line was deleted after the refactored function

New Refactorings

Simplify negative list access

This refactoring uses the fact that Python allows negative list indices to be accessed directly.

It converts this:

last_element = a[len(a) - 1]

into this:

last_element = a[-1]

Merge Comparison refactoring extended to the negative case

The merge-comparison proposal will now apply in this case:

if x != a and x != b:
    do_stuff()

This will be converted to:

if x not in [a, b]:
    do_stuff()

Bug fixes

• Ensure statements that write global state can't be hoisted out of loops (e.g. function calls)
• Do not try to remove pass statements at class level
• Do not wrap the targets of annotated assignments in brackets

Sourcery CLI

Sourcery is now available as a command line interface. This enables several new use cases:

• Refactor files on your machine without needing to run Sourcery through PyCharm or VSCode
• Check every commit is refactored using a pre-commit hook
• Run a continuous integration job to check all code is refactored

This functionality is only available with Pro/Team subscriptions. If you'd like to try it out for your team please contact us.

Installation

The Sourcery command line interface can be installed by running:

pip install sourcery

Login

Once installed you can interactively login with:

sourcery login

which will open up a browser tab and ask for confirmation.

Usage

To display suggested refactorings as a diff:

sourcery refactor {file_or_directory}

And to apply those changes to the files:

sourcery refactor --in-place {file_or_directory}

Full documentation is available here.

Code quality metrics enabled by default in the IDE plugins

You can now hover over a method definition to see a quick view of its code quality. The metrics available are:

• Cognitive complexity - a measure of how much complex nested logic is in the method
• Size - a measure of how large the method is
• Working memory - a measure of how many variables you need to keep in mind to understand the most complex parts of the method
• Quality - a combination of the above metrics to give a percentage quality score

This can be switched off in the Sourcery section of the plugin settings.

New Refactorings

Use with context manager to ensure file closure

A common way of opening and using files is:

file = open("welcome.txt")
data = file.read()
print(data)
file.close()

However if an exception is thrown in between the file being opened and closed the call to file.close() may end up being skipped. By using Python's with context manager the file is closed for you as soon as the block is exited.

with open("welcome.txt") as file:
    data = file.read()
    print(data)

Extending the list-comprehension refactoring

We have now extended this refactoring to include an additional case, where augmented assignment is used to add to the list rather than append.

The following code:

files = []
for x in file_iterator:
    if x[-4:] == ".csv":
        files += [x]

will now be refactored as:

files = [x for x in file_iterator if x[-4:] == ".csv"]

Simplify if expression by using or

Often we find ourselves setting a value if it evaluates to True, and otherwise using a default.

currency = args["currency"] if args["currency"] else DEFAULT_CURRENCY

This can be simplified to the following, which is a bit easier to read and avoids the duplication of args['currency'].

currency = args["currency"] or DEFAULT_CURRENCY

Bug Fixes

• Prevent use-assigned-variable from re-using properties

Enhanced code quality metrics in the IDE

We've been making some changes to how our code quality metrics are displayed.

When hovering over a method definition you now get more explanation of the code metrics. This means that you can see if the metric scores are good or bad at a glance.

In VS Code this is an emoji as shown above, whereas in PyCharm it is shown in writing. For each metric if the score is below average we also give a tip on how to improve it.

Right now these metrics are hidden by default in PyCharm and VS Code, but they're very easy to enable.

To set them up just add the following setting to your Sourcery config file:

Update 2022-10-13

The metrics are now enabled by default both in VSCode and PyCharm.

The config option metrics.enabled has been removed.

New Refactorings

Enumerate

Loop counters that are incremented on each iteration of the loop like so:

i = 0
for animal in animals:
    i += 1
    print(i, animal)

can be replaced with a call to enumerate with a suitable start index:

for i, animal in enumerate(animals, start=1):
    print(i, animal)

Boolean IfExp identity

The following simple but overly verbose code:

return True if some_boolean_expression else False

is refactored to:

return bool(some_boolean_expression)

The negated version:

return False if some_boolean_expression else True

is refactored to:

return not some_boolean_expression

Writing a code editor plugin documentation

Instructions for how to write a code editor plugin are now available here. As the Sourcery binary implements the Language Server Protocol it is very easy to write a plugin for a new code editor.

Website changes

• GitHub repo page allows you to select which branch to create refactor PR for
• GitHub repo page shows instructions for starting refactoring reviews if none exist
• GitHub repo page shows all refactoring jobs, including running and failed ones

Free for 3 team members on private repos

We wanted to make it easier for you to bring Sourcery into your workplace so you can improve your codebase. Starting today you can now use Sourcery for free with up to 3 team members! This is for both public and private repos.

Sourcery will review and refactor all new pull requests for these team members and give you insight to code quality. The only limitation is that refactoring whole private repos is still disabled on the free tier.

Public repos, as always, get access to all of Sourcery's features for free.

Choosing team members

One of the big takeaways from some of our conversations with you is that teams != full GitHub organisations.

Now you can pick and choose individual team members within your organisation who should have Sourcery access.

All organisation members still get full Sourcery functionality for public repositories.

More insights into code quality

Our code quality tools are now available outside of GitHub! Right now they're hidden by default in PyCharm and VS Code, but they're very easy to enable.

To set them up just add the following setting to your Sourcery config file:

metrics:
  enabled: true

We're still in the alpha stage of our code quality tools within IDEs, so we'd love to get your feedback and thoughts as we continue to make improvements.