Sourcery Pro launched

You can now purchase a Sourcery Pro subscription.

With Sourcery Pro you can:

• Remove duplicate code with our new automated method extraction
• Refactor your whole project in the IDE
• Use our GitHub bot on private repos
• Find duplicate code across your project (VS Code only at present)

Get more details here.

Duplicate code detection

You can now select multiple files or folders and scan for duplicated code within them. This will find sections of code that are 3 lines long or greater that are duplicated in multiple places. It will also find near-duplicate code where variable names have been changed but the code structure remains the same.

This feature is available in our Pro subscription, for VS Code users only initially. It is in active development so we would love your feedback.

Sublime support

Sourcery is now available in Sublime.

Get more details and setup instructions here.

New refactorings

Use str.join() instead of for loop: use-join

This will convert code like this:

result = ""
for text in long_list_of_text():
    result = result + text
return result

into:

result = "".join(long_list_of_text())
return result

Simplify Constant Sum: simplify-constant-sum

Changes this:

results = sum(1 for token in source.split() if token == 1)

to this:

results = sum(token == 1 for token in source.split())

Minor fixes

• Make simplify-boolean-comparison suggest more often
• use-count now triggers on the result of simplify-constant-sum rather than directly
• remove-redundant-if now slightly more powerful
• Improved handling of if..elif chains with repeated code
• Don't remove-redundant-if when invalidated by a context manager
• Fix issue where switch refactoring could produce incorrect result
• Fix issue with hoisting statements from loops which could produce incorrect results
• Fixed issue in PyCharm where Sourcery could lose connection to binary when it was scanning a very large function.

Improved low code quality warning message

This message now displays the name of the function where we have detected low code quality. This lets you easily see at a glance where the problems in your code are.

Bug fixes

• Improved PyCharm stability when viewing files with large numbers of functions
• Fix indentation issue when refactoring functions inside classes in PyCharm
• Fix error in GitHub bot quality report where section of code has zero weighting

Extract Method Refactoring

Sourcery can now propose refactorings to extract blocks of code into new functions. This currently proposes in two circumstances:

• When finding duplicate and near duplicate blocks of code within a single function
• When finding a large block of code in the function that would be better off in it's own function

This feature will be available in the upcoming Pro subscription.

Full project scan for refactorings

Select the project or a folder within a project in the IDE plugins, and then have Sourcery scan every file within it. All refactoring suggestions will be displayed as they are found in the Problems window.

This makes it easy to refactor a large portion of the codebase quickly without manually having to visit every file to see the Sourcery suggestions.

This feature will be available in the upcoming Pro subscription.

Improve project and module code metrics in Quality Report

Previously when aggregating metrics we simply calculated metrics for each function and then averaged them. This did not account for the differing size of functions - a 100 line function has more impact on quality than a 10 line one.

We now weight each metric (apart from method length) by the number of lines in each method when averaging them. This gives a better view of the actual file and project-level quality.

New Refactorings

Hoist code from else after guard clause: remove-else-after-guard

Hoist code from an else after a guard clause. This will happen where the main body of an if contains only a raise or return statement, and there are multiple statements in the else.

def f(a=None):
    if a is None:
        return 42
    else:
        # some long statement
        var = (i ** 2 for i in range(a))
        return sum(var)

is converted into:

def f(a=None):
    if a is None:
        return 42

    # some long statement
    var = (i ** 2 for i in range(a))
    return sum(var)

Inline variable that is immediately returned after if

Where a value is set on each branch of an if and then immediately returned, instead return it directly from each branch.

def f():
    if condition:
        val = 42
    else:
        val = 0
    return val

is converted into:

def f():
    if condition:
        return 42
    else:
        return 0

Add Command Line Interface --backup option

Use with sourcery refactor --in-place to backup changed files with given suffix:

sourcery refactor --in-place --backup .bak main.py

If any refactorings are found in main.py a copy will be made at main.py.bak before it is updated in place.

Bug Fixes and other adjustments

• Only propose for-index-replacement for lists, as it's possible __len__ and __getitem__ are implemented but __iter__ is not.
• Minor changes to the swap-if-else refactoring. This will now also be triggered where it can enable the above remove-else-after-guard refactoring. The description has also changed to reflect this.
• Fix issue where block comments above a changed line stopped refactorings being suggested
• We now don't convert an if/else to an if expression where the test contains a boolean operation. This prevents creation of hard-to-read if expression lines.
• Quality warnings in PyCharm Problems pane no longer show as raw HTML
• IDE Code metrics now have link to our documentation. We have added a title to the code metrics in the IDE with a link to our documentation. This will let you easily see how the metrics are calculated and how to improve your code.
• Fix formatting issue in Vim where a blank line was deleted after the refactored function

New Refactorings

Simplify negative list access

This refactoring uses the fact that Python allows negative list indices to be accessed directly.

It converts this:

last_element = a[len(a) - 1]

into this:

last_element = a[-1]

Merge Comparison refactoring extended to the negative case

The merge-comparison proposal will now apply in this case:

if x != a and x != b:
    do_stuff()

This will be converted to:

if x not in [a, b]:
    do_stuff()

Bug fixes

• Ensure statements that write global state can't be hoisted out of loops (e.g. function calls)
• Do not try to remove pass statements at class level
• Do not wrap the targets of annotated assignments in brackets

Sourcery CLI

Sourcery is now available as a command line interface. This enables several new use cases:

• Refactor files on your machine without needing to run Sourcery through PyCharm or VSCode
• Check every commit is refactored using a pre-commit hook
• Run a continuous integration job to check all code is refactored

This functionality is only available with Pro/Team subscriptions. If you'd like to try it out for your team please contact us.

Installation

The Sourcery command line interface can be installed by running:

pip install sourcery-cli

Login

Once installed you can interactively login with:

sourcery login

which will open up a browser tab and ask for confirmation.

Usage

To display suggested refactorings as a diff:

sourcery refactor {file_or_directory}

And to apply those changes to the files:

sourcery refactor --in-place {file_or_directory}

Full documentation is available here.

Code quality metrics enabled by default in the IDE plugins

You can now hover over a method definition to see a quick view of its code quality. The metrics available are:

• Cognitive complexity - a measure of how much complex nested logic is in the method
• Size - a measure of how large the method is
• Working memory - a measure of how many variables you need to keep in mind to understand the most complex parts of the method
• Quality - a combination of the above metrics to give a percentage quality score

This can be switched off in the Sourcery section of the plugin settings.

New Refactorings

Use with context manager to ensure file closure

A common way of opening and using files is:

file = open("welcome.txt")
data = file.read()
print(data)
file.close()

However if an exception is thrown in between the file being opened and closed the call to file.close() may end up being skipped. By using Python's with context manager the file is closed for you as soon as the block is exited.

with open("welcome.txt") as file:
    data = file.read()
    print(data)

Extending the list-comprehension refactoring

We have now extended this refactoring to include an additional case, where augmented assignment is used to add to the list rather than append.

The following code:

files = []
for x in file_iterator:
    if x[-4:] == ".csv":
        files += [x]

will now be refactored as:

files = [x for x in file_iterator if x[-4:] == ".csv"]

Simplify if expression by using or

Often we find ourselves setting a value if it evaluates to True, and otherwise using a default.

currency = args['currency'] if args['currency'] else DEFAULT_CURRENCY

This can be simplified to the following, which is a bit easier to read and avoids the duplication of args['currency'].

currency = args['currency'] or DEFAULT_CURRENCY

Bug Fixes

• Prevent use-assigned-variable from re-using properties

Enhanced code quality metrics in the IDE

We've been making some changes to how our code quality metrics are displayed.

When hovering over a method definition you now get more explanation of the code metrics. This means that you can see if the metric scores are good or bad at a glance.

In VS Code this is an emoji as shown above, whereas in PyCharm it is shown in writing. For each metric if the score is below average we also give a tip on how to improve it.

Right now these metrics are hidden by default in PyCharm and VS Code, but they're very easy to enable.

To set them up just add the following setting to your Sourcery config file:

metrics:
    enabled: True

New Refactorings

Enumerate

Loop counters that are incremented on each iteration of the loop like so:

i = 0
for animal in animals:
    i += 1
    print(i, animal)

can be replaced with a call to enumerate with a suitable start index:

for i, animal in enumerate(animals, start=1):
    print(i, animal)

Boolean IfExp identity

The following simple but overly verbose code:

return True if some_boolean_expression else False

is refactored to:

return bool(some_boolean_expression)

The negated version:

return False if some_boolean_expression else True

is refactored to:

return not some_boolean_expression

Writing a code editor plugin documentation

Instructions for how to write a code editor plugin are now available here. As the Sourcery binary implements the Language Server Protocol it is very easy to write a plugin for a new code editor.

Website changes

• GitHub repo page allows you to select which branch to create refactor PR for
• GitHub repo page shows instructions for starting refactoring reviews if none exist
• GitHub repo page shows all refactoring jobs, including running and failed ones

Free for 3 team members on private repos

We wanted to make it easier for you to bring Sourcery into your workplace so you can improve your codebase. Starting today you can now use Sourcery for free with up to 3 team members! This is for both public and private repos.

Sourcery will review and refactor all new pull requests for these team members and give you insight to code quality. The only limitation is that refactoring whole private repos is still disabled on the free tier.

Public repos, as always, get access to all of Sourcery's features for free.

Choosing team members

One of the big takeaways from some of our conversations with you is that teams != full GitHub organisations.

Now you can pick and choose individual team members within your organisation who should have Sourcery access.

All organisation members still get full Sourcery functionality for public repositories.

More insights into code quality

Our code quality tools are now available outside of GitHub! Right now they're hidden by default in PyCharm and VS Code, but they're very easy to enable.

To set them up just add the following setting to your Sourcery config file:

metrics:
    enabled: True

We're still in the alpha stage of our code quality tools within IDEs, so we'd love to get your feedback and thoughts as we continue to make improvements.