Detect clones configuration

It is now possible to configure the detect clones functionality for finding duplicate code.

This configuration must be added at the project level in the .sourcery.yaml file, under the clone_detection heading. See here for more details on the Sourcery configuration file.

• min_lines: The minimum number of lines each code section must have to be picked up. The minimum value for this is 3.
• min_duplicates: The minimum number of duplicate code sections there must be. The minimum value for this is 2.
• identical_clones_only: When this is set only exactly identical sections of duplicate code will be picked up.

clone_detection:
  min_lines: 3
  min_duplicates: 2
  identical_clones_only: false

Working memory metric improvements

The working memory metric will now only consider the most complex element within collections or call arguments, rather than incrementing for each element.

This means that when declaring a list like this one:

hats = ["BOWLER", "TOP" + "HAT", "TRILBY"]

The working memory for this statement will now be 3 (incrementing for hats, TOP and HAT) whereas before it would have been 5. This will prevent the phenomenon where creating dictionaries, lists or data classes would give functions unreasonably high working memory scores. For a full discussion of the working memory metric see our blog post here.

New refactorings

Swap Variable

Sourcery refactoring id: swap-variable

Description:

Swap variable values with tuple assignment

Before:

temp = a
a = b
b = temp

After:

a, b = b, a

Explanation:

When trying to swap the values held by two variables, we may avoid using an extra temporary variable unnecessarily by employing Python's tuple assignment. As a consequence of this refactoring, the code is not only more concise but also reflects its intention more clearly.

Merge Else If Into Elif

Sourcery refactoring id: merge-else-if-into-elif

Description:

Merge else clause's nested if statement into elif

Before:

def interpret_response(response):
    if response.status == "200":
        return response.data
    else:
        if response.status == "404":
            return "Not Found"
        else:
            return "Error"

After:

def interpret_response(response):
    if response.status == "200":
        return response.data
    elif response.status == "404":
        return "Not Found"
    else:
        return "Error"

Explanation:

Flattening if statements nested within else clauses generates code that is easier to read and expand upon.

Inline variable

Sourcery refactoring id: inline-variable

Description:

Inline variable that is only used once

Before:

thelist = []
for i in range(10):
    k = i**2
    thelist.append(k)

After:

thelist = []
for i in range(10):
    thelist.append(i**2)

Explanation:

Inlining variable can help to streamline the code, but can also make it less readable. Sourcery will only inline variables where doing so allows further readability changes to be made, such as converting a loop into a list comprehension.

Remove redundant continue

Sourcery refactoring id: remove-redundant-continue

Description:

Remove redundant continue statement

Before:

mylist2 = []
for i in mylist:
    if i != 2:
        mylist2.append(i)
    else:
        continue

After:

mylist2 = []
for i in mylist:
    if i != 2:
        mylist2.append(i)

Explanation:

If a continue is not followed by any other statements in a for or while loop then it is not necessary, so can be removed. Removing unnecessary lines declutters the code and makes it easier to understand. This refactoring will only be triggered if it unlocks further improvements.

Reintroduce else

Sourcery refactoring id: reintroduce-else

Description:

Lift code into else after break in control flow

Before:

summer_hats = []
for hat in hats:
    if hat in WINTER_HATS:
        continue
    summer_hats.append(hat)

After:

summer_hats = []
for hat in hats:
    if hat in WINTER_HATS:
        continue
    else:
        summer_hats.append(hat)

Explanation:

Where the body of an if statement ends with a break in the control flow, such as a continue, return or raise, the subsequent statements can be lifted into the else clause. On its own this change does not improve the code, so Sourcery will only suggest it where it unlocks furher improvements. In the example above once the code has been lifted into the else the conditional can be inverted and the continue removed, which then lets the for loop be converted into a list comprehension.

Remove Duplicate Key

Sourcery refactoring id: remove-duplicate-key

Description:

Remove duplicate keys in declaration of sets and dicts

Before:

addresses = {*address_list1, *address_list2, *address_list1}

After:

addresses = {*address_list2, *address_list1}

Explanation:

Keys of Sets and Dictionaries must be unique. Hence, repeated keys are redundant and can be removed from the declaration to increase the conciseness and clarity of the code.

For dictionaries that have repeated keys with differing values (i.e. {key1: 'some_value', key1: 'another_value'}), Sourcery will only remove the key-value pairs that would be removed at run time.

Flatten Nested Try

Sourcery refactoring id: flatten-nested-try

Description:

Merge nested try-statement into a single try

Before:

def testConnection(db, credentials):
    try:
        try:
            db.connect(credentials)
        except InvalidCredentials:
            return "Check your credentials"
        except ConnectionError:
            return "Error while trying to connect"
    finally:
        print("Connection attempt finished")
    return "Connection Successful"

After:

def testConnection(db, credentials):
    try:
        db.connect(credentials)
    except InvalidCredentials:
        return "Check your credentials"
    except ConnectionError:
        return "Error while trying to connect"
    finally:
        print("Connection attempt finished")
    return "Connection Successful"

Explanation:

Flattening try-except statements nested within a try-finally generates equivalent code that is easier to read and expand upon.

Collection into Set

Sourcery refactoring id: collection-into-set

Description:

Use set when checking membership of a collection of literals

Before:

if currency in ["EUR", "USD"]:
    take_payment()

After:

if currency in {"EUR", "USD"}:
    take_payment()

Explanation:

When checking if a variable is one of a collection of literals it is both more natural and more performant to use a set rather than a list or tuple to define that collection. Note that this can only be done where Sourcery can ascertain that the value being checked is of a hashable type.

Enhancements

• Sourcery will now suggest refactorings which involve the removal of comments. This allows us to make more suggestions and be more impactful on your code. These suggestions are flagged and the comments that are removed are clearly shown when hovering over.
• Simplify generators contained in calls to bytes
• Adjust return-identity proposal so that it does not propose in situations where a series of checks are being made.
• Flip Compare now works for all constant types
• Merge comparison refactoring extended to work with sets and tuples
• Ensure switch proposal is not suggested where the if conditions end with breaks of control flow
• Refinements to where the else-after-guard proposal is suggested

Bug fixes

• Fixed issue with Sourcery incorrectly removing brackets after a splat operator