Chapter 14

Algorithms

14.0 Intro · 14.1 Algorithm Concepts · 14.2 Searching · 14.3 Sorting

← → or Space to navigate · F for fullscreen

14.1 Algorithm Concepts

Big O · time & space complexity · benchmarking

Big O Notation

How an algorithm's runtime scales with input size n:

Notation Name Example
O(1) Constant Dict lookup, list append
O(log n) Logarithmic Binary search
O(n) Linear List scan, linear search
O(n log n) Linearithmic Merge sort, Timsort
O(n²) Quadratic Bubble sort, nested loops

Always prefer O(1) and O(log n) over O(n) and O(n²) when working with large data.

Benchmarking

import time

# time.perf_counter — wall clock, high precision
start = time.perf_counter()
result = sum(range(1_000_000))
elapsed = time.perf_counter() - start
print(f"{elapsed:.6f}s")
import timeit

# timeit — eliminates startup noise, repeats automatically
t = timeit.timeit(
    "sum(range(1_000_000))",
    number=10
)
print(f"avg: {t/10:.6f}s")

Best / worst / average case

Case Meaning
Best Most favourable input
Worst Most unfavourable input
Average Expected over all inputs

Big O usually refers to the worst case.

Benchmark with realistic data sizes — microbenchmarks on tiny inputs often mislead.

14.2 Searching

Linear · binary · bisect · hash-based

Linear search — O(n)

def linear_search(lst, target):
    for i, item in enumerate(lst):
        if item == target:
            return i
    return -1

Works on any list. Scans every element in the worst case.

Binary search — O(log n)

Requires a sorted list.

def binary_search(lst, target):
    lo, hi = 0, len(lst) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        if lst[mid] == target:
            return mid
        elif lst[mid] < target:
            lo = mid + 1
        else:
            hi = mid - 1
    return -1

bisect Module & Hash-Based Lookup

bisect — maintain sorted order

import bisect

data = [1, 3, 5, 7, 9]

# Find insertion point
i = bisect.bisect_left(data, 6)   # 3

# Insert maintaining sort order
bisect.insort(data, 6)
print(data)   # [1, 3, 5, 6, 7, 9]

# Lookup: is x in the sorted list?
def sorted_contains(lst, x):
    i = bisect.bisect_left(lst, x)
    return i < len(lst) and lst[i] == x

Hash-based — O(1) average

# Preprocess once
lookup = set(data)     # or dict for key→value

# O(1) membership check
print(6 in lookup)     # True

# Summary
Method Time Requirement
Linear O(n) None
Binary O(log n) Sorted
bisect O(log n) Sorted
Hash O(1) avg Hashable elements

14.3 Sorting

Built-in · insertion · bubble · merge · quicksort

Built-in Sort & Simple Algorithms

Python's Timsort — O(n log n)

lst = [3, 1, 4, 1, 5, 9, 2, 6]
lst.sort()                        # in-place
s = sorted(lst)                   # new list
s = sorted(lst, reverse=True)     # descending
s = sorted(lst, key=abs)          # custom key

Insertion sort — O(n²) worst

def insertion_sort(lst):
    for i in range(1, len(lst)):
        key = lst[i]
        j = i - 1
        while j >= 0 and lst[j] > key:
            lst[j+1] = lst[j]
            j -= 1
        lst[j+1] = key

Merge sort — O(n log n)

def merge_sort(lst):
    if len(lst) <= 1:
        return lst
    mid = len(lst) // 2
    left  = merge_sort(lst[:mid])
    right = merge_sort(lst[mid:])
    return merge(left, right)

def merge(left, right):
    result = []
    i = j = 0
    while i < len(left) and j < len(right):
        if left[i] <= right[j]:
            result.append(left[i]); i += 1
        else:
            result.append(right[j]); j += 1
    return result + left[i:] + right[j:]

Sorting Algorithm Comparison

Algorithm Best Average Worst Space Stable
Timsort (Python) O(n) O(n log n) O(n log n) O(n) ✓
Merge sort O(n log n) O(n log n) O(n log n) O(n) ✓
Quick sort O(n log n) O(n log n) O(n²) O(log n) ✗
Insertion sort O(n) O(n²) O(n²) O(1) ✓
Bubble sort O(n) O(n²) O(n²) O(1) ✓

Use Python's built-in sorted() / .sort() in practice — Timsort is exceptionally well-optimized.

Chapter 14 — Quick Reference

Concept Key syntax / notes
Big O O(1) < O(log n) < O(n) < O(n log n) < O(n²)
Benchmark time.perf_counter(), timeit.timeit()
Linear search O(n) — works on any list
Binary search O(log n) — requires sorted list
bisect bisect_left, bisect_right, insort — O(log n)
Hash lookup x in set_or_dict — O(1) average
Built-in sort lst.sort() / sorted(lst) — Timsort O(n log n)
Stable sort Equal elements keep original order

End of Chapter 14

Congratulations — you've reached the end of ThinkPy!

algorithms · data structures · Python programming