已發布·持續改進
一致性雜湊 指南 · 5/6
本章目前僅提供英文版。
Every problem below reuses hash64 and HashRing from the implementation chapter (sorted points plus the owner map from position to node). Try each one yourself before reading the solution.
Given a ring, compute what fraction of the full 2^64 space each node owns. Vary the vnode count over 1, 10, 100 and 1000 and see how far the largest and smallest shares drift from the average.
Approach: each point owns everything after the previous point up to itself. Walk the sorted points and add (p - previous) mod 2^64 to the owner's total. The predecessor of index 0 is the last point, and Python's pts[-1] handles that wrap for free. A ring with a single point gives a difference of 0, so treat it separately.
RING_SIZE = 1 << 64
def ownership(ring: HashRing) -> dict[str, float]:
pts = ring.points
if len(pts) == 1:
return {ring.owner[pts[0]]: 1.0}
share: dict[str, int] = {}
for i, p in enumerate(pts):
arc = (p - pts[i - 1]) % RING_SIZE # i == 0 wraps to the last point
node = ring.owner[p]
share[node] = share.get(node, 0) + arc
return {node: s / RING_SIZE for node, s in share.items()}
for v in (1, 10, 100, 1000):
r = HashRing(vnodes=v)
for n in range(10):
r.add(f"node-{n}")
s = ownership(r)
print(v, round(max(s.values()) * 10, 2), round(min(s.values()) * 10, 2))Multiplying by the node count normalizes the average to 1. As vnodes increase, both extremes approach 1.
Each node belongs to an availability zone. Pick R replicas for a key that are distinct physical nodes in distinct zones. If not enough qualify, return as many as you found.
Approach: walk clockwise from the key one point at a time, skipping nodes already chosen and zones already used. Stop after one full lap.
import bisect
def zone_replicas(ring: HashRing, zone_of: dict[str, str], key: str, n: int) -> list[str]:
pts = ring.points
start = bisect.bisect_left(pts, hash64(key))
chosen: list[str] = []
zones: set[str] = set()
for step in range(len(pts)):
node = ring.owner[pts[(start + step) % len(pts)]]
if node in chosen or zone_of[node] in zones:
continue
chosen.append(node)
zones.add(zone_of[node])
if len(chosen) == n:
break
return chosen
zone_of = {"a1": "az-1", "a2": "az-1", "b1": "az-2", "b2": "az-2", "c1": "az-3"}
ring = HashRing(vnodes=50)
for node in zone_of:
ring.add(node)
print(zone_replicas(ring, zone_of, "order:1001", 3)) # one node from each zoneBefore adding a node, list which hash ranges it takes over and from which existing node. Return tuples where each range excludes start and includes end.
(start, end, previous_owner)Approach: in the ring after the addition, each point p of the new node takes over the range from its predecessor up to p. The previous owner is whoever owned p in the ring before the addition. The rule still holds when several new points sit next to each other.
def migration_plan(ring: HashRing, new_node: str) -> list[tuple[int, int, str]]:
old_points = list(ring.points)
old_owner = dict(ring.owner)
ring.add(new_node)
pts = ring.points
plan = []
for i, p in enumerate(pts):
if ring.owner[p] != new_node:
continue
j = bisect.bisect_left(old_points, p) % len(old_points)
plan.append((pts[i - 1], p, old_owner[old_points[j]]))
return plan
def in_range(h: int, start: int, end: int) -> bool:
if start < end:
return start < h <= end
return h > start or h <= end # the range wraps past 0To verify, look up a few thousand keys before and after the change and check that every key that changed owner falls inside a planned range with the matching previous owner, and that no other key moved.
Nodes have capacity weights. Build a rendezvous hash that assigns keys in proportion to weight and moves only the removed node's keys when one node leaves.
Approach: for each node, turn a hash of node and key into a uniform number u between 0 and 1, and pick the node with the largest score -w / ln(u). That score wins with probability proportional to w. Using the top 53 bits plus 0.5 keeps u strictly between 0 and 1.
import math
from collections import Counter
def hrw(key: str, weights: dict[str, float]) -> str:
best, best_score = "", -math.inf
for node, w in weights.items():
u = ((hash64(f"{node}:{key}") >> 11) + 0.5) / (1 << 53) # 0 < u < 1
score = -w / math.log(u)
if score > best_score:
best, best_score = node, score
return best
weights = {"small": 1.0, "medium": 2.0, "large": 4.0}
keys = [f"obj:{i}" for i in range(70_000)]
first = {k: hrw(k, weights) for k in keys}
print(Counter(first.values())) # roughly 10000 / 20000 / 40000
del weights["medium"]
moved = [k for k in keys if hrw(k, weights) != first[k]]
print(all(first[k] == "medium" for k in moved)) # True-w / ln(u) score gives weighted rendezvous hashing with minimal movement.
0 則留言
登入 · 登入後即可留言。
來留下第一則留言吧。