Lançado · em melhoria
Guia de Hashing consistente · 2/6
Por enquanto, este capítulo está disponível apenas em inglês.
This chapter walks through a small ring by hand: lookups, adding and removing nodes, virtual nodes and replica selection. To keep the numbers readable, assume a tiny hash space from 0 to 99. Real implementations use something like 0 to 2^64 - 1.
Hashing the names of nodes A, B and C gives these positions.
| Node | Position |
|---|---|
| A | 10 |
| B | 45 |
| C | 80 |
Once the positions are sorted, each node's range follows. The rule is "the first node at or after the key", so a node owns everything after its predecessor up to and including its own position.
| Node | Range | Size |
|---|---|---|
| A | 81-99 and 0-10 | 30 |
| B | 11-45 | 35 |
| C | 46-80 | 35 |
Binary-search the sorted array for the key's position. If you run off the end, wrap to index 0.
| Key | Hash | First node at or after | Owner |
|---|---|---|---|
| user:1 | 7 | A(10) | A |
| user:2 | 33 | B(45) | B |
| user:3 | 52 | C(80) | C |
| user:4 | 91 | none, wrap to start | A |
| user:5 | 45 | B(45) | B |
import bisect
points = [10, 45, 80]
owner = {10: "A", 45: "B", 80: "C"}
def lookup(h: int) -> str:
i = bisect.bisect_left(points, h) # first point >= h
if i == len(points):
i = 0 # wrap around
return owner[points[i]]
print([lookup(h) for h in (7, 33, 52, 91, 45)]) # ['A', 'B', 'C', 'A', 'B']A new node D lands at position 60. The only range that changes is between D's predecessor B(45) and D(60), that is 46-60. That range used to belong to C, so only some of C's keys move to D. Not a single key owned by A or B moves.
| Key | Hash | Before |
|---|
| After |
|---|
| user:1 | 7 | A | A |
| user:2 | 33 | B | B |
| user:3 | 52 | C | D |
| user:4 | 91 | A | A |
The giver is always the new node's clockwise successor and the receiver is always the new node. That is the "old nodes never trade keys" property in action.
Now suppose B fails. Its range 11-45 passes to its clockwise successor; everything else stays. The catch is that a single neighbour absorbs the whole range, so its load can nearly double overnight.
Several points per node soften that problem. Hash A as A#0, A#1 and A#2 to get three points, and do the same for B and C.
| Position | Point | Node |
|---|---|---|
| 5 | B#2 | B |
| 18 | A#0 | A |
| 27 | C#1 | C |
| 41 | B#0 | B |
| 55 | A#2 | A |
| 63 | C#0 | C |
| 77 | A#1 | A |
| 86 | B#1 | B |
| 94 | C#2 | C |
If B now leaves, its three ranges go to the owners of the next points: A, A and C. With more points, the orphaned share is split across more nodes. Running the code below for 10 servers, one point per node leaves the busiest node with more than 3 times the average and another almost idle; with 100 points the shares fall within roughly 0.86 to 1.17 times the average, and with 1,000 points within about 0.93 to 1.05 (one run; exact numbers depend on the hash function and node names).
import bisect, hashlib
def h(s: str) -> int:
return int.from_bytes(hashlib.md5(s.encode()).digest()[:8], "big")
RING = 1 << 64
for v in (1, 10, 100, 1000):
pts = sorted((h(f"node{n}#{i}"), n) for n in range(10) for i in range(v))
share = [0] * 10
for j, (p, n) in enumerate(pts):
share[n] += (p - pts[j - 1][0]) % RING # arc owned by this point
ratios = [s / RING * 10 for s in share]
print(v, round(max(ratios), 2), round(min(ratios), 2))Dynamo-style stores do not keep a key on one node; they replicate it to several nodes following the ring clockwise. With virtual nodes, consecutive points can belong to the same physical machine, so skip machines already chosen and collect R distinct nodes. In the table above, two replicas for a key at position 30 are B (41) then A (55).
def replicas(points, owner, h, r):
start = bisect.bisect_left(points, h)
chosen = []
for step in range(len(points)):
node = owner[points[(start + step) % len(points)]]
if node not in chosen:
chosen.append(node)
if len(chosen) == r:
break
return chosenRendezvous hashing gets the same guarantees without a ring. For each key, compute hash(node:key) for every node and pick the top score. When a node leaves, only keys where it ranked first move, each to its runner-up; when a node joins, only keys where it now ranks first move to it. For replicas, take the top R scores.
def rendezvous(key: str, nodes: list[str]) -> str:
return max(nodes, key=lambda node: h(f"{node}:{key}"))
0 comentários
Fazer login · Faça login para deixar um comentário.
Seja o primeiro a comentar.