Datové struktury a algoritmy Část 7 Vyhledávání a vyhledávací stromy Searching and Search Trees Petr Felkel

Searching (Vyhledávání) Typical operations Typical operations Implementation in an array Sequential search Binary search Binary search tree – BST (BVS) Node representation DSA

Searching (Vyhledávání) Input: a set of n keys, a query key q Problem description: Where is q? H D N B F J S A C E G I M P V G? DSA

Searching (Vyhledávání) Input: a set of n keys, a query key q Problem description: Where is q? H D N B F J S A C E G I M P V G DSA

Searching (Vyhledávání) Input: a set of n keys, a query key q Problem description: Where is q? H D N B F J S A C E G I M P V L? DSA

Searching (Vyhledávání) Input: a set of n keys, a query key q Problem description: Where is q? H D N B F J S A C E G I M P V L not found DSA

Search space min,max empty Elem search Search space getKey Key pred, insert,delete,replace pred, succ Search space getKey Key DSA

Searching (Vyhledávání) Search space (lexicon) Static - fixed -> simpler implementation -> change means new release Dynamic - changes in time -> more complex implementation -> change by insert,delete,replace DSA

Searching (Vyhledávání) Operations: k…key, e…element with key k x…data set {subtree root node, array, table,…} Informal list: search(x,k) min(x), max(x) pred(x,e), succ(x,e) insert(x,e), delete(x,e), replace(x,e) DSA

Searching (Vyhledávání) Typical operations Implementation In array Sequential search Binary search Binary search tree – BST (BVS) Node representation DSA

Implementation Address search (přímý přístup) Associative search Compute position from key pos = f(k) Array, table,… Associative search Element located in relation to others DSA

Implementation quality P(n) = memory complexity Q(n) = time complexity of search, query I(n) time complexity of insert D(n) time complexity of delete DSA

Searching (Vyhledávání) Typical operations Implementation In an array Sequential search Binary search Binary search tree – BST (BVS) Node representation DSA

Searching in unsorted array Unsorted array P(n) = O(n) Sequential search Q(n) = O(n) insert I(n) = O(1) delete D(n) = O(n) min, max in O(n) 1 2 3 4 5 B D A C size nodeT seqSearch( key k, nodeT a[] ) { int i; while( (i < a.size) && (a[i].key != k) ) i++; if( i < a.size ) return a[i]; else return NODE_NOT_FOUND; } Java-like pseudo code DSA

Searching in unsorted array Unsorted array with sentinel Sequential search still Q(n) = O(n) 1 2 3 4 5 B D E A C size search(“E“, a) nodeT seqSearchWithSentinel( key k, nodeT a[] ) { int i; a[a.size] = createArrayElement( k ); // place a sentinel while( a[i].key != k ) // and save one test i++; // per step if( i < a.size ) return a[i]; else return NODE_NOT_FOUND; } Java-like pseudo code DSA

Searching in sorted array Binary search Q(n) = O(log(n)) insert I (n) = O(n) delete D(n) = O(n) min, max in O(1) 1 2 3 4 5 A B C E size nodeT binarySearch( key k, nodeT sortedArray[] ) { pos = bs( k, sortedArray, 0, sortedArray.size - 1 ); if( pos >= 0 ) return sortedArray[pos]; else return NODE_NOT_FOUND; // -(pos+1) contains the position // to insert the node with key k } Java-like pseudo code DSA

Binary search //Recursive version int bs( key k, nodeT a[], int first, int last ) { if( first > last ) return –(first + 1); // not found int mid = ( first + last ) / 2; if( k < a[mid].key ) return bs( k, a, first, mid – 1); if( k > a[mid].key ) return bs( k, a, mid + 1, last ); return mid; // found! } // Iterative version int bs(key k, nodeT a[], int first, int last ) { while (first <= last) { int mid = (first + last) / 2; // compute the mid point if (key > sortedArray[mid]) first = mid + 1; else if (key < sortedArray[mid]) last = mid - 1; else return mid; // found it. } return -(first + 1); // failed to find key Java-like pseudo code Java-like pseudo code DSA

Searching (Vyhledávání) Typical operations Implementation In an array Sequential search Binary search Binary search tree – BST (BVS) Node representation DSA

Binární vyhledávací strom (BVS) Kořenový binární strom uzel má 0, (1), 2 následníky Binární vyhledávací strom (BVS) uspořádaný kořenový binární strom pro všechny uzly uL z levého podstromu platí: klíč(uL)  klíč(u) pro všechny uzly uR z pravého podstromu platí: klíč(uR)  klíč(u) u je kořen DSA

Binary Search Tree (BST) Rooted binary tree node has 0, (1), 2 successors Binary Search Tree (BST) Ordered rooted binary tree for each node uL in the left sub-tree: key(uL)  key(u) for each node uR in the right sub-tree: key(u)  key(uR) u is the root DSA

Binární vyhledávací strom Binary Search Tree = 7 2 9 1 5 8 10 3 6 < > DSA

Tree node representation key left right search min, max key left right key left right DSA

Tree node representation parent key search min, max predecessor, successor left right parent parent key key left right left right DSA

Tree node representation See in Lesson6, page 17-18 public class Node { public Node parent; public Node left; public Node right; public int key; public Node(int k) { key = k; parent = null; left = null; right = null; dat = …; } public class Node { public Node left; public Node right; public int key; public Node(int k) { key = k; left = null; right = null; dat = …; } public class Tree { public Node root; public Tree() { root = null; }} DSA

Searching BST G < H G > D G > F J S G > F A C E G I M P V G = G => stop, element found DSA

Searching BST - recursively Node TreeSearch( Node x, key k ) { if(( x == null ) or ( k == x.key )) return x; if( k < x.key ) return TreeSearch( x.left, k ); else return TreeSearch( x.right, k ); } Java-like pseudo code DSA

Searching BST - iteratively Node TreeSearch( Node x, key k ) { while(( x != null ) and (k != x.key )) if( k < x.key ) x = x.left; else x = x.right; } return x; Java-like pseudo code DSA

Minimum in BST H D N B F J S A C E G I M P V DSA

Minimum in BST - iteratively Node TreeMinimum( Node x ) { if( x == null ) return null; while( x.left != null ) x = x.left; } return x; Java-like pseudo code DSA

Maximum in BST - iteratively Node TreeMaximum( Node x ) { if( x == null ) return null; while( x.right != null ) x = x.right; } return x; Java-like pseudo code DSA

Maximum in BST H D N B F J S A C E G I M P DSA

Successor in BST 1/4 in the sorted order (in-order tree walk) Two cases: Right son exists Right son is null DSA

Successor in BST 1/4 in the sorted order (in-order tree walk) 1. Right son exists H D N B F J S A C E G I M P succ(B) -> C succ(H) -> I How? DSA

Successor in BST 2/4 in the sorted order (in-order tree walk) 1. Right son exists H D N B F J S A C E G I M P succ(B) -> C succ(H) -> I Find the minimum in the right tree = min( x.right ) DSA

Successor in BST 3/4 in the sorted order (in-order tree walk) 2. Right son is null H D N B F J S A C E G I M P succ(C) -> D succ(G) -> H How? DSA

Successor in BST 4/4 in the sorted order (in-order tree walk) 2. Right son is null x = node on path y = its parent H D N B F J S A C E G I M P y x succ(C) -> D succ(G) -> H Find the minimal parent to the right (the minimal parent the node is left from) DSA

Successor in BST - recursively in the sorted order (in-order tree walk) Node TreeSuccessor( Node x ) { if( x == null ) return null; if( x.right != null ) return TreeMinimum( x.right ); y = x.parent; while( (y != null) and (x == y.right)) { x = y; y = x.parent; } return y; } Java-like pseudo code DSA

Predecessor in BST - recursively in the sorted order (in-order tree walk) Node TreePredecessor( Node x ) { if( x == null ) return null; if( x.left != null ) return TreeMaximum( x.right ); y = x.parent; while( (y != null) and (x == y.left)) { x = y; y = x.parent; } return y; } Java-like pseudo code DSA

Operational Complexity The following dynamic-set operations: Search, Maximum, Minimum, Successor, Predecessor can run in O(h) time on a binary tree of height h. …. what h? DSA

Operational Complexity H D N B F J S A C E G I M P V H D B F J A C E G I M h = log2(n) h = n!!! O(log(n)) O(n)!!! => balance the tree!!! DSA

Operational Complexity The following dynamic-set operations: Search, Maximum, Minimum, Successor, Predecessor can run in (nlog(n)) and O(n) time on a not-balanced binary tree with n nodes. DSA

Insert (vložení prvku) x = node on path y = its parent H L > H D N L < N B F J y S x A C E G I M P V insert L 1. find the parent leaf 2. connect new element as a new leaf DSA

Insert (vložení prvku) void treeInsert( Tree t, Elem e ) { x = t.root; y = null; // t is tree root if( x == null ) t.root = e; else { while(x != null) { // find the parent leaf y = x; if( e.key < x.key ) x = x.left; else x = x.right; } // add new to parent if( e.key < y.key ) y.left = e; else y.right = e; } Java-like pseudo code DSA

Delete (odstranění prvku) a) delete a leaf (smaž list) H D N B F J S A C E G I M P V H D N B F J S A C E I M P V delete(G) e=y leaf has no children -> it is simply removed DSA

Delete (odstranění prvku) b) with one child (vnitřní s potomkem) delete(F) D N e=y B F J S B E J S A C E x I M P V A C I M P V node has one child -> splice the node out (přemosti) DSA

Delete (odstranění prvku) c) with two children (se 2 potomky) B F J S A C E I M P V y e x F delete(H) D N B E J S A C I M P V node has two children -> replace it with successor with only one child DSA

Delete (odstranění prvku) Tree treeDelete( Tree t, Node e ) { Node y; // e..node to delete // find node y to splice out if(e.left == null OR e.right == null) y = e; // case a,b) 0 to 1 child else // case c) 2 children y = TreePredecessor(e); H D B F A C E y e x H D B F A C E G y = e H D B F A C E y = e a) b) C) DSA

Delete (odstranění prvku) ... // find y’s only child x if( y.left != null ) x = y.left; // non-null child of y or null else x = y.right; ... //(y will be physically removed) e H D B F A C E G y = e H D B F A C E y = e H D y B F x x A C E a) b) C) DSA

Delete (odstranění prvku) ... // link x with parent of y – link up if( x != null ) x.parent = y.parent; e H D B F A C E G y = e H D B F A C E y = e H D y B F x x A C E a) b) C) DSA

Delete (odstranění prvku) ... // link x with parent of y - link down if( y.parent == null ) t.root = x // it was root else if( y == (y.parent).left ) (y.parent).left = x; else (y.parent).right = x; ... F E y x e H H H D D D Link to null y = e y B F B F B F y = e x x A C E G A C E A C E a) b) C) DSA

Delete (odstranění prvku) ... if( y != e ) // replace e with in-order successor { e.key = y.key; // copy the key e.dat = y.dat; // copy other fields too } return y; // To be disposed // or reused // outside e e H F D D y y B F B E x A C E A C DSA

And the operational complexity?

Operational Complexity H D N B F J S A C E G I M P V H D B F J A C E G I M h = log2(n) h = n!!! O(log(n)) O(n)!!! => balance the tree!!! DSA

Tree balancing Balancing criteria Balancing criteria Rotations AVL – tree Weighted tree Balancing criteria DSA

Tree balancing Why? To get the O(log n) complexity of search,... How? By local modifications reach the global goal DSA

výška + počet potomků - 1-2 strom, ... Vyvažování stromu Silná podmínka (Ideální případ) Pro všechny uzly platí: počet uzlů vlevo = počet uzlů vpravo Slabší podmínka výška podstromů - AVL strom výška + počet potomků - 1-2 strom, ... váha podstromů (počty uzlů) - váhově vyvážený strom DSA

Tree balancing Strong criterion (Ideal case) Weaker criterion For all nodes: No of nodes left = No of nodes right Weaker criterion subtree heights - AVL tree height + number of children - 1-2 tree, ... subtree weights (No of nodes) - weighted tree DSA

Tree balancing Balancing criteria Rotations AVL – tree Weighted tree DSA

Levá rotace (Left rotation) root A B II B p1 A x I z y z h-1 x y h h+1 Node leftRotation( Node root ) { // subtree root node !!! if( root == null ) return root; Node p1 = root.right; (init) if (p1 == null) return root; root.right = p1.left; (I) p1.left = root; (II) return p1; } Java-like pseudo code DSA

Pravá rotace (right rotation) root B A II p1 A B z x I x y h-1 y z h h+1 Node rightRotation( Node root ) { // subtree root node !!! if( root == null ) return root; Node p1 = root.left; (init) if (p1 == null) return root; root.left = p1.right; (I) p1.right = root; (II) return p1; } Java-like pseudo code DSA

LR rotace (left-right rotation) root C IV B II p1 A A C 2 1 z III I B p2 x y h-1 z w w x y h h+1 Node leftRightRotation( Node root ) { if(root==null)....; Node p1 = root.left; Node p2 = p1.right; (init) root.left = p2.right; (I) p2.right = root; (II) p1.right = p2.left; (III) p2.left = p1; (IV) return p2; } Java-like pseudo code DSA

RL rotace (right- left rotation) root A II B IV C p1 A C w 2 1 I III p2 B x y h-1 w z z x y h h+1 Node rightLeftRotation( Node root ) { if(root==null)....; Node p1 = root.right; Node p2 = p1.left; (init) root.right = p2.left; (I) p2.left = root; (II) p1.left = p2.right; (III) p2.right = p1; (IV) return p2; } Java-like pseudo code DSA

Tree balancing Balancing criteria Rotations AVL tree Weighted tree DSA

Kdy použijeme kterou rotaci? AVL strom [Richta90] Výškově vyvážený strom Georgij Maximovič Adelson-Velskij a Evgenij Michajlovič Landis Výška: Prázdný strom: výška = -1 neprázdný: výška = výška delšího potomka Vyvážený strom: rozdíl výšek potomků bal = {-1, 0, 1} DSA

AVL strom (AVL tree) int height( Node t ) { if( t == null ) return -1; else return 1 + max( height( t.left ), height( t.right ) ); } int bal( Node t ) return height( t.left ) - height( t.right ); Java-like pseudo code DSA

AVL strom - výšky a rozvážení AVL tree - heights and balance -1 2 3 1 1 2 2 1 -1 -1 -1 1 1 rozvážení/ balance -1 -1 -1 -1 -1 -1 -1 -1 výška / height -1 -1 -1 -1 -1 -1 -1 -1 DSA

AVL strom před vložením uzlu AVL tree before node insertion -1 2 3 1 1 2 2 1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 DSA

AVL strom - nejmenší podstrom AVL tree - the smallest subtree Nejmenší podstrom, který se přidáním uzlu rozváží The smallest sub-tree looses its balance by insertion 1 1 -1 -1 -1 -1 -1 -1 DSA

AVL strom - vložení uzlu doleva AVL tree - node insertion left a) Podstrom se přidáním uzlu doleva rozváží The sub-tree loses its balance by node insertion - left 1 2 1 2 1 -1 -1 1 -1 -1 insert left 1 -1 -1 -1 -1 -1 -1 -1 DSA

AVL strom - pravá rotace AVL tree - right rotation a) Vložen doleva => korekce pravou rotací Node inserted to the left => balance by right rotation B A 2 2 1 1 B A 1 1 1 -1 -1 -1 R rot 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 DSA

AVL strom - vložení uzlu doprava AVL tree after insertion-right b) Podstrom se přidáním uzlu doprava rozváží The sub-tree loses its balance by node insertion - right 1 2 1 2 -1 -1 -1 1 -1 -1 insert right -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 DSA

AVL strom - pravá rotace AVL tree - right rotation b) Vložen doprava => korekce LR rotací Node inserted right => balance by the LR rotation C B 2 2 1 1 A A C 2 -1 1 -1 1 -1 -1 1 LR rot B -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 DSA

Node insertion - for general AVL sub-trees AVL tree Node insertion - for general AVL sub-trees DSA

AVL strom - nejmenší podstrom AVL tree - the smallest subtree Nejmenší podstrom, který se přidáním uzlu rozváží The smallest sub-tree looses its balance by insertion 1 Node with balance 0 n+1 n n n Sub-tree of height n n n n n Sub-tree below of height n n DSA

AVL strom - vložení uzlu doleva AVL tree - node insertion left a) Podstrom se přidáním uzlu doleva rozváží The sub-tree loses its balance by node insertion - left B B 1 2 n+1 n n+2 n A A 1 n n n n n+1 n insert left n n n n DSA

AVL strom - pravá rotace AVL tree - right rotation a) Vložen doleva => korekce pravou rotací Node inserted to the left => balance by right rotation B A 2 n+2 n n+1 n+1 B A 1 n n n n n+1 n n n n n R rot DSA

AVL strom - vložení uzlu doprava AVL tree after insertion-right b1) Podstrom se přidáním uzlu doprava rozváží The sub-tree loses its balance by node insertion - right C C 1 2 n+1 n n+2 n A A -1 n n n n n n+1 B B 1 n-1 n-1 n n-1 insert right n n-1 n-1 n n-1 n-1 DSA

AVL strom - pravá rotace AVL tree - right rotation b1) Vložen doprava => korekce LR rotací Node inserted right => balance by the LR rotation C B 2 n+2 n n+1 n+1 A A C -1 2 -1 n n n+1 n n n-1 n B n-1 1 1 n-1 n n-1 n LR rot n n-1 n-1 n DSA

AVL strom - vložení uzlu doprava AVL tree after insertion-right b2) Podstrom se přidáním uzlu doprava rozváží The sub-tree loses its balance by node insertion - right C C 1 2 n+1 n n+2 n A A -1 n n n n n n+1 B B 1 n-1 n-1 n n-1 insert right n n-1 n-1 n n-1 n-1 DSA

AVL strom - pravá rotace AVL tree - right rotation b2) Vložen doprava => korekce LR rotací Node inserted right => balance by the LR rotation C B 2 n+2 n n+1 n+1 A A C -1 2 1 n n n+1 n n-1 n n B n-1 1 1 n-1 n n-1 n LR rot n n-1 n-1 n DSA

BST Insert without balancing avlTreeInsert( Tree t, Elem e ) { x = t.root; y = null; if( x == null ) t.root = e; else { while(x != null) { // find the parent leaf y = x; if( e.key < x.key ) x = x.left else x = x.right } // add new to parent if( e.key < y.key ) y.left = e else y.right = e } Java-like pseudo code DSA

AVL Insert (with balancing) avlTreeInsert( tree t, elem e ) { // init // 1. find place for insert // 2. if( already present ) // replace the node // else // insert new node // 3.balance tree, if necessary } Java-like pseudo code DSA

AVL Insert - variables & init avlTreeInsert( Tree t, Elem e ) { Node cur, fcur; // current sub-tree and its father Node a, b; // smallest unbalanced tree and its son Bool found; // node with the same key as e found Node help; // init found = false; cur = t.root; fcur = null; a = cur, b = null; // 1. find place for insert ... Java-like pseudo code DSA

AVL Insert - find place for insert ... // 1. find a place for insert while(( cur != null ) and !found ) { if( e.key == cur.key ) found = true; else { if( e.key < cur.key ) help = cur.left; else help = cur.right; if(( help != null) and ( bal(help) != 0 )){ //remember possible place for unbalance a = help; } fcur = cur; cur = cur.help; } ... DSA

AVL Insert - replace or insert new ... // 2. if( already present ) replace the node value if( found ) setinfo( cur, e ); // replace the node value else { // insert new node to fcur help = leaf( e ); // cons ( e, null, null ); if( fcur == null ) t.root = help; // new root if( e.key < fcur.key ) fcur.left = help; else fcur.right = help; } DSA

AVL Insert - balance the subtree ... // !found continues // 3.balance tree, if necessary if( bal(a) == 2 ) { // inserted left from 1 b = a.left; if( b.key < e.key ) //and right from its son a.left = leftRotation( b ); a = rightRotation( a ); } else if( bal(a) == -2){ //inserted right from -1 b = a.right; if( e.key < b.key ) // and left from its son a.right = rightRotation( b ); a = leftRotatation( a ); } // else tree remained balanced } // !found DSA

AVL - výška stromu Pro strom S s n uzly platí Výška h(S) je max o 45% větší než u ideálně vyváženého stromu log2(n+1)  h(S)  1.4404 log2(n+2)-0.328 [Hudec96], [Honzík85] DSA

Tree balancing Balancing criteria Rotations AVL tree Weighted tree DSA

Váhově vyvážené stromy Weight balanced trees (stromy s ohraničeným vyvážením) Váha uzlu u ve stromě S: v (u) = 1/2, když je u listem v (u) = ( |UL| + 1) / ( |U| + 1 ), když u je kořen podstromu SU S UL = množina uzlů levého podstromu SU U = množina uzlů podstromu SU DSA

Váhově vyvážené stromy Weight balanced trees Stromy s ohraničeným vyvážením : Strom S má ohraničené vyvážení , 0    0,5, jestliže pro všechny uzly S platí   v(u)  1-  Výška h(S) stromu S s ohraničeným vyvážením  h(S)  (1 + log2(n+1) - 1) / log2 (1 / (1- )) [Hudec96], [Mehlhorn84] DSA

Weight balanced tree example (5+1) / (12+1) = 6/13 (3+1) / (5+1) = 2/3 H (3+1) / (6+1) = 4/7 (1+1) / (3+1) = 1/2 D N (1+1) / (2+1) = 2/3 1/2 B F J S 1/2 A C I M P 1/2 1/2 1/2 1/2 (0+1) / (1+1) = 1/2 DSA

Levá rotace (Left rotation) [Hudec96] VB ’ VA A B B VA’ VB A x z y z x y VA’ = VA / ( VA + (1- VA) . VB) VB’ = VA + (1- VA) . VB DSA

RL rotace (right- left rotation) VA VB ’ A B VA’ C VC ’ VC A C w 2 1 VB B x y w z z x y VA’ = VA / ( VA + (1- VA) VB VC) VB’ = VB (1- VC) / (1- VB VC) VC’ = VA + (1- VA) . VA VB [Hudec96] DSA