SlideShare a Scribd company logo
1 of 91
Download to read offline
Models for Hierarchical Data
         with SQL and PHP
             Bill Karwin, Percona Inc.
Me
• Software developer
• C, Java, Perl, PHP, Ruby
• SQL maven
• MySQL Consultant at Percona
• Author of SQL Antipatterns:
   Avoiding the Pitfalls of
   Database Programming




                                www.percona.com
Problem
• Store & query hierarchical data
  - Categories/subcategories
  - Bill of materials
  - Threaded discussions




                                    www.percona.com
Example: Bug Report
       Comments
                               (1) Fran:
                               What’s the cause
                               of this bug?



             (2) Ollie:                      (4) Kukla:
             I think it’s a null             We need to check
             pointer.                        valid input.



(3) Fran:                                                 (6) Fran:
                                    (5) Ollie:
No, I checked for                                         Yes, please add a
                                    Yes, that’s a bug.
that.                                                     check.




                                                          (7) Kukla:
                                                          That fixed it.



                                                                    www.percona.com
Solutions

•Adjacency list
•Path enumeration
•Nested sets
•Closure table


                      www.percona.com
Adjacency List




                 www.percona.com
Adjacency List
• Naive solution nearly everyone uses
• Each entry knows its immediate parent

     comment_id parent_id author   comment
     1           NULL     Fran     What’s the cause of this bug?
     2           1        Ollie    I think it’s a null pointer.
     3           2        Fran     No, I checked for that.
     4           1        Kukla    We need to check valid input.
     5           4        Ollie    Yes, that’s a bug.
     6           4        Fran     Yes, please add a check
     7           6        Kukla    That fixed it.

                                                            www.percona.com
Insert a New Node
INSERT INTO Comments (parent_id, author, comment)
  VALUES (5, ‘Fran’, ‘I agree!’);
                                                 (1) Fran:
                                                 What’s the cause of
                                                 this bug?




                           (2) Ollie:                            (4) Kukla:
                           I think it’s a null                   We need to check
                           pointer.                              valid input.




            (3) Fran:                                                          (6) Fran:
                                                       (5) Ollie:
            No, I checked for                                                  Yes, please add a
                                                       Yes, that’s a bug.
            that.                                                              check.




                                                                               (7) Kukla:
                                                                               That fixed it.




                                                                                                   www.percona.com
Insert a New Node
INSERT INTO Comments (parent_id, author, comment)
  VALUES (5, ‘Fran’, ‘I agree!’);
                                                 (1) Fran:
                                                 What’s the cause of
                                                 this bug?




                           (2) Ollie:                              (4) Kukla:
                           I think it’s a null                     We need to check
                           pointer.                                valid input.




            (3) Fran:                                                            (6) Fran:
                                                       (5) Ollie:
            No, I checked for                                                    Yes, please add a
                                                       Yes, that’s a bug.
            that.                                                                check.




                                                       (8) Fran:                 (7) Kukla:
                                                       I agree!                  That fixed it.




                                                                                                     www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:                                                          (6) Fran:
                                                  (5) Ollie:
       No, I checked for                                                  Yes, please add a
                                                  Yes, that’s a bug.
       that.                                                              check.




                                                                          (7) Kukla:
                                                                          That fixed it.




                                                                                              www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:
                                                  (5) Ollie:
       No, I checked for
                                                  Yes, that’s a bug.
       that.




                                                                               www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:
                                                  (5) Ollie:
       No, I checked for
                                                  Yes, that’s a bug.
       that.




                                                                               www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                              (1) Fran:
                                              What’s the cause of
                                              this bug?




                        (2) Ollie:                            (4) Kukla:
                        I think it’s a null                   We need to check
                        pointer.                              valid input.




       (3) Fran:
                                                    (5) Ollie:
       No, I checked for
                                                    Yes, that’s a bug.
       that.




       (6) Fran:
       Yes, please add a
       check.




       (7) Kukla:
       That fixed it.                                                            www.percona.com
Query Immediate Child/Parent
• Query a node’s children:
       SELECT * FROM Comments c1
        LEFT JOIN Comments c2
         ON (c2.parent_id = c1.comment_id);


• Query a node’s parent:
      SELECT * FROM Comments c1
       JOIN Comments c2
        ON (c1.parent_id = c2.comment_id);


                                        www.percona.com
Can’t Handle Deep Trees
SELECT * FROM Comments c1
  LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
  LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
  LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
  LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
  LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
  LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
  LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
  LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
  LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
  ...




                                              www.percona.com
Can’t Handle Deep Trees
SELECT * FROM Comments c1
  LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
  LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
  LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
  LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
  LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
  LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
  LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
  LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
  LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
  ...
                it still doesn’t support
                unlimited depth!



                                              www.percona.com
SQL-99 recursive syntax
WITH [RECURSIVE] CommentTree
       (comment_id, bug_id, parent_id, author, comment, depth)
  AS (
       SELECT *, 0 AS depth FROM Comments
       WHERE parent_id IS NULL
    UNION ALL
       SELECT c.*, ct.depth+1 AS depth FROM CommentTree ct
       JOIN Comments c ON (ct.comment_id = c.parent_id)
  )
  SELECT * FROM CommentTree WHERE bug_id = 1234;




   ✓
       PostgreSQL, Oracle 11g,
       IBM DB2, Microsoft SQL
       Server, Apache Derby            ✗   MySQL, SQLite, Informix,
                                           Firebird,etc.



                                                        www.percona.com
Path Enumeration




                   www.percona.com
Path Enumeration
• Store chain of ancestors in each node


     comment_id path        author   comment
     1           1/         Fran     What’s the cause of this bug?
     2           1/2/       Ollie    I think it’s a null pointer.
     3           1/2/3/     Fran     No, I checked for that.
     4           1/4/       Kukla    We need to check valid input.
     5           1/4/5/     Ollie    Yes, that’s a bug.
     6           1/4/6/     Fran     Yes, please add a check
     7           1/4/6/7/   Kukla    That fixed it.

                                                              www.percona.com
Path Enumeration
• Store chain of ancestors in each node
                                              good for
                                              breadcrumbs
     comment_id path        author   comment
     1           1/         Fran     What’s the cause of this bug?
     2           1/2/       Ollie    I think it’s a null pointer.
     3           1/2/3/     Fran     No, I checked for that.
     4           1/4/       Kukla    We need to check valid input.
     5           1/4/5/     Ollie    Yes, that’s a bug.
     6           1/4/6/     Fran     Yes, please add a check
     7           1/4/6/7/   Kukla    That fixed it.

                                                              www.percona.com
Query Ancestors and Subtrees
• Query ancestors of comment #7:
      SELECT * FROM Comments
       WHERE ‘1/4/6/7/’ LIKE path || ‘%’;


• Query descendants of comment #4:
      SELECT * FROM Comments
       WHERE path LIKE ‘1/4/%’;




                                            www.percona.com
Add a New Child of #7
INSERT INTO Comments (author, comment)
  VALUES (‘Ollie’, ‘Good job!’);
SELECT path FROM Comments
  WHERE comment_id = 7;
UPDATE Comments
  SET path = $parent_path || LAST_INSERT_ID() || ‘/’
  WHERE comment_id = LAST_INSERT_ID();




                                         www.percona.com
Nested Sets




              www.percona.com
Nested Sets
• Each comment encodes its descendants
   using two numbers:
    - A comment’s left number is less than all numbers
        used by the comment’s descendants.
    - A comment’s right number is greater than all
        numbers used by the comment’s descendants.
    - A comment’s numbers are between all
        numbers used by the comment’s ancestors.




                                          www.percona.com
What Does This Look Like?

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
What Does This Look Like?

                                        (1) Fran:
                                        What’s the
                                        cause of this
                                        bug?
                                    1                      14
                                                         (4) Kukla:
                     (2) Ollie:
                                                         We need to
                     I think it’s a null
                     pointer.                            check valid
                                                         input.
                 2                         5         6                     13

       (3) Fran:                               (5) Ollie:              (6) Fran:
       No, I checked                           Yes, that’s a           Yes, please add
       for that.                               bug.                    a check.
   3                      4                 7                   8      9                 12

                                                                       (7) Kukla:
                                                                       That fixed it.

                                                                    10                   11


                                                                                          www.percona.com
What Does This Look Like?

comment_id nsleft   nsright   author   comment
1           1       14        Fran     What’s the cause of this bug?
2           2       5         Ollie    I think it’s a null pointer.
3           3       4         Fran     No, I checked for that.
4           6       13        Kukla    We need to check valid input.
5           7       8         Ollie    Yes, that’s a bug.
6           9       12        Fran     Yes, please add a check
7           10      11        Kukla    That fixed it.




                                                        www.percona.com
What Does This Look Like?

comment_id nsleft     nsright    author   comment
1           1         14         Fran     What’s the cause of this bug?
2           2         5          Ollie    I think it’s a null pointer.
3           3         4          Fran     No, I checked for that.
4           6         13         Kukla    We need to check valid input.
5           7         8          Ollie    Yes, that’s a bug.
6           9         12         Fran     Yes, please add a check
7           10        11         Kukla    That fixed it.

                 these are not
                 foreign keys

                                                           www.percona.com
Query Ancestors of #7

                                      (1) Fran:                                       ancestors
                                      What’s the
                                      cause of this
                                      bug?
                                  1                      14
                                                       (4) Kukla:
                   (2) Ollie:
                                                       We need to
                   I think it’s a null
                   pointer.                            check valid
                                                       input.
               2                         5         6                     13                   child
     (3) Fran:                               (5) Ollie:              (6) Fran:
     No, I checked                           Yes, that’s a           Yes, please add
     for that.                               bug.                    a check.
 3                      4                 7                   8      9                 12

                                                                     (7) Kukla:
                                                                     That fixed it.

                                                                  10                   11


                                                                                        www.percona.com
Query Ancestors of #7

SELECT * FROM Comments child
 JOIN Comments ancestor ON child.nsleft
   BETWEEN ancestor.nsleft AND ancestor.nsright
 WHERE child.comment_id = 7;




                                     www.percona.com
Query Subtree Under #4

                                      (1) Fran:
                                      What’s the
                                                                                      parent
                                      cause of this
                                      bug?
                                  1                      14

                   (2) Ollie:                          (4) Kukla:                           descendants
                                                       We need to
                   I think it’s a null
                   pointer.                            check valid
                                                       input.
               2                         5         6                     13

     (3) Fran:                               (5) Ollie:              (6) Fran:
     No, I checked                           Yes, that’s a           Yes, please add
     for that.                               bug.                    a check.
 3                      4                 7                   8      9                 12

                                                                     (7) Kukla:
                                                                     That fixed it.

                                                                  10                   11


                                                                                        www.percona.com
Query Subtree Under #4

SELECT * FROM Comments parent
 JOIN Comments descendant ON descendant.nsleft
   BETWEEN parent.nsleft AND parent.nsright
 WHERE parent.comment_id = 4;




                                    www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                     13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   8      9                 12

                                                                    (7) Kukla:
                                                                    That fixed it.

                                                                 10                   11


                                                                                       www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      16
                                                        14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                   15
                                                                      13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   10 11
                                                             8 9                      14
                                                                                      12

                                                                    (7) Kukla:
                                                                    That fixed it.

                                                                12
                                                                10                    13
                                                                                      11


                                                                                       www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      16
                                                        14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                   15
                                                                      13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   10 11
                                                             8 9                      14
                                                                                      12

                                            (8) Fran:               (7) Kukla:
                                            I agree!                That fixed it.

                                        8                    9 10
                                                               12                     13
                                                                                      11


                                                                                       www.percona.com
Insert New Child of #5
UPDATE Comments
  SET nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2
     ELSE nsleft END,
     nsright = nsright+2
  WHERE nsright >= 7;
INSERT INTO Comments (nsleft, nsright, author, comment)
   VALUES (8, 9, 'Fran', 'I agree!');
• Recalculate left values for all nodes to the right of
   the new child. Recalculate right values for all
   nodes above and to the right.



                                               www.percona.com
Query Immediate Parent of #6

                                         (1) Fran:
                                         What’s the
                                         cause of this
                                         bug?
                                     1                      14
                                                          (4) Kukla:
                      (2) Ollie:
                                                          We need to
                      I think it’s a null
                      pointer.                            check valid
                                                          input.
                  2                         5         6                     13

        (3) Fran:                               (5) Ollie:              (6) Fran:
        No, I checked                           Yes, that’s a           Yes, please add
        for that.                               bug.                    a check.
    3                      4                 7                   8      9                 12

                                                                        (7) Kukla:
                                                                        That fixed it.

                                                                     10                   11


                                                                                           www.percona.com
Query Immediate Parent of #6
• Parent of #6 is an ancestor who has no
   descendant who is also an ancestor of #6.
     SELECT parent.* FROM Comments AS c
       JOIN Comments AS parent
        ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
       LEFT OUTER JOIN Comments AS in_between
        ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
         AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
       WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;




                                                           www.percona.com
Query Immediate Parent of #6
• Parent of #6 is an ancestor who has no
   descendant who is also an ancestor of #6.
     SELECT parent.* FROM Comments AS c
       JOIN Comments AS parent
        ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
       LEFT OUTER JOIN Comments AS in_between
        ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
         AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
       WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;




                   querying immediate child
                     is a similar problem

                                                           www.percona.com
Closure Table




                www.percona.com
Closure Table
CREATE TABLE TreePaths (
    ancestor    INT NOT NULL,
    descendant INT NOT NULL,
    PRIMARY KEY (ancestor, descendant),
    FOREIGN KEY(ancestor)
      REFERENCES Comments(comment_id),
    FOREIGN KEY(descendant)
      REFERENCES Comments(comment_id)
 );




                              www.percona.com
Closure Table
• Many-to-many table
• Stores every path from each node
    to each of its descendants
• A node even connects to itself




                                     www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
What Does This Look Like?
                                                   ancestor    descendant
comment_id author   comment                           1             1
                                                      1             2
    1      Fran     What’s the cause of this
                    bug?                              1             3
                                                      1             4
    2      Ollie    I think it’s a null pointer.      1             5
    3      Fran     No, I checked for that.           1             6
                                                      1             7
    4      Kukla    We need to check valid
                                                      2             2
                    input.
                                                      2             3
    5      Ollie    Yes, that’s a bug.                3             3
                                                      4             4
    6      Fran     Yes, please add a check
                                                      4             5
    7      Kukla    That fixed it.                    4             6
                                                      4             7
                                                      5             5

requires O(n²) rows                                   6
                                                      6
                                                                    6
                                                                    7
                                                      7             7




                                                              www.percona.com
What Does This Look Like?
                                                   ancestor    descendant
comment_id author   comment                           1             1
                                                      1             2
    1      Fran     What’s the cause of this
                    bug?                              1             3
                                                      1             4
    2      Ollie    I think it’s a null pointer.      1             5
    3      Fran     No, I checked for that.           1             6
                                                      1             7
    4      Kukla    We need to check valid
                                                      2             2
                    input.
                                                      2             3
    5      Ollie    Yes, that’s a bug.                3             3
                                                      4             4
    6      Fran     Yes, please add a check
                                                      4             5
    7      Kukla    That fixed it.                    4             6
                                                      4             7
                                                      5             5

requires O(n²) rows                                   6
                                                      6
                                                                    6
                                                                    7
                                                      7             7
(but far fewer in practice)
                                                              www.percona.com
Query Descendants of #4

SELECT c.* FROM Comments c
 JOIN TreePaths t
  ON (c.comment_id = t.descendant)
 WHERE t.ancestor = 4;




                                     www.percona.com
Paths Starting from #4

                              (1) Fran:
                              What’s the
                              cause of this
                              bug?


                                              (4) Kukla:
             (2) Ollie:
                                              We need to
             I think it’s a null
             pointer.                         check valid
                                              input.



 (3) Fran:                         (5) Ollie:               (6) Fran:
 No, I checked                     Yes, that’s a            Yes, please add
 for that.                         bug.                     a check.




                                                            (7) Kukla:
                                                            That fixed it.




                                                                              www.percona.com
Query Ancestors of #6

SELECT c.* FROM Comments c
 JOIN TreePaths t
  ON (c.comment_id = t.ancestor)
 WHERE t.descendant = 6;




                                   www.percona.com
Paths Terminating at #6

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                               (4) Kukla:
              (2) Ollie:
                                               We need to
              I think it’s a null
              pointer.                         check valid
                                               input.



  (3) Fran:                         (5) Ollie:               (6) Fran:
  No, I checked                     Yes, that’s a            Yes, please add
  for that.                         bug.                     a check.




                                                             (7) Kukla:
                                                             That fixed it.




                                                                               www.percona.com
Insert New Child of #5

INSERT INTO Comments
  VALUES (8, ‘Fran’, ‘I agree!’);


INSERT INTO TreePaths (ancestor, descendant)
  SELECT ancestor, 8 FROM TreePaths
  WHERE descendant = 5
  UNION ALL SELECT 8, 8;



                                    www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Delete Child #7

DELETE FROM TreePaths
  WHERE descendant = 7;




                            www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Subtree Under #4

DELETE FROM TreePaths
  WHERE descendant IN
    (SELECT descendant FROM TreePaths
     WHERE ancestor = 4);




                               www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?



               (2) Ollie:
               I think it’s a null
               pointer.



   (3) Fran:
   No, I checked
   for that.




                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:                       We need to
               I think it’s a null              check valid
               pointer.                         input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Path Length
• Add a length column                      ancestor   descendant   length

• MAX(length) is depth of tree
                                              1           1          0
                                              1           2          1
                                              1           3          2

• Makes it easier to query                    1
                                              1
                                                          4
                                                          5
                                                                     1
                                                                     2
   immediate parent or child:                 1           6          2
                                              1           7          3
     SELECT c.*                               2           2          0

       FROM Comments c                        2           3          1

       JOIN TreePaths t                       3           3          0
                                              4           4          0
        ON (c.comment_id = t.descendant)      4           5          1
       WHERE t.ancestor = 4                   4           6          1
        AND t.length = 1;                     4           7          2
                                              5           5          0
                                              6           6          0
                                              6           7          1
                                              7           7          0




                                                  www.percona.com
Path Length
• Add a length column                      ancestor   descendant   length

• MAX(length) is depth of tree
                                              1           1          0
                                              1           2          1
                                              1           3          2

• Makes it easier to query                    1
                                              1
                                                          4
                                                          5
                                                                     1
                                                                     2
   immediate parent or child:                 1           6          2
                                              1           7          3
     SELECT c.*                               2           2          0

       FROM Comments c                        2           3          1

       JOIN TreePaths t                       3           3          0
                                              4           4          0
        ON (c.comment_id = t.descendant)      4           5          1
       WHERE t.ancestor = 4                   4           6          1
        AND t.length = 1;                     4           7          2
                                              5           5          0
                                              6           6          0
                                              6           7          1
                                              7           7          0




                                                  www.percona.com
Choosing the Right Design

Design        Tables Query Query     Delete   Insert    Move     Referential
                     Child Subtree   Node     Node     Subtree    Integrity
Adjacency       1    Easy   Hard      Easy     Easy     Easy         Yes
List
Path            1    Hard   Easy     Easy     Easy      Easy         No
Enumeration
Nested Sets     1    Hard   Easy     Hard     Hard      Hard         No

Closure         2    Easy   Easy     Easy     Easy      Easy        Yes
Table




                                                           www.percona.com
PHP Demo
of Closure Table



                   www.percona.com
Hierarchical Test Data
• Integrated Taxonomic Information System
  - http://itis.gov/
  - Free authoritative taxonomic information on plants,
     animals, fungi, microbes
  - 518,756 scientific names (as of Feb 2011)




                                             www.percona.com
California Poppy
Kingdom:    Plantae
Division:   Tracheobionta
Class:      Magnoliophyta
Order:      Magnoliopsida
unranked:   Magnoliidae
unranked:   Papaverales
Family:     Papaveraceae
Genus:      Eschscholzia
Species:    Eschscholzia californica




                                       www.percona.com
California Poppy
Kingdom:    Plantae
Division:   Tracheobionta
Class:      Magnoliophyta
Order:      Magnoliopsida
unranked:   Magnoliidae
unranked:   Papaverales
Family:     Papaveraceae
Genus:      Eschscholzia
Species:    Eschscholzia californica


    id=18956

                                       www.percona.com
California Poppy: ITIS Entry

           SELECT * FROM Hierarchy
        WHERE hierarchy_string LIKE ‘%-18956’;

hierarchy_string
202422-564824-18061-18063-18064-18879-18880-18954-18956




                                                  www.percona.com
California Poppy: ITIS Entry

           SELECT * FROM Hierarchy
        WHERE hierarchy_string LIKE ‘%-18956’;

hierarchy_string
202422-564824-18061-18063-18064-18879-18880-18954-18956



    ITIS data uses                  ...but I converted
    path enumeration                it to closure table

                                                  www.percona.com
Hierarchical Data Classes
abstract class ZendX_Db_Table_TreeTable
      extends Zend_Db_Table_Abstract
  {
      public function fetchTreeByRoot($rootId, $expand)
      public function fetchBreadcrumbs($leafId)
  }




                                                 www.percona.com
Hierarchical Data Classes
class ZendX_Db_Table_Row_TreeRow
       extends Zend_Db_Table_Row_Abstract
   {
       public function addChildRow($childRow)
       public function getChildren()
   }
class ZendX_Db_Table_Rowset_TreeRowset
       extends Zend_Db_Table_Rowset_Abstract
   {
       public function append($row)
   }




                                                www.percona.com
Using TreeTable
class ItisTable extends ZendX_Db_Table_TreeTable
   {
       protected $_name = “longnames”;
       protected $_closureName = “treepaths”;
   }
$itis = new ItisTable();




                                               www.percona.com
Breadcrumbs
$breadcrumbs = $itis->fetchBreadcrumbs(18956);
foreach ($breadcrumbs as $crumb) {
      print $crumb->completename . “ > ”;
   }


Plantae > Tracheobionta > Magnoliophyta > Magnoliopsida >
  Magnoliidae > Papaverales > Papaveraceae > Eschscholzia >
  Eschscholzia californica >




                                                 www.percona.com
Breadcrumbs SQL

SELECT a.* FROM longnames AS a
 INNER JOIN treepaths AS c ON a.tsn = c.a
 WHERE (c.d = 18956)
 ORDER BY c.l DESC




                                   www.percona.com
How Does it Perform?
• Query profile = 0.0006 sec
• MySQL EXPLAIN:
table type     key       ref     rows   extra

c     ref      tree_dl   const   9      Using where; Using index

a     eq_ref   primary   c.a     1




                                                    www.percona.com
Dump Tree
$tree = $itis->fetchTreeByRoot(18880); // Papaveraceae
print_tree($tree);


function print_tree($tree, $prefix = ‘’)
  {
    print “{$prefix} {$tree->completename}n”;
    foreach ($tree->getChildren() as $child) {
      print_tree($child, “{$prefix} ”);
    }
  }




                                                 www.percona.com
Dump Tree Result
Papaveraceae                 Romneya
 Platystigma                      Romneya coulteri
   Platystigma linearis           Romneya trichocalyx
 Glaucium                       Dendromecon
   Glaucium corniculatum          Dendromecon harfordii
   Glaucium flavum                Dendromecon rigida
 Chelidonium                    Eschscholzia
   Chelidonium majus              Eschscholzia californica
 Bocconia                         Eschscholzia glyptosperma
   Bocconia frutescens            Eschscholzia hypecoides
 Stylophorum                      Eschscholzia lemmonii
   Stylophorum diphyllum          Eschscholzia lobbii
 Stylomecon                       Eschscholzia minutiflora
   Stylomecon heterophylla        Eschscholzia parishii
 Canbya                           Eschscholzia ramosa
   Canbya aurea                   Eschscholzia rhombipetala
   Canbya candida                 Eschscholzia caespitosa
 Chlidonium                  etc...
   Chlidonium majus
                                                www.percona.com
Dump Tree SQL
SELECT d.*, p.a AS _parent
 FROM treepaths AS c
 INNER JOIN longnames AS d ON c.d = d.tsn
 LEFT JOIN treepaths AS p ON p.d = d.tsn
    AND p.a IN (202422, 564824, 18053, 18020)
    AND p.l = 1
 WHERE (c.a = 202422)
    AND (p.a IS NOT NULL OR d.tsn = 202422)
 ORDER BY c.l, d.completename;




                                        www.percona.com
Dump Tree SQL
                                     show children
SELECT d.*, p.a AS _parent           of these nodes
 FROM treepaths AS c
 INNER JOIN longnames AS d ON c.d = d.tsn
 LEFT JOIN treepaths AS p ON p.d = d.tsn
    AND p.a IN (202422, 564824, 18053, 18020)
    AND p.l = 1
 WHERE (c.a = 202422)
    AND (p.a IS NOT NULL OR d.tsn = 202422)
 ORDER BY c.l, d.completename;




                                        www.percona.com
How Does it Perform?
• Query profile = 0.20 sec on Macbook Pro
• MySQL EXPLAIN:
table type     key        ref     rows     extra

c     ref      tree_adl   const   114240   Using index; Using
                                           temporary; Using filesort
d     eq_ref   primary    c.d     1

p     ref      tree_dl    c.d,    1        Using where; Using index
                          const




                                                        www.percona.com
SHOW CREATE TABLE
CREATE TABLE `treepaths` (
    `a` int(11) NOT NULL DEFAULT '0',
    `d` int(11) NOT NULL DEFAULT '0',
    `l` tinyint(3) unsigned NOT NULL DEFAULT '0',
    PRIMARY KEY (`a`,`d`),
    KEY `tree_adl` (`a`,`d`,`l`),
    KEY `tree_dl` (`d`,`l`),
    CONSTRAINT FOREIGN KEY (`a`)
      REFERENCES `longnames` (`tsn`),
    CONSTRAINT FOREIGN KEY (`d`)
      REFERENCES `longnames` (`tsn`)
  ) ENGINE=InnoDB




                                                    www.percona.com
SHOW TABLE STATUS
Name:             treepaths
 Engine:          InnoDB
 Version:         10
 Row_format:      Compact
 Rows:            4600439
 Avg_row_length: 62
 Data_length:     288276480
 Max_data_length: 0
 Index_length:    273137664
 Data_free:       7340032



                              www.percona.com
Demo Time!




             www.percona.com
SQL Antipatterns




http://www.pragprog.com/titles/bksqla/
                                 www.percona.com
Models for hierarchical data

More Related Content

What's hot

What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...Simplilearn
 
Docker Networking Overview
Docker Networking OverviewDocker Networking Overview
Docker Networking OverviewSreenivas Makam
 
An introduction to terraform
An introduction to terraformAn introduction to terraform
An introduction to terraformJulien Pivotto
 
#APIOps- Agile API Development powered by API Connect
#APIOps- Agile API Development powered by API Connect#APIOps- Agile API Development powered by API Connect
#APIOps- Agile API Development powered by API Connectpramodvallanur
 
Managing multiple event types in a single topic with Schema Registry | Bill B...
Managing multiple event types in a single topic with Schema Registry | Bill B...Managing multiple event types in a single topic with Schema Registry | Bill B...
Managing multiple event types in a single topic with Schema Registry | Bill B...HostedbyConfluent
 
Continuous Testing with Molecule, Ansible, and GitHub Actions
Continuous Testing with Molecule, Ansible, and GitHub ActionsContinuous Testing with Molecule, Ansible, and GitHub Actions
Continuous Testing with Molecule, Ansible, and GitHub ActionsJeff Geerling
 
Capabilities for Resources and Effects
Capabilities for Resources and EffectsCapabilities for Resources and Effects
Capabilities for Resources and EffectsMartin Odersky
 
모두가 성장하는 스터디 만들기
모두가 성장하는 스터디 만들기모두가 성장하는 스터디 만들기
모두가 성장하는 스터디 만들기BYUNGHOKIM10
 
Best Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformBest Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformDevOps.com
 
Terraform AWS modules and some best practices - September 2019
Terraform AWS modules and some best practices - September 2019Terraform AWS modules and some best practices - September 2019
Terraform AWS modules and some best practices - September 2019Anton Babenko
 
API Testing following the Test Pyramid
API Testing following the Test PyramidAPI Testing following the Test Pyramid
API Testing following the Test PyramidElias Nogueira
 
Unique ID generation in distributed systems
Unique ID generation in distributed systemsUnique ID generation in distributed systems
Unique ID generation in distributed systemsDave Gardner
 
MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks EDB
 
How to Automate API Testing
How to Automate API TestingHow to Automate API Testing
How to Automate API TestingBruno Pedro
 
REST API Design & Development
REST API Design & DevelopmentREST API Design & Development
REST API Design & DevelopmentAshok Pundit
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetesRishabh Indoria
 

What's hot (20)

What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
 
Docker Networking Overview
Docker Networking OverviewDocker Networking Overview
Docker Networking Overview
 
An introduction to terraform
An introduction to terraformAn introduction to terraform
An introduction to terraform
 
Final terraform
Final terraformFinal terraform
Final terraform
 
#APIOps- Agile API Development powered by API Connect
#APIOps- Agile API Development powered by API Connect#APIOps- Agile API Development powered by API Connect
#APIOps- Agile API Development powered by API Connect
 
Managing multiple event types in a single topic with Schema Registry | Bill B...
Managing multiple event types in a single topic with Schema Registry | Bill B...Managing multiple event types in a single topic with Schema Registry | Bill B...
Managing multiple event types in a single topic with Schema Registry | Bill B...
 
Continuous Testing with Molecule, Ansible, and GitHub Actions
Continuous Testing with Molecule, Ansible, and GitHub ActionsContinuous Testing with Molecule, Ansible, and GitHub Actions
Continuous Testing with Molecule, Ansible, and GitHub Actions
 
Capabilities for Resources and Effects
Capabilities for Resources and EffectsCapabilities for Resources and Effects
Capabilities for Resources and Effects
 
모두가 성장하는 스터디 만들기
모두가 성장하는 스터디 만들기모두가 성장하는 스터디 만들기
모두가 성장하는 스터디 만들기
 
Best Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformBest Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with Terraform
 
Terraform AWS modules and some best practices - September 2019
Terraform AWS modules and some best practices - September 2019Terraform AWS modules and some best practices - September 2019
Terraform AWS modules and some best practices - September 2019
 
Api testing
Api testingApi testing
Api testing
 
The basics of fluentd
The basics of fluentdThe basics of fluentd
The basics of fluentd
 
API Testing following the Test Pyramid
API Testing following the Test PyramidAPI Testing following the Test Pyramid
API Testing following the Test Pyramid
 
Terraform
TerraformTerraform
Terraform
 
Unique ID generation in distributed systems
Unique ID generation in distributed systemsUnique ID generation in distributed systems
Unique ID generation in distributed systems
 
MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks
 
How to Automate API Testing
How to Automate API TestingHow to Automate API Testing
How to Automate API Testing
 
REST API Design & Development
REST API Design & DevelopmentREST API Design & Development
REST API Design & Development
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 

Viewers also liked

SQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークSQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークke-m kamekoopa
 
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現gree_tech
 
SQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーSQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーke-m kamekoopa
 
Database Anti Patterns
Database Anti PatternsDatabase Anti Patterns
Database Anti PatternsRobert Treat
 
Fax With Sangoma Gateway
Fax With Sangoma GatewayFax With Sangoma Gateway
Fax With Sangoma GatewayHossein Yavari
 
10 system.security.cryptography
10 system.security.cryptography10 system.security.cryptography
10 system.security.cryptographyMohammad Alyan
 
1 first lesson -assemblies
1  first lesson -assemblies1  first lesson -assemblies
1 first lesson -assembliesMohammad Alyan
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributesMohammad Alyan
 
Login System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlLogin System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlthesoftwareguy7
 
8 memory managment & pointers
8 memory managment & pointers8 memory managment & pointers
8 memory managment & pointersMohammad Alyan
 
3 third lesson-reflection
3 third lesson-reflection3 third lesson-reflection
3 third lesson-reflectionMohammad Alyan
 
4 fourth lesson-deployment
4 fourth lesson-deployment4 fourth lesson-deployment
4 fourth lesson-deploymentMohammad Alyan
 

Viewers also liked (18)

Recursive Query Throwdown
Recursive Query ThrowdownRecursive Query Throwdown
Recursive Query Throwdown
 
SQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークSQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォーク
 
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
 
SQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーSQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリー
 
Database Anti Patterns
Database Anti PatternsDatabase Anti Patterns
Database Anti Patterns
 
Fax With Sangoma Gateway
Fax With Sangoma GatewayFax With Sangoma Gateway
Fax With Sangoma Gateway
 
5 fifth lesson -xml
5 fifth lesson -xml5 fifth lesson -xml
5 fifth lesson -xml
 
7 multi threading
7 multi threading7 multi threading
7 multi threading
 
10 system.security.cryptography
10 system.security.cryptography10 system.security.cryptography
10 system.security.cryptography
 
1 first lesson -assemblies
1  first lesson -assemblies1  first lesson -assemblies
1 first lesson -assemblies
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributes
 
Introduction To ERP
Introduction To ERPIntroduction To ERP
Introduction To ERP
 
Login System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlLogin System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysql
 
8 memory managment & pointers
8 memory managment & pointers8 memory managment & pointers
8 memory managment & pointers
 
3 third lesson-reflection
3 third lesson-reflection3 third lesson-reflection
3 third lesson-reflection
 
Coursera
CourseraCoursera
Coursera
 
4 fourth lesson-deployment
4 fourth lesson-deployment4 fourth lesson-deployment
4 fourth lesson-deployment
 
9 networking
9 networking9 networking
9 networking
 

More from Karwin Software Solutions LLC (17)

How to Use JSON in MySQL Wrong
How to Use JSON in MySQL WrongHow to Use JSON in MySQL Wrong
How to Use JSON in MySQL Wrong
 
Load Data Fast!
Load Data Fast!Load Data Fast!
Load Data Fast!
 
InnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick FiguresInnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick Figures
 
SQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and ProfitSQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and Profit
 
Extensible Data Modeling
Extensible Data ModelingExtensible Data Modeling
Extensible Data Modeling
 
Sql query patterns, optimized
Sql query patterns, optimizedSql query patterns, optimized
Sql query patterns, optimized
 
Survey of Percona Toolkit
Survey of Percona ToolkitSurvey of Percona Toolkit
Survey of Percona Toolkit
 
How to Design Indexes, Really
How to Design Indexes, ReallyHow to Design Indexes, Really
How to Design Indexes, Really
 
Schemadoc
SchemadocSchemadoc
Schemadoc
 
Percona toolkit
Percona toolkitPercona toolkit
Percona toolkit
 
MySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB StatusMySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB Status
 
Requirements the Last Bottleneck
Requirements the Last BottleneckRequirements the Last Bottleneck
Requirements the Last Bottleneck
 
Mentor Your Indexes
Mentor Your IndexesMentor Your Indexes
Mentor Your Indexes
 
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
 
Full Text Search In PostgreSQL
Full Text Search In PostgreSQLFull Text Search In PostgreSQL
Full Text Search In PostgreSQL
 
Practical Object Oriented Models In Sql
Practical Object Oriented Models In SqlPractical Object Oriented Models In Sql
Practical Object Oriented Models In Sql
 
Sql Antipatterns Strike Back
Sql Antipatterns Strike BackSql Antipatterns Strike Back
Sql Antipatterns Strike Back
 

Recently uploaded

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 

Recently uploaded (20)

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 

Models for hierarchical data

  • 1. Models for Hierarchical Data with SQL and PHP Bill Karwin, Percona Inc.
  • 2. Me • Software developer • C, Java, Perl, PHP, Ruby • SQL maven • MySQL Consultant at Percona • Author of SQL Antipatterns: Avoiding the Pitfalls of Database Programming www.percona.com
  • 3. Problem • Store & query hierarchical data - Categories/subcategories - Bill of materials - Threaded discussions www.percona.com
  • 4. Example: Bug Report Comments (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 5. Solutions •Adjacency list •Path enumeration •Nested sets •Closure table www.percona.com
  • 6. Adjacency List www.percona.com
  • 7. Adjacency List • Naive solution nearly everyone uses • Each entry knows its immediate parent comment_id parent_id author comment 1 NULL Fran What’s the cause of this bug? 2 1 Ollie I think it’s a null pointer. 3 2 Fran No, I checked for that. 4 1 Kukla We need to check valid input. 5 4 Ollie Yes, that’s a bug. 6 4 Fran Yes, please add a check 7 6 Kukla That fixed it. www.percona.com
  • 8. Insert a New Node INSERT INTO Comments (parent_id, author, comment) VALUES (5, ‘Fran’, ‘I agree!’); (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 9. Insert a New Node INSERT INTO Comments (parent_id, author, comment) VALUES (5, ‘Fran’, ‘I agree!’); (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 10. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 11. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. www.percona.com
  • 12. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. www.percona.com
  • 13. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. (6) Fran: Yes, please add a check. (7) Kukla: That fixed it. www.percona.com
  • 14. Query Immediate Child/Parent • Query a node’s children: SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id); • Query a node’s parent: SELECT * FROM Comments c1 JOIN Comments c2 ON (c1.parent_id = c2.comment_id); www.percona.com
  • 15. Can’t Handle Deep Trees SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id) LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id) LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id) LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id) LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id) LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id) LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id) LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id) LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id) ... www.percona.com
  • 16. Can’t Handle Deep Trees SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id) LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id) LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id) LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id) LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id) LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id) LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id) LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id) LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id) ... it still doesn’t support unlimited depth! www.percona.com
  • 17. SQL-99 recursive syntax WITH [RECURSIVE] CommentTree (comment_id, bug_id, parent_id, author, comment, depth) AS ( SELECT *, 0 AS depth FROM Comments WHERE parent_id IS NULL UNION ALL SELECT c.*, ct.depth+1 AS depth FROM CommentTree ct JOIN Comments c ON (ct.comment_id = c.parent_id) ) SELECT * FROM CommentTree WHERE bug_id = 1234; ✓ PostgreSQL, Oracle 11g, IBM DB2, Microsoft SQL Server, Apache Derby ✗ MySQL, SQLite, Informix, Firebird,etc. www.percona.com
  • 18. Path Enumeration www.percona.com
  • 19. Path Enumeration • Store chain of ancestors in each node comment_id path author comment 1 1/ Fran What’s the cause of this bug? 2 1/2/ Ollie I think it’s a null pointer. 3 1/2/3/ Fran No, I checked for that. 4 1/4/ Kukla We need to check valid input. 5 1/4/5/ Ollie Yes, that’s a bug. 6 1/4/6/ Fran Yes, please add a check 7 1/4/6/7/ Kukla That fixed it. www.percona.com
  • 20. Path Enumeration • Store chain of ancestors in each node good for breadcrumbs comment_id path author comment 1 1/ Fran What’s the cause of this bug? 2 1/2/ Ollie I think it’s a null pointer. 3 1/2/3/ Fran No, I checked for that. 4 1/4/ Kukla We need to check valid input. 5 1/4/5/ Ollie Yes, that’s a bug. 6 1/4/6/ Fran Yes, please add a check 7 1/4/6/7/ Kukla That fixed it. www.percona.com
  • 21. Query Ancestors and Subtrees • Query ancestors of comment #7: SELECT * FROM Comments WHERE ‘1/4/6/7/’ LIKE path || ‘%’; • Query descendants of comment #4: SELECT * FROM Comments WHERE path LIKE ‘1/4/%’; www.percona.com
  • 22. Add a New Child of #7 INSERT INTO Comments (author, comment) VALUES (‘Ollie’, ‘Good job!’); SELECT path FROM Comments WHERE comment_id = 7; UPDATE Comments SET path = $parent_path || LAST_INSERT_ID() || ‘/’ WHERE comment_id = LAST_INSERT_ID(); www.percona.com
  • 23. Nested Sets www.percona.com
  • 24. Nested Sets • Each comment encodes its descendants using two numbers: - A comment’s left number is less than all numbers used by the comment’s descendants. - A comment’s right number is greater than all numbers used by the comment’s descendants. - A comment’s numbers are between all numbers used by the comment’s ancestors. www.percona.com
  • 25. What Does This Look Like? (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 26. What Does This Look Like? (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 27. What Does This Look Like? comment_id nsleft nsright author comment 1 1 14 Fran What’s the cause of this bug? 2 2 5 Ollie I think it’s a null pointer. 3 3 4 Fran No, I checked for that. 4 6 13 Kukla We need to check valid input. 5 7 8 Ollie Yes, that’s a bug. 6 9 12 Fran Yes, please add a check 7 10 11 Kukla That fixed it. www.percona.com
  • 28. What Does This Look Like? comment_id nsleft nsright author comment 1 1 14 Fran What’s the cause of this bug? 2 2 5 Ollie I think it’s a null pointer. 3 3 4 Fran No, I checked for that. 4 6 13 Kukla We need to check valid input. 5 7 8 Ollie Yes, that’s a bug. 6 9 12 Fran Yes, please add a check 7 10 11 Kukla That fixed it. these are not foreign keys www.percona.com
  • 29. Query Ancestors of #7 (1) Fran: ancestors What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 child (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 30. Query Ancestors of #7 SELECT * FROM Comments child JOIN Comments ancestor ON child.nsleft BETWEEN ancestor.nsleft AND ancestor.nsright WHERE child.comment_id = 7; www.percona.com
  • 31. Query Subtree Under #4 (1) Fran: What’s the parent cause of this bug? 1 14 (2) Ollie: (4) Kukla: descendants We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 32. Query Subtree Under #4 SELECT * FROM Comments parent JOIN Comments descendant ON descendant.nsleft BETWEEN parent.nsleft AND parent.nsright WHERE parent.comment_id = 4; www.percona.com
  • 33. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 34. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 16 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 15 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 10 11 8 9 14 12 (7) Kukla: That fixed it. 12 10 13 11 www.percona.com
  • 35. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 16 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 15 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 10 11 8 9 14 12 (8) Fran: (7) Kukla: I agree! That fixed it. 8 9 10 12 13 11 www.percona.com
  • 36. Insert New Child of #5 UPDATE Comments SET nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2 ELSE nsleft END, nsright = nsright+2 WHERE nsright >= 7; INSERT INTO Comments (nsleft, nsright, author, comment) VALUES (8, 9, 'Fran', 'I agree!'); • Recalculate left values for all nodes to the right of the new child. Recalculate right values for all nodes above and to the right. www.percona.com
  • 37. Query Immediate Parent of #6 (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 38. Query Immediate Parent of #6 • Parent of #6 is an ancestor who has no descendant who is also an ancestor of #6. SELECT parent.* FROM Comments AS c JOIN Comments AS parent ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright) LEFT OUTER JOIN Comments AS in_between ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright) WHERE c.comment_id = 6 AND in_between.comment_id IS NULL; www.percona.com
  • 39. Query Immediate Parent of #6 • Parent of #6 is an ancestor who has no descendant who is also an ancestor of #6. SELECT parent.* FROM Comments AS c JOIN Comments AS parent ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright) LEFT OUTER JOIN Comments AS in_between ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright) WHERE c.comment_id = 6 AND in_between.comment_id IS NULL; querying immediate child is a similar problem www.percona.com
  • 40. Closure Table www.percona.com
  • 41. Closure Table CREATE TABLE TreePaths ( ancestor INT NOT NULL, descendant INT NOT NULL, PRIMARY KEY (ancestor, descendant), FOREIGN KEY(ancestor) REFERENCES Comments(comment_id), FOREIGN KEY(descendant) REFERENCES Comments(comment_id) ); www.percona.com
  • 42. Closure Table • Many-to-many table • Stores every path from each node to each of its descendants • A node even connects to itself www.percona.com
  • 43. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 44. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 45. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 46. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 47. What Does This Look Like? ancestor descendant comment_id author comment 1 1 1 2 1 Fran What’s the cause of this bug? 1 3 1 4 2 Ollie I think it’s a null pointer. 1 5 3 Fran No, I checked for that. 1 6 1 7 4 Kukla We need to check valid 2 2 input. 2 3 5 Ollie Yes, that’s a bug. 3 3 4 4 6 Fran Yes, please add a check 4 5 7 Kukla That fixed it. 4 6 4 7 5 5 requires O(n²) rows 6 6 6 7 7 7 www.percona.com
  • 48. What Does This Look Like? ancestor descendant comment_id author comment 1 1 1 2 1 Fran What’s the cause of this bug? 1 3 1 4 2 Ollie I think it’s a null pointer. 1 5 3 Fran No, I checked for that. 1 6 1 7 4 Kukla We need to check valid 2 2 input. 2 3 5 Ollie Yes, that’s a bug. 3 3 4 4 6 Fran Yes, please add a check 4 5 7 Kukla That fixed it. 4 6 4 7 5 5 requires O(n²) rows 6 6 6 7 7 7 (but far fewer in practice) www.percona.com
  • 49. Query Descendants of #4 SELECT c.* FROM Comments c JOIN TreePaths t ON (c.comment_id = t.descendant) WHERE t.ancestor = 4; www.percona.com
  • 50. Paths Starting from #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 51. Query Ancestors of #6 SELECT c.* FROM Comments c JOIN TreePaths t ON (c.comment_id = t.ancestor) WHERE t.descendant = 6; www.percona.com
  • 52. Paths Terminating at #6 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 53. Insert New Child of #5 INSERT INTO Comments VALUES (8, ‘Fran’, ‘I agree!’); INSERT INTO TreePaths (ancestor, descendant) SELECT ancestor, 8 FROM TreePaths WHERE descendant = 5 UNION ALL SELECT 8, 8; www.percona.com
  • 54. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 55. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 56. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 57. Delete Child #7 DELETE FROM TreePaths WHERE descendant = 7; www.percona.com
  • 58. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 59. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 60. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. www.percona.com
  • 61. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 62. Delete Subtree Under #4 DELETE FROM TreePaths WHERE descendant IN (SELECT descendant FROM TreePaths WHERE ancestor = 4); www.percona.com
  • 63. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 64. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 65. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (2) Ollie: I think it’s a null pointer. (3) Fran: No, I checked for that. www.percona.com
  • 66. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null check valid pointer. input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 67. Path Length • Add a length column ancestor descendant length • MAX(length) is depth of tree 1 1 0 1 2 1 1 3 2 • Makes it easier to query 1 1 4 5 1 2 immediate parent or child: 1 6 2 1 7 3 SELECT c.* 2 2 0 FROM Comments c 2 3 1 JOIN TreePaths t 3 3 0 4 4 0 ON (c.comment_id = t.descendant) 4 5 1 WHERE t.ancestor = 4 4 6 1 AND t.length = 1; 4 7 2 5 5 0 6 6 0 6 7 1 7 7 0 www.percona.com
  • 68. Path Length • Add a length column ancestor descendant length • MAX(length) is depth of tree 1 1 0 1 2 1 1 3 2 • Makes it easier to query 1 1 4 5 1 2 immediate parent or child: 1 6 2 1 7 3 SELECT c.* 2 2 0 FROM Comments c 2 3 1 JOIN TreePaths t 3 3 0 4 4 0 ON (c.comment_id = t.descendant) 4 5 1 WHERE t.ancestor = 4 4 6 1 AND t.length = 1; 4 7 2 5 5 0 6 6 0 6 7 1 7 7 0 www.percona.com
  • 69. Choosing the Right Design Design Tables Query Query Delete Insert Move Referential Child Subtree Node Node Subtree Integrity Adjacency 1 Easy Hard Easy Easy Easy Yes List Path 1 Hard Easy Easy Easy Easy No Enumeration Nested Sets 1 Hard Easy Hard Hard Hard No Closure 2 Easy Easy Easy Easy Easy Yes Table www.percona.com
  • 70. PHP Demo of Closure Table www.percona.com
  • 71. Hierarchical Test Data • Integrated Taxonomic Information System - http://itis.gov/ - Free authoritative taxonomic information on plants, animals, fungi, microbes - 518,756 scientific names (as of Feb 2011) www.percona.com
  • 72. California Poppy Kingdom: Plantae Division: Tracheobionta Class: Magnoliophyta Order: Magnoliopsida unranked: Magnoliidae unranked: Papaverales Family: Papaveraceae Genus: Eschscholzia Species: Eschscholzia californica www.percona.com
  • 73. California Poppy Kingdom: Plantae Division: Tracheobionta Class: Magnoliophyta Order: Magnoliopsida unranked: Magnoliidae unranked: Papaverales Family: Papaveraceae Genus: Eschscholzia Species: Eschscholzia californica id=18956 www.percona.com
  • 74. California Poppy: ITIS Entry SELECT * FROM Hierarchy WHERE hierarchy_string LIKE ‘%-18956’; hierarchy_string 202422-564824-18061-18063-18064-18879-18880-18954-18956 www.percona.com
  • 75. California Poppy: ITIS Entry SELECT * FROM Hierarchy WHERE hierarchy_string LIKE ‘%-18956’; hierarchy_string 202422-564824-18061-18063-18064-18879-18880-18954-18956 ITIS data uses ...but I converted path enumeration it to closure table www.percona.com
  • 76. Hierarchical Data Classes abstract class ZendX_Db_Table_TreeTable extends Zend_Db_Table_Abstract { public function fetchTreeByRoot($rootId, $expand) public function fetchBreadcrumbs($leafId) } www.percona.com
  • 77. Hierarchical Data Classes class ZendX_Db_Table_Row_TreeRow extends Zend_Db_Table_Row_Abstract { public function addChildRow($childRow) public function getChildren() } class ZendX_Db_Table_Rowset_TreeRowset extends Zend_Db_Table_Rowset_Abstract { public function append($row) } www.percona.com
  • 78. Using TreeTable class ItisTable extends ZendX_Db_Table_TreeTable { protected $_name = “longnames”; protected $_closureName = “treepaths”; } $itis = new ItisTable(); www.percona.com
  • 79. Breadcrumbs $breadcrumbs = $itis->fetchBreadcrumbs(18956); foreach ($breadcrumbs as $crumb) { print $crumb->completename . “ > ”; } Plantae > Tracheobionta > Magnoliophyta > Magnoliopsida > Magnoliidae > Papaverales > Papaveraceae > Eschscholzia > Eschscholzia californica > www.percona.com
  • 80. Breadcrumbs SQL SELECT a.* FROM longnames AS a INNER JOIN treepaths AS c ON a.tsn = c.a WHERE (c.d = 18956) ORDER BY c.l DESC www.percona.com
  • 81. How Does it Perform? • Query profile = 0.0006 sec • MySQL EXPLAIN: table type key ref rows extra c ref tree_dl const 9 Using where; Using index a eq_ref primary c.a 1 www.percona.com
  • 82. Dump Tree $tree = $itis->fetchTreeByRoot(18880); // Papaveraceae print_tree($tree); function print_tree($tree, $prefix = ‘’) { print “{$prefix} {$tree->completename}n”; foreach ($tree->getChildren() as $child) { print_tree($child, “{$prefix} ”); } } www.percona.com
  • 83. Dump Tree Result Papaveraceae Romneya Platystigma Romneya coulteri Platystigma linearis Romneya trichocalyx Glaucium Dendromecon Glaucium corniculatum Dendromecon harfordii Glaucium flavum Dendromecon rigida Chelidonium Eschscholzia Chelidonium majus Eschscholzia californica Bocconia Eschscholzia glyptosperma Bocconia frutescens Eschscholzia hypecoides Stylophorum Eschscholzia lemmonii Stylophorum diphyllum Eschscholzia lobbii Stylomecon Eschscholzia minutiflora Stylomecon heterophylla Eschscholzia parishii Canbya Eschscholzia ramosa Canbya aurea Eschscholzia rhombipetala Canbya candida Eschscholzia caespitosa Chlidonium etc... Chlidonium majus www.percona.com
  • 84. Dump Tree SQL SELECT d.*, p.a AS _parent FROM treepaths AS c INNER JOIN longnames AS d ON c.d = d.tsn LEFT JOIN treepaths AS p ON p.d = d.tsn AND p.a IN (202422, 564824, 18053, 18020) AND p.l = 1 WHERE (c.a = 202422) AND (p.a IS NOT NULL OR d.tsn = 202422) ORDER BY c.l, d.completename; www.percona.com
  • 85. Dump Tree SQL show children SELECT d.*, p.a AS _parent of these nodes FROM treepaths AS c INNER JOIN longnames AS d ON c.d = d.tsn LEFT JOIN treepaths AS p ON p.d = d.tsn AND p.a IN (202422, 564824, 18053, 18020) AND p.l = 1 WHERE (c.a = 202422) AND (p.a IS NOT NULL OR d.tsn = 202422) ORDER BY c.l, d.completename; www.percona.com
  • 86. How Does it Perform? • Query profile = 0.20 sec on Macbook Pro • MySQL EXPLAIN: table type key ref rows extra c ref tree_adl const 114240 Using index; Using temporary; Using filesort d eq_ref primary c.d 1 p ref tree_dl c.d, 1 Using where; Using index const www.percona.com
  • 87. SHOW CREATE TABLE CREATE TABLE `treepaths` ( `a` int(11) NOT NULL DEFAULT '0', `d` int(11) NOT NULL DEFAULT '0', `l` tinyint(3) unsigned NOT NULL DEFAULT '0', PRIMARY KEY (`a`,`d`), KEY `tree_adl` (`a`,`d`,`l`), KEY `tree_dl` (`d`,`l`), CONSTRAINT FOREIGN KEY (`a`) REFERENCES `longnames` (`tsn`), CONSTRAINT FOREIGN KEY (`d`) REFERENCES `longnames` (`tsn`) ) ENGINE=InnoDB www.percona.com
  • 88. SHOW TABLE STATUS Name: treepaths Engine: InnoDB Version: 10 Row_format: Compact Rows: 4600439 Avg_row_length: 62 Data_length: 288276480 Max_data_length: 0 Index_length: 273137664 Data_free: 7340032 www.percona.com
  • 89. Demo Time! www.percona.com