A brief overview of several alternatives for Distributed Counting and Sorting.

1. Counting, Sorting and Distributed Coordination Luis Galárraga Saarland University

3. Concurrent objects and basic concepts

6. Sample sorting

9. Avoid memory contention!!!

11. Not serious for reads.. but for writes

12. Concurrent Objects

15. Quiescent consistency

17. Method calls by different threads are unrelated by program order.

18. Sequential Consistency

21. If two calls overlap, then their order is ambiguous and we are free to order them in any convenient way.

22. Linearization points: points where the method seems to take effect.

23. Linearizability

25. Distributed Counting

27. Perform the increment

28. Free the lock

30. Test and set (with/without Exponential Backoff)

31. Queue locks (MCS, CLH, etc... )

34. Some threads become responsible of gathering other increments and combine them.

35. Some hierarchical data structure must be used.

36. Binary trees

39. Each thread is assigned to a leaf. A leaf can be assigned to 2 threads at most.

40. The value of the counter is stored at the root

42. FIRST: One thread has visited this node and becomes the master

43. SECOND: A second thread (slave) is waiting for the master to combine.

44. Software Combining Trees

46. Linearizable alternative

48. How long to wait for other threads to combine?

50. Balancing networks are constructed connecting balancer's outputs to other balancer's inputs.

55. For k > 1, k is a power of 2:

58. For k > 1, a sequence of lg(2k) block networks: log(2k) blocks

63. BM or Busch-Mavronikolas, w inputs, p*w outputs for some p>1

64. BM outperforms Bitonic under most conditions

66. Distributed Sorting

69. Proof in [3]

72. Table of size d * w/2 storing which entries (wires) compare in each layer

73. Synchronization via Barriers, see [4]

74. Each thread/processor does s comparisons in every round, p*s is a power of 2

75. Bitonic Sort

80. No cache efficient!!!

84. Take s samples then sort them using BitonicSort

87. Step 3: Each thread sequentially sorts the items in its bucket.

91. Parallel Merge Sorting

93. Partitioned Parallel Radix Sort [6]

94. An equilibrium between fairness in terms of work distribution and communication effort.

95. Sources [1] M. Herlihy and N. Shavit, “Concurrent objects,” in The Art of Multiprocessor Programming , pp. 45–69, Burlington, USA: Elsevier Inc., 2008. [2] E. N. Klein, C. Busch, and D. R. Musser, “An experimental analysis of counting networks,” Journal of the ACM , pp. 1020–1048, September 1994. [3] J. Aspnes, M. Herlihy, and N. Shavit, “Counting networks,” Journal of the ACM , 1994. [4] M. Herlihy and N. Shavit, “Barriers,” in The Art of Multiprocessor Programming , pp. 397–415, Burlington, USA: Elsevier Inc., 2008. [5] A. Sohn and Y. Kodama, “Load balanced parallel radix sort,” in ICS ’98: Proceedings of the 12th international conference on Supercomputing , (New York, NY, USA), pp. 305–312, ACM, 1998. [6] S.-J. Lee, M. Jeon, D. Kim, and A. Sohn, “Partitioned parallel radix sort,” J. Parallel Distrib. Comput. , vol. 62, no. 4, pp. 656–668, 2002.