libmc is a memcached client library for Python without any other dependencies in runtime. It's mainly written in C++ and Cython. https://github.com/douban/libmc

1. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Reinventing the wheel: libmc
panmiaocai@douban.com
Jan 23, 2015

2. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
The memcached family
▶ memcached (the memcached server-side software)
▶ memcache (typo and nickname)
▶ mc (nickname)
▶ libmemcached ( C/C++ client library)
▶ libmemcache (yet another C client library, DEAD since 2006)
▶ python-memcached (pure python client library)
▶ python-libmemcached (Cython client, library based on
libmemcached)
▶ cmemcached (alias of python-libmemcached)
▶ pylibmc (C-Extensions for Python by hand, based on
libmemcached)

5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Bad things of libmemcached
▶ Divergence of libmemcached between upstream and the
branch we use. (the “large split” feature)
▶ Complex
▶ Buggy
▶ (lib)memcached is a weird creature.
http://hoborglabs.com/en/blog/2013/memcached-php

7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Retrieval Commands
Request and Response
get <key>*rn
VALUE <key> <flags> <bytes>rn
<data block>rn
Example
get foo bar bazrn
VALUE foo 0 3rn
abcrn
VALUE baz 0 5rn
simp.rn
ENDrn

9. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Storage Commands
Request and Response
<key> <flags> <exptime> <bytes>rn
<data block>rn
STOREDrn | NOT STOREDrn
Example
set dou 0 0 10rn
0123456789rn
set ban 0 0 4rn
aoeirn
NOT STOREDrn
STOREDrn

10. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

11. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Design of libmc
PyClient
|
Client
| (1:1)
ConnectionPool
| (1:N)
Connection
/ | (1:1)
BufferReader Parser BufferWriter
| (1:N)
DataBlock

12. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

13. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BufferWriter
s s i z e t sendmsg ( i n t socket , const s t r u c t msghdr ∗
message , i n t f l a g s ) ;
s t r u c t i o v e c {
void ∗ i o v b a s e ;
s i z e t i o v l e n ;
};
s t r u c t msghdr {
. . .
s t r u c t i o v e c ∗ msg iov ;
s i z e t msg iovlen ;
. . .
};
▶ UIO MAXIOV
▶ commitRead

14. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

15. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BufferReader
+ Fixed + Flexible
+------------+------------+-----------------------+---------------
| 8192 recved| 8192 recved| more than 8192 recved | DataBlock ...
+----+----+-------+-------+---------------------------------------
|XXXX|OOOO|BB|BBBB|CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC| DataBlockSlice
|XXXX|OOOO|BB|BBBB|CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC| ...
+------------+------+------------+------------+-------------------
|VALU|foo | 0 0 20| value of foo value of foo valu| Token
|E | | | e of foo .....................| ...
|----+----+-------+-------------------------------+---------------
▶ peek / readUntil / skipUtil / readUnsigned / readBytes /
skipBytes / (no rewind)
▶ token.size() in 0, 1, 2 / SmallVector
▶ predictive DataBlock size

16. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

17. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Other parts
▶ Parser as a finite-state machine (FSM)
▶ poll in ConnectionPool

18. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

19. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.

20. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.

21. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.

22. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.

23. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.

24. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error

25. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing

26. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.

27. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.

28. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.

29. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.

30. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.
▶ write the right benchmark. bench order? dependency?

31. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.
▶ write the right benchmark. bench order? dependency?
▶ md5 is slow. bugfree fnv1 is hard to implement.

32. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.
▶ write the right benchmark. bench order? dependency?
▶ md5 is slow. bugfree fnv1 is hard to implement.
▶ vim: Rip-Rip/clang complete + cpplint.py

33. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

34. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Great Toolchain
▶ CMake
▶ Valgrind
▶ Callgrind(valgrind –tool=callgrind) +
kcachegrind/qcachegrind
▶ googletest

35. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Links I
libmc: https://github.com/douban/libmc
benchmark of libmc: https:
//gist.github.com/mckelvin/8ded053f6523931765d4
Improving performance of std::vector:
http://fahrenheit2539.blogspot.com/2012/06/
introduction-in-depths-look-at.html
How to profile C++ application with Callgrind / KCacheGrind:
http://baptiste-wicht.com/posts/2011/09/
profile-c-application-with-callgrind-kcachegrind.
html