Earlier today Matt Ingenthron blogged about the new tool memcapable I wrote a while back. In his blog Matt mentions some of the reasons why we want such a tool, but he didn't actually mention the reason for why I actually sat down to create the tool.

If you follow my blog you might remember my entry "Callback based protocol parser in libmemcached?". Before I could start implementing the parser, I really needed a tool to:

  1. send all of the defined packet structures to the server
  2. verify the response packet generated from the server
I didn't have the need for the textual protocol at the time I wrote the initial version of memcapable, so right now memcapable can only be used to test the binary protocol (not all variants of all commands are implemented in the initial version).

So how does it work? In it's simplest form you can use it to test the memcached server running on the default port on the same computer:

trond@storm> ./memcapable noop [pass] quit [pass] quitq [pass] set [pass] setq [pass] flush [pass] flushq [pass] add [pass] addq [pass] replace [pass] replaceq [pass] delete [pass] deleteq [pass] get [pass] getq [pass] getk [pass] getkq [pass] incr [pass] incrq [pass] decr [pass] decrq [pass] version [pass] append [pass] appendq [pass] prepend [pass] prependq [pass] stat [pass] illegal [pass] All tests passed Now this looks really nice doesn't it, but let's try to run it on a 1.2.8 version:

trond@storm> ./memcapable noop [FAIL] quit [FAIL] quitq [FAIL] set [FAIL] setq [FAIL] flush [FAIL] flushq [FAIL] add [FAIL] addq [FAIL] replace [FAIL] replaceq [FAIL] delete [FAIL] deleteq [FAIL] get [FAIL] getq [FAIL] getk [FAIL] getkq [FAIL] incr [FAIL] incrq [FAIL] decr [FAIL] decrq [FAIL] version [FAIL] append [FAIL] appendq [FAIL] prepend [FAIL] prependq [FAIL] stat [FAIL] illegal [FAIL] 28 of 28 tests failed This shouldn't come as a big surprise, because the binary protocol isn't implemented in 1.2.8. Getting [FAIL] isn't really that informative, because it doesn't help you as a developer to figure out what's wrong. I have added a couple of options to the program that may help you to track down the real problem: -v and -c.

-v Print out the assertion that failed -c Create a coredump when an assertion fails Let's start the memcached server and disable the use of CAS and re-run memcapable with -v -c

trond@storm> ./memcapable -v -c noop [pass] quit [pass] quitq [pass] set memcapable.c:493: rsp->plain.message.header.response.cas != 0 zsh: IOT instruction (core dumped) ./memcapable -v -c As you can see it expects the response packet to have a CAS value set for the operation. If you would like to inspect the response packet you could load it into your debugger and poke around:

trond@storm> dbx - core Corefile specified executable: "/source/libmemcached/memcapable/clients/memcapable" Reading memcapable core file header read successfully Reading ld.so.1 Reading libm.so.2 Reading libnsl.so.1 Reading libsocket.so.1 Reading libpthread.so.1 Reading libthread.so.1 Reading libc.so.1 t@1 (l@1) program terminated by signal ABRT (Abort) 0xfffffd7fff2842aa: _lwp_kill+0x000a: jae _lwp_kill+0x18 [ 0xfffffd7fff2842b8, .+0xe ] Current function is ensure 212 abort(); (dbx) where current thread: t@1 [1] _lwp_kill(0x1, 0x6, 0xffffff01cdf92ae0, 0xfffffd7fff284c0e, 0x12, 0x0), at 0xfffffd7fff2842aa [2] thr_kill(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff2788cd [3] raise(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff227511 [4] abort(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff1fda41 =>[5] ensure(val = 0, expression = 0x408498 "rsp->plain.message.header.response.cas != 0", file = 0x408198 "memcapable.c", line = 493), line 212 in "memcapable.c" [6] do_validate_response_header(rsp = 0xfffffd7fffdfeed8, cc = '\001', status = 0), line 493 in "memcapable.c" [7] test_binary_set_impl(key = 0x4087b0 "test_binary_set", cc = '\001'), line 621 in "memcapable.c" [8] test_binary_set(), line 651 in "memcapable.c" [9] main(argc = 3, argv = 0xfffffd7fffdff798), line 1196 in "memcapable.c" (dbx) frame 6 Current function is do_validate_response_header 493 verify(rsp->plain.message.header.response.cas != 0); (dbx) print rsp->plain.message.header.response rsp->plain.message.header.response = { magic = '?' opcode = '\001' keylen = 0 extlen = '\0' datatype = '\0' status = 0 bodylen = 0 opaque = 3735928559U cas = 0 } I noticed earlier today that there are some issues with the -v flag if you don't include -c (it just prints out the assertion and reports everything back up as success ;-)). I'll try to address that in the next release.



More...