No title Revision 326466346437 (Fri Jun 02 2017 at 23:21) - Diff Link to this snippet: https://friendpaste.com/59jNoNCWLSrPOyTCQK76Rq Embed: manni perldoc borland colorful default murphy trac fruity autumn bw emacs pastie friendly Show line numbers Wrap lines 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523[root@localhost ~]# more /data/core.beam.smp.7965.log Basic crash dump analysis of /data/core.beam.smp.7965.Please send the file to support@couchbase.com--------------------------------------------------------------------------------File information:-rwxr-xr-x. 1 bin bin 12181134 Oct 10 15:10 /opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp2d761bc628b84d9891c5c0e36875aaa3 /opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp-rw-------. 1 couchbase couchbase 321896448 Oct 11 00:00 /data/core.beam.smp.79650c5a52fdde860c8e1222ba8c474ffccb /data/core.beam.smp.79651.4.4_209_g7b9e75fversion1.4.4_598_g8587311version1.4.4_598_g8587311--------------------------------------------------------------------------------Core file callstacks:GNU gdb (GDB) Red Hat Enterprise Linux (7.2-56.el6)Copyright (C) 2010 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>This is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law. Type "show copying"and "show warranty" for details.This GDB was configured as "x86_64-redhat-linux-gnu".For bug reporting instructions, please see:<http://www.gnu.org/software/gdb/bugs/>...Reading symbols from /opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp...done.[New Thread 7969][New Thread 7971][New Thread 7974][New Thread 7980][New Thread 7972][New Thread 7987][New Thread 7981][New Thread 7989][New Thread 7970][New Thread 7973][New Thread 7982][New Thread 7978][New Thread 7975][New Thread 7979][New Thread 27725][New Thread 7983][New Thread 7976][New Thread 7967][New Thread 7977][New Thread 7968][New Thread 27722][New Thread 7985][New Thread 7986][New Thread 7965][New Thread 7984][New Thread 7966][New Thread 7995][New Thread 7988]Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libutil.so.1Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libdl.so.2Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.Loaded symbols for /lib64/libm.so.6Reading symbols from /lib64/libncurses.so.5...(no debugging symbols found)...done.Loaded symbols for /lib64/libncurses.so.5Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.[Thread debugging using libthread_db enabled]Loaded symbols for /lib64/libpthread.so.0Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/librt.so.1Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.Loaded symbols for /lib64/libc.so.6Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/ld-linux-x86-64.so.2Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done.Loaded symbols for /lib64/libtinfo.so.5Reading symbols from /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so...done.Loaded symbols for /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.soReading symbols from /usr/lib64/libcrypto.so.6...(no debugging symbols found)...done.Loaded symbols for /usr/lib64/libcrypto.so.6Reading symbols from /usr/lib64/libssl.so.6...(no debugging symbols found)...done.Loaded symbols for /usr/lib64/libssl.so.6Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libz.so.1Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libgssapi_krb5.so.2Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.Loaded symbols for /lib64/libkrb5.so.3Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libcom_err.so.2Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done.Loaded symbols for /lib64/libk5crypto.so.3Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done.Loaded symbols for /lib64/libkrb5support.so.0Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libkeyutils.so.1Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libresolv.so.2Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libselinux.so.1Reading symbols from /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.soReading symbols from /opt/couchbase/lib/libv8.so...done.Loaded symbols for /opt/couchbase/lib/libv8.soReading symbols from /opt/couchbase/lib/libicuuc.so.44...done.Loaded symbols for /opt/couchbase/lib/libicuuc.so.44Reading symbols from /opt/couchbase/lib/libicudata.so.44...(no debugging symbols found)...done.Loaded symbols for /opt/couchbase/lib/libicudata.so.44Reading symbols from /opt/couchbase/lib/libicui18n.so.44...done.Loaded symbols for /opt/couchbase/lib/libicui18n.so.44Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libcrypt.so.1Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done.Loaded symbols for /usr/lib64/libstdc++.so.6Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libgcc_s.so.1Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.Loaded symbols for /lib64/libfreebl3.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.soCore was generated by `/opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp -A 16 -sbt u -P 327680 -K tru'.Program terminated with signal 11, Segmentation fault.#0 0x00007fe075b6d1ab in memcpy () from /lib64/libc.so.6Thread 28 (Thread 0x7fe072143700 (LWP 7988)):#0 mseg_dealloc (atype=11, seg=0x7fe01e7b1000, size=262144, opt=0x2207f50) at sys/common/erl_mseg.c:765#1 0x000000000054ab21 in erts_mseg_dealloc_opt (atype=11, seg=0x7fe01e7b1000, size=262144, opt=0x2207f50) at sys/common/erl_mseg.c:1384#2 0x0000000000436d39 in alcu_mseg_dealloc (allctr=0x2207ec0, blk=<value optimized out>) at beam/erl_alloc_util.c:517#3 destroy_carrier (allctr=0x2207ec0, blk=<value optimized out>) at beam/erl_alloc_util.c:1796#4 0x000000000043833f in do_erts_alcu_free (type=<value optimized out>, unused=<value optimized out>, p=0x7fe01e7b1030) at beam/erl_alloc_util.c:2898#5 erts_alcu_free_thr_pref (type=<value optimized out>, unused=<value optimized out>, p=0x7fe01e7b1030) at beam/erl_alloc_util.c:2949#6 0x000000000053cde2 in destroy (s=0x2219b80) at drivers/common/gzio.c:442#7 0x000000000054e325 in file_stop (e=<value optimized out>) at drivers/common/efile_drv.c:698#8 0x000000000047fef3 in terminate_port (prt=0x7fe073d42228) at beam/io.c:1849#9 0x0000000000482908 in erts_do_exit_port (p=0x7fe073d42228, from=<value optimized out>, reason=<value optimized out>) at beam/io.c:2059#10 0x0000000000497143 in doit_exit_link (lnk=0x7fe0178bed20, vpcontext=<value optimized out>) at beam/erl_process.c:7256#11 0x00000000004fab87 in erts_sweep_links (root=<value optimized out>, doit=0x496e90 <doit_exit_link>, context=0x7fe072142bd0) at beam/erl_monitors.c:859#12 0x0000000000498a05 in continue_exit_process (p=0x7fe0178be9e0, pix_lock=<value optimized out>) at beam/erl_process.c:7646#13 0x000000000051db94 in terminate_proc (c_p=0x7fe0178be9e0, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#14 handle_error (c_p=0x7fe0178be9e0, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#15 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#16 0x000000000048d496 in sched_thread_func (vesdp=0x7fe074c4b140) at beam/erl_process.c:3782#17 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b50) at pthread/ethread.c:106#18 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#19 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 27 (Thread 0x7fe01f3ac700 (LWP 7995)):#0 0x00007fe07608dd2d in nanosleep () from /lib64/libpthread.so.0#1 0x00007fe01f3b13da in terminatorLoop (args=<value optimized out>) at mapreduce_nif.cc:450#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fe072b43ae0) at pthread/ethread.c:106#3 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 26 (Thread 0x7fe0756c1700 (LWP 7966)):#0 0x00007fe07608d54d in read () from /lib64/libpthread.so.0#1 0x000000000053810e in signal_dispatcher_thread_func (unused=<value optimized out>) at sys/unix/sys.c:2906#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12a70) at pthread/ethread.c:106#3 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 25 (Thread 0x7fe0735ad700 (LWP 7984)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c750) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c750) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c750) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 24 (Thread 0x7fe076d5e700 (LWP 7965)):#0 0x00007fe075bc4d03 in select () from /lib64/libc.so.6#1 0x0000000000537ba0 in erts_sys_main_thread () at sys/unix/sys.c:3049#2 0x0000000000444761 in erl_start (argc=54, argv=<value optimized out>) at beam/erl_init.c:1441#3 0x0000000000426219 in main (argc=<value optimized out>, argv=<value optimized out>) at sys/unix/erl_main.c:29Thread 23 (Thread 0x7fe073545700 (LWP 7986)):#0 0x00007fe0760870ad in pthread_join () from /lib64/libpthread.so.0#1 0x00007fe021c85887 in v8::internal::ContextSwitcher::StopPreemption () at src/v8threads.cc:445#2 0x00007fe01f3b38b3 in destroyContext (ctx=0x7fe013199030) at mapreduce.cc:345#3 0x00000000005138fb in nif_resource_dtor (bin=<value optimized out>) at beam/erl_nif.c:1217#4 0x000000000048904b in erts_bin_free (offheap=<value optimized out>) at beam/erl_binary.h:303#5 erts_cleanup_offheap (offheap=<value optimized out>) at beam/erl_message.c:174#6 0x0000000000498a81 in delete_process (p=0x7fe0178b36a0, pix_lock=<value optimized out>) at beam/erl_process.c:6621#7 continue_exit_process (p=0x7fe0178b36a0, pix_lock=<value optimized out>) at beam/erl_process.c:7663#8 0x000000000051db94 in terminate_proc (c_p=0x7fe0178b36a0, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#9 handle_error (c_p=0x7fe0178b36a0, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#10 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#11 0x000000000048d496 in sched_thread_func (vesdp=0x7fe074c2efc0) at beam/erl_process.c:3782#12 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b50) at pthread/ethread.c:106#13 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#14 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 22 (Thread 0x7fe07358b700 (LWP 7985)):#0 0x00007fe07608e09d in waitpid () from /lib64/libpthread.so.0#1 0x000000000053906f in child_waiter (unused=<value optimized out>) at sys/unix/sys.c:2821#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b30) at pthread/ethread.c:106#3 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 21 (Thread 0x7fe076b79700 (LWP 27722)):#0 0x00007fe07608c720 in sem_wait () from /lib64/libpthread.so.0#1 0x00007fe021c0518d in v8::internal::LinuxSemaphore::Wait (this=0x7fe06c109b20) at src/platform-linux.cc:869#2 0x00007fe021c1c45c in v8::internal::RuntimeProfiler::WaitForSomeIsolateToEnterJS () at src/runtime-profiler.cc:386#3 0x00007fe021c06136 in v8::internal::SignalSender::Run (this=0x7fe028039720) at src/platform-linux.cc:1126#4 0x00007fe021c05580 in v8::internal::ThreadEntry (arg=0x7fe028039720) at src/platform-linux.cc:745#5 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#6 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 20 (Thread 0x7fe0749ad700 (LWP 7968)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x0000000000462ad9 in erts_cnd_wait (unused=<value optimized out>) at beam/erl_threads.h:697#3 erts_smp_cnd_wait (unused=<value optimized out>) at beam/erl_smp.h:472#4 sys_msg_dispatcher_func (unused=<value optimized out>) at beam/erl_trace.c:3238#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b80) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 19 (Thread 0x7fe07369b700 (LWP 7977)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c408) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c408) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c408) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 18 (Thread 0x7fe074cc0700 (LWP 7967)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x000000000046a1cf in erts_cnd_wait (unused=<value optimized out>) at beam/erl_threads.h:697#3 erts_smp_cnd_wait (unused=<value optimized out>) at beam/erl_smp.h:472#4 emergency_watchdog (unused=<value optimized out>) at beam/utils.c:3945#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b80) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 17 (Thread 0x7fe0736bd700 (LWP 7976)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c390) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c390) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c390) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 16 (Thread 0x7fe0735cf700 (LWP 7983)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c6d8) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c6d8) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c6d8) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 15 (Thread 0x7fe014d1e700 (LWP 27725)):#0 0x00007fe075b903bd in nanosleep () from /lib64/libc.so.6#1 0x00007fe075bc5574 in usleep () from /lib64/libc.so.6#2 0x00007fe021c85848 in v8::internal::ContextSwitcher::Run (this=0x7fe024134460) at src/v8threads.cc:457#3 0x00007fe021c05580 in v8::internal::ThreadEntry (arg=0x7fe024134460) at src/platform-linux.cc:745#4 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#5 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 14 (Thread 0x7fe073657700 (LWP 7979)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c4f8) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c4f8) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c4f8) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 13 (Thread 0x7fe0736df700 (LWP 7975)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c318) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c318) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c318) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 12 (Thread 0x7fe073679700 (LWP 7978)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c480) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c480) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c480) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 11 (Thread 0x7fe0735f1700 (LWP 7982)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c660) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c660) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c660) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 10 (Thread 0x7fe073723700 (LWP 7973)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c228) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c228) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c228) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 9 (Thread 0x7fe073789700 (LWP 7970)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c0c0) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c0c0) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c0c0) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 8 (Thread 0x7fe071742700 (LWP 7989)):#0 0x00007fe07608d1e5 in __lll_unlock_wake () from /lib64/libpthread.so.0#1 0x00007fe0760896f7 in _L_unlock_657 () from /lib64/libpthread.so.0#2 0x00007fe07608965f in pthread_mutex_unlock () from /lib64/libpthread.so.0#3 0x000000000049ceb5 in ethr_event_set (p=<value optimized out>, trnsfr_lcks=<value optimized out>, pix_lock=0x86fb40, unlock=1) at ../include/internal/pthread/ethr_event.h:111#4 erts_tse_set (p=<value optimized out>, trnsfr_lcks=<value optimized out>, pix_lock=0x86fb40, unlock=1) at beam/erl_threads.h:1795#5 transfer_locks (p=<value optimized out>, trnsfr_lcks=<value optimized out>, pix_lock=0x86fb40, unlock=1) at beam/erl_process_lock.c:417#6 0x0000000000497527 in doit_exit_monitor (mon=0x7fe01789c4b8, vpcontext=0x7fe071741c10) at beam/erl_process.c:7192#7 0x00000000004faad7 in erts_sweep_monitors (root=<value optimized out>, doit=0x497280 <doit_exit_monitor>, context=0x7fe071741c10) at beam/erl_monitors.c:823#8 0x0000000000498a23 in continue_exit_process (p=0x7fe01802d670, pix_lock=<value optimized out>) at beam/erl_process.c:7653#9 0x000000000051db94 in terminate_proc (c_p=0x7fe01802d670, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#10 handle_error (c_p=0x7fe01802d670, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#11 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#12 0x000000000048d496 in sched_thread_func (vesdp=0x7fe074c59200) at beam/erl_process.c:3782#13 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b50) at pthread/ethread.c:106#14 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#15 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 7 (Thread 0x7fe073613700 (LWP 7981)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c5e8) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c5e8) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c5e8) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 6 (Thread 0x7fe072b44700 (LWP 7987)):#0 0x0000000000488feb in erts_bin_free (offheap=<value optimized out>) at beam/erl_binary.h:302#1 erts_cleanup_offheap (offheap=<value optimized out>) at beam/erl_message.c:174#2 0x0000000000498a81 in delete_process (p=0x7fe0178b7c90, pix_lock=<value optimized out>) at beam/erl_process.c:6621#3 continue_exit_process (p=0x7fe0178b7c90, pix_lock=<value optimized out>) at beam/erl_process.c:7663#4 0x000000000051db94 in terminate_proc (c_p=0x7fe0178b7c90, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#5 handle_error (c_p=0x7fe0178b7c90, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#6 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#7 0x000000000048d496 in sched_thread_func (vesdp=0x7fe074c3d080) at beam/erl_process.c:3782#8 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b50) at pthread/ethread.c:106#9 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#10 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 5 (Thread 0x7fe073745700 (LWP 7972)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c1b0) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c1b0) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c1b0) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 4 (Thread 0x7fe073635700 (LWP 7980)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c570) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c570) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c570) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 3 (Thread 0x7fe073701700 (LWP 7974)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c2a0) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c2a0) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c2a0) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 2 (Thread 0x7fe073767700 (LWP 7971)):#0 0x00007fe07608a43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fe073f1c138) at beam/erl_threads.h:697#3 async_get (arg=0x7fe073f1c138) at beam/erl_async.c:207#4 async_main (arg=0x7fe073f1c138) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#6 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fe075bcc11d in clone () from /lib64/libc.so.6Thread 1 (Thread 0x7fe0737ab700 (LWP 7969)):#0 0x00007fe075b6d1ab in memcpy () from /lib64/libc.so.6#1 0x000000000053c7fe in erts_gzread (file=0x2225af8, buf=<value optimized out>, len=16384) at drivers/common/gzio.c:473#2 0x000000000054e208 in invoke_read (data=0x2221ff0) at drivers/common/efile_drv.c:966#3 0x00000000004e8f55 in async_main (arg=0x7fe073f1c048) at beam/erl_async.c:281#4 0x0000000000584d89 in thr_wrapper (vtwd=0x7fff86e12b10) at pthread/ethread.c:106#5 0x00007fe076086851 in start_thread () from /lib64/libpthread.so.0#6 0x00007fe075bcc11d in clone () from /lib64/libc.so.6--------------------------------------------------------------------------------Module information:/lib64/libutil.so.1:lrwxrwxrwx. 1 root root 15 Oct 5 13:43 /lib64/libutil.so.1 -> libutil-2.12.soaae98c3065bf167a8e867e830d1c8817 /lib64/libutil.so.1/lib64/libdl.so.2:lrwxrwxrwx. 1 root root 13 Oct 5 13:43 /lib64/libdl.so.2 -> libdl-2.12.so88e56138964e07d30dfc44e0287a0f69 /lib64/libdl.so.2/lib64/libm.so.6:lrwxrwxrwx. 1 root root 12 Oct 5 13:43 /lib64/libm.so.6 -> libm-2.12.soe9a818815675dbfd19b291272de00f44 /lib64/libm.so.6/lib64/libncurses.so.5:lrwxrwxrwx. 1 root root 17 Sep 20 13:34 /lib64/libncurses.so.5 -> libncurses.so.5.7c7f1fe3252dd7381ab04c84d8889f352 /lib64/libncurses.so.5/lib64/libpthread.so.0:lrwxrwxrwx. 1 root root 18 Oct 5 13:43 /lib64/libpthread.so.0 -> libpthread-2.12.sofa8af7a3f05f54ac461c40ca500147b7 /lib64/libpthread.so.0/lib64/librt.so.1:lrwxrwxrwx. 1 root root 13 Oct 5 13:43 /lib64/librt.so.1 -> librt-2.12.sof2cedf89cd9f49580afdb9f91b434b97 /lib64/librt.so.1/lib64/libc.so.6:lrwxrwxrwx. 1 root root 12 Oct 5 13:43 /lib64/libc.so.6 -> libc-2.12.so8ec284d3097552c0d5e9630b279cfcc4 /lib64/libc.so.6/lib64/ld-linux-x86-64.so.2:lrwxrwxrwx. 1 root root 10 Oct 5 13:43 /lib64/ld-linux-x86-64.so.2 -> ld-2.12.soee467ccc70564df4de2d2e0eac104fa5 /lib64/ld-linux-x86-64.so.2/lib64/libtinfo.so.5:lrwxrwxrwx. 1 root root 15 Sep 20 13:34 /lib64/libtinfo.so.5 -> libtinfo.so.5.7183199f39b8490ee65db7f12ca553db3 /lib64/libtinfo.so.5/opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so:-rwxr-xr-x. 1 bin bin 105367 Oct 10 15:10 /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so6e54a6b6dcaa9d8ad139be11e0d4b6a4 /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so/usr/lib64/libcrypto.so.6:lrwxrwxrwx. 1 root root 19 Sep 20 14:51 /usr/lib64/libcrypto.so.6 -> libcrypto.so.0.9.8e78dc020d4c5906ef8d38fa8772745aec /usr/lib64/libcrypto.so.6/usr/lib64/libssl.so.6:lrwxrwxrwx. 1 root root 16 Sep 20 14:51 /usr/lib64/libssl.so.6 -> libssl.so.0.9.8ea02f2de91ba1e75330034bd0d52ca968 /usr/lib64/libssl.so.6/lib64/libz.so.1:lrwxrwxrwx. 1 root root 13 Sep 20 13:34 /lib64/libz.so.1 -> libz.so.1.2.32437e7ab120ea1ad0b8496301dfd08f1 /lib64/libz.so.1/lib64/libgssapi_krb5.so.2:lrwxrwxrwx. 1 root root 21 Sep 20 13:34 /lib64/libgssapi_krb5.so.2 -> libgssapi_krb5.so.2.24a07896ad5cf7e42559312cc6a645f90 /lib64/libgssapi_krb5.so.2/lib64/libkrb5.so.3:lrwxrwxrwx. 1 root root 14 Sep 20 13:34 /lib64/libkrb5.so.3 -> libkrb5.so.3.3bd4bc41781b2ad8160f2ef6f1c6b69c7 /lib64/libkrb5.so.3/lib64/libcom_err.so.2:lrwxrwxrwx. 1 root root 17 Sep 20 13:34 /lib64/libcom_err.so.2 -> libcom_err.so.2.1ec4544049cb88f5284f7ba57fe22087e /lib64/libcom_err.so.2/lib64/libk5crypto.so.3:lrwxrwxrwx. 1 root root 18 Sep 20 13:34 /lib64/libk5crypto.so.3 -> libk5crypto.so.3.1656a9cc782142aeab1f0f659b3d39734 /lib64/libk5crypto.so.3/lib64/libkrb5support.so.0:lrwxrwxrwx. 1 root root 21 Sep 20 13:34 /lib64/libkrb5support.so.0 -> libkrb5support.so.0.17149fe73335280ee240e5b66920ce013 /lib64/libkrb5support.so.0/lib64/libkeyutils.so.1:lrwxrwxrwx. 1 root root 18 Sep 20 13:34 /lib64/libkeyutils.so.1 -> libkeyutils.so.1.38002611a776e9de0ccf7832721b86d95 /lib64/libkeyutils.so.1/lib64/libresolv.so.2:lrwxrwxrwx. 1 root root 17 Oct 5 13:43 /lib64/libresolv.so.2 -> libresolv-2.12.so80455b98f518df830373b2db62c4e19c /lib64/libresolv.so.2/lib64/libselinux.so.1:-rwxr-xr-x. 1 root root 122040 Dec 7 2011 /lib64/libselinux.so.1d498609d23563a027254e14d299375ab /lib64/libselinux.so.1/opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.so:-rwxr-xr-x. 1 bin bin 191090 Oct 10 15:19 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.sod558f5aa32038379008c1d9725c68397 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.so/opt/couchbase/lib/libv8.so:-rwxr-xr-x. 1 bin bin 85345184 Oct 10 15:14 /opt/couchbase/lib/libv8.sob6c7f7cd39d687a268b6b7c5272d36cd /opt/couchbase/lib/libv8.so/opt/couchbase/lib/libicuuc.so.44:lrwxrwxrwx. 1 bin bin 16 Oct 10 15:52 /opt/couchbase/lib/libicuuc.so.44 -> libicuuc.so.44.042f69b9a8effcfd45a1c70aea9dcb66a /opt/couchbase/lib/libicuuc.so.44/opt/couchbase/lib/libicudata.so.44:lrwxrwxrwx. 1 bin bin 18 Oct 10 15:52 /opt/couchbase/lib/libicudata.so.44 -> libicudata.so.44.0797015967c9d62eb4ce64e861e45f500 /opt/couchbase/lib/libicudata.so.44/opt/couchbase/lib/libicui18n.so.44:lrwxrwxrwx. 1 bin bin 18 Oct 10 15:52 /opt/couchbase/lib/libicui18n.so.44 -> libicui18n.so.44.0906d4788c04365237419e085f34cc982 /opt/couchbase/lib/libicui18n.so.44/lib64/libcrypt.so.1:lrwxrwxrwx. 1 root root 16 Oct 5 13:43 /lib64/libcrypt.so.1 -> libcrypt-2.12.so8100ffde3f3465c0086c61e50ed06c14 /lib64/libcrypt.so.1/usr/lib64/libstdc++.so.6:lrwxrwxrwx. 1 root root 19 Sep 20 13:34 /usr/lib64/libstdc++.so.6 -> libstdc++.so.6.0.135b6a438cd9d564af737b1d3f14271ab3 /usr/lib64/libstdc++.so.6/lib64/libgcc_s.so.1:lrwxrwxrwx. 1 root root 28 Oct 5 13:43 /lib64/libgcc_s.so.1 -> libgcc_s-4.4.6-20120305.so.1fa27e0c9fbddbeece6d909b9f2c088e3 /lib64/libgcc_s.so.1/lib64/libfreebl3.so:-rwxr-xr-x. 1 root root 383504 Dec 7 2011 /lib64/libfreebl3.so9fe8811e93e403a2681f99156170b5f5 /lib64/libfreebl3.so/opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so:-rwxr-xr-x. 1 bin bin 411836 Oct 10 15:20 /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so1b715bac077be8bf2e638d3c1cb26061 /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so/opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so:-rwxr-xr-x. 1 bin bin 673632 Oct 10 15:19 /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so5af42359c199dd26548c0383e70d4933 /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so/opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so:-rwxr-xr-x. 1 bin bin 344520 Oct 10 15:20 /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so5c4a39422733f2cefa699dd14bc80500 /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so/opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so:-rwxr-xr-x. 1 bin bin 577011 Oct 10 15:19 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so0ed1f41631963e85232351d1ae3d9852 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so