No title Revision 643230653934 (Thu Jun 29 2017 at 19:10) - Diff Link to this snippet: https://friendpaste.com/59jNoNCWLSrPOyTCQKWoCO Embed: manni perldoc borland colorful default murphy trac fruity autumn bw emacs pastie friendly Show line numbers Wrap lines 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533[root@localhost ~]# more /data/core.beam.smp.16438.log Basic crash dump analysis of /data/core.beam.smp.16438.Please send the file to support@couchbase.com--------------------------------------------------------------------------------File information:-rwxr-xr-x. 1 bin bin 12181270 Oct 11 15:45 /opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smpf76dbd0c5dddd3731f18ee215769087d /opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp-rw-------. 1 couchbase couchbase 453914624 Oct 11 23:42 /data/core.beam.smp.1643806894bec152c3eb7f2407bb595d8c135 /data/core.beam.smp.16438version1.4.4_598_g8587311version1.4.4_598_g85873111.4.4_209_g7b9e75fversion1.4.4_598_g8587311version1.4.4_598_g8587311--------------------------------------------------------------------------------Core file callstacks:GNU gdb (GDB) Red Hat Enterprise Linux (7.2-56.el6)Copyright (C) 2010 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>This is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law. Type "show copying"and "show warranty" for details.This GDB was configured as "x86_64-redhat-linux-gnu".For bug reporting instructions, please see:<http://www.gnu.org/software/gdb/bugs/>...Reading symbols from /opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp...done.[New Thread 16460][New Thread 5467][New Thread 5561][New Thread 16468][New Thread 16438][New Thread 16442][New Thread 5559][New Thread 16456][New Thread 16443][New Thread 16439][New Thread 16457][New Thread 16444][New Thread 16453][New Thread 16440][New Thread 16462][New Thread 16461][New Thread 16446][New Thread 16450][New Thread 16451][New Thread 16452][New Thread 16454][New Thread 16458][New Thread 16455][New Thread 16449][New Thread 16459][New Thread 16448][New Thread 16447][New Thread 16445][New Thread 16441]Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libutil.so.1Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libdl.so.2Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.Loaded symbols for /lib64/libm.so.6Reading symbols from /lib64/libncurses.so.5...(no debugging symbols found)...done.Loaded symbols for /lib64/libncurses.so.5Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.[Thread debugging using libthread_db enabled]Loaded symbols for /lib64/libpthread.so.0Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/librt.so.1Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.Loaded symbols for /lib64/libc.so.6Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/ld-linux-x86-64.so.2Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done.Loaded symbols for /lib64/libtinfo.so.5Reading symbols from /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so...done.Loaded symbols for /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.soReading symbols from /usr/lib64/libcrypto.so.6...(no debugging symbols found)...done.Loaded symbols for /usr/lib64/libcrypto.so.6Reading symbols from /usr/lib64/libssl.so.6...(no debugging symbols found)...done.Loaded symbols for /usr/lib64/libssl.so.6Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libz.so.1Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libgssapi_krb5.so.2Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.Loaded symbols for /lib64/libkrb5.so.3Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libcom_err.so.2Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done.Loaded symbols for /lib64/libk5crypto.so.3Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done.Loaded symbols for /lib64/libkrb5support.so.0Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libkeyutils.so.1Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libresolv.so.2Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libselinux.so.1Reading symbols from /opt/couchbase/lib/libv8.so...done.Loaded symbols for /opt/couchbase/lib/libv8.soReading symbols from /opt/couchbase/lib/libicuuc.so.44...done.Loaded symbols for /opt/couchbase/lib/libicuuc.so.44Reading symbols from /opt/couchbase/lib/libicudata.so.44...(no debugging symbols found)...done.Loaded symbols for /opt/couchbase/lib/libicudata.so.44Reading symbols from /opt/couchbase/lib/libicui18n.so.44...done.Loaded symbols for /opt/couchbase/lib/libicui18n.so.44Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libcrypt.so.1Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done.Loaded symbols for /usr/lib64/libstdc++.so.6Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libgcc_s.so.1Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.Loaded symbols for /lib64/libfreebl3.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/couch_view_parser-1.0/priv/couch_view_parser_nif.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/couch_view_parser-1.0/priv/couch_view_parser_nif.soReading symbols from /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.so...done.Loaded symbols for /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.soCore was generated by `/opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp -A 16 -sbt u -P 327680 -K tru'.Program terminated with signal 11, Segmentation fault.#0 0x000000000043ec6b in tree_delete (allctr=<value optimized out>, del=0x264ca48, flags=<value optimized out>) at beam/erl_bestfit_alloc.c:435435 beam/erl_bestfit_alloc.c: No such file or directory. in beam/erl_bestfit_alloc.cThread 29 (Thread 0x7fd722101700 (LWP 16441)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x0000000000462ad9 in erts_cnd_wait (unused=<value optimized out>) at beam/erl_threads.h:697#3 erts_smp_cnd_wait (unused=<value optimized out>) at beam/erl_smp.h:472#4 sys_msg_dispatcher_func (unused=<value optimized out>) at beam/erl_trace.c:3238#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc246320) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 28 (Thread 0x7fd720e99700 (LWP 16445)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd7216701b0) at beam/erl_threads.h:697#3 async_get (arg=0x7fd7216701b0) at beam/erl_async.c:207#4 async_main (arg=0x7fd7216701b0) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 27 (Thread 0x7fd720e55700 (LWP 16447)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd7216702a0) at beam/erl_threads.h:697#3 async_get (arg=0x7fd7216702a0) at beam/erl_async.c:207#4 async_main (arg=0x7fd7216702a0) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 26 (Thread 0x7fd720e33700 (LWP 16448)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670318) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670318) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670318) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 25 (Thread 0x7fd720c99700 (LWP 16459)):#0 0x00007fd7237db0ad in pthread_join () from /lib64/libpthread.so.0#1 0x00007fd6cf5d6887 in v8::internal::ContextSwitcher::StopPreemption () at src/v8threads.cc:445#2 0x00007fd6ccd048b3 in destroyContext (ctx=0x2618518) at mapreduce.cc:345#3 0x00000000005138fb in nif_resource_dtor (bin=<value optimized out>) at beam/erl_nif.c:1217#4 0x000000000048904b in erts_bin_free (offheap=<value optimized out>) at beam/erl_binary.h:303#5 erts_cleanup_offheap (offheap=<value optimized out>) at beam/erl_message.c:174#6 0x0000000000498a81 in delete_process (p=0x7fd6c74be460, pix_lock=<value optimized out>) at beam/erl_process.c:6621#7 continue_exit_process (p=0x7fd6c74be460, pix_lock=<value optimized out>) at beam/erl_process.c:7663#8 0x000000000051db94 in terminate_proc (c_p=0x7fd6c74be460, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#9 handle_error (c_p=0x7fd6c74be460, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#10 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#11 0x000000000048d496 in sched_thread_func (vesdp=0x7fd722382fc0) at beam/erl_process.c:3782#12 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462f0) at pthread/ethread.c:106#13 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#14 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 24 (Thread 0x7fd720e11700 (LWP 16449)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670390) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670390) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670390) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 23 (Thread 0x7fd720d45700 (LWP 16455)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670660) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670660) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670660) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 22 (Thread 0x7fd720cdf700 (LWP 16458)):#0 0x00007fd7237e209d in waitpid () from /lib64/libpthread.so.0#1 0x000000000053906f in child_waiter (unused=<value optimized out>) at sys/unix/sys.c:2821#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462d0) at pthread/ethread.c:106#3 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 21 (Thread 0x7fd720d67700 (LWP 16454)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd7216705e8) at beam/erl_threads.h:697#3 async_get (arg=0x7fd7216705e8) at beam/erl_async.c:207#4 async_main (arg=0x7fd7216705e8) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 20 (Thread 0x7fd720dab700 (LWP 16452)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd7216704f8) at beam/erl_threads.h:697#3 async_get (arg=0x7fd7216704f8) at beam/erl_async.c:207#4 async_main (arg=0x7fd7216704f8) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 19 (Thread 0x7fd720dcd700 (LWP 16451)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670480) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670480) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670480) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 18 (Thread 0x7fd720def700 (LWP 16450)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670408) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670408) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670408) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 17 (Thread 0x7fd720e77700 (LWP 16446)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670228) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670228) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670228) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 16 (Thread 0x7fd71f897700 (LWP 16461)):#0 bf_link_free_block (allctr=<value optimized out>, block=0x7fd6b64a2018, flags=<value optimized out>) at beam/erl_bestfit_alloc.c:719#1 0x000000000043833f in do_erts_alcu_free (type=<value optimized out>, unused=<value optimized out>, p=0x7fd6b64a2028) at beam/erl_alloc_util.c:2898#2 erts_alcu_free_thr_pref (type=<value optimized out>, unused=<value optimized out>, p=0x7fd6b64a2028) at beam/erl_alloc_util.c:2949#3 0x0000000000489022 in erts_free (offheap=<value optimized out>) at beam/erl_alloc.h:226#4 erts_bin_free (offheap=<value optimized out>) at beam/erl_binary.h:307#5 erts_cleanup_offheap (offheap=<value optimized out>) at beam/erl_message.c:174#6 0x0000000000498a81 in delete_process (p=0x7fd6cac33938, pix_lock=<value optimized out>) at beam/erl_process.c:6621#7 continue_exit_process (p=0x7fd6cac33938, pix_lock=<value optimized out>) at beam/erl_process.c:7663#8 0x000000000051db94 in terminate_proc (c_p=0x7fd6cac33938, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#9 handle_error (c_p=0x7fd6cac33938, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#10 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#11 0x000000000048d496 in sched_thread_func (vesdp=0x7fd72239f140) at beam/erl_process.c:3782#12 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462f0) at pthread/ethread.c:106#13 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#14 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 15 (Thread 0x7fd71ee96700 (LWP 16462)):#0 process_main () at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:876#1 0x000000000048d496 in sched_thread_func (vesdp=0x7fd7223ad200) at beam/erl_process.c:3782#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462f0) at pthread/ethread.c:106#3 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 14 (Thread 0x7fd722414700 (LWP 16440)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x000000000046a1cf in erts_cnd_wait (unused=<value optimized out>) at beam/erl_threads.h:697#3 erts_smp_cnd_wait (unused=<value optimized out>) at beam/erl_smp.h:472#4 emergency_watchdog (unused=<value optimized out>) at beam/utils.c:3945#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc246320) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 13 (Thread 0x7fd720d89700 (LWP 16453)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670570) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670570) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670570) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 12 (Thread 0x7fd720ebb700 (LWP 16444)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670138) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670138) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670138) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 11 (Thread 0x7fd720d01700 (LWP 16457)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670750) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670750) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670750) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 10 (Thread 0x7fd722e15700 (LWP 16439)):#0 0x00007fd7237e154d in read () from /lib64/libpthread.so.0#1 0x000000000053810e in signal_dispatcher_thread_func (unused=<value optimized out>) at sys/unix/sys.c:2906#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc246210) at pthread/ethread.c:106#3 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 9 (Thread 0x7fd720edd700 (LWP 16443)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd7216700c0) at beam/erl_threads.h:697#3 async_get (arg=0x7fd7216700c0) at beam/erl_async.c:207#4 async_main (arg=0x7fd7216700c0) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 8 (Thread 0x7fd720d23700 (LWP 16456)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd7216706d8) at beam/erl_threads.h:697#3 async_get (arg=0x7fd7216706d8) at beam/erl_async.c:207#4 async_main (arg=0x7fd7216706d8) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 7 (Thread 0x7fd6b26bc700 (LWP 5559)):#0 0x00007fd7232e43bd in nanosleep () from /lib64/libc.so.6#1 0x00007fd723319574 in usleep () from /lib64/libc.so.6#2 0x00007fd6cf5d6848 in v8::internal::ContextSwitcher::Run (this=0x7fd6d40cd310) at src/v8threads.cc:457#3 0x00007fd6cf556580 in v8::internal::ThreadEntry (arg=0x7fd6d40cd310) at src/platform-linux.cc:745#4 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#5 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 6 (Thread 0x7fd720eff700 (LWP 16442)):#0 0x00007fd7237de43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0#1 0x0000000000582e29 in ethr_cond_wait (cnd=<value optimized out>, mtx=<value optimized out>) at common/ethr_mutex.c:1350#2 0x00000000004e8efb in erts_cnd_wait (arg=0x7fd721670048) at beam/erl_threads.h:697#3 async_get (arg=0x7fd721670048) at beam/erl_async.c:207#4 async_main (arg=0x7fd721670048) at beam/erl_async.c:274#5 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462b0) at pthread/ethread.c:106#6 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#7 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 5 (Thread 0x7fd7244b2700 (LWP 16438)):#0 0x00007fd723318d03 in select () from /lib64/libc.so.6#1 0x0000000000537ba0 in erts_sys_main_thread () at sys/unix/sys.c:3049#2 0x0000000000444761 in erl_start (argc=54, argv=<value optimized out>) at beam/erl_init.c:1441#3 0x0000000000426219 in main (argc=<value optimized out>, argv=<value optimized out>) at sys/unix/erl_main.c:29Thread 4 (Thread 0x7fd6cccfd700 (LWP 16468)):#0 0x00007fd7237e1d2d in nanosleep () from /lib64/libpthread.so.0#1 0x00007fd6ccd023da in terminatorLoop (args=<value optimized out>) at mapreduce_nif.cc:450#2 0x0000000000584d89 in thr_wrapper (vtwd=0x7fd71ee95ae0) at pthread/ethread.c:106#3 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#4 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 3 (Thread 0x7fd6c6070700 (LWP 5561)):#0 0x00007fd7232e43bd in nanosleep () from /lib64/libc.so.6#1 0x00007fd723319574 in usleep () from /lib64/libc.so.6#2 0x00007fd6cf5d6848 in v8::internal::ContextSwitcher::Run (this=0x7fd6c00f7400) at src/v8threads.cc:457#3 0x00007fd6cf556580 in v8::internal::ThreadEntry (arg=0x7fd6c00f7400) at src/platform-linux.cc:745#4 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#5 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 2 (Thread 0x7fd71dc50700 (LWP 5467)):#0 0x00007fd7237e0720 in sem_wait () from /lib64/libpthread.so.0#1 0x00007fd6cf55618d in v8::internal::LinuxSemaphore::Wait (this=0x7fd6d4009b10) at src/platform-linux.cc:869#2 0x00007fd6cf56d45c in v8::internal::RuntimeProfiler::WaitForSomeIsolateToEnterJS () at src/runtime-profiler.cc:386#3 0x00007fd6cf557136 in v8::internal::SignalSender::Run (this=0x7fd6d40726c0) at src/platform-linux.cc:1126#4 0x00007fd6cf556580 in v8::internal::ThreadEntry (arg=0x7fd6d40726c0) at src/platform-linux.cc:745#5 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#6 0x00007fd72332011d in clone () from /lib64/libc.so.6Thread 1 (Thread 0x7fd720298700 (LWP 16460)):#0 0x000000000043ec6b in tree_delete (allctr=<value optimized out>, del=0x264ca48, flags=<value optimized out>) at beam/erl_bestfit_alloc.c:435#1 0x0000000000437091 in mbc_free (allctr=0x26412c0, p=<value optimized out>) at beam/erl_alloc_util.c:867#2 0x000000000043833f in do_erts_alcu_free (type=<value optimized out>, unused=<value optimized out>, p=0x264cb90) at beam/erl_alloc_util.c:2898#3 erts_alcu_free_thr_pref (type=<value optimized out>, unused=<value optimized out>, p=0x264cb90) at beam/erl_alloc_util.c:2949#4 0x000000000053cde2 in destroy (s=0x264c988) at drivers/common/gzio.c:442#5 0x000000000054e325 in file_stop (e=<value optimized out>) at drivers/common/efile_drv.c:698#6 0x000000000047fef3 in terminate_port (prt=0x7fd72112dfd8) at beam/io.c:1849#7 0x0000000000482908 in erts_do_exit_port (p=0x7fd72112dfd8, from=<value optimized out>, reason=<value optimized out>) at beam/io.c:2059#8 0x0000000000497143 in doit_exit_link (lnk=0x7fd6cac19038, vpcontext=<value optimized out>) at beam/erl_process.c:7256#9 0x00000000004fab87 in erts_sweep_links (root=<value optimized out>, doit=0x496e90 <doit_exit_link>, context=0x7fd720297bd0) at beam/erl_monitors.c:859#10 0x0000000000498a05 in continue_exit_process (p=0x7fd6cb835b58, pix_lock=<value optimized out>) at beam/erl_process.c:7646#11 0x000000000051db94 in terminate_proc (c_p=0x7fd6cb835b58, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5633#12 handle_error (c_p=0x7fd6cb835b58, pc=<value optimized out>, reg=<value optimized out>, bf=<value optimized out>) at beam/beam_emu.c:5508#13 0x0000000000520cdf in process_main () at beam/beam_emu.c:3273#14 0x000000000048d496 in sched_thread_func (vesdp=0x7fd722391080) at beam/erl_process.c:3782#15 0x0000000000584d89 in thr_wrapper (vtwd=0x7fffcc2462f0) at pthread/ethread.c:106#16 0x00007fd7237da851 in start_thread () from /lib64/libpthread.so.0#17 0x00007fd72332011d in clone () from /lib64/libc.so.6--------------------------------------------------------------------------------Module information:/lib64/libutil.so.1:lrwxrwxrwx. 1 root root 15 Oct 5 13:44 /lib64/libutil.so.1 -> libutil-2.12.soaae98c3065bf167a8e867e830d1c8817 /lib64/libutil.so.1/lib64/libdl.so.2:lrwxrwxrwx. 1 root root 13 Oct 5 13:44 /lib64/libdl.so.2 -> libdl-2.12.so88e56138964e07d30dfc44e0287a0f69 /lib64/libdl.so.2/lib64/libm.so.6:lrwxrwxrwx. 1 root root 12 Oct 5 13:44 /lib64/libm.so.6 -> libm-2.12.soe9a818815675dbfd19b291272de00f44 /lib64/libm.so.6/lib64/libncurses.so.5:lrwxrwxrwx. 1 root root 17 Sep 19 17:48 /lib64/libncurses.so.5 -> libncurses.so.5.7c7f1fe3252dd7381ab04c84d8889f352 /lib64/libncurses.so.5/lib64/libpthread.so.0:lrwxrwxrwx. 1 root root 18 Oct 5 13:44 /lib64/libpthread.so.0 -> libpthread-2.12.sofa8af7a3f05f54ac461c40ca500147b7 /lib64/libpthread.so.0/lib64/librt.so.1:lrwxrwxrwx. 1 root root 13 Oct 5 13:44 /lib64/librt.so.1 -> librt-2.12.sof2cedf89cd9f49580afdb9f91b434b97 /lib64/librt.so.1/lib64/libc.so.6:lrwxrwxrwx. 1 root root 12 Oct 5 13:44 /lib64/libc.so.6 -> libc-2.12.so8ec284d3097552c0d5e9630b279cfcc4 /lib64/libc.so.6/lib64/ld-linux-x86-64.so.2:lrwxrwxrwx. 1 root root 10 Oct 5 13:44 /lib64/ld-linux-x86-64.so.2 -> ld-2.12.soee467ccc70564df4de2d2e0eac104fa5 /lib64/ld-linux-x86-64.so.2/lib64/libtinfo.so.5:lrwxrwxrwx. 1 root root 15 Sep 19 17:48 /lib64/libtinfo.so.5 -> libtinfo.so.5.7183199f39b8490ee65db7f12ca553db3 /lib64/libtinfo.so.5/opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so:-rwxr-xr-x. 1 bin bin 105391 Oct 11 15:45 /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.sodc63afef083fc7b813a220cd78794144 /opt/couchbase/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so/usr/lib64/libcrypto.so.6:lrwxrwxrwx. 1 root root 19 Sep 19 19:34 /usr/lib64/libcrypto.so.6 -> libcrypto.so.0.9.8e78dc020d4c5906ef8d38fa8772745aec /usr/lib64/libcrypto.so.6/usr/lib64/libssl.so.6:lrwxrwxrwx. 1 root root 16 Sep 19 19:34 /usr/lib64/libssl.so.6 -> libssl.so.0.9.8ea02f2de91ba1e75330034bd0d52ca968 /usr/lib64/libssl.so.6/lib64/libz.so.1:lrwxrwxrwx. 1 root root 13 Sep 19 17:48 /lib64/libz.so.1 -> libz.so.1.2.32437e7ab120ea1ad0b8496301dfd08f1 /lib64/libz.so.1/lib64/libgssapi_krb5.so.2:lrwxrwxrwx. 1 root root 21 Sep 19 17:48 /lib64/libgssapi_krb5.so.2 -> libgssapi_krb5.so.2.24a07896ad5cf7e42559312cc6a645f90 /lib64/libgssapi_krb5.so.2/lib64/libkrb5.so.3:lrwxrwxrwx. 1 root root 14 Sep 19 17:48 /lib64/libkrb5.so.3 -> libkrb5.so.3.3bd4bc41781b2ad8160f2ef6f1c6b69c7 /lib64/libkrb5.so.3/lib64/libcom_err.so.2:lrwxrwxrwx. 1 root root 17 Sep 19 17:48 /lib64/libcom_err.so.2 -> libcom_err.so.2.1ec4544049cb88f5284f7ba57fe22087e /lib64/libcom_err.so.2/lib64/libk5crypto.so.3:lrwxrwxrwx. 1 root root 18 Sep 19 17:48 /lib64/libk5crypto.so.3 -> libk5crypto.so.3.1656a9cc782142aeab1f0f659b3d39734 /lib64/libk5crypto.so.3/lib64/libkrb5support.so.0:lrwxrwxrwx. 1 root root 21 Sep 19 17:48 /lib64/libkrb5support.so.0 -> libkrb5support.so.0.17149fe73335280ee240e5b66920ce013 /lib64/libkrb5support.so.0/lib64/libkeyutils.so.1:lrwxrwxrwx. 1 root root 18 Sep 19 17:48 /lib64/libkeyutils.so.1 -> libkeyutils.so.1.38002611a776e9de0ccf7832721b86d95 /lib64/libkeyutils.so.1/lib64/libresolv.so.2:lrwxrwxrwx. 1 root root 17 Oct 5 13:44 /lib64/libresolv.so.2 -> libresolv-2.12.so80455b98f518df830373b2db62c4e19c /lib64/libresolv.so.2/lib64/libselinux.so.1:-rwxr-xr-x. 1 root root 122040 Dec 7 2011 /lib64/libselinux.so.1d498609d23563a027254e14d299375ab /lib64/libselinux.so.1/opt/couchbase/lib/libv8.so:-rwxr-xr-x. 1 bin bin 85345192 Oct 11 15:50 /opt/couchbase/lib/libv8.soe061e6d32762ade7a40b4a92af499947 /opt/couchbase/lib/libv8.so/opt/couchbase/lib/libicuuc.so.44:lrwxrwxrwx. 1 bin bin 16 Oct 11 16:56 /opt/couchbase/lib/libicuuc.so.44 -> libicuuc.so.44.0a0ed6347d68900c78789b5a22591d41c /opt/couchbase/lib/libicuuc.so.44/opt/couchbase/lib/libicudata.so.44:lrwxrwxrwx. 1 bin bin 18 Oct 11 16:56 /opt/couchbase/lib/libicudata.so.44 -> libicudata.so.44.0797015967c9d62eb4ce64e861e45f500 /opt/couchbase/lib/libicudata.so.44/opt/couchbase/lib/libicui18n.so.44:lrwxrwxrwx. 1 bin bin 18 Oct 11 16:56 /opt/couchbase/lib/libicui18n.so.44 -> libicui18n.so.44.0e237dad82e9fa1d5bcb44d6bb7ecff48 /opt/couchbase/lib/libicui18n.so.44/lib64/libcrypt.so.1:lrwxrwxrwx. 1 root root 16 Oct 5 13:44 /lib64/libcrypt.so.1 -> libcrypt-2.12.so8100ffde3f3465c0086c61e50ed06c14 /lib64/libcrypt.so.1/usr/lib64/libstdc++.so.6:lrwxrwxrwx. 1 root root 19 Sep 19 17:48 /usr/lib64/libstdc++.so.6 -> libstdc++.so.6.0.135b6a438cd9d564af737b1d3f14271ab3 /usr/lib64/libstdc++.so.6/lib64/libgcc_s.so.1:lrwxrwxrwx. 1 root root 28 Oct 5 13:44 /lib64/libgcc_s.so.1 -> libgcc_s-4.4.6-20120305.so.1fa27e0c9fbddbeece6d909b9f2c088e3 /lib64/libgcc_s.so.1/lib64/libfreebl3.so:-rwxr-xr-x. 1 root root 383504 Dec 7 2011 /lib64/libfreebl3.so9fe8811e93e403a2681f99156170b5f5 /lib64/libfreebl3.so/opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so:-rwxr-xr-x. 1 bin bin 411908 Oct 11 15:57 /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.sob3f1040e670bef35c1fbff8822febb6f /opt/couchbase/lib/couchdb/erlang/lib/snappy-1.0.4/priv/snappy_nif.so/opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so:-rwxr-xr-x. 1 bin bin 673672 Oct 11 15:56 /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.soe92698fc2627b06207aab2f9e18b9203 /opt/couchbase/lib/couchdb/erlang/lib/mapreduce-1.0/priv/mapreduce_nif.so/opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so:-rwxr-xr-x. 1 bin bin 344616 Oct 11 15:56 /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so9b36f5e9f8933bb01d7ebc4cd409befc /opt/couchbase/lib/couchdb/erlang/lib/ejson-0.1.0/priv/ejson.so/opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so:-rwxr-xr-x. 1 bin bin 577043 Oct 11 15:56 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.soccf4f68c122506b83c4e85afcedef3c3 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_ejson_compare.so/opt/couchbase/lib/couchdb/erlang/lib/couch_view_parser-1.0/priv/couch_view_parser_nif.so:-rwxr-xr-x. 1 bin bin 800853 Oct 11 15:57 /opt/couchbase/lib/couchdb/erlang/lib/couch_view_parser-1.0/priv/couch_view_parser_nif.so41a4fff32b510e4867843ea91fa70268 /opt/couchbase/lib/couchdb/erlang/lib/couch_view_parser-1.0/priv/couch_view_parser_nif.so/opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.so:-rwxr-xr-x. 1 bin bin 191106 Oct 11 15:56 /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.sod55e6c1c92a89c31d7575a4c5d3ed4ca /opt/couchbase/lib/couchdb/erlang/lib/couch-1.2.0a-091f7f4-git/priv/lib/couch_icu_driver.so