crash after running notmuch new

classic Classic list List threaded Threaded
6 messages Options
Matt Matt
Reply | Threaded
Open this post in threaded view
|

crash after running notmuch new

hi,

I have a script that runs mbsync followed by notmuch new.
It ends with `terminate called after throwing an instance of
'Xapian::DatabaseCorruptError`.

Any idea what could cause this. Here is a trace without symbols thoug
====
#0  0x00007f1cb90d315a in raise () from
/nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib/libc.so.6
#1  0x00007f1cb90bd548 in abort () from
/nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib/libc.so.6
#2  0x00007f1cb8d3e643 in ?? () from
/nix/store/lzyjivyzrp6agilxhgh0h40pwwg18agy-gcc-9.2.0-lib/lib/libstdc++.so.6
#3  0x00007f1cb8d4a066 in ?? () from
/nix/store/lzyjivyzrp6agilxhgh0h40pwwg18agy-gcc-9.2.0-lib/lib/libstdc++.so.6
#4  0x00007f1cb8d4a0b1 in std::terminate() () from
/nix/store/lzyjivyzrp6agilxhgh0h40pwwg18agy-gcc-9.2.0-lib/lib/libstdc++.so.6
#5  0x00007f1cb8d4a336 in __cxa_rethrow () from
/nix/store/lzyjivyzrp6agilxhgh0h40pwwg18agy-gcc-9.2.0-lib/lib/libstdc++.so.6
#6  0x00007f1cb8ec1f08 in
Xapian::TermIterator::TermIterator(Xapian::TermIterator::Internal*)
[clone .cold] ()
   from /nix/store/5afyzmgp6rsn1digmh1hf47g7ppwp25v-xapian-1.4.15/lib/libxapian.so.30
#7  0x00007f1cb8ef9546 in
Xapian::Database::allterms_begin(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&) const () from
/nix/store/5afyzmgp6rsn1digmh1hf47g7ppwp25v-xapian-1.4.15/lib/libxapian.so.30
#8  0x00007f1cb94a81bf in
_create_filenames_for_terms_with_prefix(void*, _notmuch_database*,
char const*) [clone .isra.0] ()
   from /nix/store/f4jm5j94pamm5xkbqrk1r70zjvjn0mch-notmuch-0.29.3/lib/libnotmuch.so.5
#9  0x00007f1cb94a8395 in notmuch_directory_get_child_directories ()
   from /nix/store/f4jm5j94pamm5xkbqrk1r70zjvjn0mch-notmuch-0.29.3/lib/libnotmuch.so.5
#10 0x00000000004105df in add_files ()
#11 0x000000000040fe9e in add_files ()
#12 0x000000000040fe9e in add_files ()
#13 0x000000000040fe9e in add_files ()
#14 0x0000000000411693 in notmuch_new_command ()
===

Cheers
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
David Bremner-2 David Bremner-2
Reply | Threaded
Open this post in threaded view
|

Re: crash after running notmuch new

Matt <[hidden email]> writes:

> hi,
>
> I have a script that runs mbsync followed by notmuch new.
> It ends with `terminate called after throwing an instance of
> 'Xapian::DatabaseCorruptError`.
>

Have you tried running xapian-check?

d
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Matt Matt
Reply | Threaded
Open this post in threaded view
|

Re: crash after running notmuch new

thanks didn't know about xapian-check !
the output
===
docdata:
blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260
B-tree checked okay
termlist table structure checked OK

postlist:
blocksize=8K items=2598971 firstunused=61412 revision=421 levels=2 root=49814
xapian-check: DatabaseCorruptError: Db block overwritten - are there
multiple writers?
===
suggests there is an error but I couldn't find a fix for it. Should I
just remove the xapian folder and rerun `notmuch new` ?

Thanks !

2020年4月7日(火) 15:03 David Bremner <[hidden email]>:

>
> Matt <[hidden email]> writes:
>
> > hi,
> >
> > I have a script that runs mbsync followed by notmuch new.
> > It ends with `terminate called after throwing an instance of
> > 'Xapian::DatabaseCorruptError`.
> >
>
> Have you tried running xapian-check?
>
> d
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
David Bremner-2 David Bremner-2
Reply | Threaded
Open this post in threaded view
|

Re: crash after running notmuch new

Matt <[hidden email]> writes:

> thanks didn't know about xapian-check !
> the output
> ===
> docdata:
> blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2
> B-tree checked okay
> docdata table structure checked OK
>
> termlist:
> blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260
> B-tree checked okay
> termlist table structure checked OK
>
> postlist:
> blocksize=8K items=2598971 firstunused=61412 revision=421 levels=2 root=49814
> xapian-check: DatabaseCorruptError: Db block overwritten - are there
> multiple writers?
> ===
> suggests there is an error but I couldn't find a fix for it. Should I
> just remove the xapian folder and rerun `notmuch new` ?

If you have a backup of your tags from notmuch-dump, then yes that's
probably a good way forward.   I've put the xapian developers in copy in
case they are interested in trying to debug this corruption. For those
just joining us, this is notmuch 0.29.3 linked against xapian 1.4.15

d
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Olly Betts Olly Betts
Reply | Threaded
Open this post in threaded view
|

Re: crash after running notmuch new

On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote:
> Matt <[hidden email]> writes:
[...]

> > termlist:
> > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260
> > B-tree checked okay
> > termlist table structure checked OK
> >
> > postlist:
> > blocksize=8K items=2598971 firstunused=61412 revision=421 levels=2 root=49814
> > xapian-check: DatabaseCorruptError: Db block overwritten - are there
> > multiple writers?
> > ===
> > suggests there is an error but I couldn't find a fix for it. Should I
> > just remove the xapian folder and rerun `notmuch new` ?
>
> If you have a backup of your tags from notmuch-dump, then yes that's
> probably a good way forward.

If you don't have a current dump, you may be able to rescue a dump of
tags from a broken notmuch database using:

https://git.xapian.org/?p=xapian;a=blob;f=README.notmuch;hb=refs/heads/notmuch-tag-rescue-hack

That should work if the termlist table is undamaged (as the above
appears to show), and may work even if it's damaged.

> I've put the xapian developers in copy in
> case they are interested in trying to debug this corruption. For those
> just joining us, this is notmuch 0.29.3 linked against xapian 1.4.15

Was the database created with 1.4.15 too?

If it's reproducible, I'm definitely interested.

If it isn't reproducible (and/or the data is sensitive) it's much more
difficult to usefully investigate.  And it may also be due to a
non-Xapian issue (a bug in something else or a hardware problem).

Cheers,
    Olly
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Matt Matt
Reply | Threaded
Open this post in threaded view
|

Re: crash after running notmuch new

thanks for the support,
since the notmuch database must have been ~7 years long with lots of
upgrade, unexpected shutdowns, configuration fiddling I thought it
would have been very hard to track down the root cause. If I had had a
better idea of the root cause I would have liked to help but all my
tags are automatically set so it was simpler to just regenerate the
database and that's what I ended up doing.

Thanks for the proposition though.

Matt

2020年4月8日(水) 0:20 Olly Betts <[hidden email]>:

>
> On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote:
> > Matt <[hidden email]> writes:
> [...]
> > > termlist:
> > > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260
> > > B-tree checked okay
> > > termlist table structure checked OK
> > >
> > > postlist:
> > > blocksize=8K items=2598971 firstunused=61412 revision=421 levels=2 root=49814
> > > xapian-check: DatabaseCorruptError: Db block overwritten - are there
> > > multiple writers?
> > > ===
> > > suggests there is an error but I couldn't find a fix for it. Should I
> > > just remove the xapian folder and rerun `notmuch new` ?
> >
> > If you have a backup of your tags from notmuch-dump, then yes that's
> > probably a good way forward.
>
> If you don't have a current dump, you may be able to rescue a dump of
> tags from a broken notmuch database using:
>
> https://git.xapian.org/?p=xapian;a=blob;f=README.notmuch;hb=refs/heads/notmuch-tag-rescue-hack
>
> That should work if the termlist table is undamaged (as the above
> appears to show), and may work even if it's damaged.
>
> > I've put the xapian developers in copy in
> > case they are interested in trying to debug this corruption. For those
> > just joining us, this is notmuch 0.29.3 linked against xapian 1.4.15
>
> Was the database created with 1.4.15 too?
>
> If it's reproducible, I'm definitely interested.
>
> If it isn't reproducible (and/or the data is sensitive) it's much more
> difficult to usefully investigate.  And it may also be due to a
> non-Xapian issue (a bug in something else or a hardware problem).
>
> Cheers,
>     Olly
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch