Database error

classic Classic list List threaded Threaded
3 messages Options
Varac Varac
Reply | Threaded
Open this post in threaded view
|

Database error

Hi,

I get a DatabaseError right after initial indexing.
I can reproduce this with my mails every time.
First, I remove the whole .notmuch/xapian directory and
run "notmuch new", which results in a DatabaseError when
using "xapian-check":


  » notmuch new
  Welcome to a new version of notmuch! Your database will now be upgraded.
  This process is safe to interrupt.
  Backing up tags to /home/varac/.offlineimap/Mail/.notmuch/dump-20180419T160138.gz...
  Your notmuch database has now been upgraded.
  Processed 148440 total files in 13m 38s (181 files/sec.).
  Added 140351 new messages to the database.

  » xapian-check .notmuch/xapian
  docdata:
  blocksize=8K items=564 firstunused=7 revision=7 levels=1 root=2
  B-tree checked okay
  docdata table structure checked OK

  termlist:
  blocksize=8K items=305882 firstunused=64506 revision=7 levels=2 root=63491
  xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0


  » notmuch compact
  Compacting database...
  compacting table postlist
       Reduced by 34% 130352K (373696K -> 243344K)
  compacting table docdata
       Reduced by 28% 16K (56K -> 40K)
  compacting table termlist
       Reduced by 48% 250928K (516048K -> 265120K)
  compacting table position
       Reduced by 44% 455456K (1034784K -> 579328K)
  compacting table spelling
       doesn't exist
  compacting table synonym
       doesn't exist
  Done.


  » xapian-check .notmuch/xapian
  docdata:
  blocksize=8K items=564 firstunused=5 revision=1 levels=1 root=2
  B-tree checked okay
  docdata table structure checked OK

  termlist:
  blocksize=8K items=305882 firstunused=33140 revision=1 levels=2 root=746
  B-tree checked okay
  termlist table structure checked OK

  postlist:
  blocksize=8K items=2845658 firstunused=30418 revision=1 levels=2 root=578
  B-tree checked okay
  postlist table structure checked OK

  position:
  blocksize=8K items=27789188 firstunused=72416 revision=1 levels=2 root=466
  B-tree checked okay
  position table structure checked OK

  spelling:
  Lazily created, and not yet used.

  synonym:
  Lazily created, and not yet used.

  No errors found


  » dpkg -l |egrep '(notmuch|xapian)'
  ii  afew                                                        1.3.0-1                                                    all          Tagging script for notmuch mail
  ii  libnotmuch-dev                                              0.26-1ubuntu3                                              amd64        thread-based email index, search and tagging (development)
  ii  libnotmuch5                                                 0.26-1ubuntu3                                              amd64        thread-based email index, search and tagging (runtime)
  ii  libxapian30:amd64                                           1.4.5-1                                                    amd64        Search engine library
  ii  notmuch                                                     0.26-1ubuntu3                                              amd64        thread-based email index, search and tagging
  ii  notmuch-addrlookup                                          9-1                                                        amd64        Address lookup tool for Notmuch
  ii  python-notmuch                                              0.26-1ubuntu2                                              all          Python interface to the notmuch mail search and index library
  ii  python3-notmuch                                             0.26-1ubuntu3                                              all          Python 3 interface to the notmuch mail search and index library
  ii  xapian-tools                                                1.4.5-1                                                    amd64        Basic tools for Xapian search engine library

After this I try to tun "afew --tag --new -v" but it fails with (I don't know if
this is related though):

  Traceback (most recent call last):
    File "/usr/bin/afew", line 11, in <module>
      load_entry_point('afew==1.3.0', 'console_scripts', 'afew')()
    File "/usr/lib/python3/dist-packages/afew/commands.py", line 159, in main
      inner_main(args, database, query_string)
    File "/usr/lib/python3/dist-packages/afew/main.py", line 23, in main
      filter_.run(query_string)
    File "/usr/lib/python3/dist-packages/afew/filters/BaseFilter.py", line 59, in run
      for message in self.database.get_messages(query):
    File "/usr/lib/python3/dist-packages/afew/Database.py", line 93, in get_messages
      for message in self.do_query(query).search_messages():
    File "/usr/lib/python3/dist-packages/notmuch/query.py", line 182, in search_messages
      raise NotmuchError(status)
  notmuch.errors.XapianError

Please help! I kind of lost here.

Greetings, Varac

_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch

signature.asc (817 bytes) Download Attachment
David Bremner-2 David Bremner-2
Reply | Threaded
Open this post in threaded view
|

Re: Database error

Varac <[hidden email]> writes:

> Hi,
>
> I get a DatabaseError right after initial indexing.
> I can reproduce this with my mails every time.
> First, I remove the whole .notmuch/xapian directory and
> run "notmuch new", which results in a DatabaseError when
> using "xapian-check":
>
>
>   » notmuch new
>   Welcome to a new version of notmuch! Your database will now be upgraded.
>   This process is safe to interrupt.
>   Backing up tags to /home/varac/.offlineimap/Mail/.notmuch/dump-20180419T160138.gz...
>   Your notmuch database has now been upgraded.
>   Processed 148440 total files in 13m 38s (181 files/sec.).
>   Added 140351 new messages to the database.
>
>   » xapian-check .notmuch/xapian
>   docdata:
>   blocksize=8K items=564 firstunused=7 revision=7 levels=1 root=2
>   B-tree checked okay
>   docdata table structure checked OK
>
>   termlist:
>   blocksize=8K items=305882 firstunused=64506 revision=7 levels=2 root=63491
>   xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0


This is almost certainly a known and recently fixed xapian bug. notmuch
compact is the recommended workaround (or just ignore the message).

>
> After this I try to tun "afew --tag --new -v" but it fails with (I don't know if
> this is related though):
>
>   Traceback (most recent call last):
>     File "/usr/bin/afew", line 11, in <module>
>       load_entry_point('afew==1.3.0', 'console_scripts', 'afew')()
>     File "/usr/lib/python3/dist-packages/afew/commands.py", line 159, in main
>       inner_main(args, database, query_string)
>     File "/usr/lib/python3/dist-packages/afew/main.py", line 23, in main
>       filter_.run(query_string)
>     File "/usr/lib/python3/dist-packages/afew/filters/BaseFilter.py", line 59, in run
>       for message in self.database.get_messages(query):
>     File "/usr/lib/python3/dist-packages/afew/Database.py", line 93, in get_messages
>       for message in self.do_query(query).search_messages():
>     File "/usr/lib/python3/dist-packages/notmuch/query.py", line 182, in search_messages
>       raise NotmuchError(status)
>   notmuch.errors.XapianError
>

It's hard to say for sure, but it's possible this would be fixed by the
patch

        https://mid.mail-archive.com/20180414014610.15438-4-david@...

That should be part of a bug-fix release of notmuch fairly soon. You can
also try to find a subset of mail reproducing the problem. I don't
know what that afew invokation is trying to do. Can you reproduce the
problem using the notmuch CLI?

d
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Varac Varac
Reply | Threaded
Open this post in threaded view
|

Re: Database error

Quoting David Bremner (2018-04-20 14:44:25)

> >   » xapian-check .notmuch/xapian docdata:
> >   blocksize=8K items=564 firstunused=7 revision=7 levels=1 root=2
> >   B-tree checked okay
> >   docdata table structure checked OK
> >
> >   termlist:
> >   blocksize=8K items=305882 firstunused=64506 revision=7 levels=2 root=63491
> >   xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0
>
>
> This is almost certainly a known and recently fixed xapian bug. notmuch
> compact is the recommended workaround (or just ignore the message).
Ok, I'll ignore it for now.

>
> >
> > After this I try to tun "afew --tag --new -v" but it fails with (I don't know if
> > this is related though):
> >
> >   Traceback (most recent call last):
> >     File "/usr/bin/afew", line 11, in <module>
> >       load_entry_point('afew==1.3.0', 'console_scripts', 'afew')()
> >     File "/usr/lib/python3/dist-packages/afew/commands.py", line 159, in main
> >       inner_main(args, database, query_string)
> >     File "/usr/lib/python3/dist-packages/afew/main.py", line 23, in main
> >       filter_.run(query_string)
> >     File "/usr/lib/python3/dist-packages/afew/filters/BaseFilter.py", line 59, in run
> >       for message in self.database.get_messages(query):
> >     File "/usr/lib/python3/dist-packages/afew/Database.py", line 93, in get_messages
> >       for message in self.do_query(query).search_messages():
> >     File "/usr/lib/python3/dist-packages/notmuch/query.py", line 182, in search_messages
> >       raise NotmuchError(status)
> >   notmuch.errors.XapianError
> >
>
> It's hard to say for sure, but it's possible this would be fixed by the
> patch
>
>         https://mid.mail-archive.com/20180414014610.15438-4-david@...
>
> That should be part of a bug-fix release of notmuch fairly soon. You can
> also try to find a subset of mail reproducing the problem. I don't
> know what that afew invokation is trying to do. Can you reproduce the
> problem using the notmuch CLI?
It turned out that this was a bad syntax in afew, where I quoted a query in
single quotes.

Greetings, Varac

_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch

signature.asc (817 bytes) Download Attachment