session keys, version 2

classic Classic list List threaded Threaded
29 messages Options
12
Daniel Kahn Gillmor Daniel Kahn Gillmor
Reply | Threaded
Open this post in threaded view
|

[PATCH v2 20/21] docs: clean up documentation about decryption policies

Now that the range of sensible decryption policies has come into full
view, we take a bit of space to document the distinctions.

Most people will use either "auto" or "true" -- but we provide "false"
and "nostash" to handle use cases that might reasonably be requested.

Note also that these can be combined in sensible ways.  Like, if your
mail comes in regularly to a service that doesn't have access to your
secret keys, but does have access to your index, and you feel
comfortable adding selected encrypted messages to the index after
you've read them, you could stay in "auto" normally, and then when you
find yourself reading an indexable message (e.g. one you want to be
able to search for in the future, and that you don't mind exposing to
whatever entities have access to your inde), you can do:

    notmuch reindex --decrypt=true id:[hidden email]

That leaves your default the same (still "auto") but you get the
cleartext index and stashed session key benefits for that particular
message.
---
 doc/man1/notmuch-config.rst | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 773fd9da..3ba849b2 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,7 +142,9 @@ The available configuration items are described below.
 
         **[STORED IN DATABASE]**
 
-        One of ``false``, ``auto``, ``nostash``, or ``true``.
+        Policy for decrypting encrypted messages during indexing.
+        Must be one of: ``false``, ``auto``, ``nostash``, or
+        ``true``.
 
         When indexing an encrypted e-mail message, if this variable is
         set to ``true``, notmuch will try to decrypt the message and
@@ -156,6 +158,34 @@ The available configuration items are described below.
         ``nostash`` is the same as ``true`` except that it will not
         stash newly-discovered session keys in the database.
 
+        From the command line (i.e. during **notmuch-new(1)**,
+        **notmuch-insert(1)**, or **notmuch-reindex(1)**), the user
+        can override the database's stored decryption policy with the
+        ``--decrypt=`` option.
+
+        Here is a table that summarizes the functionality of each of
+        these policies:
+
+        +------------------------+-------+------+---------+------+
+        |                        | false | auto | nostash | true |
+        +========================+=======+======+=========+======+
+        | Index cleartext using  |       |  X   |    X    |  X   |
+        | stashed session keys   |       |      |         |      |
+        +------------------------+-------+------+---------+------+
+        | Index cleartext        |       |      |    X    |  X   |
+        | using secret keys      |       |      |         |      |
+        +------------------------+-------+------+---------+------+
+        | Stash session keys     |       |      |         |  X   |
+        +------------------------+-------+------+---------+------+
+        | Delete stashed session |   X   |      |         |      |
+        | keys on reindex        |       |      |         |      |
+        +------------------------+-------+------+---------+------+
+
+        Stashed session keys are kept in the database as properties
+        associated with the message.  See ``session-key`` in
+        **notmuch-properties(7)** for more details about how they can
+        be useful.
+
         Be aware that the notmuch index is likely sufficient (and a
         stashed session key is certainly sufficient) to reconstruct
         the cleartext of the message itself, so please ensure that the
@@ -201,5 +231,6 @@ SEE ALSO
 **notmuch-restore(1)**,
 **notmuch-search(1)**,
 **notmuch-search-terms(7)**,
+**notmuch-properties(7)**,
 **notmuch-show(1)**,
 **notmuch-tag(1)**
--
2.15.0

_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Daniel Kahn Gillmor Daniel Kahn Gillmor
Reply | Threaded
Open this post in threaded view
|

[PATCH v2 21/21] python: add decrypt_policy argument to Database.index_file()

In reply to this post by Daniel Kahn Gillmor
We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.
---
 bindings/python/notmuch/database.py | 46 +++++++++++++++++++++++++++++++++++--
 bindings/python/notmuch/globals.py  |  5 ++++
 2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index 1279804a..ef370839 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -28,6 +28,7 @@ from .globals import (
     _str,
     NotmuchDatabaseP,
     NotmuchDirectoryP,
+    NotmuchIndexoptsP,
     NotmuchMessageP,
     NotmuchTagsP,
 )
@@ -72,6 +73,9 @@ class Database(object):
     MODE = Enum(['READ_ONLY', 'READ_WRITE'])
     """Constants: Mode in which to open the database"""
 
+    DECRYPTION_POLICY = Enum(['FALSE', 'TRUE', 'AUTO', 'NOSTASH'])
+    """Constants: policies for decrypting messages during indexing"""
+
     """notmuch_database_get_directory"""
     _get_directory = nmlib.notmuch_database_get_directory
     _get_directory.argtypes = [NotmuchDatabaseP, c_char_p, POINTER(NotmuchDirectoryP)]
@@ -400,13 +404,25 @@ class Database(object):
         # return the Directory, init it with the absolute path
         return Directory(abs_dirpath, dir_p, self)
 
+    _get_default_indexopts = nmlib.notmuch_database_get_default_indexopts
+    _get_default_indexopts.argtypes = [NotmuchDatabaseP]
+    _get_default_indexopts.restype = NotmuchIndexoptsP
+
+    _indexopts_set_decrypt_policy = nmlib.notmuch_indexopts_set_decrypt_policy
+    _indexopts_set_decrypt_policy.argtypes = [NotmuchIndexoptsP, c_uint]
+    _indexopts_set_decrypt_policy.restype = None
+
+    _indexopts_destroy = nmlib.notmuch_indexopts_destroy
+    _indexopts_destroy.argtypes = [NotmuchIndexoptsP]
+    _indexopts_destroy.restype = None
+
     _index_file = nmlib.notmuch_database_index_file
     _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
                              c_void_p,
                              POINTER(NotmuchMessageP)]
     _index_file.restype = c_uint
 
-    def index_file(self, filename, sync_maildir_flags=False):
+    def index_file(self, filename, sync_maildir_flags=False, decrypt_policy=None):
         """Adds a new message to the database
 
         :param filename: should be a path relative to the path of the
@@ -427,6 +443,23 @@ class Database(object):
             API. You might want to look into the underlying method
             :meth:`Message.maildir_flags_to_tags`.
 
+        :param decrypt_policy: If the message contains any encrypted
+            parts, and decrypt_policy is set to
+            :attr:`DECRYPTION_POLICY`.TRUE, notmuch will try to
+            decrypt the message and index the cleartext, stashing any
+            discovered session keys.  If it is set to
+            :attr:`DECRYPTION_POLICY`.FALSE, it will never try to
+            decrypt during indexing.  If it is set to
+            :attr:`DECRYPTION_POLICY`.AUTO, then it will try to use
+            any stashed session keys it knows about, but will not try
+            to access the user's secret keys.
+            :attr:`DECRYPTION_POLICY`.NOSTASH behaves the same as
+            :attr:`DECRYPTION_POLICY`.TRUE except that no session keys
+            are stashed in the database.  If decrypt_policy is set to
+            None (the default), then the database itself will decide
+            whether to decrypt, based on the `index.decrypt`
+            configuration setting (see notmuch-config(1)).
+
         :returns: On success, we return
 
            1) a :class:`Message` object that can be used for things
@@ -454,10 +487,19 @@ class Database(object):
               :attr:`STATUS`.READ_ONLY_DATABASE
                       Database was opened in read-only mode so no message can
                       be added.
+
         """
         self._assert_db_is_initialized()
         msg_p = NotmuchMessageP()
-        status = self._index_file(self._db, _str(filename), c_void_p(None), byref(msg_p))
+        indexopts = c_void_p(None)
+        if decrypt_policy is not None:
+            indexopts = self._get_default_indexopts(self._db)
+            self._indexopts_set_decrypt_policy(indexopts, decrypt_policy)
+
+        status = self._index_file(self._db, _str(filename), indexopts, byref(msg_p))
+
+        if indexopts:
+            self._indexopts_destroy(indexopts)
 
         if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
             raise NotmuchError(status)
diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py
index b1eec2cf..71426c84 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -88,3 +88,8 @@ NotmuchDirectoryP = POINTER(NotmuchDirectoryS)
 class NotmuchFilenamesS(Structure):
     pass
 NotmuchFilenamesP = POINTER(NotmuchFilenamesS)
+
+
+class NotmuchIndexoptsS(Structure):
+    pass
+NotmuchIndexoptsP = POINTER(NotmuchIndexoptsS)
--
2.15.0

_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Daniel Kahn Gillmor Daniel Kahn Gillmor
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 21/21] python: add decrypt_policy argument to Database.index_file()

On Thu 2017-11-30 03:59:46 -0500, Daniel Kahn Gillmor wrote:
> @@ -454,10 +487,19 @@ class Database(object):
>                :attr:`STATUS`.READ_ONLY_DATABASE
>                        Database was opened in read-only mode so no message can
>                        be added.
> +
>          """
>          self._assert_db_is_initialized()

ugh, somehow i missed this gratuitous whitespace addition the second
time around.  This is fixed in my gitlab session-keys branch [0], but it
doesn't seem worth re-posting the entire 21-message series here to fix
it.

        --dkg

[0] https://gitlab.com/dkg/notmuch

_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Daniel Kahn Gillmor Daniel Kahn Gillmor
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 04/21] crypto: use stashed session-key properties for decryption, if available

In reply to this post by Daniel Kahn Gillmor
On Thu 2017-11-30 03:59:29 -0500, Daniel Kahn Gillmor wrote:
> +    hexidecimal representation of the algorithm-specific key.  For

ugh, this should be hexadecimal, not hexidecimal.

This is fixed in my gitlab session-keys branch [0], but doesn't seem
worth re-posting the entire series for. :)

      --dkg

[0] https://gitlab.com/dkg/notmuch
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
David Bremner-2 David Bremner-2
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 04/21] crypto: use stashed session-key properties for decryption, if available

Daniel Kahn Gillmor <[hidden email]> writes:

> On Thu 2017-11-30 03:59:29 -0500, Daniel Kahn Gillmor wrote:
>> +    hexidecimal representation of the algorithm-specific key.  For
>
> ugh, this should be hexadecimal, not hexidecimal.
>
> This is fixed in my gitlab session-keys branch [0], but doesn't seem
> worth re-posting the entire series for. :)
>

I've amended it by hand, but for future reference the usual convention
is to post an amended patch as a reply, and to mark the original as
obsolete in nmbug

d
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
David Bremner-2 David Bremner-2
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

In reply to this post by Daniel Kahn Gillmor

Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
concerns/suggestions Jamie had, so I'm hoping he can have a quick look
at those.

d



_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Daniel Kahn Gillmor Daniel Kahn Gillmor
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

On Mon 2017-12-04 21:59:18 -0400, David Bremner wrote:
> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
> at those.

to be fair, i thought Jamie's concerns were correct -- the normalized
interface is better.  i was only a bit leery about the change
introducing an extra delay.  But Jamie's concerns were also corroborated
by id:[hidden email] -- maybe you could ask the author of
that message to review?  :P I think Jamie is currently slammed by
non-notmuch work, unfortunately.

   --dkg
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
Jameson Graef Rollins Jameson Graef Rollins
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

In reply to this post by David Bremner-2
On Mon, Dec 04 2017, David Bremner <[hidden email]> wrote:
> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
> at those.

Yes, this new series is great and definitely addresses all my concerns.
I'm stoked to see that the first part of it has been pushed, and looking
forward to the full series!

This is really great progress, Daniel.  Thanks for pushing on this.

jamie.

_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch

signature.asc (847 bytes) Download Attachment
Daniel Kahn Gillmor Daniel Kahn Gillmor
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

On Thu 2017-12-07 00:20:12 -0800, Jameson Graef Rollins wrote:

> On Mon, Dec 04 2017, David Bremner <[hidden email]> wrote:
>> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
>> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
>> at those.
>
> Yes, this new series is great and definitely addresses all my concerns.
> I'm stoked to see that the first part of it has been pushed, and looking
> forward to the full series!
>
> This is really great progress, Daniel.  Thanks for pushing on this.

Thanks for the review!  I've just pushed v3 of what remains of this
series, which is basically the same as the remaining patches here, with
a couple minor cleanups.

you can find it starting at
id:[hidden email]

        --dkg
_______________________________________________
notmuch mailing list
[hidden email]
https://notmuchmail.org/mailman/listinfo/notmuch
12