[PATCH 1/2] test: add a function to run Python tests

classic Classic list List threaded Threaded
12 messages Options
Thomas Jost Thomas Jost
Reply | Threaded
Open this post in threaded view
|

[PATCH 1/2] test: add a function to run Python tests

The new test_python() function makes writing Python tests a little easier:
- it sets the environment variables as needed
- it redirects stdout to the OUTPUT file (like test_emacs()).

This commit also declares python as an external prereq.

The stdout redirection is required to avoid trouble when running commands like
"python 'script' | sort > OUTPUT": in such a case, any error due to a missing
external prereq would be "swallowed" by sort, resulting to a failed test instead
of a skipped one.
---
 test/python      |    6 ++----
 test/test-lib.sh |    9 +++++++++
 2 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/test/python b/test/python
index f737749..c3aa726 100755
--- a/test/python
+++ b/test/python
@@ -5,9 +5,7 @@ test_description="python bindings"
 add_email_corpus
 
 test_begin_subtest "compare thread ids"
-LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib \
-PYTHONPATH=$TEST_DIRECTORY/../bindings/python \
-python <<EOF | sort > OUTPUT
+test_python <<EOF
 import notmuch
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
 q_new = notmuch.Query(db, 'tag:inbox')
@@ -15,5 +13,5 @@ for t in q_new.search_threads():
     print t.get_thread_id()
 EOF
 notmuch search --output=threads tag:inbox | sed s/^thread:// | sort > EXPECTED
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file <(sort OUTPUT) EXPECTED
 test_done
diff --git a/test/test-lib.sh b/test/test-lib.sh
index a975957..519bd84 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -919,6 +919,14 @@ test_emacs () {
  emacsclient --socket-name="$EMACS_SERVER" --eval "(progn $@)"
 }
 
+test_python() {
+ export LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib
+ export PYTHONPATH=$TEST_DIRECTORY/../bindings/python
+
+ (echo "import sys; _orig_stdout=sys.stdout; sys.stdout=open('OUTPUT', 'w')"; cat) \
+ | python -
+}
+
 test_reset_state_ () {
  test -z "$test_init_done_" && test_init_
 
@@ -1148,3 +1156,4 @@ test_declare_external_prereq emacs
 test_declare_external_prereq emacsclient
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
+test_declare_external_prereq python
--
1.7.8

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
Thomas Jost Thomas Jost
Reply | Threaded
Open this post in threaded view
|

[PATCH 2/2] test: use python2 instead of python if available

Some distros (Arch Linux) ship Python as python2 and Python 3 as python.
Checking for python2 is necessary for the Python tests to work on these
platforms.
---
 test/test-lib.sh |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 519bd84..155ad3c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -923,8 +923,14 @@ test_python() {
  export LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib
  export PYTHONPATH=$TEST_DIRECTORY/../bindings/python
 
+ # Some distros (e.g. Arch Linux) ship Python 2.* as /usr/bin/python2,
+ # most others as /usr/bin/python. So first try python2, and fallback to
+ # python if python2 doesn't exist.
+ cmd=python2
+ [[ "$test_missing_external_prereq_python2_" = t ]] && cmd=python
+
  (echo "import sys; _orig_stdout=sys.stdout; sys.stdout=open('OUTPUT', 'w')"; cat) \
- | python -
+ | $cmd -
 }
 
 test_reset_state_ () {
@@ -1157,3 +1163,4 @@ test_declare_external_prereq emacsclient
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
 test_declare_external_prereq python
+test_declare_external_prereq python2
--
1.7.8

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
David Bremner-2 David Bremner-2
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 1/2] test: add a function to run Python tests

In reply to this post by Thomas Jost
On Wed,  7 Dec 2011 10:46:17 +0100, Thomas Jost <[hidden email]> wrote:
> The new test_python() function makes writing Python tests a little easier:
> - it sets the environment variables as needed
> - it redirects stdout to the OUTPUT file (like test_emacs()).

pushed both, after rebasing.
_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
Dmitry Kurochkin Dmitry Kurochkin
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 1/2] test: add a function to run Python tests

In reply to this post by Thomas Jost
Hi Thomas.

On Wed,  7 Dec 2011 10:46:17 +0100, Thomas Jost <[hidden email]> wrote:

> The new test_python() function makes writing Python tests a little easier:
> - it sets the environment variables as needed
> - it redirects stdout to the OUTPUT file (like test_emacs()).
>
> This commit also declares python as an external prereq.
>
> The stdout redirection is required to avoid trouble when running commands like
> "python 'script' | sort > OUTPUT": in such a case, any error due to a missing
> external prereq would be "swallowed" by sort, resulting to a failed test instead
> of a skipped one.
> ---
>  test/python      |    6 ++----
>  test/test-lib.sh |    9 +++++++++
>  2 files changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/test/python b/test/python
> index f737749..c3aa726 100755
> --- a/test/python
> +++ b/test/python
> @@ -5,9 +5,7 @@ test_description="python bindings"
>  add_email_corpus
>  
>  test_begin_subtest "compare thread ids"
> -LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib \
> -PYTHONPATH=$TEST_DIRECTORY/../bindings/python \
> -python <<EOF | sort > OUTPUT
> +test_python <<EOF
>  import notmuch
>  db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
>  q_new = notmuch.Query(db, 'tag:inbox')
> @@ -15,5 +13,5 @@ for t in q_new.search_threads():
>      print t.get_thread_id()
>  EOF
>  notmuch search --output=threads tag:inbox | sed s/^thread:// | sort > EXPECTED
> -test_expect_equal_file OUTPUT EXPECTED
> +test_expect_equal_file <(sort OUTPUT) EXPECTED
>  test_done
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index a975957..519bd84 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -919,6 +919,14 @@ test_emacs () {
>   emacsclient --socket-name="$EMACS_SERVER" --eval "(progn $@)"
>  }
>  
> +test_python() {
> + export LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib
> + export PYTHONPATH=$TEST_DIRECTORY/../bindings/python
> +
> + (echo "import sys; _orig_stdout=sys.stdout; sys.stdout=open('OUTPUT', 'w')"; cat) \
> + | python -

Perhaps we should have a test-lib.py for test-specific stuff like this
(similar to test-lib.el)?  I think it would be cleaner and makes it easy
to add more Python test auxiliary functions later.

Regards,
  Dmitry

> +}
> +
>  test_reset_state_ () {
>   test -z "$test_init_done_" && test_init_
>  
> @@ -1148,3 +1156,4 @@ test_declare_external_prereq emacs
>  test_declare_external_prereq emacsclient
>  test_declare_external_prereq gdb
>  test_declare_external_prereq gpg
> +test_declare_external_prereq python
> --
> 1.7.8
>
> _______________________________________________
> notmuch mailing list
> [hidden email]
> http://notmuchmail.org/mailman/listinfo/notmuch
_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
Thomas Jost Thomas Jost
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 1/2] test: add a function to run Python tests

On Sun, 11 Dec 2011 18:58:18 +0400, Dmitry Kurochkin <[hidden email]> wrote:
> Perhaps we should have a test-lib.py for test-specific stuff like this
> (similar to test-lib.el)?  I think it would be cleaner and makes it easy
> to add more Python test auxiliary functions later.

Well, right now that would probably be overkill: the stdout-to-file
redirection takes 3 lines at most. Besides we would still need to set
some environment variables to run python with the correct directories.
But of course if we need to add more helper functions for running python
tests, then a test-lib.py would be nice.

Maybe we could even think about using one of Python's unit test libs for
that: doctest, unittest (both in the standard library) and nose seem to
be the most popular ones.

Regards,

--
Thomas/Schnouki

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

attachment0 (499 bytes) Download Attachment
Patrick Totzke Patrick Totzke
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 1/2] test: add a function to run Python tests

Quoting Thomas Jost (2011-12-13 17:21:34)

>On Sun, 11 Dec 2011 18:58:18 +0400, Dmitry Kurochkin <[hidden email]> wrote:
>> Perhaps we should have a test-lib.py for test-specific stuff like this
>> (similar to test-lib.el)?  I think it would be cleaner and makes it easy
>> to add more Python test auxiliary functions later.
>
>Well, right now that would probably be overkill: the stdout-to-file
>redirection takes 3 lines at most. Besides we would still need to set
>some environment variables to run python with the correct directories.
>But of course if we need to add more helper functions for running python
>tests, then a test-lib.py would be nice.

I agree that a test-lib python module that contains some helpers would be useful.
Specifically, we could use a "test_expect_success" helper that
wraps a given script into a try/except block.

>Maybe we could even think about using one of Python's unit test libs for
>that: doctest, unittest (both in the standard library) and nose seem to
>be the most popular ones.
never used any of these.

I cleaned up the initial test a bit (see the following messages for the patches).
Here are a few suggestions for tests we should implement later:

test_begin_subtest "tag messages" # add_tag/remove_tag
test_begin_subtest "add messages" # compare msg.get_filename to original
test_begin_subtest "find message"
test_begin_subtest "remove messages"
test_begin_subtest "list tags" # get_all_tags()
test_begin_subtest "count messages" # count_messages()
test_begin_subtest "count threads" # MISSING in the bindings!  count_messages()
test_begin_subtest "count messages in thread" # get_total_messages compare with notmuch search messages |wc
test_begin_subtest "exceptions" # ReadOnlyDatabaseError, XapianError.
test_begin_subtest "format message content" # format_message_as_text(indent=0)

best,
/p
_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
Patrick Totzke Patrick Totzke
Reply | Threaded
Open this post in threaded view
|

[PATCH 1/2] clean up "compare thread ids" python test

This makes the test script open the database in READ_ONLY mode
and use the libraries own sorting methods instead of "sort".
---
 test/python |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/test/python b/test/python
index c3aa726..c318cc1 100755
--- a/test/python
+++ b/test/python
@@ -7,11 +7,12 @@ add_email_corpus
 test_begin_subtest "compare thread ids"
 test_python <<EOF
 import notmuch
-db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
 q_new = notmuch.Query(db, 'tag:inbox')
+q_new.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
 for t in q_new.search_threads():
     print t.get_thread_id()
 EOF
-notmuch search --output=threads tag:inbox | sed s/^thread:// | sort > EXPECTED
-test_expect_equal_file <(sort OUTPUT) EXPECTED
+notmuch search --sort=oldest-first --output=threads tag:inbox | sed s/^thread:// > EXPECTED
+test_expect_equal_file OUTPUT EXPECTED
 test_done
--
1.7.5.4

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
Patrick Totzke Patrick Totzke
Reply | Threaded
Open this post in threaded view
|

[PATCH 2/2] python test "compare message ids"

Introduces a second (trivial) test for the python
bindings that searches for message ids and compares
the output with that of `notmuch search`.
---
 test/python |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/test/python b/test/python
index c318cc1..6018c2d 100755
--- a/test/python
+++ b/test/python
@@ -15,4 +15,17 @@ for t in q_new.search_threads():
 EOF
 notmuch search --sort=oldest-first --output=threads tag:inbox | sed s/^thread:// > EXPECTED
 test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "compare message ids"
+test_python <<EOF
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
+q_new = notmuch.Query(db, 'tag:inbox')
+q_new.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
+for m in q_new.search_messages():
+    print m.get_message_id()
+EOF
+notmuch search --sort=oldest-first --output=messages tag:inbox | sed s/^id:// > EXPECTED
+test_expect_equal_file OUTPUT EXPECTED
+
 test_done
--
1.7.5.4

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch
spaetz spaetz
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 1/2] clean up "compare thread ids" python test

In reply to this post by Patrick Totzke
On Mon,  2 Jan 2012 14:51:26 +0000, Patrick Totzke <[hidden email]> wrote:
> This makes the test script open the database in READ_ONLY mode
> and use the libraries own sorting methods instead of "sort".

+1 I don't want to fudge the tests (I don't know a thing about them), so
I can't judge the test outcome, but it is certainly good from the python side of
things. So I'll push this one.

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

attachment0 (203 bytes) Download Attachment
spaetz spaetz
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 2/2] python test "compare message ids"

In reply to this post by Patrick Totzke
On Mon,  2 Jan 2012 14:51:27 +0000, Patrick Totzke <[hidden email]> wrote:
> Introduces a second (trivial) test for the python
> bindings that searches for message ids and compares
> the output with that of `notmuch search`.

Test passes, so it went in.

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

attachment0 (203 bytes) Download Attachment
spaetz spaetz
Reply | Threaded
Open this post in threaded view
|

notmuch git's disk is full! ERROR pushing.

In reply to this post by Patrick Totzke
Actually trying to push the above 2 patches, I get

Writing objects: 100% (8/8), 1.25 KiB, done.
Total 8 (delta 5), reused 0 (delta 0)
error: unable to create temporary sha1 filename : No space left on device

fatal: failed to write object
error: unpack failed: unpack-objects abnormal exit
To ssh://[hidden email]/git/notmuch
 ! [remote rejected] master -> master (n/a (unpacker error))
error: failed to push some refs to 'ssh://[hidden email]/git/notmuch'

Sebastian

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

attachment0 (203 bytes) Download Attachment
Carl Worth-2 Carl Worth-2
Reply | Threaded
Open this post in threaded view
|

Re: notmuch git's disk is full! ERROR pushing.

On Sun, 08 Jan 2012 13:55:53 +0100, Sebastian Spaeth <[hidden email]> wrote:
> Actually trying to push the above 2 patches, I get
>
> Writing objects: 100% (8/8), 1.25 KiB, done.
> Total 8 (delta 5), reused 0 (delta 0)
> error: unable to create temporary sha1 filename : No space left on
> device

My apologies!

I've relieved the immediate problem for now. (And I'm hoping to soon
move notmuchmail.org to a different server with a disk that's a couple
of orders of magnitude larger.)

-Carl

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

attachment0 (203 bytes) Download Attachment