[PATCH] Document the dependency on dtach in the test suite

[Ethan Glasser-Camp]

From: Ethan Glasser-Camp <[hidden email]>

dtach (and before that, screen) are used to run emacs with more
verisimilitude, but this is only mentioned in the NEWS. Mention this
in the test/README file, to make it easier to find out why a bunch of
tests are failing/skipped.

Signed-off-by: Ethan Glasser-Camp <[hidden email]>
---

Just cutting my teeth on the codebase, and it took me a little while
to figure this out, so I thought I'd document it.

 test/README |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/test/README b/test/README
index bde6db0..29afb3b 100644
--- a/test/README
+++ b/test/README
@@ -6,6 +6,11 @@ When fixing bugs or enhancing notmuch, you are strongly encouraged to
 add tests in this directory to cover what you are trying to fix or
 enhance.
 
+Prerequisites
+-------------
+emacs is run within dtach, so you need to have that installed to run
+emacs tests (or crypto tests, which rely on emacs to run).
+
 Running Tests
 -------------
 The easiest way to run tests is to say "make test", (or simply run the
--
1.7.5.4

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

[Dmitry Kurochkin]

On Mon, 16 Jan 2012 12:16:08 -0500, Ethan Glasser-Camp <[hidden email]> wrote:
> From: Ethan Glasser-Camp <[hidden email]>
>
> dtach (and before that, screen) are used to run emacs with more
> verisimilitude, but this is only mentioned in the NEWS. Mention this
> in the test/README file, to make it easier to find out why a bunch of
> tests are failing/skipped.
>

Can you please add other prerequisites as well?  There is gpg and
probably more.

Can we just make it an alphabetically-sorted list?  That would make it
easier to read and add more dependencies in the future.  I.e.:

  Some tests require the following external dependencies to run:

    * emacs(1)
    * emacsclient(1)
    ...

Regards,
  Dmitry

> Signed-off-by: Ethan Glasser-Camp <[hidden email]>
> ---
>
> Just cutting my teeth on the codebase, and it took me a little while
> to figure this out, so I thought I'd document it.
>
>  test/README |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/test/README b/test/README
> index bde6db0..29afb3b 100644
> --- a/test/README
> +++ b/test/README
> @@ -6,6 +6,11 @@ When fixing bugs or enhancing notmuch, you are strongly encouraged to
>  add tests in this directory to cover what you are trying to fix or
>  enhance.
>  
> +Prerequisites
> +-------------
> +emacs is run within dtach, so you need to have that installed to run
> +emacs tests (or crypto tests, which rely on emacs to run).
> +
>  Running Tests
>  -------------
>  The easiest way to run tests is to say "make test", (or simply run the
> --
> 1.7.5.4
>
> _______________________________________________
> notmuch mailing list
> [hidden email]
> http://notmuchmail.org/mailman/listinfo/notmuch

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

[Pieter Praet]

When tests are skipped due to missing prereqs, those prereqs are only
displayed when running with the `--verbose' option.  This is essential
information when troubleshooting, so always send to stdout.

---

 test/test-lib.sh |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index d1fbc05..6560628 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -673,8 +673,8 @@ test_skip () {
 
 test_check_missing_external_prereqs_ () {
  if test -n "$test_subtest_missing_external_prereqs_"; then
- say_color skip >&3 "missing prerequisites:"
- echo "$test_subtest_missing_external_prereqs_" >&3
+ say_color skip >&1 "missing prerequisites:"
+ echo "$test_subtest_missing_external_prereqs_" >&1
  test_report_skip_ "$@"
  else
  false
--
1.7.8.1

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

[Tomi Ollila-2]

On Wed, 18 Jan 2012 13:19:41 +0100, Pieter Praet <[hidden email]> wrote:
> When tests are skipped due to missing prereqs, those prereqs are only
> displayed when running with the `--verbose' option.  This is essential
> information when troubleshooting, so always send to stdout.
>
> ---

Instead of this the test suite could be enchanged so that
full log of the execution is stored somewhere and path
to that file is displayed at the end of test. Generally,
all output could stay available after tests exit so that
those could be investigated furtner.

(I tried to grep 'missing prerequisites' after running test
but did not find the info. the file opened for >&3 has been
deleted already?)

Now I just see huge amounts of missing prerequisites: emacs(1)
in my output (I presume that doesn't happen due to my
hack to "hide" emacs).

But well, I'm not against applying this patch -- just that
the test system starts to get so rotten touching it gives
shivers...

Tomi

>
>  test/test-lib.sh |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index d1fbc05..6560628 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -673,8 +673,8 @@ test_skip () {
>  
>  test_check_missing_external_prereqs_ () {
>   if test -n "$test_subtest_missing_external_prereqs_"; then
> - say_color skip >&3 "missing prerequisites:"
> - echo "$test_subtest_missing_external_prereqs_" >&3
> + say_color skip >&1 "missing prerequisites:"
> + echo "$test_subtest_missing_external_prereqs_" >&1
>   test_report_skip_ "$@"
>   else
>   false
> --
> 1.7.8.1

_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

[Pieter Praet]

On Wed, 18 Jan 2012 15:53:59 +0200, Tomi Ollila <[hidden email]> wrote:

> On Wed, 18 Jan 2012 13:19:41 +0100, Pieter Praet <[hidden email]> wrote:
> > When tests are skipped due to missing prereqs, those prereqs are only
> > displayed when running with the `--verbose' option.  This is essential
> > information when troubleshooting, so always send to stdout.
> >
> > ---
>
> Instead of this the test suite could be enchanged so that
> full log of the execution is stored somewhere and path
> to that file is displayed at the end of test. Generally,
> all output could stay available after tests exit so that
> those could be investigated furtner.
>

Indeed.  AFAIK, this used to be possible using the `--tee' option,
but the files it produces appear to be deleted when the tests finish.

> (I tried to grep 'missing prerequisites' after running test
> but did not find the info. the file opened for >&3 has been
> deleted already?)
>

Hmmm, odd.

Are you sure you were using my patch and/or the `--verbose' option?

Works here, using this highly sophisticated Emacs-hiding wizardry:

  #+begin_src sh
    sudo mv /usr/bin/emacs /usr/bin/emacs.bak
    make test | grep 'missing prerequisites'
    sudo mv /usr/bin/emacs.bak /usr/bin/emacs
  #+end_src

(patent pending... ;)

> Now I just see huge amounts of missing prerequisites: emacs(1)
> in my output (I presume that doesn't happen due to my
> hack to "hide" emacs).
>

When using my patch, you mean?

Most likely because "$test_subtest_missing_external_prereqs_" is only
reset at the beginning of every test series, so if an Emacs-dependent
subtest is skipped, all subsequent skipped subtests will report that
dependency as well, even if they were skipped due to a missing vi :).

> But well, I'm not against applying this patch -- just that
> the test system starts to get so rotten touching it gives
> shivers...
>

I wouldn't say it's getting rotten, but yeah, "tread lightly" should
probably be mandatory advice to those aspiring to explore its bowels...

> Tomi
>
> >
> >  test/test-lib.sh |    4 ++--
> >  1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/test/test-lib.sh b/test/test-lib.sh
> > index d1fbc05..6560628 100644
> > --- a/test/test-lib.sh
> > +++ b/test/test-lib.sh
> > @@ -673,8 +673,8 @@ test_skip () {
> >  
> >  test_check_missing_external_prereqs_ () {
> >   if test -n "$test_subtest_missing_external_prereqs_"; then
> > - say_color skip >&3 "missing prerequisites:"
> > - echo "$test_subtest_missing_external_prereqs_" >&3
> > + say_color skip >&1 "missing prerequisites:"
> > + echo "$test_subtest_missing_external_prereqs_" >&1
> >   test_report_skip_ "$@"
> >   else
> >   false
> > --
> > 1.7.8.1

Peace

--
Pieter
_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

[Pieter Praet]

On Fri, 20 Jan 2012 01:24:02 +0100, Pieter Praet <[hidden email]> wrote:
> [...]
> Most likely because "$test_subtest_missing_external_prereqs_" is only
> reset at the beginning of every test series, so if an Emacs-dependent
> subtest is skipped, all subsequent skipped subtests will report that
> dependency as well, even if they were skipped due to a missing vi :).
> [...]

That's not right at all.

"$test_subtest_missing_external_prereqs_" is reset at every invocation
of `test_begin_subtest', using `test_reset_state_'.

Apologies for the misinformation.


Peace

--
Pieter
_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch

[David Bremner-2]

On Wed, 18 Jan 2012 13:19:41 +0100, Pieter Praet <[hidden email]> wrote:
> When tests are skipped due to missing prereqs, those prereqs are only
> displayed when running with the `--verbose' option.  This is essential
> information when troubleshooting, so always send to stdout.

pushed,

d
_______________________________________________
notmuch mailing list
[hidden email]
http://notmuchmail.org/mailman/listinfo/notmuch