Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: Tests for packaging
PostPosted: Mon Jul 30, 2018 2:22 pm 

Joined: Wed Jul 25, 2018 12:05 am
Posts: 8
I just noticed that the upgrade to ntl 11 broke singular ntl integration (as reported in GitHub issue #874). To avoid problems like that in the future, I'd like to add the singular test suite to the singular package.

However from the documentation its not entirely clear to me which tests I should run. Apparently there is `make check`, `make distcheck` and `Tst/regress.cmd`. And then `Tst/regress.cmd` has a *lot* of tests. Should I run all three of them, using the recommended list "Old/universal.lst Buch/buch.lst Plural/short.lst Plural/dmod.lst Short/ok_s.lst Long/ok_l.lst" for regress?

Edit: Also I just noticed that the `Tst` folder is not included in the tarballs found at the uni-kl.de ftp server. Since the patch releases aren't tagged on GitHub, that is currently the only reliable source for the latest release of singular.


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Tue Jul 31, 2018 4:28 pm 

Joined: Wed May 25, 2005 4:16 pm
Posts: 275
make check: very simple basic tests, should work also without externel libs (like NTL, flint.cddlib)

make distcheck: make check + checking the completeness of the result of make dist

regress.cmd Old/universal.lst Buch/buch.lst Short/ok_s.lst Plural/short.lst - recommended tests (which also tests stuff with NTL, flint)

regress.cmd Old/universal.lst Buch/buch.lst Short/ok_s.lst Plural/short.lst Plural/dmod.lst Plural/doc.lst Manual/s.lst Plural.lst BuchDL/buchdl.lst New.lst Long/ok_l.lst Short.lst Manual/m.lst Long/ok_l2.lst - complete list of tests


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Tue Jul 31, 2018 5:57 pm 

Joined: Wed Jul 25, 2018 12:05 am
Posts: 8
Okay, thank you. And what would be the best way to get the source of a release with the tests included? As I said fetching it from GitHub would be perfect, but the releases are not tagged there.


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Wed Aug 01, 2018 4:07 pm 

Joined: Wed May 25, 2005 4:16 pm
Posts: 275
ftp://www.mathematik.uni-kl.de/pub/Math ... 1p3.tar.gz provides the missing test files corresponding to singular-4.1.1p3.tar.gz


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Fri Aug 03, 2018 2:43 pm 

Joined: Wed Jul 25, 2018 12:05 am
Posts: 8
Do you plan to continue including `-tst` files for future releases? Or maybe include the `Tst` folder in the regular releases?

I have the tests working now. However if I create a test failure (by not passing `--enable-gfanlib`), regress.cmd apparently doesn't exit with exit code 1:

```
perl ./regress.cmd -s "$out/bin/Singular" \
Old/universal.lst \
Buch/buch.lst \
Short/ok_s.lst \
Plural/short.lst || exit 1
echo "Exit status $?"
```

This script doesn't exit early and reports "Exit status 0". Is that intended?

Some feedback: I needed to add "sharutils" as a dependency (for unencode and undecode), which isn't documented anywhere. Also it would be perfect if those tests could be run by `make installcheck`.

Off topic: Is it somehow possible to enable email notifications for this forum? I've enabled "Notify me upon replies by default" in "Board preferences -> posting defaults", but I still don't get emails.


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Tue Aug 07, 2018 3:48 pm 

Joined: Wed May 25, 2005 4:16 pm
Posts: 275
Somehow we will make the test files available.

But it will not become part of usual tar files, and not part of make:
- it would add additional dependencies (gunzip, uudecode/uuencode, perl)
- a difference in the test files is not necessary a bug: for example diff between 32bit version and 64bit


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Tue Aug 07, 2018 10:45 pm 

Joined: Wed Jul 25, 2018 12:05 am
Posts: 8
But the dependencies wouldn't affect users that don't run the tests, so I don't see the harm in including the tests.

> a difference in the test files is not necessary a bug: for example diff between 32bit version and 64bit

That sounds like the tests aren't a good fit for packaging after all. If the packaging tests fail, that should indicate a bug / packaging failure / mismatched dependency. Many false-positives will just make people ignore the tests.

So is there a subset of the tests that you would recommend running for that purpose?

Also, what about the exit code issue I mentioned in the last post? Is that a known problem or did I do something wrong?


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Re: Tests for packaging
PostPosted: Sun Aug 19, 2018 2:44 pm 

Joined: Wed Jul 25, 2018 12:05 am
Posts: 8
Bumping this, I hope that isn't against forum etiquette.


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

It is currently Fri May 13, 2022 10:59 am
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group