Singular
https://www.singular.uni-kl.de/forum/

Tests for packaging
https://www.singular.uni-kl.de/forum/viewtopic.php?f=10&t=2773		 Page 1 of 1
Author:  timokau [ Mon Jul 30, 2018 2:22 pm ]
Post subject:  Tests for packaging
I just noticed that the upgrade to ntl 11 broke singular ntl integration (as reported in GitHub issue #874). To avoid problems like that in the future, I'd like to add the singular test suite to the singular package.

However from the documentation its not entirely clear to me which tests I should run. Apparently there is `make check`, `make distcheck` and `Tst/regress.cmd`. And then `Tst/regress.cmd` has a *lot* of tests. Should I run all three of them, using the recommended list "Old/universal.lst Buch/buch.lst Plural/short.lst Plural/dmod.lst Short/ok_s.lst Long/ok_l.lst" for regress?

Edit: Also I just noticed that the `Tst` folder is not included in the tarballs found at the uni-kl.de ftp server. Since the patch releases aren't tagged on GitHub, that is currently the only reliable source for the latest release of singular.
Author:  hannes [ Tue Jul 31, 2018 4:28 pm ]
Post subject:  Re: Tests for packaging
make check: very simple basic tests, should work also without externel libs (like NTL, flint.cddlib)

make distcheck: make check + checking the completeness of the result of make dist

regress.cmd Old/universal.lst Buch/buch.lst Short/ok_s.lst Plural/short.lst - recommended tests (which also tests stuff with NTL, flint)

regress.cmd Old/universal.lst Buch/buch.lst Short/ok_s.lst Plural/short.lst Plural/dmod.lst Plural/doc.lst Manual/s.lst Plural.lst BuchDL/buchdl.lst New.lst Long/ok_l.lst Short.lst Manual/m.lst Long/ok_l2.lst - complete list of tests
Author:  timokau [ Tue Jul 31, 2018 5:57 pm ]
Post subject:  Re: Tests for packaging
Okay, thank you. And what would be the best way to get the source of a release with the tests included? As I said fetching it from GitHub would be perfect, but the releases are not tagged there.
Author:  hannes [ Wed Aug 01, 2018 4:07 pm ]
Post subject:  Re: Tests for packaging
ftp://www.mathematik.uni-kl.de/pub/Math ... 1p3.tar.gz provides the missing test files corresponding to singular-4.1.1p3.tar.gz
Author:  timokau [ Fri Aug 03, 2018 2:43 pm ]
Post subject:  Re: Tests for packaging
Do you plan to continue including `-tst` files for future releases? Or maybe include the `Tst` folder in the regular releases?

I have the tests working now. However if I create a test failure (by not passing `--enable-gfanlib`), regress.cmd apparently doesn't exit with exit code 1:

```
perl ./regress.cmd -s "$out/bin/Singular" \
Old/universal.lst \
Buch/buch.lst \
Short/ok_s.lst \
Plural/short.lst || exit 1
echo "Exit status $?"
```

This script doesn't exit early and reports "Exit status 0". Is that intended?

Some feedback: I needed to add "sharutils" as a dependency (for unencode and undecode), which isn't documented anywhere. Also it would be perfect if those tests could be run by `make installcheck`.

Off topic: Is it somehow possible to enable email notifications for this forum? I've enabled "Notify me upon replies by default" in "Board preferences -> posting defaults", but I still don't get emails.
Author:  hannes [ Tue Aug 07, 2018 3:48 pm ]
Post subject:  Re: Tests for packaging
Somehow we will make the test files available.

But it will not become part of usual tar files, and not part of make:
- it would add additional dependencies (gunzip, uudecode/uuencode, perl)
- a difference in the test files is not necessary a bug: for example diff between 32bit version and 64bit
Author:  timokau [ Tue Aug 07, 2018 10:45 pm ]
Post subject:  Re: Tests for packaging
But the dependencies wouldn't affect users that don't run the tests, so I don't see the harm in including the tests.

> a difference in the test files is not necessary a bug: for example diff between 32bit version and 64bit

That sounds like the tests aren't a good fit for packaging after all. If the packaging tests fail, that should indicate a bug / packaging failure / mismatched dependency. Many false-positives will just make people ignore the tests.

So is there a subset of the tests that you would recommend running for that purpose?

Also, what about the exit code issue I mentioned in the last post? Is that a known problem or did I do something wrong?
Author:  timokau [ Sun Aug 19, 2018 2:44 pm ]
Post subject:  Re: Tests for packaging
Bumping this, I hope that isn't against forum etiquette.
Page 1 of 1 All times are UTC + 1 hour [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/