Two points:

1. There is a test/README, which is more like advice/reminders to ourselves when we write test recipes
2. I think we need to ask the question "is this helpful for the user?", as the stuff that you're moving is intended for use consumption. Does it help them to have to go looking in another file when it comes to testing details?

Travis doesn't seem relevant.

With the hyperlinks, going across to the other file is easy. I'm fine with this. @levitte, dismiss my review if you feel strongly.

@levitte,

1. There is a test/README, which is more like advice/reminders to ourselves when we write test recipes.

Ah, yes! I somehow recalled that there was a README already, but then I overlooked it.
I suggest that I'll append that more internal info to the end of the new test/README.md, but to do this after this PR and #12109 have been merged because otherwise there would be a heavy merge conflict.
I suggest moving also the contents of NOTES.VALGRIND into test/README.md on that occasion.

BTW, would be good if #12109 gets approved soon..

2. I think we need to ask the question "is this helpful for the user?", as the stuff that you're moving is intended for use consumption. Does it help them to have to go looking in another file when it comes to testing details?

I agree with @paulidale that due to the hyperlinks going to the other file should be convenient to do. Moreover, many users will expect info on using the tests in test/ folder, not within ./INSTALL.md.

Yesterday I had started cleaning up of references and code/symbol quotation layout in INSTALL.md, and I've just completed this, adding a new commit for that.
Can you please re-approve?

hyperlinks

When you get a tarball and there's an WHATEVER.md, how do you read it? I usually run less or similar on it. Hyperlinks aren't the best in that case. A browser? If it knows a way to render markdown...

Moreover, many users will expect info on using the tests in test/ folder, not within ./INSTALL.md.

I do not know what users you speak for. To me, that signals that testing isn't that important.

hyperlinks

When you get a tarball and there's an WHATEVER.md, how do you read it? I usually run less or similar on it.

Me too.

Hyperlinks aren't the best in that case.

Agreed.
An alternative would be to put the same text in both places, which is even more sub-optimal in my view.

A browser? If it knows a way to render markdown...

If I use Firefox for file browsing with a URL like this: file:///home/david/openssl/prepare/
and then click on a .md file (or anything else with 'unknown' file extension) that silly thing does not even attempt rendering this as text. Instead, it asks me to open the file in an editor.
Chrome/Chromium does better here: it renders such files as plain text.

Yet when browsing the file via GitHub, e.g., via
https://github.com/siemens/openssl/blob/add_test_README.md/INSTALL.md#test-failures
everything works fine.

Moreover, many users will expect info on using the tests in test/ folder, not within ./INSTALL.md.

I do not know what users you speak for. To me, that signals that testing isn't that important.

When I as a user wanna know details about testing, I'd expect them in the test/ folder README, not buried under ./INSTALL.md.
And I did not say that all users think this way but just many of them 😉

We do still stress the importance of running the tests in INSTALL.md at the points where it makes most sense:

Test OpenSSL

After a successful build, and before installing, the libraries should be tested. Run:

make test                                      # Unix
mms test                                       ! OpenVMS
nmake test                                     # Windows

Warning: you MUST run the tests from an unprivileged account (or disable your privileges temporarily if your platform allows it).

See the file test/README.md for further details.

and

Running Selected Tests

You can specify a set of tests to be performed using the make variable TESTS.

See the section Running Selected Tests of test/README.md.

and

Test Failures

If some tests fail, look at the output. There may be reasons for the failure that isn't a problem in OpenSSL itself (like an OS malfunction or a Perl issue).

You may want increased verbosity, that can be accomplished as described in section Test Failures of test/README.md.

If you find a problem with OpenSSL itself, try removing any compiler optimization flags from the CFLAGS line in the Makefile and run make clean; make or corresponding.

To report a bug please open an issue on GitHub, at https://github.com/openssl/openssl/issues.

For more details on how the make variables TESTS can be used, see section Running Selected Tests of test/README.md.

Note also that I did not remove any of these sections regarding tests but just moved the details (namely on selecting specific tests and choosing verbosity) to test/README.md.

@romen since you brought up the idea for such a change, WDYT about the current state of this PR?

An alternative would be to put the same text in both places, which is even more sub-optimal in my view.

No, please, that's a maintenance issue waiting to happen.

Yet when browsing the file via GitHub, e.g., via
https://github.com/siemens/openssl/blob/add_test_README.md/INSTALL.md#test-failures
everything works fine.

Yeeeeeah, and the user will intuitively know to go there when they see INSTALL.md in their source tree. Right.

An alternative would be to put the same text in both places, which is even more sub-optimal in my view.

No, please, that's a maintenance issue waiting to happen.

Good that we agree here.

Yet when browsing the file via GitHub, e.g., via
https://github.com/siemens/openssl/blob/add_test_README.md/INSTALL.md#test-failures
everything works fine.

Yeeeeeah, and the user will intuitively know to go there when they see INSTALL.md in their source tree. Right.

Sure 😉

Of course this is not what we can always expect.
Yet whenever a user starts looking at the project via the natural entry point https://github.com/openssl/openssl all those .md files are rendered well, including their hyperlinks.

There is the inherent problem after we started switching to .md files that local file browser support is (so far) bad. Non-clickable references, being discussed here, is just one of the resulting nuisances.

Yet whenever a user starts looking at the project via the natural entry point https://github.com/openssl/openssl all those .md files are rendered well, including their hyperlinks.

Please stop. We announce tarballs when releasing, they are downloadable from https://www.openssl.org/source/, that's what most users are going to go for. From that point of view, there's nothing "natural" about the github repo.

Yet whenever a user starts looking at the project via the natural entry point https://github.com/openssl/openssl all those .md files are rendered well, including their hyperlinks.

Please stop. We announce tarballs when releasing, they are downloadable from https://www.openssl.org/source/, that's what most users are going to go for. From that point of view, there's nothing "natural" about the github repo.

Well, I was not thinking about the way users obtain the sources (I agree that for most of them it will be via the sources linked from https://www.openssl.org/, but could also be via git etc.),
but how new users would approach OpenSSL. When googling for 'openssl', the GitHub repo is already the 3nd hit after the normal project home page and the Wikipedia entry), and more and more folks will be using that for its convenience.

You are right we cannot generally expect people to use the online browsing approach.
BUT, following this logic, we should not have converted any of the doc/text files to MarkDown.
As mentioned, their offline file browsing support is not yet wide-spread,
and this pertains not only to hyperlinks but also to their general rendering (layout etc.).
This is an inherent problem, by far not only affecting the changes proposed in this PR.

BUT, following this logic, we should not have converted any of the doc/text files to MarkDown.

Why not? They are still readable as plain text, that's actually the most central feature with MarkDown!

Why not? They are still readable as plain text, that's actually the most central feature with MarkDown!

Ok, for most part they are pretty well readable.
Yet already the existing (intra- and inter-file) cross-references are hard to read and use manually, in particular at the beginning of INSTALL.md:

Table of Contents
=================

 - [Prerequisites](#prerequisites)
 - [Notational Conventions](#notational-conventions)
 - [Quick Installation Guide](#quick-installation-guide)
   - [Building OpenSSL](#building-openssl)
   - [Installing OpenSSL](#installing-openssl)
 - [Configuration Options](#configuration-options)
   - [API Level](#api-level)
   - [Cross Compile Prefix](#cross-compile-prefix)
   - [Build Type](#build-type)
   - [Directories](#directories)
   - [Compiler Warnings](#compiler-warnings)
   - [ZLib Flags](#zlib-flags)
   - [Seeding the Random Generator](#seeding-the-random-generator)
   - [Enable and Disable Features](#enable-and-disable-features)
   - [Displaying configuration data](#displaying-configuration-data)
 - [Installation Steps in Detail](#installation-steps-in-detail)
   - [Configure](#configure-openssl)
   - [Build](#build-openssl)
   - [Test](#test-openssl)
   - [Install](#install-openssl)
 - [Advanced Build Options](#advanced-build-options)
   - [Environment Variables](#environment-variables)
   - [Makefile Targets](#makefile-targets)
   - [Running Selected Tests](#running-selected-tests)
 - [Troubleshooting](#troubleshooting)
   - [Configuration Problems](#configuration-problems)
   - [Build Failures](#build-failures)
   - [Test Failures](#test-failures)
 - [Notes](#notes)
   - [Notes on multi-threading](#notes-on-multi-threading)
   - [Notes on shared libraries](#notes-on-shared-libraries)
   - [Notes on random number generation](#notes-on-random-number-generation)

Prerequisites
=============

To install OpenSSL, you will need:

 * A make implementation
 * Perl 5 with core modules (please read [NOTES.PERL](NOTES.PERL))
 * The Perl module Text::Template (please read [NOTES.PERL](NOTES.PERL))
 * an ANSI C compiler
 * a development environment in the form of development libraries and C
   header files
 * a supported operating system

For additional platform specific requirements, solutions to specific
issues and other details, please read one of these:

 * [NOTES.UNIX](NOTES.UNIX) - notes for Unix like systems
 * [NOTES.VMS](NOTES.VMS) - notes related to OpenVMS
 * [NOTES.WIN](NOTES.WIN) - notes related to the Windows platform
 * [NOTES.DJGPP](NOTES.DJGPP) - building for DOS with DJGPP
 * [NOTES.ANDROID](NOTES.ANDROID) - building for Android platforms (using NDK)
 * [NOTES.VALGRIND](NOTES.VALGRIND) - testing with Valgrind
 * [NOTES.PERL](NOTES.PERL) - some notes on Perl

It's a matter of taste... the titles in the table of contents match the headings further down, and it's true that we have separate NOTES files for the non-common stuff.

This still doesn't change my opinion on moving the test specific stuff (which is common for all platforms, I might add)

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set. Please review the updates and set the label manually.

Re: hyperlinks

I might be weird, but even when reading markdown on the terminal, if I find an hyperlink to another file in the Repo I have no problem in pointing my editor/viewer/pager to the file I am pointed to, and actually I prefer to have logical separation among different files (when it is worth it) instead of a monolithic single long file.

E.g. I'd rather avoid to merge the current content of test/README or test/NOTES.VALGRIND into test/README.md and I would rather provide links.

My preference for individual files is that I tend to read the whole file only once (case in which split files is annoying) and many more times I am more likely to be looking for some details about a specific topic. In the latter case having separate files means that I can still use git grep (or a recursive grep/ack/ag/pt/whatever if dealing with a source tarball without git info) if memory fails me and I want to quickly check which file covered which topic, and grepping for keywords within that file leads to less false positives than grepping for a pattern in a single monolithic file.

Re: hyperlinks

I might be weird, but even when reading markdown on the terminal, if I find an hyperlink to another file in the Repo I have no problem in pointing my editor/viewer/pager to the file I am pointed to,

Interesting - with which offline tool does this work for you?
It does not work for me with less run in a Bash shell,
neither with Emacs within the terminal or on a separate window.

and actually I prefer to have logical separation among different files (when it is worth it) instead of a monolithic single long file.

Me too. This is another reason I don't like all those details on tests in INSTALL.md.

E.g. I'd rather avoid to merge the current content of test/README or test/NOTES.VALGRIND into test/README.md and I would rather provide links.

Fine by me.

My preference for individual files is that I tend to read the whole file only once (case in which split files is annoying) and many more times I am more likely to be looking for some details about a specific topic. In the latter case having separate files means that I can still use git grep (or a recursive grep/ack/ag/pt/whatever if dealing with a source tarball without git info) if memory fails me and I want to quickly check which file covered which topic, and grepping for keywords within that file leads to less false positives than grepping for a pattern in a single monolithic file.

Good point.

[foo/bar.md](foo/bar.md) is equivalent to just [foo/bar.md]

Let's put that to the test, shall we?

• [foo/bar.md](foo/bar.md) : foo/bar.md
• [foo/bar.md] : [foo/bar.md]

I would prefer to simply append the current contents of test/README to test/README.md,

Two very different audiences. I would rather not.

I would prefer to simply append the current contents of test/README to test/README.md,

Two very different audiences. I would rather not.

All right.

Re: readability of links

[foo/bar.md](foo/bar.md) is equivalent to just [foo/bar.md], but in many instances we are still using the explicit link form: readability without markdown renderers could be improved switching to the implicit form.

Argh, after doing this in all of INSTALL.md I found that although this seems to be possible according to https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#links

That's because you misunderstand. [link text itself] is a reference link, the actual link is a few lines down, looking like this:

[link text itself]: http://www.reddit.com

Note that we have external links in this form

Re: readability of links
[foo/bar.md](foo/bar.md) is equivalent to just [foo/bar.md], but in many instances we are still using the explicit link form: readability without markdown renderers could be improved switching to the implicit form.

Argh, after doing this in all of INSTALL.md I found that although this seems to be possible according to https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#links

That's because you misunderstand. [link text itself] is a reference link, the actual link is a few lines down, looking like this:

[link text itself]: http://www.reddit.com

Note that we have external links in this form

Got it - thanks @levitte for the clarification!

I am sorry for the confusion and the extra work I caused about implicit links!

I was clearly mistaken!

All my feedback was addressed and the leftover items are fine by me if they stay as currently proposed or not.

Thanks @romen for your further comments and approval.

I just changed the compiler name references as mentioned above,
and reverted the removal of the $ command prefix that I had done also in this PR because @levitte and others did not like that (as discussed e.g. in #12257).

What still remains open here:

@levitte, you questioned the idea of moving (as I clarified, just the details of) the information about testing out of INSTALL.md into test/README.md. After having discussed this a bit, is this meanwhile acceptable to you?

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set. Please review the updates and set the label manually.

I've rebased this PR to solve merge conflicts on INSTALL.md
and did a further little improvement on its Test Failures section.

Could any OTC member please re-approve.

@levitte, you questioned the idea of moving (as I clarified, just the details of) the information about testing out of INSTALL.md into test/README.md. After having discussed this a bit, is this meanwhile acceptable to you?

I haven't changed my mind, but am clearly a minority, and I don't care enough to fight harder. Que sera, sera...

@levitte, you questioned the idea of moving (as I clarified, just the details of) the information about testing out of INSTALL.md into test/README.md. After having discussed this a bit, is this meanwhile acceptable to you?

I haven't changed my mind, but am clearly a minority, and I don't care enough to fight harder. Que sera, sera...

Thanks - so I'm taking this as an implicit approval, or at least no objection against the two existing ones :)

Rebased, solved conflicts, and merged.
Thanks to all who took part discussing this.