Skip to content

Infrastructure for templated doc in POD files#10159

Closed
richsalz wants to merge 2 commits intoopenssl:masterfrom
richsalz:doc-template
Closed

Infrastructure for templated doc in POD files#10159
richsalz wants to merge 2 commits intoopenssl:masterfrom
richsalz:doc-template

Conversation

@richsalz
Copy link
Copy Markdown
Contributor

This adds infrastructure so that we can use Perl variables for common flags.
The build system, and manpages, aren't wired up to it yet.
This replaces #10156 which I am about to close.
This is ready for review.

@richsalz
Copy link
Copy Markdown
Contributor Author

I added more error-checking to the dofile script; I wasted time because there were no errors output.

I think dofile should have another flag that means "for input file foo.in write to foo" That would make the makefiles simpler and, given the error-checking addition, more correct now.

Comment thread doc/perlvars.pl Outdated
@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 13, 2019

You haven't elaborated how this would be integrated in the build system. Remember, the first token in a GENERATE value is the generator, and that's the .in file name. We do not have the language to add flags before that name (between util/dofile.pl and whatever.in). Of course, given the possibility for build.info attributes, it's not impossible.

Something I don't understand, though, is how the introduction of -i gives less complicated Makefiles. You have basically traded this (that everyone understands):

util/dofile.pm foo.in > foo

for this (which isn't what I would call intuitive):

util/dofile.pl -i foo.in

Right?

A totally different thing, though, is the error handling. That is nicely fixed, and quite frankly, if there was a PR with only that fix, I would approve it on the spot. Sometimes, dividing things onto one-theme PRs is a good thing, ya know 😉

Comment thread util/dofile.pl Outdated
Comment thread util/dofile.pl Outdated
@richsalz
Copy link
Copy Markdown
Contributor Author

Something I don't understand, though, is how the introduction of -i gives less complicated Makefiles

util/dofile -Mconfigdata -i.in -s doc/perlvars.pl doc/man1/*.in

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 13, 2019

That doesn't help parallell jobs much. Also, build.info has zero support for wildcards

Comment thread Configurations/unix-Makefile.tmpl Outdated
Comment thread util/dofile.pl Outdated
Comment thread util/dofile.pl Outdated
@richsalz
Copy link
Copy Markdown
Contributor Author

Fixup commit pushed. I don't think you'll like the way I did the Unix makefile, but I couldn't make my way through the depth of the templates. And anyway we still need to resolve the other system.s

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 13, 2019

Regarding your Makefile template hack, you're right that I strongly dislike it. That's one more hack that we must remember to duplicate across all build file templates, so that's the wrong direction to follow. Specifying the file generation via GENERATE lines in build.info is the way to make it platform agnostic, and also means that we have a clear declaration of everything we want to build one way or another, with minimal hidden hacks that will be hard to see a year or two later.

As it is right now, the GENERATE line just takes the value and uses that as arguments with no regard for anything, so for example, this line:

GENERATE[foo]=foo.in whatever

Translates to (on my setup):

$(PERL) "-I$(BLDDIR)" -Mconfigdata "../master/util/dofile.pl" \
	    "-oMakefile" ../master/foo.in whatever > "foo"

However, what we could do to support your kind of construct is to translate something like this:

GENERATE[foo]=foo.in
DEPEND[foo.in]=perlvars.pm

to:

$(PERL) "-I$(BLDDIR)" -Mconfigdata -I../master/ -Mperlvars "../master/util/dofile.pl" \
	    "-oMakefile" ../master/foo.in whatever > "foo"

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 13, 2019

(as a matter of fact, that build.info construct is already supported, all that's needed is a small change to allow anything ending with .pm to get loaded by perl with -M)

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 13, 2019

#10162 allows the following construct:

GENERATE[doc/man1/openssl-s_client.pod]=doc/man1/openssl-s_client.pod.in
DEPEND[doc/man1/openssl-s_client.pod]=doc/perlvars.pm
GENERATE[doc/man1/openssl-s_server.pod]=doc/man1/openssl-s_server.pod
DEPEND[doc/man1/openssl-s_server.pod]=doc/perlvars.pm

Add that to the top build.info and see what happens (merged with my PR, of course). Oh, you'll have to rename doc/perlvars.pl to doc/perlvars.pm...

@richsalz
Copy link
Copy Markdown
Contributor Author

I will rebase and use the your technique once #10162 is merged.

@richsalz
Copy link
Copy Markdown
Contributor Author

So additional commit pushed. Now I just need to know which build.info file ? Should I add a new one in doc? If so, how do I do that.

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 14, 2019

So far, I've touch the top one. However, if you prefer to add one in doc/man1, feel free. What you need to do is to make sure there's a SUBDIRS statement that leads there. For example, this in the top build.info will do what you want:

SUBDIRS=doc/man1

@richsalz
Copy link
Copy Markdown
Contributor Author

And add a target/dependency to build_generated?

@richsalz
Copy link
Copy Markdown
Contributor Author

(Adding that dependency means modifying the three makefile templates, right?)

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 14, 2019

Nope, that's done "magically" with this dependency:

DEPEND[]=whatever

(the empty brackets are significant for this)

@richsalz
Copy link
Copy Markdown
Contributor Author

I don't think my question was clear: how do I make sure that the templated-docs are built at the same time all other generated files are built? Are you saying, if I just put

   DEPEND[] = doc/man1/openssl-s_client.pod ...etc...

then it will work?

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 14, 2019

Yes, that's right. Now, for your stuff specifically, since you depend on doc/perlvars.pm, you will also need this:

DEPEND[doc/man1/openssl-s_client.pod]=doc/perlvars.pm

@richsalz
Copy link
Copy Markdown
Contributor Author

this works, except @autowarntext isn't populated. thoughts?

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 14, 2019

Catch 22. perlvars.pm is loaded before dofile.pl starts and thereby before @autowarntext is created. I didn't think of that. Although, I also am not sure why you can't just have this in your .pod.in files:

=begin comment

{- join("\n", @autowarntext) -}

=end comment

It's not quite as "magic"

@richsalz
Copy link
Copy Markdown
Contributor Author

putting the text explicitly seems the simplest thing to do (for now :).

@richsalz
Copy link
Copy Markdown
Contributor Author

also set, ready for review. thanks.

@richsalz
Copy link
Copy Markdown
Contributor Author

It would be great if this could get reviewed and merged next. I will then rebase all the other doc refactoring PR's to use this. It will make reviewing those other half-dozen PR's much simpler, as it replaces whole clumps of line with a perl variable reference.

Okay, I exaggerate. It's only five other PR's. But still...

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 15, 2019

One, two, three, gazillion 😉

@richsalz
Copy link
Copy Markdown
Contributor Author

I am starting to backport #10118 (as a new PR) to use this. So yeah, a pico-gazillion at least.

@levitte levitte added branch: master Applies to master branch approval: review pending This pull request needs review by a committer labels Oct 24, 2019
@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 26, 2019

You do know, right, that you don't have to rebase all the time? It's only necessary when github shows you there's a merge conflict

@richsalz
Copy link
Copy Markdown
Contributor Author

Everyone should have a hobby :)

@levitte
Copy link
Copy Markdown
Member

levitte commented Oct 26, 2019

Yeah, so reviewers get to wonder if there's anything new to review, might realise that there isn't, possibly after looking at the latest force-pushed link... either way, that takes extra time, which may slow down approval. So for a hobby, it's not a smart one...

@richsalz
Copy link
Copy Markdown
Contributor Author

I'll try to refrain.

Use new doc-build capabilities
Add -i flag to dofile.
Add doc/man1 to SUBDIRS for the new templated doc files
Rewrite commit a397aca (merged from PR 10118) to use the doc-template stuff.
Put template references in common place
Template options and text come at the end of command-specific options:
opt_x, opt_trust, opt_r (in that order).
Refactor xchain options.
Do doc-nits after building generated sources.
Comment thread .gitignore Outdated
Copy link
Copy Markdown
Member

@t8m t8m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from the typo and nit looks ok.

Comment thread doc/man1/openssl-ca.pod.in Outdated
Comment thread doc/man1/openssl-cms.pod.in Outdated
Copy link
Copy Markdown
Member

@t8m t8m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved now, @levitte does your approval still hold?

@t8m t8m added approval: omc review pending and removed approval: review pending This pull request needs review by a committer labels Oct 29, 2019
@t8m t8m added approval: done This pull request has the required number of approvals and removed approval: omc review pending labels Oct 30, 2019
@t8m t8m self-assigned this Oct 30, 2019
@mattcaswell mattcaswell added approval: ready to merge The 24 hour grace period has passed, ready to merge and removed approval: done This pull request has the required number of approvals labels Oct 31, 2019
openssl-machine pushed a commit that referenced this pull request Oct 31, 2019
Use new doc-build capabilities
Add -i flag to dofile.
Add doc/man1 to SUBDIRS for the new templated doc files
Rewrite commit a397aca (merged from PR 10118) to use the doc-template stuff.
Put template references in common place
Template options and text come at the end of command-specific options:
opt_x, opt_trust, opt_r (in that order).
Refactor xchain options.
Do doc-nits after building generated sources.

Reviewed-by: Tomas Mraz <[email protected]>
Reviewed-by: Richard Levitte <[email protected]>
(Merged from #10159)
@t8m
Copy link
Copy Markdown
Member

t8m commented Oct 31, 2019

Merged to master as 9fcb970

@t8m t8m closed this Oct 31, 2019
@richsalz richsalz deleted the doc-template branch October 31, 2019 13:24
@richsalz
Copy link
Copy Markdown
Contributor Author

Thanks for your feedback (and perl assistance) in getting this done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: ready to merge The 24 hour grace period has passed, ready to merge branch: master Applies to master branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants