Skip to content

Generate man pages from the Command description#23825

Merged
icecrime merged 3 commits intomoby:masterfrom
dnephin:auto-gen-man-page
Jul 19, 2016
Merged

Generate man pages from the Command description#23825
icecrime merged 3 commits intomoby:masterfrom
dnephin:auto-gen-man-page

Conversation

@dnephin
Copy link
Member

@dnephin dnephin commented Jun 21, 2016

spf13/cobra includes support for generating a man page. It makes use of cpuguy83/go-md2man which we already use to generate the existing man pages from markdown.

The cobra generated man page is very similar to the output of our current generation. There are a few small differences. If any of them are considered to be significant, we can update the generation code to output what we want.

Notable differences:

  • the SYNOPSIS string only says OPTIONS instead of the full list of flags
  • the command group page (docker volume, docker network, etc) lists the child commands under "SEE ALSO" on a single line, instead of listing them out one-per-line in a "COMMANDS" section
  • the date is displayed in the footer of the page (where it currently says "Docker User Manuals") instead of in the header

By keeping the command description and examples in the code it should make it easier to keep the man pages up-to-date and accurate. It makes it easier to add new commands, as we can generate the man pages automatically without adding any new files. We can also use this Long (description) text to generate the command reference used by the online docs.

The existing markdown files have not been removed. They will override the generated files if one exists for the same command. If the design of this PR is accepted I can start moving the text for more of the commands into the code.

This PR makes the following changes:

  • update the man/Dockerfile to include all the new required dependencies
  • update the packaging scripts to call the new man/generate.sh to generate all the man pages
  • remove the files for volume commands, and move the description into the Command.Long field.
  • adds man pages for all the new 1.12 commands. These man pages will be a little bare until we add a Long description to the commands.

cc @cpuguy83, @icecrime

@dnephin dnephin added this to the 1.12.0 milestone Jun 21, 2016
@dnephin dnephin removed this from the 1.12.0 milestone Jun 21, 2016
@LK4D4
Copy link
Contributor

LK4D4 commented Jun 21, 2016

@dnephin failures are real

@LK4D4 LK4D4 added the status/failing-ci Indicates that the PR in its current state fails the test suite label Jun 21, 2016
@dnephin dnephin force-pushed the auto-gen-man-page branch from cd79a09 to 90982d3 Compare June 22, 2016 14:13
@dnephin dnephin removed the status/failing-ci Indicates that the PR in its current state fails the test suite label Jun 22, 2016
@LK4D4
Copy link
Contributor

LK4D4 commented Jun 22, 2016

@dnephin Do you know why we need go-md2man in main Dockerfile?

@LK4D4
Copy link
Contributor

LK4D4 commented Jun 22, 2016

LGTM

@vdemeester
Copy link
Member

hum, make manpages failed for me :

[ERROR] Failed to set version on github.com/spf13/cobra to dc45219961f875acff5ee07ed263e5dc19e0c5f1: exit status 128
[INFO] Setting version for github.com/spf13/viper to c1ccc378a054ea8d4e38d8c67f6938d4760b53dd.
[INFO] Setting version for golang.org/x/sys to 62bee037599929a6e9146f29d10dd5208c43507d.
[INFO] Setting version for gopkg.in/yaml.v2 to a83829b6f1293c91addabc89d0571c246397bbf4.
An Error has occurred
The command '/bin/sh -c glide install && mv vendor src' returned a non-zero code: 2

@dnephin
Copy link
Member Author

dnephin commented Jun 23, 2016

I was waiting for the spf13/cobra PR (which included some bug fixes for this) to get merged before tagging. I has been merged now, so I'll update the sha here.

@dnephin dnephin force-pushed the auto-gen-man-page branch from 90982d3 to 8c9fa36 Compare June 23, 2016 14:17
@dnephin
Copy link
Member Author

dnephin commented Jun 23, 2016

Fixed the tag. I'm using v1.3 as the version now.

@albers
Copy link
Member

albers commented Jun 23, 2016

I think the representation of options with arguments in the synopsis is wrong.
Given that expressions inside of brackets are optional, the general syntax of an option with a mandatory argument should be

[--option[=]value]

in order to describe both legal syntax variants --option value and --option=value.

Here is an example from the generated man pages:

[--net[="bridge"]]

This means that --net=bridge and --net are both valid argument expressions, which is not true:

$ docker run --net
flag needs an argument: --net

The wrong syntax is also used in most current static man pages, so I understand why the generated output picked up this syntax. This is a good moment to tidy this up.

@albers
Copy link
Member

albers commented Jun 23, 2016

Please ignore my previous comment.
I did not notice that only selected commands have been ported so far and picked a non-generated one.

@dnephin dnephin force-pushed the auto-gen-man-page branch from 8c9fa36 to 390221a Compare June 24, 2016 19:04
@dnephin dnephin added this to the 1.12.0 milestone Jun 29, 2016
@dnephin dnephin added area/cli Client priority/P2 Normal priority: default priority applied. labels Jun 29, 2016
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https:// ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, this was just copied from the old markdown man page

@dnephin dnephin force-pushed the auto-gen-man-page branch from 390221a to 9f677e5 Compare July 3, 2016 15:23
@icecrime
Copy link
Contributor

icecrime commented Jul 5, 2016

That's really awesome 👍 LGTM (cc @tianon @SvenDowideit who might be interested too).

@SvenDowideit
Copy link
Contributor

LGTM - we'll want to consume this into the cli docs eventually too

@thaJeztah
Copy link
Member

I'm still a bit on the fence on this one. I'm happy with it being generated
automatically, but on the other hand, there's some things "attached" to it.
I was writing down some things, but not finished, so I'll paste my quick notes here;

  • making "documentation" PR's becomes more difficult; to make
    changes to this part of the documentation now requires contributors
    to touch the actual code
  • related to the above; we still have to maintain the online documentation
    /reference (which has a lot of the same content). Can we still copy/paste
    between the Markdown files and the embedded docs (looking at it,
    back-tics will break?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we stick to 80-chars for these strings as well, to be consistent with our markdown?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @dnephin

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this from the markdown without reformatting it. So it is consistent with the old markdown at least :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We stick to 80 chars, perhaps this slipped through during review, but let's not inherit that mistake

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I fixed it

@vdemeester
Copy link
Member

I kinda like that but I'm wondering how to handle more complicated manpages, like docker-ps.1.md or docker-run.1.md where we have longer description for some/most of the flags than what we have/describe in the code (and thus in --help). For example the list of possible --filter for ps (or --format) or the big description for --cpu-shares in run, …

There is also some manpages where we have an EXAMPLES section that should be taken care of.

So I'm torn 😅. I like that but I feel we should see how to handle these more complex command manpages before doing anything.

@dnephin
Copy link
Member Author

dnephin commented Jul 13, 2016

If we need additional text, we can add it to the description. It's not uncommon for man pages to have sections for things that require more explanation. If we really want a more verbose man page for some commands we can keep the md around for those. There is always an override from .md file.

The default should really be automatic generation from the source. If that's insufficient for some flags or commands we have the ability to extend the generated docs, but I think that is more the exception than the rule.

For example the list of possible --filter for ps

This is an excellent example. We can document that by adding the following section to the description:


## Filters

exited=<int>
  filter to containers which have an exit code of <int>
...

Putting this detail into a section in the description gives us more space to describe things.

There is also some manpages where we have an EXAMPLES section

cobra supports the example section as well. I use it in this PR for volume rm. In the end this is just another section, so it could just as easily go into the description.

I feel we should see how to handle these more complex command manpages before doing anything.

Hopefully that covers it, but if not I'd be happy to provide more examples.

@dnephin dnephin force-pushed the auto-gen-man-page branch from 9f677e5 to 5116bdc Compare July 13, 2016 18:57
@dnephin dnephin force-pushed the auto-gen-man-page branch from 5116bdc to ee3f562 Compare July 13, 2016 19:00
@dnephin
Copy link
Member Author

dnephin commented Jul 19, 2016

@thaJeztah any outstanding concerns on this?

dnephin added 3 commits July 19, 2016 12:00
Use the generate.sh script instead of md2man directly.
Update Dockerfile for generating man pages.

Signed-off-by: Daniel Nephin <[email protected]>
@dnephin dnephin force-pushed the auto-gen-man-page branch from ee3f562 to 47cca88 Compare July 19, 2016 16:01
@thaJeztah
Copy link
Member

@dnephin I still have concerns, but perhaps the best way is to test/try it and see if it works well in our workflow. We can evaluate after a while. So "lgtm"

@thaJeztah
Copy link
Member

@dnephin really appreciated though, please don't take it the wrong way 👍

@icecrime
Copy link
Contributor

Alright let's go for it then 😇 LGTM

@icecrime icecrime added status/4-merge process/cherry-pick and removed area/cli Client priority/P2 Normal priority: default priority applied. status/2-code-review labels Jul 19, 2016
@icecrime
Copy link
Contributor

Not sure if we want to cherry-pick this.

@tiborvass @thaJeztah Adding the label, but up to you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants