Skip to content

User manual#446

Merged
jedbrown merged 4 commits intomasterfrom
user-manual
Mar 8, 2020
Merged

User manual#446
jedbrown merged 4 commits intomasterfrom
user-manual

Conversation

@valeriabarra
Copy link
Copy Markdown
Contributor

@valeriabarra valeriabarra commented Jan 22, 2020

Our very first user manual!

It uses sphinx, which is able to read both Markdown (although I still have to figure out some small things see below) and ReStructuredText files. Math formulas are rendered nicely in html directly from ReStructuredText files using the mathjax extension (a bit of reformatting using the proper directives is necessary if you want to convert a latex file straight to an rst file).

TO DOs:
- [ ] Figure out how to properly parse and render existing \snippet from md files (see, e.g., libCEEDapi.md:238). this seems to not be possible, but I have converted the md files to rst files and used different directives for code snippet inclusions.

  • Include man pages from code with breathe.
  • Add release notes page
  • Figure out citations with sphinxcontrib-bibtex or other available extensions.
  • Conclude description of all existing examples.
  • Add getting started page

@codecov
Copy link
Copy Markdown

codecov bot commented Jan 22, 2020

Codecov Report

Merging #446 into master will decrease coverage by 0.65%.
The diff coverage is 97.51%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #446      +/-   ##
==========================================
- Coverage   97.22%   96.57%   -0.66%     
==========================================
  Files         218      222       +4     
  Lines       12546    13038     +492     
==========================================
+ Hits        12198    12591     +393     
- Misses        348      447      +99
Flag Coverage Δ
#backends 94.16% <ø> (ø) ⬆️
#examples 79.87% <ø> (?)
#gallery 100% <ø> (ø) ⬆️
#interface 94.95% <97.51%> (ø) ⬆️
#tests 99.4% <ø> (ø) ⬆️
Impacted Files Coverage Δ
interface/ceed-tensor.c 100% <ø> (ø) ⬆️
interface/ceed-vector.c 96.74% <100%> (ø) ⬆️
interface/ceed-elemrestriction.c 100% <100%> (ø) ⬆️
interface/ceed-qfunction.c 100% <100%> (ø) ⬆️
interface/ceed.c 92.54% <85%> (ø) ⬆️
interface/ceed-operator.c 97.09% <95.16%> (ø) ⬆️
interface/ceed-basis.c 99.14% <99.31%> (ø) ⬆️
examples/ceed/ex2-surface.h 49.18% <0%> (ø)
examples/ceed/ex1-volume.h 70% <0%> (ø)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed264d0...2517cf5. Read the comment docs.

@valeriabarra
Copy link
Copy Markdown
Contributor Author

@jedbrown I have integrated our doxygen code documentation with breathe. Unfortunately, the lowest-maintainability option that they provide with the directive doxygenindex is very messy (it outputs all doxygen documentation in one section, with no semantic separation—see this for instance, or this). So I had to use the doxygengroup directive to separate things in sections, at least. This creates a bit of maintenance, in case new semantic objects are added to the library.

@valeriabarra valeriabarra added this to the v0.6 milestone Jan 30, 2020
@valeriabarra
Copy link
Copy Markdown
Contributor Author

Update:

Since we have two pretty lengthy markdown files with API documentation and developer notes, namely, libCEEDapi.md and libCEEDdev.md, I have tried to reuse them, instead of converting them to rst files (which would need reformatting all math and other features). But I have encountered a major obstacle.

The only outstanding issue in this PR right now is the ability to parse previously existing doxygen markdown directives, for instance \include and \snippet directives (see e.g., libCEEDapi.md:238).
I have tried different markdown parsers available for sphinx, such as recommonmark and markdown-parser, but since the \include and \snippet directives are not standard markdown commands, but doxygen ones, they are not parsed properly. I have also tried to include the corresponding doxygen-generated file and use the doxygenfile directive in breathe, but it didn't work either.

@valeriabarra valeriabarra removed the 0-WIP label Feb 4, 2020
@valeriabarra valeriabarra changed the title WIP: User manual User manual Feb 4, 2020
@valeriabarra
Copy link
Copy Markdown
Contributor Author

Further update:

Files libCEEDapi.md and libCEEDdev.md have been converted to libCEEDapi.rst and libCEEDdev.rst, respectively. This allowed to include code as in libCEEDapi.rst:314 or blocks of code as in libCEEDapi.rst:320. However, the code blocks do not have cross-refs as the original libCEEDapi.md file had through the doxygen directives. If this is not desired, I can revert this change.

At this point, this is ready for review. Thanks.

@valeriabarra
Copy link
Copy Markdown
Contributor Author

CodeCov is confused but CI passes.

@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Feb 6, 2020

Do you get these?

$ make html                                                                                                                               
Running Sphinx v2.2.1                                                                                                                                                                          
/usr/lib/python3.8/site-packages/sphinx/util/docutils.py:284: RemovedInSphinx30Warning: function based directive support is now deprecated. Use class based directive instead.                 
  warnings.warn('function based directive support is now deprecated. '                                                                                                                         
making output directory... done                                                                                                                                                                WARNING: html_static_path entry '_static' does not exist
loading intersphinx inventory from https://docs.python.org/objects.inv...                                                                                                                      
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv                                                     
building [mo]: targets for 0 po files that are out of date                                                                                                                                     
building [html]: targets for 9 source files that are out of date                                                                                                                               
updating environment: [new config] 9 added, 0 changed, 0 removed                                                                                                                               
checking for /home/jed/src/libceed/doc/sphinx/source/references.bib in bibtex cache... not found                                                                                               
parsing bibtex file /home/jed/src/libceed/doc/sphinx/source/references.bib... parsed 1 entries
checking for /home/jed/src/libceed/doc/sphinx/source/references.bib in bibtex cache... up to date                                                                                              
reading sources... [100%] releasenotes         
/home/jed/src/libceed/doc/sphinx/source/FEMtheory.rst:4: WARNING: duplicate label theoretical framework, other instance in /home/jed/src/libceed/doc/sphinx/source/FEMtheory.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:18: WARNING: doxygengroup: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:29: WARNING: doxygengroup: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:40: WARNING: doxygengroup: Cannot find file: /home/jed/src/libceed/xml/index.xml                                                              /home/jed/src/libceed/doc/sphinx/source/code.rst:51: WARNING: doxygengroup: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:62: WARNING: doxygengroup: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:73: WARNING: doxygengroup: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:84: WARNING: doxygenenum: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:87: WARNING: doxygenenum: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:90: WARNING: doxygenenum: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:93: WARNING: doxygenenum: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:96: WARNING: doxygenenum: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:99: WARNING: doxygenenum: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:107: WARNING: doxygentypedef: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:110: WARNING: doxygentypedef: Cannot find file: /home/jed/src/libceed/xml/index.xml
/home/jed/src/libceed/doc/sphinx/source/code.rst:4: WARNING: duplicate label code, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:16: WARNING: duplicate label ceed, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:27: WARNING: duplicate label ceedvector, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:38: WARNING: duplicate label ceedelemrestriction, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:49: WARNING: duplicate label ceedbasis, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:60: WARNING: duplicate label ceedqfunction, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:71: WARNING: duplicate label ceedoperator, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:82: WARNING: duplicate label enumerations, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/code.rst:105: WARNING: duplicate label typedefs, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:31: WARNING: duplicate label ceed, other instance in /home/jed/src/libceed/doc/sphinx/source/code.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:4: WARNING: duplicate label examples, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:31: WARNING: duplicate label ceed, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:123: WARNING: duplicate label area, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:138: WARNING: duplicate label cube, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:179: WARNING: duplicate label sphere, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:376: WARNING: duplicate label advection, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/examples.rst:399: WARNING: duplicate label density current, other instance in /home/jed/src/libceed/doc/sphinx/source/examples.rst
/home/jed/src/libceed/doc/sphinx/source/intro.rst:4: WARNING: duplicate label introduction, other instance in /home/jed/src/libceed/doc/sphinx/source/intro.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:4: WARNING: duplicate label implementation, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:16: WARNING: duplicate label finite element operator decomposition, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi
.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:64: WARNING: duplicate label terminology and notation, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:188: WARNING: duplicate label partial assembly, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:224: WARNING: duplicate label parallel decomposition, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:271: WARNING: duplicate label api description, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDapi.rst:487: WARNING: duplicate label interface principles and evolution, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDapi.r
st
/home/jed/src/libceed/doc/sphinx/source/libCEEDdev.rst:4: WARNING: duplicate label developer notes, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDdev.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDdev.rst:9: WARNING: duplicate label shape, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDdev.rst
/home/jed/src/libceed/doc/sphinx/source/libCEEDdev.rst:50: WARNING: duplicate label clang-tidy, other instance in /home/jed/src/libceed/doc/sphinx/source/libCEEDdev.rst
/home/jed/src/libceed/doc/sphinx/source/references.rst:4: WARNING: duplicate citation Brown:2010, other instance in /home/jed/src/libceed/doc/sphinx/source/intro.rst
/home/jed/src/libceed/doc/sphinx/source/references.rst:4: WARNING: duplicate label bibliography, other instance in /home/jed/src/libceed/doc/sphinx/source/references.rst
WARNING: duplicate label for keys Brown:2010 and Brown:2010
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] releasenotes          
generating indices...  genindexdone
writing additional pages...  search/usr/lib/python3.8/site-packages/sphinx_rtd_theme/search.html:21: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert
 a <script> tag directly in your theme instead. 
  {% endblock %}
done
copying images... [100%] ../../img/L-vector-AMR.svg                                            
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 47 warnings.

The HTML pages are in build/html.

@valeriabarra
Copy link
Copy Markdown
Contributor Author

RemovedInSphinx30Warning: function based directive support is now deprecated. Use class based directive instead.
warnings.warn('function based directive support is now deprecated. '

Unfortunately, I searched for the first warning message RemovedInSphinx30Warning: function based directive support is now deprecated. Use class based directive instead. warnings.warn('function based directive support is now deprecated. ' and couldn't find anything useful to get rid of it.

Have you built the xml files first by running make doc at top level? I do not get the Cannot find file: ones.

I do get some warnings for duplicate labels, unfortunately I wasn't able to figure out how to remove them yet.

@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Feb 6, 2020

Looks like the "function based directive" warning is a breathe issue: breathe-doc/breathe#467

There are some other warnings, some of which are related to redundant tags.

Site is live here: https://libceed.readthedocs.io/en/latest/

@jeremylt
Copy link
Copy Markdown
Member

jeremylt commented Feb 6, 2020

This is looking great! One thought - is there a way to integrate the code of conduct and contributing info? It seems like those would be appropriate additions to the "Developer Notes" section.

@jeremylt
Copy link
Copy Markdown
Member

jeremylt commented Feb 6, 2020

Am I correct that the plan is to remove the codedocs site and update the top level README with the new readthedocs site?

@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Feb 6, 2020

@jeremylt I think yes, this should replace the codedocs website, though we may still want Doxygen code listings because it can hyperlink all functions -- we don't know how to do that in Sphinx.

Copy link
Copy Markdown
Contributor Author

@valeriabarra valeriabarra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Strange that now the Contributing page does not appear in the TOC. It only appears after clicking on any other page. Fixed with clearing cached builds

@jedbrown jedbrown force-pushed the user-manual branch 5 times, most recently from a5c0f2b to 65a6697 Compare March 3, 2020 19:30
@jeremylt
Copy link
Copy Markdown
Member

jeremylt commented Mar 6, 2020

This will need the updates on master, but otherwise it looks like it might be ready?

@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Mar 6, 2020

@jeremylt What is your opinion of this "mermaid" diagram? Keep or drop? https://libceed.readthedocs.io/en/latest/api/index.html

The other issue that needs checking is the author list after upstream updates.

I would prefer the examples.rst to be split into each directory -- it's already pretty long and will get longer if we do a legit explanation of the methodology in Navier-Stokes, for example.

@jeremylt
Copy link
Copy Markdown
Member

jeremylt commented Mar 6, 2020

I think that the mermaid is a useful visualization to explain how we plug into user code. I like it.

@jedbrown jedbrown force-pushed the user-manual branch 2 times, most recently from cb2763f to d9898d4 Compare March 6, 2020 17:42
@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Mar 6, 2020

@valeriabarra Thought we should make it nicer (using TikZ or an SVG editor). I lean toward the simplicity of being able to create them like this.

My recent push fixes authorship maintenance.

@valeriabarra
Copy link
Copy Markdown
Contributor Author

@valeriabarra Thought we should make it nicer (using TikZ or an SVG editor). I lean toward the simplicity of being able to create them like this.

My recent push fixes authorship maintenance.

I'm a graphics fan but that's fine if it is the preferred way. Should we at least caption the image saying that it represents the schematic of the API classes or the role of libCEED?

@valeriabarra
Copy link
Copy Markdown
Contributor Author

@valeriabarra Thought we should make it nicer (using TikZ or an SVG editor). I lean toward the simplicity of being able to create them like this.

My recent push fixes authorship maintenance.

Thanks for the AUTHORS update. Will the AUTHORS file be automatically updated?

@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Mar 6, 2020

AUTHORS is automatically updated when people run make doc from the top level. If the information isn't correct, one should update .mailmap to get preferred author names.

@jedbrown
Copy link
Copy Markdown
Member

jedbrown commented Mar 6, 2020

I just like simple and maintainable. If you want to make a nicer figure, that'd be great, but I think there are probably more impactful ways to spend time.

Copy link
Copy Markdown
Member

@jedbrown jedbrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with merging. Once this and #463 are merged, we can split the examples.rst into separate files and make more complete documentation for the cubed sphere and Navier-Stokes cases. Let me know if you'd like help with Git logistics for squashing and merging coherently.

@valeriabarra
Copy link
Copy Markdown
Contributor Author

Yes please. I would be very thankful if you can make some git magic.

jedbrown and others added 4 commits March 8, 2020 17:13
The parser handles line comments fine.  This also avoids malfunction
in case // comments appear in ceed.h (the former rule excluded all
lines containing '///' and ignored '//' lines, resulting in all
subsequent lines being ignored).
Some technical notes about the expanded docs/process:

* Doxygen is now restricted to API documentation.  Sphinx docs are
  incomplete surrogates due to not hyperlinking code listings, though
  we hope the Doxygen site can be removed eventually.

* PDF output uses XeTeX (unicode and OTF fonts) with TeX Gyre Pagella
  and DejaVu Sans Mono.  You may need to install these fonts in order
  to build the PDF users manual.

* Obtaining the author list from Git has not worked on
  ReadTheDocs (`git shortlog` is empty even though `git log` is not),
  so we automatically generate it on developer machines with `make
  doc`.  If the author information is incorrect/mangled, edit .mailmap
  to fix the mapping.

* Math semantics are based on the description here
  sphinx-doc/sphinx#3785

    Sphinx chooses one of two setups for the maths.

      * If there is a single equation, then we wrap the maths in
        \begin{equation*} (or equation).

      * If there are multiple equations (as determined by the presence of
        an empty line), then we wrap the maths first in \begin{aligned}
        and then in \begin{align*} (or align).

Co-authored-by: Jeremy L. Thompson <[email protected]>
Co-authored-by: Jed Brown <[email protected]>
Breathe and/or Sphinx lacks support for restrict qualifiers, so use
the workaround suggested here to hide the restrict qualifier from
Breathe/Sphinx.

breathe-doc/breathe#477 (comment)

An alternative is to use the C domain for *.c and *.h files.  This
handles restrict, but reduces enum names to their numeric value.  Once
that Breathe/Sphinx bug is fixed, we could skip the restrict elision
in favor of using the C domain.
@jedbrown jedbrown merged commit da62e0a into master Mar 8, 2020
@jeremylt jeremylt deleted the user-manual branch March 13, 2020 17:57
@valeriabarra valeriabarra mentioned this pull request Mar 18, 2020
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants