Fix formatting and rendering issues in the autoreport by mahendrark · Pull Request #643 · rl-institut/multi-vector-simulator

mahendrark · 2020-11-09T10:46:08Z

Fix #634

Changes proposed in this pull request:

Correct the case of headings of some sections
Fit the ES graph within the page boundaries
Fix the issue of dash tables not being rendered in the PDF report

The following steps were realized, as well (if applies):

Use in-line comments to explain your code
Write docstrings to your code (example docstring)
For new functionalities: Explain in readthedocs
Write test(s) for your new patch of code (pytests, assertion debug messages)
Update the CHANGELOG.md
Apply black (black . --exclude docs/)
Check if benchmark tests pass locally (EXECUTE_TESTS_ON=master pytest)

Please mark above checkboxes as following:

Open
Done

❌ Check not applicable to this PR

_{For more information on how to contribute check the CONTRIBUTING.md.}

mahendrark · 2020-11-09T11:54:38Z

So at the moment, the tables are being rendered sometimes and not sometimes. It seems to be very random :/

Bachibouzouk · 2020-11-09T12:41:21Z

So at the moment, the tables are being rendered sometimes and not sometimes. It seems to be very random :/

We can fix this in _print_pdf_from_chrome function for F2

Right now the pdf creator waits to see the div with the id main-div in the dash app, it might be that when this div is built, the tables did not have time to be rendered themselves

To fix this you can try to change for the div of a table in the line which has await page.waitForSelector("#main-div")

run the report's dash app with mvs_report -i <path of your MVS output folder> to look at the name of the table div there

mahendrark · 2020-11-09T13:32:58Z

First, I changed the parameter to the ID of the last table. I found that by doing inspect element. But this always fails.

Next, I tried this. I changed the parameter to:

await page.waitForSelector("#dash_data_tables")

I just inserted a div element around the last make_dash_table function call and gave it the id "dash_data_tables". The PDF is generated, but no tables again

Bachibouzouk · 2020-11-09T14:11:02Z

I just inserted a div element around the last make_dash_table function call and gave it the id "dash_data_tables". The PDF is generated, but no tables again

You have to provide id to a datatable via Dash component and use this, or take the id a component within a dashtable, if you add an empty div, this one is going to get rendered very fast.

We can also select classes by replacing # by .

mahendrark · 2020-11-09T16:09:13Z

await page.goto("http://127.0.0.1:8050", {"waitUntil": "networkidle0", "timeout": "120000"})

this one goes on forever

I tried a lot of other ways, but none seem to guarantee that the images will be displayed every time. I selected the .tableplay class and tried with that, but no, does not work.

Bachibouzouk · 2020-11-09T17:23:39Z

    await page.goto(
        "http://127.0.0.1:8050", {"waitUntil": "domcontentloaded", "timeout": 120000}
    )
    resp = await page.waitForSelector(".dash-cell")  #only changed here
    print(resp)

This worked for me (there is sill the problem of the asset folder somehow not being found, but at least the tables get printed to the pdf)

mahendrark · 2020-11-09T17:59:29Z

@smartie2076

I will fix the energy demands issue in another PR

Please review this and let me know if the rendering and formatting appear good in your computer

Bachibouzouk

It worked for me. I would like to let someone with windows trying it as well

Bachibouzouk · 2020-11-09T21:51:47Z

src/multi_vector_simulator/F2_autoreport.py

    )
-    await page.waitForSelector("#main-div")
+    await page.waitForSelector(
+        ".dash-cell", {"visible": "true",},


this is the important part, you might want to explain what happens here, and maybe add a full entry for it in the changelog

smartie2076 · 2020-11-10T09:10:09Z

Mhm, interessant... ich muss noch irgendwas falsch gemacht haben mit dem setup:

(D:\PycharmProjects\mvs_eland\win_venv) D:\PycharmProjects\mvs_eland>python mvs_tool.py -i tests/inputs -f -pdf -ext csv
10:09:00-INFO-Path for logging: D:\PycharmProjects\mvs_eland\MVS_outputs\mvs_logfile.log
10:09:00-INFO-

 Multi-Vector Simulation Tool (MVS) V0.5.0
 Version: 2020-10-05
 Part of the toolbox of H2020 project "E-LAND", Integrated multi-vector management system for Energy isLANDs
 Coded at: Reiner Lemoine Institute (Berlin)
 Contributors: Martha M. Hoffmann


10:09:00-INFO-loading and converting all csv's from tests/inputs\csv_elements into one json
10:09:00-INFO-Json file created successfully from csv's and stored into tests/inputs\csv_elements\mvs_csv_config.json


10:09:00-INFO-The energy system modelled includes following energy vectors / sectors: Electricity
10:09:00-INFO-Pre-processing all assets in asset group energyProviders.
10:09:00-INFO-Peak demand pricing is taking place 2 times per year, ie. every 6.0 months.
10:09:00-INFO-Asset Electricity_grid_DSO_consumption_source was added to the energyProduction assets.
10:09:00-INFO-Pre-processing all assets in asset group energyConversion.
10:09:00-INFO-Pre-processing all assets in asset group energyStorage.
10:09:00-INFO-Pre-processing all assets in asset group energyProduction.
10:09:00-INFO-NumExpr defaulting to 8 threads.
10:09:00-INFO-Pre-processing all assets in asset group energyConsumption.
10:09:00-INFO-Processed cost data and added economic values.
10:09:00-ERROR-Energy system bus Electricity (DSO)_pdp_bus has too few assets connected to it. The minimal number of assets that need to be connected so that the bus is not a de
ad end should be two, excluding the excess sink. These are the connected assets: Electricity_grid_DSO_consumption
Traceback (most recent call last):
  File "mvs_tool.py", line 4, in <module>
    main()
  File "d:\pycharmprojects\mvs_eland\src\multi_vector_simulator\cli.py", line 137, in main
    data_processing.all(dict_values)
  File "d:\pycharmprojects\mvs_eland\src\multi_vector_simulator\C0_data_processing.py", line 88, in all
    "json_input_processed",
  File "d:\pycharmprojects\mvs_eland\src\multi_vector_simulator\F0_output.py", line 282, in store_as_json
    indent=4,
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\encoder.py", line 201, in encode
    chunks = list(chunks)
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "D:\PycharmProjects\mvs_eland\win_venv\lib\json\encoder.py", line 438, in _iterencode
    o = _default(o)
  File "d:\pycharmprojects\mvs_eland\src\multi_vector_simulator\B0_data_input_json.py", line 158, in convert_from_special_types_to_json
    answer = {DATA_TYPE_JSON_KEY: TYPE_SERIES, VALUE: o.to_list()}
  File "C:\Users\Martha.Hoffmann\AppData\Roaming\Python\Python37\site-packages\pandas\core\generic.py", line 4376, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'to_list'

... ich denke dass ich mal das MVS neu einrichten sollte. Vielleicht löst sich dann das Problem.

smartie2076 · 2020-11-10T10:03:29Z

Ahja, mit pip install -e . the problem solved itself - the simulation runs though. Still, I have problems with the html and pdf.

Note: You also address #535 (eventhough you dont fix all)

Here are my installed packages:
And here are the versions that are different from @mahendrark´s (as per #535), including only those that both of us have:
installed_packages_diff.txt
installed_packages_smartie2076.txt

Notes on the dash app:

The E-Land logo is not loaded
I have two maps appearing, with slightly different formatting
Tables appear
Headings and so on don`t have color anymore
The plots are printed twice - once with pagewidth and with 1/3 pagewidth
Log messages are printed twice:

Notes on pdf report:

No colors in headings, no page margins
One empty and one filled map of the location
ES graph appears correctly
plots appear only once
Logging warning/error messages appear twice

Bachibouzouk · 2020-11-10T10:15:47Z

Notes on pdf report:

* No colors in headings, no page margins

* One empty and one filled map of the location

* ES graph appears correctly

* plots appear only once

* Logging warning/error messages appear twice

This is all the same origin, it seems that the asset folder is not accessible to you on windows, therefore no logo and no styles from .css file, this should be fixed in #608

smartie2076

Okay, it were installation issues on my side. Everything good now!

CHANGELOG.md

Co-authored-by: smartie2076 <[email protected]>

mahendrark added 2 commits November 9, 2020 11:44

Fix sentence formatting for several headings

d0d12c4

Add CSS code to fit the ES system image within the parent div

b7df1d1

mahendrark added the enhancement New feature or request label Nov 9, 2020

mahendrark requested a review from smartie2076 November 9, 2020 11:37

mahendrark changed the title ~~Fix formatting issues in the autoreport~~ Fix formatting and rendering issues in the autoreport Nov 9, 2020

mahendrark force-pushed the fix/autoreport_issues branch from 29fe380 to b7df1d1 Compare November 9, 2020 12:33

mahendrark added 3 commits November 9, 2020 18:54

Fix the assets folder not found issue

f5b62a0

Fix the issue of dash tables not rendering in the PDF

93fb3c8

Update CHANGELOG

2d3072d

mahendrark self-assigned this Nov 9, 2020

Bachibouzouk mentioned this pull request Nov 9, 2020

Finalize packaging mvs #608

Merged

7 tasks

Bachibouzouk reviewed Nov 9, 2020

View reviewed changes

smartie2076 approved these changes Nov 10, 2020

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

Improve CHANGELOG update

e985e67

Co-authored-by: smartie2076 <[email protected]>

mahendrark merged commit fa7f55e into dev Nov 10, 2020

mahendrark deleted the fix/autoreport_issues branch November 10, 2020 12:56

Conversation

mahendrark commented Nov 9, 2020 • edited by Bachibouzouk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mahendrark commented Nov 9, 2020

Uh oh!

Bachibouzouk commented Nov 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mahendrark commented Nov 9, 2020

Uh oh!

Bachibouzouk commented Nov 9, 2020

Uh oh!

mahendrark commented Nov 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bachibouzouk commented Nov 9, 2020

Uh oh!

mahendrark commented Nov 9, 2020

Uh oh!

Bachibouzouk left a comment

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Nov 9, 2020

Choose a reason for hiding this comment

Uh oh!

smartie2076 commented Nov 10, 2020

Uh oh!

smartie2076 commented Nov 10, 2020

Notes on the dash app:

Notes on pdf report:

Uh oh!

Bachibouzouk commented Nov 10, 2020

Notes on pdf report:

Uh oh!

smartie2076 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mahendrark commented Nov 9, 2020 •

edited by Bachibouzouk

Loading

Bachibouzouk commented Nov 9, 2020 •

edited

Loading

mahendrark commented Nov 9, 2020 •

edited

Loading