Skip to content

Conversation

@fanquake
Copy link
Member

@fanquake fanquake commented Jul 28, 2019

I frequently run make deploy while testing on macOS to get a properly light themed .app. With a brew installed Qt, this currently results in a pretty bloated executable:

branch .app size .dmg size make deploy time
master (febf3a8) 235mb 86mb 38s
This PR (da98f6d470d236c027b7eb8b5f5552fdca04e803) 51mb 21mb 22s

Similar change to dd367ff.

                         'QtGui.framework'],
  'pluginPath': '/usr/local/opt/qt/plugins',
  'qtPath': '/usr/local/opt/qt'}
-[('platforminputcontexts', 'libqtvirtualkeyboardplugin.dylib'),
- ('geoservices', 'libqtgeoservices_esri.dylib'),
- ('geoservices', 'libqtgeoservices_mapboxgl.dylib'),
- ('geoservices', 'libqtgeoservices_nokia.dylib'),
- ('geoservices', 'libqtgeoservices_itemsoverlay.dylib'),
- ('geoservices', 'libqtgeoservices_osm.dylib'),
- ('geoservices', 'libqtgeoservices_mapbox.dylib'),
- ('sceneparsers', 'libgltfsceneexport.dylib'),
- ('sceneparsers', 'libgltfsceneimport.dylib'),
- ('platforms', 'libqwebgl.dylib'),
+[('platforms', 'libqwebgl.dylib'),
  ('platforms', 'libqoffscreen.dylib'),
  ('platforms', 'libqminimal.dylib'),
  ('platforms', 'libqcocoa.dylib'),
  ('platformthemes', 'libqxdgdesktopportal.dylib'),
- ('printsupport', 'libcocoaprintersupport.dylib'),
- ('webview', 'libqtwebview_webengine.dylib'),
- ('webview', 'libqtwebview_darwin.dylib'),
- ('geometryloaders', 'libdefaultgeometryloader.dylib'),
- ('geometryloaders', 'libgltfgeometryloader.dylib'),
  ('styles', 'libqmacstyle.dylib'),
- ('canbus', 'libqttinycanbus.dylib'),
- ('canbus', 'libqtpassthrucanbus.dylib'),
- ('canbus', 'libqtvirtualcanbus.dylib'),
- ('canbus', 'libqtpeakcanbus.dylib'),
  ('bearer', 'libqgenericbearer.dylib'),
- ('imageformats', 'libqgif.dylib'),
- ('imageformats', 'libqwbmp.dylib'),
- ('imageformats', 'libqwebp.dylib'),
- ('imageformats', 'libqico.dylib'),
- ('imageformats', 'libqmacheif.dylib'),
- ('imageformats', 'libqjpeg.dylib'),
- ('imageformats', 'libqtiff.dylib'),
- ('imageformats', 'libqicns.dylib'),
- ('imageformats', 'libqtga.dylib'),
- ('imageformats', 'libqmacjp2.dylib'),
- ('texttospeech', 'libqtexttospeech_speechosx.dylib'),
- ('generic', 'libqtuiotouchplugin.dylib'),
- ('renderplugins', 'libscene2d.dylib'),
- ('gamepads', 'libdarwingamepad.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_thai.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_openwnn.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_hangul.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_pinyin.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_tcime.dylib')]
+ ('generic', 'libqtuiotouchplugin.dylib')]

@promag
Copy link
Contributor

promag commented Jul 28, 2019

Concept ACK, nice.

@fanquake fanquake force-pushed the slim_macdeployqtplus branch from da98f6d to 1ac7b7f Compare July 29, 2019 01:36
@fanquake
Copy link
Member Author

Updated to skip more plugins, including virtualkeyboard and imageformats.

@laanwj
Copy link
Member

laanwj commented Jul 29, 2019

Huh, it's interesting how no longer referencing dynamic libraries results in such a change in executable size. Do they all get included in the app image?

@fanquake
Copy link
Member Author

Huh, it's interesting how no longer referencing dynamic libraries results in such a change in executable size. Do they all get included in the app image?

Yep the macdeployqtplus script copies .framework and .dylib files into the Bitcoin-Qt.app bundle. Using master, with Qt 5.13.0 installed via brew, that ends up looking like the below. QtWebEngineCore.framework is to blame for most of the bloat..

/Volumes/Bitcoin-Core/Bitcoin-Qt.app/Contents/Frameworks:

166M	QtWebEngineCore.framework
5.8M	QtCore.framework
5.6M	QtWidgets.framework
5.3M	QtGui.framework
4.2M	QtQml.framework
4.0M	QtQuick.framework
2.5M	Qt3DRender.framework
1.7M	libcrypto.1.0.0.dylib
1.7M	QtLocation.framework
1.3M	libdb_cxx-4.8.dylib
1.3M	QtNetwork.framework
788K	QtVirtualKeyboard.framework
636K	Qt3DExtras.framework
492K	QtPositioning.framework
476K	QtDBus.framework
416K	QtWebEngine.framework
404K	Qt3DInput.framework
396K	Qt3DCore.framework
392K	libzmq.5.dylib
308K	QtPrintSupport.framework
228K	libevent-2.1.6.dylib
220K	QtSerialBus.framework
212K	QtSql.framework
172K	Qt3DQuick.framework
160K	QtWebSockets.framework
128K	QtWebChannel.framework
120K	QtGamepad.framework
112K	QtPositioningQuick.framework
100K	Qt3DQuickScene2D.framework
 92K	libboost_filesystem.dylib
 88K	QtWebView.framework
 88K	QtSerialPort.framework
 84K	libboost_thread-mt.dylib
 60K	Qt3DLogic.framework
 52K	QtTextToSpeech.framework
 40K	libqrencode.4.dylib
 36K	libboost_chrono-mt.dylib
 36K	QtConcurrent.framework
 16K	libevent_pthreads-2.1.6.dylib
8.0K	libboost_system.dylib

/Volumes/Bitcoin-Core/Bitcoin-Qt.app/Contents/PlugIns:

5.3M	geoservices
4.9M	platforms
3.0M	virtualkeyboard
1.7M	imageformats
392K	sceneparsers
204K	styles
184K	canbus
140K	geometryloaders
104K	webview
 80K	generic
 68K	platformthemes
 64K	gamepads
 52K	platforminputcontexts
 52K	bearer
 44K	texttospeech
 44K	renderplugins
 28K	printsupport

@laanwj
Copy link
Member

laanwj commented Jul 30, 2019

That's … the worst of both worlds.
Concept ACK anyway

@DrahtBot
Copy link
Contributor

Gitian builds for commit 68da549 (master):

Gitian builds for commit e5c174621ec8e85ab95055bb1846cb16ea8c548f (master and this pull):

@laanwj
Copy link
Member

laanwj commented Jul 31, 2019

ACK 1ac7b7f (purely Python code review and the fact that this passes travis, cannot run this on a mac)

With all the PRs shaving bits off Qt, it would be nice to have a document about what are all the different places a library needs to be added in case someone does want to add a qt module dependency.

@hebasto
Copy link
Member

hebasto commented Aug 27, 2019

Concept ACK.

@dongcarl
Copy link
Contributor

dongcarl commented Sep 9, 2019

tested ACK 1ac7b7f

Tested that the dmg built correctly, Bitcoin Core ran, and the various pages of the GUI looks like they're working.

fanquake added a commit that referenced this pull request Sep 10, 2019
…tplus

1ac7b7f scripts: filter more qt plugins we don't use in macdeployqtplus (fanquake)
57cdd06 scripts: misc cleanups in macdeployqtplus (fanquake)
51729a4 scripts: use format() in macdeployqtplus (fanquake)
1c37e81 scripts: add type annotations to macdeployqtplus (fanquake)

Pull request description:

  I frequently run `make deploy` while testing on macOS to get a properly light themed .app. With a brew installed Qt, this currently results in a pretty bloated executable:

  | branch | .app size | .dmg size | `make deploy` time |
  | ------- | --------- | --------- | --------------------- |
  | master (febf3a8) | 235mb | 86mb | 38s |
  | This PR (da98f6d470d236c027b7eb8b5f5552fdca04e803) | 51mb | 21mb | 22s |

  Similar change to dd367ff.

  ```diff
                           'QtGui.framework'],
    'pluginPath': '/usr/local/opt/qt/plugins',
    'qtPath': '/usr/local/opt/qt'}
  -[('platforminputcontexts', 'libqtvirtualkeyboardplugin.dylib'),
  - ('geoservices', 'libqtgeoservices_esri.dylib'),
  - ('geoservices', 'libqtgeoservices_mapboxgl.dylib'),
  - ('geoservices', 'libqtgeoservices_nokia.dylib'),
  - ('geoservices', 'libqtgeoservices_itemsoverlay.dylib'),
  - ('geoservices', 'libqtgeoservices_osm.dylib'),
  - ('geoservices', 'libqtgeoservices_mapbox.dylib'),
  - ('sceneparsers', 'libgltfsceneexport.dylib'),
  - ('sceneparsers', 'libgltfsceneimport.dylib'),
  - ('platforms', 'libqwebgl.dylib'),
  +[('platforms', 'libqwebgl.dylib'),
    ('platforms', 'libqoffscreen.dylib'),
    ('platforms', 'libqminimal.dylib'),
    ('platforms', 'libqcocoa.dylib'),
    ('platformthemes', 'libqxdgdesktopportal.dylib'),
  - ('printsupport', 'libcocoaprintersupport.dylib'),
  - ('webview', 'libqtwebview_webengine.dylib'),
  - ('webview', 'libqtwebview_darwin.dylib'),
  - ('geometryloaders', 'libdefaultgeometryloader.dylib'),
  - ('geometryloaders', 'libgltfgeometryloader.dylib'),
    ('styles', 'libqmacstyle.dylib'),
  - ('canbus', 'libqttinycanbus.dylib'),
  - ('canbus', 'libqtpassthrucanbus.dylib'),
  - ('canbus', 'libqtvirtualcanbus.dylib'),
  - ('canbus', 'libqtpeakcanbus.dylib'),
    ('bearer', 'libqgenericbearer.dylib'),
  - ('imageformats', 'libqgif.dylib'),
  - ('imageformats', 'libqwbmp.dylib'),
  - ('imageformats', 'libqwebp.dylib'),
  - ('imageformats', 'libqico.dylib'),
  - ('imageformats', 'libqmacheif.dylib'),
  - ('imageformats', 'libqjpeg.dylib'),
  - ('imageformats', 'libqtiff.dylib'),
  - ('imageformats', 'libqicns.dylib'),
  - ('imageformats', 'libqtga.dylib'),
  - ('imageformats', 'libqmacjp2.dylib'),
  - ('texttospeech', 'libqtexttospeech_speechosx.dylib'),
  - ('generic', 'libqtuiotouchplugin.dylib'),
  - ('renderplugins', 'libscene2d.dylib'),
  - ('gamepads', 'libdarwingamepad.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_thai.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_openwnn.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_hangul.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_pinyin.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_tcime.dylib')]
  + ('generic', 'libqtuiotouchplugin.dylib')]
  ```

ACKs for top commit:
  laanwj:
    ACK 1ac7b7f (purely Python code review and the fact that this passes travis, cannot run this on a mac)
  dongcarl:
    tested ACK 1ac7b7f

Tree-SHA512: 5974eeaf7229bb5bde2b283c1331ec57ee87f624db146401f6b77dee4ee5502e0bd669958a46205f10398a371f8e6c91ddacb9f0e1943f9f7d042fb6de7957a8
@fanquake fanquake merged commit 1ac7b7f into bitcoin:master Sep 10, 2019
@fanquake fanquake removed the request for review from theuni September 10, 2019 00:55
@fanquake fanquake deleted the slim_macdeployqtplus branch September 10, 2019 01:10
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Sep 10, 2019
…deployqtplus

1ac7b7f scripts: filter more qt plugins we don't use in macdeployqtplus (fanquake)
57cdd06 scripts: misc cleanups in macdeployqtplus (fanquake)
51729a4 scripts: use format() in macdeployqtplus (fanquake)
1c37e81 scripts: add type annotations to macdeployqtplus (fanquake)

Pull request description:

  I frequently run `make deploy` while testing on macOS to get a properly light themed .app. With a brew installed Qt, this currently results in a pretty bloated executable:

  | branch | .app size | .dmg size | `make deploy` time |
  | ------- | --------- | --------- | --------------------- |
  | master (febf3a8) | 235mb | 86mb | 38s |
  | This PR (da98f6d470d236c027b7eb8b5f5552fdca04e803) | 51mb | 21mb | 22s |

  Similar change to dd367ff.

  ```diff
                           'QtGui.framework'],
    'pluginPath': '/usr/local/opt/qt/plugins',
    'qtPath': '/usr/local/opt/qt'}
  -[('platforminputcontexts', 'libqtvirtualkeyboardplugin.dylib'),
  - ('geoservices', 'libqtgeoservices_esri.dylib'),
  - ('geoservices', 'libqtgeoservices_mapboxgl.dylib'),
  - ('geoservices', 'libqtgeoservices_nokia.dylib'),
  - ('geoservices', 'libqtgeoservices_itemsoverlay.dylib'),
  - ('geoservices', 'libqtgeoservices_osm.dylib'),
  - ('geoservices', 'libqtgeoservices_mapbox.dylib'),
  - ('sceneparsers', 'libgltfsceneexport.dylib'),
  - ('sceneparsers', 'libgltfsceneimport.dylib'),
  - ('platforms', 'libqwebgl.dylib'),
  +[('platforms', 'libqwebgl.dylib'),
    ('platforms', 'libqoffscreen.dylib'),
    ('platforms', 'libqminimal.dylib'),
    ('platforms', 'libqcocoa.dylib'),
    ('platformthemes', 'libqxdgdesktopportal.dylib'),
  - ('printsupport', 'libcocoaprintersupport.dylib'),
  - ('webview', 'libqtwebview_webengine.dylib'),
  - ('webview', 'libqtwebview_darwin.dylib'),
  - ('geometryloaders', 'libdefaultgeometryloader.dylib'),
  - ('geometryloaders', 'libgltfgeometryloader.dylib'),
    ('styles', 'libqmacstyle.dylib'),
  - ('canbus', 'libqttinycanbus.dylib'),
  - ('canbus', 'libqtpassthrucanbus.dylib'),
  - ('canbus', 'libqtvirtualcanbus.dylib'),
  - ('canbus', 'libqtpeakcanbus.dylib'),
    ('bearer', 'libqgenericbearer.dylib'),
  - ('imageformats', 'libqgif.dylib'),
  - ('imageformats', 'libqwbmp.dylib'),
  - ('imageformats', 'libqwebp.dylib'),
  - ('imageformats', 'libqico.dylib'),
  - ('imageformats', 'libqmacheif.dylib'),
  - ('imageformats', 'libqjpeg.dylib'),
  - ('imageformats', 'libqtiff.dylib'),
  - ('imageformats', 'libqicns.dylib'),
  - ('imageformats', 'libqtga.dylib'),
  - ('imageformats', 'libqmacjp2.dylib'),
  - ('texttospeech', 'libqtexttospeech_speechosx.dylib'),
  - ('generic', 'libqtuiotouchplugin.dylib'),
  - ('renderplugins', 'libscene2d.dylib'),
  - ('gamepads', 'libdarwingamepad.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_thai.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_openwnn.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_hangul.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_pinyin.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_tcime.dylib')]
  + ('generic', 'libqtuiotouchplugin.dylib')]
  ```

ACKs for top commit:
  laanwj:
    ACK 1ac7b7f (purely Python code review and the fact that this passes travis, cannot run this on a mac)
  dongcarl:
    tested ACK 1ac7b7f

Tree-SHA512: 5974eeaf7229bb5bde2b283c1331ec57ee87f624db146401f6b77dee4ee5502e0bd669958a46205f10398a371f8e6c91ddacb9f0e1943f9f7d042fb6de7957a8
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 16, 2020
Summary:
```
I frequently run make deploy while testing on macOS to get a properly
light themed .app. With a brew installed Qt, this currently results in a
pretty bloated executable:
branch 	.app size 	.dmg size 	make deploy time
master (febf3a8) 	235mb 	86mb 	38s
This PR (da98f6d) 	51mb 	21mb 	22s

Similar change to dd367ff.

                         'QtGui.framework'],
  'pluginPath': '/usr/local/opt/qt/plugins',
  'qtPath': '/usr/local/opt/qt'}
-[('platforminputcontexts', 'libqtvirtualkeyboardplugin.dylib'),
- ('geoservices', 'libqtgeoservices_esri.dylib'),
- ('geoservices', 'libqtgeoservices_mapboxgl.dylib'),
- ('geoservices', 'libqtgeoservices_nokia.dylib'),
- ('geoservices', 'libqtgeoservices_itemsoverlay.dylib'),
- ('geoservices', 'libqtgeoservices_osm.dylib'),
- ('geoservices', 'libqtgeoservices_mapbox.dylib'),
- ('sceneparsers', 'libgltfsceneexport.dylib'),
- ('sceneparsers', 'libgltfsceneimport.dylib'),
- ('platforms', 'libqwebgl.dylib'),
+[('platforms', 'libqwebgl.dylib'),
  ('platforms', 'libqoffscreen.dylib'),
  ('platforms', 'libqminimal.dylib'),
  ('platforms', 'libqcocoa.dylib'),
  ('platformthemes', 'libqxdgdesktopportal.dylib'),
- ('printsupport', 'libcocoaprintersupport.dylib'),
- ('webview', 'libqtwebview_webengine.dylib'),
- ('webview', 'libqtwebview_darwin.dylib'),
- ('geometryloaders', 'libdefaultgeometryloader.dylib'),
- ('geometryloaders', 'libgltfgeometryloader.dylib'),
  ('styles', 'libqmacstyle.dylib'),
- ('canbus', 'libqttinycanbus.dylib'),
- ('canbus', 'libqtpassthrucanbus.dylib'),
- ('canbus', 'libqtvirtualcanbus.dylib'),
- ('canbus', 'libqtpeakcanbus.dylib'),
  ('bearer', 'libqgenericbearer.dylib'),
- ('imageformats', 'libqgif.dylib'),
- ('imageformats', 'libqwbmp.dylib'),
- ('imageformats', 'libqwebp.dylib'),
- ('imageformats', 'libqico.dylib'),
- ('imageformats', 'libqmacheif.dylib'),
- ('imageformats', 'libqjpeg.dylib'),
- ('imageformats', 'libqtiff.dylib'),
- ('imageformats', 'libqicns.dylib'),
- ('imageformats', 'libqtga.dylib'),
- ('imageformats', 'libqmacjp2.dylib'),
- ('texttospeech', 'libqtexttospeech_speechosx.dylib'),
- ('generic', 'libqtuiotouchplugin.dylib'),
- ('renderplugins', 'libscene2d.dylib'),
- ('gamepads', 'libdarwingamepad.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_thai.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_openwnn.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_hangul.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_pinyin.dylib'),
- ('virtualkeyboard', 'libqtvirtualkeyboard_tcime.dylib')]
+ ('generic', 'libqtuiotouchplugin.dylib')]
```

Backport of core [[bitcoin/bitcoin#16477 | PR16477]].

Test Plan:
On OSX:
  ninja osx-dmg
On my machine the DMG was ~89MB and is now ~21MB.

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Subscribers: deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7964
random-zebra added a commit to random-zebra/PIVX that referenced this pull request Jun 11, 2021
a68c7d5 depends: mac_alias 2.2.0 (fanquake)
d8e2baf doc: Add explicit macdeployqtplus dependencies install step (Hennadii Stepanov)
013305d macdeploy: use Python 3.6 (fanquake)
faf77c3 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run (fanquake)
8bcfd58 macdeploy: remove existing PIVX-Core.dmg if present (fanquake)
023d3ca macdeploy: move qt_conf to where it's used (fanquake)
7cdb5bb macdeploy: consolidate .DS_Store generation (fanquake)
4da04d7 macdeploy: assume plistlib is available (fanquake)
56ab77a macdeploy: have a single level of logging output (fanquake)
d111cdf macdeploy: remove add-resources argument (fanquake)
4312410 macdeploy: remove codesigning argument (fanquake)
c2ee635 build: automatically determine macOS translations (fanquake)
1c44ecf scripts: filter more qt plugins we don't use in macdeployqtplus (fanquake)
c854f78 scripts: misc cleanups in macdeployqtplus (fanquake)
a3873ea scripts: use format() in macdeployqtplus (fanquake)
a65bea5 scripts: add type annotations to macdeployqtplus (fanquake)
ba179e5 build: Drop macports support (Ben Woosley)

Pull request description:

  This is a companion to PIVX-Project#2272 that focuses on on the `.dmg` creation aspect of macOS builds (ie, `make deploy`). The following upstream PRs are backported here:
   - bitcoin#15175
   - bitcoin#16477
   - bitcoin#20422
   - bitcoin#20890
   - bitcoin#21658

  Also worth mentioning: This drops support for MacPorts entirely, which has been antiquated and un-maintained for quite some time, and never actually used by any PIVX macOS build doc.

ACKs for top commit:
  furszy:
    Tested using depends, ACK a68c7d5.
  random-zebra:
    utACK a68c7d5 and merging...

Tree-SHA512: 3e9fa81a905ca3e90f07ff1213ec69dd1220a19a6a215f256ab67f2594476dc95e8fe88f15a1c9f3314b1757a7a2e5d8e6d7a790d85c117bf4236a3833757430
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants