A few questions on the implementation #3
Description
Hello, great to see this!
I've also been working on this on and off over the past month. So far it's all custom code in a single site. Here's a few notes and challenges I ran into, and I'm curious if you've hit them yet. I don't think you have because I don't see workarounds in the code.
- For writing reports, how are permissions working? While I posted How to get the $uid / $gid variables in custom docker compose services? ddev/ddev#4733 about ZAP, we had exactly the same issue with Playwright.
- Have you run into this bug yet with Firefox? This plus the above led me to stacking Playwright into the web container, instead of a separate service: [BUG] Running Playwright on Firefox inside a Docker container always hangs forever microsoft/playwright#16491
- Some tools like https://github.com/chanzuckerberg/axe-storybook-testing also have a Playwright dependency. Any thoughts on how to run them inside the playwright container without having to double-download browsers?
At a high level, what we've ended up with is:
- A separate yarn project for playwright in the project repo in
test/playwright. - An additional dockerfile for the web service that is enabled via a
ddev playwrightcommand that copies it in. As well, a pre-start host hook that updates it if it already exists. Unfortunately, symlinks and hardlinks aren't working. - Adding a VNC server and xvfb for headed operations.
- Lots of use of of build caches to speed up development and local rebuilds:
# Debian images by default disable apt caching, so turn it on until we finish
# the build.
RUN mv /etc/apt/apt.conf.d/docker-clean /etc/apt/docker-clean-disabled
# Install Playwright browsers and system dependencies.
#
# We want to install whatever browsers are used by the currently locked version
# of playwright in test/playwright. What this does is:
#
# 1. Creates a copy of the playwright directory in the image. We need this
# because Playwright keeps a hidden .links/<hash> file with the path of
# what browsers are installed by what apps. Without it, Playwright will re-
# download browsers even though they are in the directory.
# 2. $username doesn't work in --mount... lines as it's a shell variable set by
# ddev. Instead, we mount the .cache cache into /dot-cache, and symlink it
# in.
# 3. Then we remove the symlink, and copy the cache directory into ~/.cache so
# it is persisted in the image.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/dot-cache,sharing=locked \
--mount=type=bind,source=./test/playwright,target=/playwright \
sudo -u $username mkdir -p /var/www/html/test \
&& cp -a /playwright /var/www/html/test \
&& chown -R $username:$username /var/www/html/test/playwright \
&& chown -R $username:$username /dot-cache \
&& sudo -u $username ln -s /dot-cache /home/$username/.cache \
&& cd /var/www/html/test/playwright \
&& sudo -u $username yarn playwright install-deps \
&& sudo -u $username yarn playwright install \
&& rm -rf /var/www/html/test/playwright \
&& rm /home/$username/.cache \
&& sudo -u $username mkdir -p /home/$username/.cache \
&& sudo -u $username cp -a /dot-cache/* /home/$username/.cache/
# Install a window manager and VNC server.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
sudo apt-get install -y fluxbox \
xauth \
x11vnc
# We're done with apt so disable caching again for the final image.
RUN mv /etc/apt/docker-clean-disabled /etc/apt/apt.conf.d/docker-clean
# Install novnc to access the GUI over a browser.
ENV NOVNC_VERSION="1.4.0" \
WEBSOCKIFY_VERSION="0.11.0"
RUN wget -nv -O noVNC.zip \
"https://github.com/novnc/noVNC/archive/refs/tags/v${NOVNC_VERSION}.zip" \
&& unzip -x noVNC.zip \
&& mkdir -p /opt/bin \
&& mv noVNC-${NOVNC_VERSION} /opt/bin/noVNC \
&& cp /opt/bin/noVNC/vnc.html /opt/bin/noVNC/index.html \
&& rm noVNC.zip \
&& wget -nv -O websockify.zip \
"https://github.com/novnc/websockify/archive/refs/tags/v${WEBSOCKIFY_VERSION}.zip" \
&& unzip -x websockify.zip \
&& rm websockify.zip \
&& rm -rf websockify-${WEBSOCKIFY_VERSION}/tests \
&& mv websockify-${WEBSOCKIFY_VERSION} /opt/bin/noVNC/utils/websockify
# Copy start scripts for the x server, x11vnc, and novnc.
COPY .ddev/web-build/start-*.sh /usr/local/bin
RUN chmod +rx /usr/local/bin/start-*.sh
# novnc requires a password, so we set it to secret matching Selenium's images.
RUN sudo -u $username mkdir -p /home/$username/.vnc
RUN sudo -u $username x11vnc -storepasswd secret /home/$username/.vnc/passwd