Note: 6.7.1 can be downloaded from https://github.com/IQSS/dataverse/releases/tag/v6.7.1

What this PR does / why we need it:

After deploying 6.7 to Harvard Dataverse, we found a serious performance problem:

• Severe Performance Regression in "Create Dataset" Page – Excessive SQL Queries (360k+) #11698

The problem is being caused by this PR:

• blocked deposit page, hide host field when only one collection (e.g. root) #11301

We decided to create a 6.7.1 hotfix release and revert the problematic PR above.

Which issue(s) this PR closes:

• Closes Severe Performance Regression in "Create Dataset" Page – Excessive SQL Queries (360k+) #11698

Special notes for your reviewer:

We haven't done a hotfix release in a while, since 5.11.1. Things to note:

• Yes, we are intentionally merging to master. "If you are making a hotfix release, make the pull request against the “master” branch. Do not delete the branch after merging because we will later merge it into the “develop” branch to pick up the hotfix. More on this later." -- https://guides.dataverse.org/en/6.7/developers/making-releases.html#prepare-release-branch
• I had to revert blocked deposit page, hide host field when only one collection (e.g. root) #11301 manually. I think I got it right. However, please review it carefully. Update: I got it slightly wrong at first but between 48bad38 and 7cbd4ba it should be fine now. Thank you to @sekmiller for finding my mistake during code review! ❤️
• This PR contains the normal "bump version and change base image" commits for any release. This is for images to be pushed to Docker Hub when we release.
• I already bumped the version to 6.7.1 in Jenkins so I'm hoping API tests will pass. Please confirm this. This allowed the API tests to pass: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-11700/2/testReport/edu.harvard.iq.dataverse.api/ . Pull requests that still have 6.7 as the version will fail until we do the step to merge this the 6.7.1 changes into develop. Stay tuned.
• I'm making a PR to update the "making releases" doc based on this experience: tweak "making releases" doc after 6.7.1 hotfix #11701

Suggestions on how to test this:

On a server with a lot of collections (such as the QA server) try bringing up the "create dataset" and "edit dataset" screens on 6.7. This should be quite slow (for me on Harvard Dataverse it timed out after about 2.5 minutes). Compare with this PR.

Also, on 6.7 the "host dataverse" field will sometimes be absent. "Do not show the host dataverse field when there is nothing to choose." As of this PR the "host dataverse" field will always be present. See the release note snippet in #11301 (there's also language about not erasing your input).

Try browsing to a collection you can create datasets in and click "add dataset". Then change the "host dataverse" to another collection you can create datasets it. Create the dataset. Did it get created in the right collection?

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Yes. See above.

Is there a release notes update needed for this change?:

We'll write the release notes as we draft the release. Our last hotfix was https://github.com/IQSS/dataverse/releases/tag/v5.11.1 and here's a suggested released note based on it.

Dataverse 6.7.1

This is a bug fix release for Dataverse 6.7.1 that fixes a performance problem when loading the "create dataset" and "edit dataset pages". For details see #11700.

Complete List of Changes

For the complete list of code changes in this release, see the 6.7.1 milestone in GitHub.

For help with upgrading, installing, or general questions please post to the Dataverse Community Google Group or email [email protected].

Upgrade Instructions

These instructions assume that you've already upgraded through all the 5.x releases and are now running Dataverse 6.7.

0. These instructions assume that you are upgrading from the immediate previous version. If you are running an earlier version, the only supported way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to this version.

If you are running Payara as a non-root user (and you should be!), remember not to execute the commands below as root. By default, Payara runs as the dataverse user. In the commands below, we use sudo to run the commands as a non-root user.

Also, we assume that Payara 6 is installed in /usr/local/payara6. If not, adjust as needed.

export PAYARA=/usr/local/payara6

(or setenv PAYARA /usr/local/payara6 if you are using a csh-like shell)

1. List deployed applications

$PAYARA/bin/asadmin list-applications

2. Undeploy the previous version (should match "list-applications" above)

$PAYARA/bin/asadmin undeploy dataverse-6.7

3. Download and deploy this version

wget https://github.com/IQSS/dataverse/releases/download/v6.7/dataverse-6.7.1.war
$PAYARA/bin/asadmin deploy dataverse-6.7.1.war

Note: if you have any trouble deploying, stop Payara, remove the following directories, start Payara, and try to deploy again.

sudo service payara stop
sudo rm -rf $PAYARA/glassfish/domains/domain1/generated
sudo rm -rf $PAYARA/glassfish/domains/domain1/osgi-cache
sudo rm -rf $PAYARA/glassfish/domains/domain1/lib/databases
sudo service payara start

Additional documentation:

None.