Harvest: map publisher tag to distributorName #9013

tcoupin · 2022-10-03T07:26:53Z

What this PR does / why we need it:

Which issue(s) this PR closes:

Closes Publisher tag in oai_dc format #8739

scolapasta · 2024-04-23T23:32:24Z

If you are still interested in this PR, can you please merge and resolve any merge conflicts with the latest from develop? If so, we can prioritize reviewing and QAing the changes. If we don’t hear from you by May 22, 2024, we’ll go ahead and close this PR (it can always be reopened after that date, if there is still interest).

plecor · 2024-05-03T08:15:16Z

The branch is now up to date with develop and the sql migration file was renamed to match the latest version - please tell me if there is any convention I'm missing.

pdurbin · 2024-05-03T11:46:16Z

@plecor can you please add a release note snippet that describes what this pull request does? Please see https://guides.dataverse.org/en/6.2/developers/version-control.html#writing-release-note-snippets

Also, it would be nice to add something to the guides but I'm not sure where. Maybe we should add a changelog to https://guides.dataverse.org/en/6.2/admin/harvestserver.html ? Would that make sense?

Finally, tests are always nice but not strictly required. There are some in HarvestingServerIT.java.

plecor · 2024-05-14T14:01:25Z

Thanks @pdurbin.

I created a release note and added the relevant test to HarvestingClientsIT.
I'm really not sure however where this would fit in the docs.

pdurbin · 2024-05-14T18:05:20Z

@plecor I went ahead and added a new changelog for harvesting clients: https://dataverse-guide--9013.org.readthedocs.build/en/9013/admin/harvestclients.html#harvesting-client-changelog

In review we'll see what others think.

coveralls · 2024-05-14T18:09:45Z

coverage: 22.754%. remained the same
when pulling 7e72b25 on tcoupin:8739-publisher-during-harvesting
into 4dfb9e7 on IQSS:develop.

pdurbin

I added some feedback.

pdurbin · 2024-11-25T20:37:01Z

doc/sphinx-guides/source/admin/harvestclients.rst

+Harvesting Client Changelog
+---------------------------
+
+- As of Dataverse 6.3, the publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This change affects all newly harvested datasets. For more information, see https://github.com/IQSS/dataverse/pull/9013


Suggested change

- As of Dataverse 6.3, the publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This change affects all newly harvested datasets. For more information, see https://github.com/IQSS/dataverse/pull/9013

- As of Dataverse 6.6, the publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This change affects all newly harvested datasets. For more information, see https://github.com/IQSS/dataverse/pull/9013

I don't think this will made the cut for 6.5, sorry!

Also, please resolve conflicts.

Finally, I need to check with @landreev to see what he thinks of this whole "harvesting client changelog" thing I came up with. 😅 This: https://dataverse-guide--9013.org.readthedocs.build/en/9013/admin/harvestclients.html#harvesting-client-changelog

I like the idea, of having a dedicated changelog.

src/main/resources/db/migration/V6.2.0.2__8739-publisher-during-harvesting.sql

pdurbin · 2024-11-25T20:40:36Z

src/test/java/edu/harvard/iq/dataverse/api/HarvestingClientsIT.java

        int sz = jsonPath.getInt("data.items.size()");
        for(int idx = 0; idx < sz; idx++) {
            globalIdList.add(jsonPath.getString("data.items["+idx+"].global_id"));
        }


Hmm, it's been so long that https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-9013/7/display/redirect shows a failure but that job is now a 404 so I can't see any details.

After you merge the latest from develop let's keep an eye on the new Jenkins run.

I'm still seeing failures. I just kicked off another run. Fingers crossed: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-9013/12/

Last time all checks were green except for "continuous-integration/jenkins/pr-merge " that was pending. I'm not clear on what it is doing.

Ok! Now that the app is being deployed following the SQL script renaming, I can see that a couple harvesting-related tests are failing:

edu.harvard.iq.dataverse.api.HarvestingClientsIT.testHarvestingClientRun_AllowHarvestingMissingCVV_False(HarvestingClientsIT.java:187)

edu.harvard.iq.dataverse.api.HarvestingClientsIT.testHarvestingClientRun_AllowHarvestingMissingCVV_True(HarvestingClientsIT.java:191)

@plecor can you please take a look? Do you need help with how to run these tests?

Hi @pdurbin ,

I've re-run the tests from scratch and I wonder if it could be a chicken and egg situation. The new assertion fails if I run the tests from this branch because the sql migration file it's adding hasn't been run. If I manually update the db then the tests run fine. Could it be something similar going on with Jenkins?

Ok, the issue is that in a new install, the corresponding data is initialized by afterMigrate__1-7256-upsert-referenceData.sql after the other migrations, so the data we're trying to update doesn't exist yet.

I updated this file with the same change as the sql migration file.

Interesting. Thanks.

@poikilotherm judging from https://github.com/IQSS/dataverse/blame/c8499ba9553ac46cf3adc56d1b9e56f0c781d30f/src/main/resources/db/migration/afterMigrate__1-7256-upsert-referenceData.sql you added that "after migrate" file. What do you think? Any risk in changing it? 🤔

@qqmyers took a look and had this to say:

"that afterMigrate script appears to run after everything else, every time you restart, rather than being one of the scripts tracked by hash value in the flyway_schema_history table. So - it looks OK to me to update it. (Mostly from looking at https://documentation.red-gate.com/fd/callback-concept-184127466.html and related and verifying that I don't see it in the flyway_schema_history table)."

doc/release-notes/8739-publisher-during-harvesting.md

pdurbin · 2024-11-25T20:45:08Z

src/main/resources/db/migration/V6.2.0.2__8739-publisher-during-harvesting.sql

@@ -0,0 +1 @@
+update foreignmetadatafieldmapping set datasetfieldname = 'distributorName' where foreignfieldxpath = ':publisher';


What about datasets that are already harvested? The docs say, "This change affects all newly harvested datasets."

The plan is to have a mix?

Or should we add to the release note that all datasets should be re-harvested? Is that crazy?

Indeed, the fix isn't retroactive, unless all datasets are re-harvested. This is not great but I have no idea how we could do this another way.

Maybe we could only suggest re-harvesting only if people are affected by/care about the discrepancy it fixes?

@plecor for now, can you please add something to the release note. "All datasets should be re-harvested to pick up this change." Or whatever makes sense to you. Thanks.

Ok. To reflect this, the full text of the release note is now:

The publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This improves the citation associated with these datasets, but the change only affects newly harvested datasets. All datasets should be re-harvested if you wish to pick up this change on already harvested datasets. For more information, see the guides, #8739, and #9013.

Co-authored-by: Philip Durbin <[email protected]>

…g-harvesting

…' into 8739-publisher-during-harvesting

src/main/resources/db/migration/V6.4.0.1__8739-publisher-during-harvesting.sql

ofahimIQSS · 2025-01-06T18:40:59Z

This branch has some conflicts to resolve before I can QA.

…g-harvesting

plecor · 2025-01-07T07:35:21Z

@ofahimIQSS I updated the branch to resolve the conflict.

doc/sphinx-guides/source/admin/harvestclients.rst

Co-authored-by: Omer Fahim <[email protected]>

ofahimIQSS · 2025-01-15T16:35:36Z

what harvesting client should this be tested with?

ofahimIQSS · 2025-01-22T18:00:34Z

Tested Harvesting, performed regression testing in internal - no issues found. Conflict needs to be resolved.

pdurbin · 2025-01-22T18:58:02Z

@ofahimIQSS I resolved the merge conflicts.

ofahimIQSS · 2025-01-23T14:08:06Z

Tested after conflict resolution - no issues found. Merging PR

@ejb

* title case * fix links * add cross links * resolving conflict with the newly-merged flyway script #10909 * resolving a merge conflict with a flyway script that's been merged #10909 * bump sql script #10217 * use Title Case to match other fields #10217 * reword source name help text #10217 * always show source name help text #10217 * duplicate ui:define body * Revert "10476 display on create field option" * undo change to sql script * adding replace file test * update flyway name * remove the need to pass the datafile version to the API * restore "parameters" minus displayOnCreate #11306 * remove flyway merge issue * update flyway * fix for unrelated issue found in QA * flyway change * rename sql * rename sql * typo * one more * Merge develop without revert * Restore displayOnCreate field option changes * keep renaming the flyway script... #10909 * Add displayOnCreate option for dataset field types - Added @column annotation for displayOnCreate field in DatasetFieldType - Updated JsonPrinter to include displayOnCreate in display condition * Update DataversesIT test to modify metadata block listing parameter * Update native API documentation for displayOnCreate field option - Clarify that required fields are always displayed regardless of displayOnCreate setting * bug fix - only create cite once, not on refresh of cslOutput * flyway number change * flyway again * Implement null support for displayOnCreate field option - Change displayOnCreate to nullable Boolean in DatasetFieldType - Update API and service methods to handle null displayOnCreate values - Modify native API documentation to explain null displayOnCreate behavior - Add null checks in MetadataBlockServiceBean queries * fix: make displayOnCreate nullable in DatasetFieldType - Modified JsonPrinter to handle null values for displayOnCreate - Updated BriefJsonPrinter to use null-safe comparison * setDisplayOnCreate astroInstrument back to false #10476 * Handle null displayOnCreate in MetadataBlock and JsonPrinter - Update MetadataBlock to safely check displayOnCreate with null values - Modify JsonPrinter to default displayOnCreate to false when null - Ensure consistent null-safe handling of displayOnCreate across components * move permission check out of loop * rename api from /versions/list to /versionDifferences * Refine displayOnCreate logic in JsonPrinter * Update displayOnCreate method call in metadataFragment.xhtml - Change method call from property access to method invocation for displayOnCreate - Maintain consistent null-safe handling of displayOnCreate field option * adding todo for future refactoring * Add getter method for displayOnCreate in DatasetFieldType - Implement a simple getter method for the displayOnCreate field - Maintain consistency with existing setter method * note second callback needed. * fix create mode bug * Update doc/release-notes/6.2025.2_update.md Co-authored-by: Omer Fahim <[email protected]> * Update doc/release-notes/6.2025.2_update.md Co-authored-by: Philip Durbin <[email protected]> * Update doc/release-notes/6.2025.2_update.md Co-authored-by: Omer Fahim <[email protected]> * fixing datafile persistentId * localhost and longer key * use original cert/key names * add comment * test(mail): update test SSL certs for maildev to be RSA 2028 again It is not necessary to use 4096 bit, as the error was about the wrong common name in the cert. Detailed explanation: Before, the cert presented to Jakarta Mail neither contained a commonName=localhost nor subjectAltName=localhost. This worked before with Payara 6.2024.6 because Eclipse Angus Mail 1.0.0 was still in use (see list of managed dependencies on the 6.2024.6 BOM). Certificate host identity check was introduced as default=on with Eclipse Angus Mail 1.1.0 (see their release notes). Now that Payara 6.2025.2 is in use on develop, we upgraded to Eclipse Angus Mail 2.3.0, making the error appear. * test(mail): update test SSL cert generation docs Including an openssl.conf makes it much easier and less error prone to generate the certs. Rewriting the necessary shell command steps in the test comment to document the process. * test(mail): pin down port in TC waiting strategy Without the explicit addition of which port to use for the HTTP based waiting strategy, someone may end up at random with the SMTP port being picked. This will obviously fail and result in aborted tests, making results flaky. Pinning the port down should prevent this for good. * extend size of deaccessionlink column * stub out 6.6 release notes #11183 * Payara upgrade #11183 * upgrade Solr #11183 * move collection * license enhancements * range search * datasets without files * license enhancements * anon preview url * ontoportal * ror * bug fixes * boolean * more bugs * don't use in any edit mode * typos * 3d objects * delete files, citation * search api fields for files * signposting * collection featured items * dataset types linked to metadata blocks * fix heading levels * Improve metadata block field selection logic for display and required fields - Modify MetadataBlockServiceBean to include required fields in query selection - Update JsonPrinter to simplify field display logic for metadata blocks - Ensure fields are correctly filtered based on display and required status * harvesting * locks and permalink * 3 more * #10476 add "override" test. * remind those with text customizations to update properties files See https://dataverse.zulipchat.com/#narrow/channel/378866-troubleshooting/topic/upgrade.20from.206.2E3.20to.206.2E4/near/504689559 * fix npe from breadcrumbs * pass param if sent * Fix input levels update preserving inherited metadata blocks When updating input levels for a collection, preserve inherited metadata blocks and existing input levels while allowing collection-specific configurations. * unrelated - lower log level * docker updates * 3 more * typo * API updates * make inputlevel displayOnCreate Boolean * guides * 6 more * archival block, etalab * PR #9013 was merged for 6.6 not 6.5 * fix header level for harvesting non-OIA-PMH * move 6.6 and 5.13 mentions to changelog * reharvest, guestbook bugs * try separate transient doc, revert dsft doc to boolean * missed classes from previous commit * metrics and sorting fixes * postgres * source name harvesting * Add displayOnCreate field to dataset schema and update field display logic - Update dataset-schema.json to include a new "displayOnCreate" field - Modify DataverseServiceBean to set default displayOnCreate behavior * merge issue * samesite * show role * quick test fix * API updates * PID formatting * remove assert temporarily * external tools * #10476 fix command test * send feedback to contacts * dft fix * perms * indexing cvoc * API for classic download count * move version note to 6.6 where it belongs #11068 * API Guide: don't interleave "notes" with "types" #11068 * optional version note * remove scary line from optional version note * make text under Cite Data not bold #11163 * csl * delete snippet without adding to notes, too in the weeds * harvest from datacite * globus * bearer tokens * fix heading level * html sanitize JSON CSL elements * ORCID stuff * how to re-harvest * fix link * #10476 update tests json printer * fix null handling * #10476 code/test cleanup * Update 6.6-release-notes.md Fixed some typos and made some changes to improve readability * Update doc/sphinx-guides/source/admin/harvestclients.rst Co-authored-by: Omer Fahim <[email protected]> * Update doc/sphinx-guides/source/admin/harvestclients.rst Co-authored-by: Omer Fahim <[email protected]> * Update doc/sphinx-guides/source/admin/harvestclients.rst Co-authored-by: Omer Fahim <[email protected]> * Update doc/sphinx-guides/source/user/find-use-data.rst Co-authored-by: Omer Fahim <[email protected]> * fix ORCIDs * mention video talking about highlights * #10476 fix DataversesIT * remove duplicates from merge * tweaks * #11183 fix typo * fix link to war file * Update doc/release-notes/6.6-release-notes.md Co-authored-by: Omer Fahim <[email protected]> * Update doc/release-notes/6.6-release-notes.md Co-authored-by: Omer Fahim <[email protected]> * typo: cheat Co-authored-by: Omer Fahim <[email protected]> * fix link to issue * typos * Remove default displayOnCreate setting for required fields in DataverseServiceBean * fix 500 err * unused @ejb, cleanup * fixes * add test for setting input level w/o optional displayOnCreate * handle levels in printer for API calls * display on create * fix LocalDisplay logic * typo Co-authored-by: Omer Fahim <[email protected]> * don't include when include is false * fix ownerdv null case * get level once, setlocaldisplayoncreate correctly * Try order by id * @GPortas edits * add index * updates per review * comment tweak * Release 6.6 * Update doc/sphinx-guides/source/versions.rst Co-authored-by: Philip Durbin <[email protected]> --------- Co-authored-by: Philip Durbin <[email protected]> Co-authored-by: Leonid Andreev <[email protected]> Co-authored-by: Jim Myers <[email protected]> Co-authored-by: Philip Durbin <[email protected]> Co-authored-by: Omer Fahim <[email protected]> Co-authored-by: Alexis Guanche <[email protected]> Co-authored-by: qqmyers <[email protected]> Co-authored-by: Oliver Bertuch <[email protected]> Co-authored-by: Stephen Kraffmiller <[email protected]> Co-authored-by: Julian Gautier <[email protected]>

@ejb

* fix links * add cross links * resolving conflict with the newly-merged flyway script #10909 * resolving a merge conflict with a flyway script that's been merged #10909 * bump sql script #10217 * use Title Case to match other fields #10217 * reword source name help text #10217 * always show source name help text #10217 * duplicate ui:define body * Revert "10476 display on create field option" * undo change to sql script * adding replace file test * update flyway name * remove the need to pass the datafile version to the API * restore "parameters" minus displayOnCreate #11306 * remove flyway merge issue * update flyway * fix for unrelated issue found in QA * flyway change * rename sql * rename sql * typo * one more * Merge develop without revert * Restore displayOnCreate field option changes * keep renaming the flyway script... #10909 * Add displayOnCreate option for dataset field types - Added @column annotation for displayOnCreate field in DatasetFieldType - Updated JsonPrinter to include displayOnCreate in display condition * Update DataversesIT test to modify metadata block listing parameter * Update native API documentation for displayOnCreate field option - Clarify that required fields are always displayed regardless of displayOnCreate setting * bug fix - only create cite once, not on refresh of cslOutput * flyway number change * flyway again * Implement null support for displayOnCreate field option - Change displayOnCreate to nullable Boolean in DatasetFieldType - Update API and service methods to handle null displayOnCreate values - Modify native API documentation to explain null displayOnCreate behavior - Add null checks in MetadataBlockServiceBean queries * fix: make displayOnCreate nullable in DatasetFieldType - Modified JsonPrinter to handle null values for displayOnCreate - Updated BriefJsonPrinter to use null-safe comparison * setDisplayOnCreate astroInstrument back to false #10476 * Handle null displayOnCreate in MetadataBlock and JsonPrinter - Update MetadataBlock to safely check displayOnCreate with null values - Modify JsonPrinter to default displayOnCreate to false when null - Ensure consistent null-safe handling of displayOnCreate across components * move permission check out of loop * rename api from /versions/list to /versionDifferences * Refine displayOnCreate logic in JsonPrinter * Update displayOnCreate method call in metadataFragment.xhtml - Change method call from property access to method invocation for displayOnCreate - Maintain consistent null-safe handling of displayOnCreate field option * adding todo for future refactoring * Add getter method for displayOnCreate in DatasetFieldType - Implement a simple getter method for the displayOnCreate field - Maintain consistency with existing setter method * note second callback needed. * fix create mode bug * Update doc/release-notes/6.2025.2_update.md Co-authored-by: Omer Fahim <[email protected]> * Update doc/release-notes/6.2025.2_update.md Co-authored-by: Philip Durbin <[email protected]> * Update doc/release-notes/6.2025.2_update.md Co-authored-by: Omer Fahim <[email protected]> * fixing datafile persistentId * localhost and longer key * use original cert/key names * add comment * test(mail): update test SSL certs for maildev to be RSA 2028 again It is not necessary to use 4096 bit, as the error was about the wrong common name in the cert. Detailed explanation: Before, the cert presented to Jakarta Mail neither contained a commonName=localhost nor subjectAltName=localhost. This worked before with Payara 6.2024.6 because Eclipse Angus Mail 1.0.0 was still in use (see list of managed dependencies on the 6.2024.6 BOM). Certificate host identity check was introduced as default=on with Eclipse Angus Mail 1.1.0 (see their release notes). Now that Payara 6.2025.2 is in use on develop, we upgraded to Eclipse Angus Mail 2.3.0, making the error appear. * test(mail): update test SSL cert generation docs Including an openssl.conf makes it much easier and less error prone to generate the certs. Rewriting the necessary shell command steps in the test comment to document the process. * test(mail): pin down port in TC waiting strategy Without the explicit addition of which port to use for the HTTP based waiting strategy, someone may end up at random with the SMTP port being picked. This will obviously fail and result in aborted tests, making results flaky. Pinning the port down should prevent this for good. * extend size of deaccessionlink column * stub out 6.6 release notes #11183 * Payara upgrade #11183 * upgrade Solr #11183 * move collection * license enhancements * range search * datasets without files * license enhancements * anon preview url * ontoportal * ror * bug fixes * boolean * more bugs * don't use in any edit mode * typos * 3d objects * delete files, citation * search api fields for files * signposting * collection featured items * dataset types linked to metadata blocks * fix heading levels * Improve metadata block field selection logic for display and required fields - Modify MetadataBlockServiceBean to include required fields in query selection - Update JsonPrinter to simplify field display logic for metadata blocks - Ensure fields are correctly filtered based on display and required status * harvesting * locks and permalink * 3 more * #10476 add "override" test. * remind those with text customizations to update properties files See https://dataverse.zulipchat.com/#narrow/channel/378866-troubleshooting/topic/upgrade.20from.206.2E3.20to.206.2E4/near/504689559 * fix npe from breadcrumbs * pass param if sent * Fix input levels update preserving inherited metadata blocks When updating input levels for a collection, preserve inherited metadata blocks and existing input levels while allowing collection-specific configurations. * unrelated - lower log level * docker updates * 3 more * typo * API updates * make inputlevel displayOnCreate Boolean * guides * 6 more * archival block, etalab * PR #9013 was merged for 6.6 not 6.5 * fix header level for harvesting non-OIA-PMH * move 6.6 and 5.13 mentions to changelog * reharvest, guestbook bugs * try separate transient doc, revert dsft doc to boolean * missed classes from previous commit * metrics and sorting fixes * postgres * source name harvesting * Add displayOnCreate field to dataset schema and update field display logic - Update dataset-schema.json to include a new "displayOnCreate" field - Modify DataverseServiceBean to set default displayOnCreate behavior * merge issue * samesite * show role * quick test fix * API updates * PID formatting * remove assert temporarily * external tools * #10476 fix command test * send feedback to contacts * dft fix * perms * indexing cvoc * API for classic download count * move version note to 6.6 where it belongs #11068 * API Guide: don't interleave "notes" with "types" #11068 * optional version note * remove scary line from optional version note * make text under Cite Data not bold #11163 * csl * delete snippet without adding to notes, too in the weeds * harvest from datacite * globus * bearer tokens * fix heading level * html sanitize JSON CSL elements * ORCID stuff * how to re-harvest * fix link * #10476 update tests json printer * fix null handling * #10476 code/test cleanup * Update 6.6-release-notes.md Fixed some typos and made some changes to improve readability * Update doc/sphinx-guides/source/admin/harvestclients.rst Co-authored-by: Omer Fahim <[email protected]> * Update doc/sphinx-guides/source/admin/harvestclients.rst Co-authored-by: Omer Fahim <[email protected]> * Update doc/sphinx-guides/source/admin/harvestclients.rst Co-authored-by: Omer Fahim <[email protected]> * Update doc/sphinx-guides/source/user/find-use-data.rst Co-authored-by: Omer Fahim <[email protected]> * fix ORCIDs * mention video talking about highlights * #10476 fix DataversesIT * remove duplicates from merge * tweaks * #11183 fix typo * fix link to war file * Update doc/release-notes/6.6-release-notes.md Co-authored-by: Omer Fahim <[email protected]> * Update doc/release-notes/6.6-release-notes.md Co-authored-by: Omer Fahim <[email protected]> * typo: cheat Co-authored-by: Omer Fahim <[email protected]> * fix link to issue * typos * Remove default displayOnCreate setting for required fields in DataverseServiceBean * fix 500 err * unused @ejb, cleanup * fixes * add test for setting input level w/o optional displayOnCreate * handle levels in printer for API calls * display on create * fix LocalDisplay logic * typo Co-authored-by: Omer Fahim <[email protected]> * don't include when include is false * fix ownerdv null case * get level once, setlocaldisplayoncreate correctly * Try order by id * @GPortas edits * add index * updates per review * comment tweak * Release 6.6 * Update doc/sphinx-guides/source/versions.rst Co-authored-by: Philip Durbin <[email protected]> * 6.6 release fix (#11353) --------- Co-authored-by: Philip Durbin <[email protected]> Co-authored-by: Leonid Andreev <[email protected]> Co-authored-by: Jim Myers <[email protected]> Co-authored-by: Philip Durbin <[email protected]> Co-authored-by: Omer Fahim <[email protected]> Co-authored-by: Alexis Guanche <[email protected]> Co-authored-by: qqmyers <[email protected]> Co-authored-by: Oliver Bertuch <[email protected]> Co-authored-by: Stephen Kraffmiller <[email protected]> Co-authored-by: Julian Gautier <[email protected]>

mreekie added the bk2211 label Nov 1, 2022

mreekie removed the bk2211 label Jan 11, 2023

pdurbin added the Size: 10 A percentage of a sprint. 7 hours. label Feb 28, 2024

tcoupin and others added 2 commits May 3, 2024 10:06

Harvest: map publisher tag to distributorName

a0a7c8d

Rename migration file

d92d048

plecor force-pushed the 8739-publisher-during-harvesting branch from 95d9c69 to d92d048 Compare May 3, 2024 08:10

plecor added 2 commits May 14, 2024 15:55

Add use case to HarvestingClientsIT

d6a6e56

Add release note

64b69b9

add change in behavior to new harvesting client changlog IQSS#8739

5dfb01e

pdurbin added the Type: Feature a feature request label Oct 9, 2024

cmbz added GREI 3 Search and Browse Feature: Harvesting FY25 Sprint 11 FY25 Sprint 11 (2024-11-20 - 2024-12-04) labels Nov 18, 2024

pdurbin self-assigned this Nov 25, 2024

pdurbin reviewed Nov 25, 2024

View reviewed changes

pdurbin assigned plecor Nov 25, 2024

plecor and others added 6 commits December 3, 2024 16:44

Update doc/sphinx-guides/source/admin/harvestclients.rst

09fd6c1

Co-authored-by: Philip Durbin <[email protected]>

Update doc/release-notes/8739-publisher-during-harvesting.md

8b273e8

Co-authored-by: Philip Durbin <[email protected]>

Merge remote-tracking branch 'iqss/develop' into 8739-publisher-durin…

a3255b7

…g-harvesting

Rename migration script

bc33fbf

Merge remote-tracking branch 'origin/8739-publisher-during-harvesting…

c955183

…' into 8739-publisher-during-harvesting

Improve release note

b94d53f

pdurbin reviewed Dec 4, 2024

View reviewed changes

src/main/resources/db/migration/V6.4.0.1__8739-publisher-during-harvesting.sql Show resolved Hide resolved

cmbz added the FY25 Sprint 12 FY25 Sprint 12 (2024-12-04 - 2024-12-18) label Dec 5, 2024

Applies sql change to upsert script

c8499ba

pdurbin requested a review from poikilotherm December 9, 2024 19:26

pdurbin assigned landreev Dec 16, 2024

pdurbin approved these changes Dec 19, 2024

View reviewed changes

pdurbin unassigned pdurbin, landreev and plecor Dec 19, 2024

cmbz added the FY25 Sprint 14 FY25 Sprint 14 (2025-01-02 - 2025-01-15) label Jan 2, 2025

plecor added 2 commits January 7, 2025 08:30

Merge remote-tracking branch 'iqss/develop' into 8739-publisher-durin…

3440ca5

…g-harvesting

Add comment to sql file

2415ed7

ofahimIQSS self-assigned this Jan 7, 2025

ofahimIQSS reviewed Jan 7, 2025

View reviewed changes

doc/sphinx-guides/source/admin/harvestclients.rst Outdated Show resolved Hide resolved

Update Dataverse version in changelog

b1db18e

Co-authored-by: Omer Fahim <[email protected]>

cmbz added the FY25 Sprint 15 FY25 Sprint 15 (2025-01-15 - 2025-01-29) label Jan 15, 2025

pdurbin added 2 commits January 22, 2025 13:57

bump sql script version IQSS#8739

acda458

Merge branch 'develop' into 8739-publisher-during-harvesting IQSS#8739

7e72b25

ofahimIQSS merged commit 24143fd into IQSS:develop Jan 23, 2025
12 checks passed

ofahimIQSS removed their assignment Jan 23, 2025

pdurbin added this to the 6.6 milestone Jan 23, 2025

cmbz mentioned this pull request Jan 27, 2025

GREI 3: HDV Task - Improve OAI-PMH Harvesting IQSS/dataverse-pm#171

Open

58 tasks

pdurbin added a commit that referenced this pull request Mar 11, 2025

PR #9013 was merged for 6.6 not 6.5

5095b00

pdurbin mentioned this pull request Mar 11, 2025

tweaks to harvest client docs #11332

Merged

	- As of Dataverse 6.3, the publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This change affects all newly harvested datasets. For more information, see https://github.com/IQSS/dataverse/pull/9013
	- As of Dataverse 6.6, the publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This change affects all newly harvested datasets. For more information, see https://github.com/IQSS/dataverse/pull/9013

		@@ -0,0 +1 @@
		update foreignmetadatafieldmapping set datasetfieldname = 'distributorName' where foreignfieldxpath = ':publisher';

Harvest: map publisher tag to distributorName #9013

Harvest: map publisher tag to distributorName #9013

Uh oh!

Conversation

tcoupin commented Oct 3, 2022 • edited by pdurbin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

scolapasta commented Apr 23, 2024

Uh oh!

plecor commented May 3, 2024

Uh oh!

pdurbin commented May 3, 2024

Uh oh!

plecor commented May 14, 2024

Uh oh!

pdurbin commented May 14, 2024

Uh oh!

coveralls commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pdurbin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ofahimIQSS commented Jan 6, 2025

Uh oh!

plecor commented Jan 7, 2025

Uh oh!

Uh oh!

ofahimIQSS commented Jan 15, 2025

Uh oh!

ofahimIQSS commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pdurbin commented Jan 22, 2025

Uh oh!

ofahimIQSS commented Jan 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

tcoupin commented Oct 3, 2022 •

edited by pdurbin

Loading

coveralls commented May 14, 2024 •

edited

Loading

ofahimIQSS commented Jan 22, 2025 •

edited

Loading