--- title: "Duplicate Detection setup" canonical: "https://support.soutron.net/space/KNOW/48300045/Duplicate%20Detection%20setup" format: markdown --- Duplicate detection can be enabled in two modes. 1. Manual record creation 1. Record Import from a file   Although duplicate detection is not enforced in Auto cataloguing module it is now possible to enable record updating if matches are found for imported auto cat records.   **Setup**   If you wish to use manual record detection open system configuration maintenance and enable the option '**Enable Manually created record Duplication Check' **and also set** 'Duplicate detection match (%)' **to your preferred value, the default setting is 90%. Next open field maintenance and set the 'Duplicate Detection Weight' for each field you wish to be included in the duplicate detection matching process. Values should be decimal in the range of 0-1. If you want to only match when the full value is matched enter 1. if you wanted to match titles where only some of the words were the same enter a value in the range of 0.5-0.8.     **Manual Record Detection**   Once all set-up is complete the next time you create a new record that matches an existing title based on your settings you will see a pop-up prompt upon clicking the save new record button. This will provide you with a display of the record(s) the system found as a potential match. You can then decide to continue to save the record or decide to cancel the new record creation.   ![image](media://81547e81-70f1-4701-9ee0-e48c1659002d)   **Record Import Detection**   File imports continue to be configured via Import Maintenance, however there are some additional options in the set-up page. A new column has been added which now allows you to specify the duplicate detection value for each mapped field. Setting this value overrides field maintenance. Values here should be entered in the range of 0-100, 100 being a complete match. This column is indicated by the label 'DD'.   ![image](media://4c627c9d-3312-41df-a076-d93e0bfc5f89)   At the bottom of the import template are two additional options under the 'Import Options' section.  **'Run duplicate detection when import completes' **- when enabled this option will perform the duplicate check as part of the import process.  **'Duplicate detection level (1-100%)'** - This value overrides the Duplicate detection match value you set in the system configuration. This is to allow you to have different match rates for manual and imported records.   ![image](media://9b4bfe3a-0455-4103-a12a-5a7e6f8b4b2a)   Once the template is configured you can elect to run the import either via a scheduled task or using the run now option.   Running via Run now   The run now pop-up now has an additional tab for 'Duplicate detection' This only has one option listed and when enabled will ensure duplicates are matched upon import. Having the option in here means it can be enabled disabled even if it was or was not selected in the template, and so you can be selective as to weather duplicate detection runs for certain imports.     **Managing duplicated record found during import**   Once the import has completed you can navigate to Modules > Cataloguing > Duplications. This page will display details of all records the system believes are a match for an existing record. At the top of the page it will display the total number of duplicates waiting to be processed, and below that the import template the record was imported by and the date & time-stamp of the import.   Below that is the record display of the imported (potentially duplicated record), to the right of this record will be the record the system has identified as an existing match. If several records are thought to be matched you will be able to navigate through them all.   ![image](media://dea639cd-53ec-4f51-baf4-d073a6b9fd35)     Once you have reviewed the two records you have four options.   1. Add duplicate - This will save the duplicated record currently displayed on screen to the system and give it a new unique CID. 2. Delete duplicate - This will delete the duplicate currently displayed on screen. 3. Approve all - If you are confident none of the records matched and displayed are duplicates you can opt to save all records to the system, each record will be allocated a new unique CID number. 4. Delete all - If you are confident all of the records matched and displayed are duplicates you can opt to delete all records from duplicate maintenance.   **How duplicates are calculated**   The actual formula used to calculate the duplicate weighting is quite complex, below we have tried to explain this in a simple example.   The weighting assigned to each field is converted to a whole number, if that field is then matched it is added to a total weighting, if that total weight equals or is greater than the match percent set in system configuration or the import template the record is treated as a match.   Example: - Title is assigned value of 70 in import template - Edition is assigned value of 15 in import template - Publication Date is assigned a value of 20% in import template - Match percentage is set to 90%   When this record is created lets assume the Title and edition match but the publication date entered is different.   The total for this record will be calculated as: 70+15. Giving us a match total of 85. Because of match percent is set to 90 this record will not be recognized as a match.