# IWS5.Importer.Worker
The IWS5.Importer.Worker project is a worker service responsible for importing work items in bulk to the Process Director system.

The IWS5.Importer.Worker project was ported to the iWS5 solution from the [WorkItemImporter_LPNU](https://github.com/Greenfield-Investments/WorkItemImporter_LPNU) project, and exists only in the FEMA branch of the iWS5 solution. The below documentation is for the FEMA branch only. To read the documentation for WorkItemImporter_LPNU, see its README in its repository root.

## Contents
The IWS5.Importer.Worker project is somewhat self-contained, as it was ported from the old standalone project near the end of the active Process Director development cycle. There is a lot of code which is already implemented elsewhere in the iWS5 solution, meaning there is duplicate code.

The important code is as follows:

1. [Processors](Processors/) - Contains classes with logic for processing files and work items during import events.
2. [Workers](Workers/) - Contains classes with definitions for listener workers which find work items to import.

## Development & Configuration
The IWS5.Importer.Worker project has a single `appsettings.json` file which contains configuration variables that must be set/updated for the worker to run:

1. Connection strings (used for database connection):
    1. `GreenfieldPD` - The full connection string to authenticate with a local or production copy of the Process Director database. For example: `Data Source=MY-DATABASE-SERVER;User ID=PDAdmin;Password=MY_SECURE_PASSWORD;Initial Catalog=ProcessDirector;Encrypt=True;Trust Server Certificate=True`. In general, this string is dependent on the SQL Server configuration, but the PDAdmin user (with the `sysadmin` role on the server) should be used.

The [IWS5.Importer.Worker setup package script](https://github.com/Greenfield-Investments/setup-packages/tree/main/IWS5.Importer.Worker) modifies the `appsettings.json` file with the values for the database connection provided by the user during installation.

The rest of the required configuration variables for IWS5.Importer.Worker are set in the database via the classic iAdmin application, under the "Configure" -> "Import Settings" -> "Default Settings" section. Example configuration:

| Configuration Key | Configuration Value |
| --- | --- |
| Name | Production |
| Document Path | C:\Importer\ImportDir |
| Archive Path | C:\Importer\Backup |
| Error Path | C:\Importer\ImportError |
| Log Path | C:\Logs\Importer\log.txt |
| Loop Pause (min) | 30 |
| Log Retention (days) | 30 |
| Log File Max Size (MB) | 1000000 |
| CPU Usage Limit (%) | 25 |
| Max PDF Size (MB) | 25 |
| Clean Archive (days) | 30 |
| Is Active | (Checked) |

Make sure that the "Document Path", "Archive Path", and "Error Path" directories exist on the server where the importer worker will run.

Additionally, a valid Syncfusion Essential Studio license key must be supplied in [Program.cs](Program.cs) for PDF processing functionalities to work properly.

## Usage
To use the IWS5.Importer.Worker application, first set the importer configuration in iAdmin as described in the previous section. Then, to import work items, follow these steps:

1. In the "Document Path" directory (e.x.: `C:\Importer\ImportDir`), create a new directory with the name of the process to import work items into. For example: `MyProcess`.
2. In the newly-created process directory, optionally create an XML file with the name of the process (e.x.: `MyProcess.xml`) following the import XML schema below. This file will apply to all work items being imported for the process, except those where a work item-specific XML file is found.
3. In the newly-created process directory, create a new directory for each work item that should be imported. The directories' names should match the work item's name, so if you want a work item called "Work Item 1" then use that name for the directory.
4. In each of the newly-created work item directories, add any document/image files that should be imported for the given work item.
5. In each of the newly-created work item directories, optionally create an XML file with the name of the work item (e.x.: `Work Item 1.xml`) following the import XML schema below. This file will apply to a single work item.
6. In each of the newly-created work item directories, create an empty file called `done.txt` to mark the work item as ready for import.

## Import XML Schema
The import XML schema is as follows:
```xml
<?xml version="1.0" encoding="utf-8"?>
<ROOT>
	<PROCESS>
		<PROCESS_NAME>{PROCESS_NAME}</PROCESS_NAME>
		<PROCESS_VER>{PROCESS_VERSION}</PROCESS_VER>
		<WORKITEM_NAME>{WORK_ITEM_NAME}</WORKITEM_NAME>
	</PROCESS>
	<FOLDER>
		<FIELDS>
			<!-- Add keys for folder fields using their database column names -->
		</FIELDS>
	</FOLDER>
	<DOCUMENT>
		<DOC_CABINET>{DOCUMENT_TEMPLATE_NAME}</DOC_CABINET>
		<FIELDS>
			<!-- Add keys for document template fields using their database column names -->
		</FIELDS>
	</DOCUMENT>
</ROOT>
```

Folder field and document template field key names should match their database column names. For example: if your process has a folder field by the name of `FOLDER_FIELD_STRING_1`, then the key would be `<FOLDER_FIELD_STRING_1></FOLDER_FIELD_STRING_1>`. The values for document template fields supplied in XML will apply to all documents being imported.

When creating an XML file at the process level (not for individual work items), you may wish to omit the `<WORKITEM_NAME>` key. Otherwise, all imported work items for that process will have the same name.

## Additional Notes
The importer worker will fail to start successfully if an import configuration is not found in the database. The only way to stop the importer worker from running is to end the process.

The importer worker's folder and document template field caching implementation may be broken. In cases where a work item is imported and has a work item-level XML file to define field values, the next work item may inherit those values if it does not have its own XML file.

There is a known issue with document template field processing when importing work items that may cause an error like: `Field DOC_CABINET with value {DOCUMENT_TEMPLATE_NAME} is not valid for cabinet {DOCUMENT_TEMPLATE_ID}`. This error does not appear to affect field value assignment and can most likely be ignored.