What is ITpipes Sync and ITpipes Host

ITpipes Sync

1. From a field data collection system (CCTV truck) to an office Master database.
2. From an interim QA/QC “temp” project/database to a centralized project/database.
3. From an ITpipes centralized database into an AMS.
   

Note: Please that to transfer information into multiple databases, for example, a single project/database syncing into more than one project/database, is currently not supported. This feature may be part of our upcoming Dashboards Module in ITpipes Web. 

Set up Requirements

Note: Sync modules communicate using port 8101 TCP protocols. This port will need to be opened in

the Firewall for proper communication.

1. (ITpipes Host) Server-side: Microsoft SQL Server 2012 or higher
2. (ITpipes Host) Server-side: Windows Server 2012 or higher
3. (ITpipes Sync) Client side: Windows 7 or higher
4. ITpipes Sync requires that ITpipes Host be either installed on the same computer or if on a domain, on a centralized server.
5. (ITpipes Sync) Anti-virus exception/whitelist allowing Sync to run on the Client side.

Note: For transferring into an AMS, integration to that AMS would need setup. For example, from an ITpipes centralized project/database into an AMS system such as Cityworks, Maximo, etc. Contact an ITpipes Sales Representative for current integrations. This document refers to Asset Management Systems (AMS), Centralized Maintenance Management Systems (CMMS), Work order systems (WOS), and ERP (Enterprise Resource Planning Systems). All are generically referred to as “AMS”.

Recommendations

ITpipes Sync Python Scripting

How Python’s Scripts work within ITpipes Sync

Mobile System

1. ITSync Client installed
2. ITpipes Host service installed
3. IronPython installed
   

Server-Side

1. Script folder structure created with appropriate permissions
   

Folder Structure

1. Create a folder in a shared directory where the python scripts will reside in.
1. I.e. \\NetworkShare\ITpipesScripts
2. Create the following folders in the newly created folder:
1. PostJobProcessingEvents
2. PostProcessingEvents
3. PreJobProcessingEvents
4. PreProcessingEvents
3. Download or transfer the scripts and place them in the appropriate folders that were created above. Sync will automatically download the scripts and run them.
   

Workflow

1. ITSync Client executes to upload new inspections
   
2. CCTV Truck daily upload
   
3. Contractor upload
4. Sync downloads any new scripts from the server location
5. Pre Processing Python scripts execute
6. Pre Job Processing Python scripts execute
7. Data transfer job process occurs including AMS integration
8. Post Job Processing Python scripts execute
9. Post Processing Python scripts execute

ITpipes Python Scripting Execution Level

1. Preprocessing Events:
1. Scripts running at this level are intended to run once per Sync process. The execution of scripts in this level occurs before any data synchronization between the databases being synchronized. At this level of script execution, database manipulation or editing is not permitted.
2. PreJobprocessing Events:
1. Scripts running at this level are intended to run once per project/database. The execution occurs before any data synchronization between the databases being synchronized. At this level of script execution, data manipulation or editing is permitted.
3. PostJobprocessing Events:
1. Scripts running at this level are intended to run once per project/database. The execution occurs after any data synchronization between the databases is synchronized. At this level of execution, data manipulation or editing is permitted.
4. Postprocessing Events:
1. Scripts running at this level are intended to run once per sync process. The execution occurs after any data synchronization between the databases is synchronized. At this level of script execution, database manipulation or editing is not permitted.

Examples of what can be done with ITpipes Python’s Scripts

1. Pre or Post-processing execution level scripts.
1. Update GIS map or Inspection template on mobile units
   
2. ITpipesSyncTemplates.py: Allows the ability to update or place the template in the template folder from the source location to the target location. For example, from a centralized office location onto a mobile unit.
   
3. These scripts are some examples of what can be accomplished with python scripts. If there is a special
   
4. need that is required. Contact an ITpipes Sales Representative to discuss the option for python scripts.
   
5. Establish network connection and map network virtual drive before data transfer is attempted
   
6.  Delete video files that are older than a specified number of months or days
   
7. ITpipesMp4CheckandDelete: Goes through the chosen path and will check to see if the mp4 is within a certain percentage of length to the WMV. The script also can delete videos that are older than a certain amount of days.
   
8. Release mapped network drive
   
9. Shut down the mobile unit computer
10. Back up certain files or folders from the truck to the server.
2. Pre or Post Job execution level scripts.
1. Create new field values
2. Inspection percentage
3. ITpipesMLIPercentage.py: Places a percentage of the inspection length based upon the total length on a specific field.
4. Length Check
5. If “inspection percentage” is greater than or less than a designated value, a Y/N flag is added 
   
6. Tap Count
   
7. Segment length update
8. ITpipesSectionLengthUpdate.py: Updates the Section length of the asset with the actual inspection length.
9. Incomplete report
10. ITpipesNotCompletetoTextFile: Goes into the database and creates a CSV file that includes all the assets that have not been set to complete at the end of the inspection.
11. ITpipesSeatacReportFix: This is a specific script for a client that copies over information from custom fields to the fields that show on the report.

ITpipes Host & ITpipes Host Manager

1. ITpipes Sync relies on ITpipes Host for the credential information to the target ITpipes project database.
2. ITpipes Host contains the global Sync settings and a list of valid target databases.
3. ITpipes Sync will lock the ITpipes Host service when the ITpipes Sync process begins so only one ITpipes Sync may occur at the same time. If ITpipes Host is locked, ITpipes Sync will wait until the ITpipes Host unlocks before automatically starting the sync process.
4. If the target location is a SQL database. The password to the database is encrypted to maintain security.
5. ITpipes Host contains AMS system information necessary to perform the AMS Integration into an AMS system.
6. Host Manager is the GUI (Graphical User Interface) used to set up and maintain ITpipes Host. ITpipes
7. Host Manager is installed at the same time as ITpipes Host.
   

Workflow

1. ITpipes Host is installed.
2. ITpipes Host Manager is used to configure the ITpipes Host service. This includes adding database connections and AMS integrations to the list of target databases for ITpipes Sync.
3. ITpipes Sync is installed and pointed to the system in which the ITpipes Host service was installed.
4. ITpipes Sync is configured to pair ITpipes projects on the inspection system with their appropriate target database.
5. At a scheduled time every day, or upon being manually forced, ITpipes Sync will iterate through each project defined in the settings. ITpipes Sync will verify that the data and media in the source project exist in the target project.
6. If an AMS project is defined in Host Manager, ITpipes Sync or ITpipes Host, depending on which AMS system is being used, will upload new inspections to the AMS system. Sync will also download new open work orders from the AMS system, if configured to do so.
7. If configured to download from the target project, ITpipes Sync will download the information from the target project to the source project.

ITpipes Host Installation and Setup

ITpipes Host Installation

Prerequisites

1. ITpipes Master SQL Server database has been created.
2. Required information including client-specific details that the client typically sets up and maintains. ITpipes may have a record of this information, but we are unable to track any changes or updates made to this information so we highly recommend having this available from internal sources before installation.
3. SQL Server instance name
4. If located on a domain. The host is recommended to be installed on a server. ITpipes Sync is to be installed on the mobile units. To install this, administrative access is required.
5. Individual requirements vary by the AMS system. The requirements are defined in another document.
   
6. SQL Server database credentials
   
7. Database name set during ITpipes master database creation

Installation

1. If a new install:
   
1. Unzip and run the ITpipes Host installation program that was downloaded from Google Drive.
   
2. Download the zip file containing the updated files.
   
3. Unblock the downloaded file.
   
1. Right-click the downloaded file.
   
2. Select Properties.
   
3. Select the checkmark next to unblock.
   
4. Select Ok.
   
4. Unzip the downloaded file
   
5. Copy and paste the files inside the unzipped folder to the correct folder location.
   
1. Server -> C:\Program Files (x86)\Infrastructure Technologies\ITpipes Host
   
2. Manage -> C:\Program Files (x86)\Infrastructure Technologies\ITpipes Host Manager
6. Proceed to setup.
   
2. If updating:
   
1. Download the zip file containing the updated files.
   
2. Unblock the downloaded file.
   
1. Right-click the downloaded file.
   
2. Select Properties.
   
3. Select the checkmark next to unblock.
   
4. Select Ok.
   
3. Unzip the downloaded file
   
4. Copy and paste the files inside the unzipped folder to the correct folder location.
   
1. Server -> C:\Program Files (x86)\Infrastructure Technologies\ITpipes Host
   
2. Manage -> C:\Program Files (x86)\Infrastructure Technologies\ITpipes Host Manager
5. Proceed to setup.

ITpipes Host Setup (ITpipes Host Manager)

1. Before running ITpipes Host Manager, proper permissions will need to be set for the “C:\Program Files (x86)\Infrastructure Technologies” directory. Users will need “Modify” access for the directory for ITpipes Host Manager to function properly.

To open ITpipes Host Manager:

Note: If ITpipes Sync does not start syncing before the end time has been reached. ITpipes Sync will not start the syncing process.

1. Select ITpipes Host Manager through the start menu to launch the stand-alone application.
1. ITpipes Host Manager can also be started by going to “C:\Program Files (x86)\Infrastructure Technologies\Host Manager” and running the Host Manager.exe program.
2. Settings (Gear Icon):
1. Host Settings - Provide a computer name or IP Address to run the client. ex: localhost. This will need to be able to connect to define the rest of the settings.
2. Sync Initialization Settings - Choose when ITpipes Sync is to automatically start and how often.
1. Frequency - How often ITpipes Sync will automatically start.
2. Time Range - When Sync will try and start the sync process. (Note: If running multiple trucks. ITpipes Host will lock while one truck is syncing. ITpipes Sync will check periodically for Host availability.
3. Email Settings - enabling email alerts will provide the recipients an email of the sync log. Note: This feature requires a Username and Password.
1. Outgoing Server: The SMTP Mail Server.
2. Port: Port with which the SMTP server communicates, this is usually Port “21”.
3. Username: Username of a login to the SMTP Server.
4. Email Address: Email address of the Username
5. Password: Password of the username.
6. Send alerts to: This lists the recipients of the email. To send to multiple people separate the emails by a comma.
4. Sync Settings -
1. Sync Asset Types: Which asset types that ITpipes Sync will sync to and/or from the target database. 
   
2. Data Case Options: If the desire to have all the information coming from the source database will exist in a specific case, i.e. upper, lower, or Formal. For example, if Title is selected. “title” becomes “Title.
3. Match Criteria: The fields that will be matched for sync to merge source information to target information for an inspection.  All fields defined in the match criteria have to match to combine inspections onto another asset. For example, if ML_Name, US_MH, and DS_MH are selected. Sync will not merge the inspection onto the asset unless ML_Name AND US_MH AND DS_MH fields match. To switch between asset types. Select the button next to the desired asset type.
5. Upload Settings - configure what will be uploaded to the target project.
1. Disable: Disables the upload feature of ITpipes Sync.
2. All: Uploads all inspections from the source database.
3. Complete inspections: Only uploads inspections that are marked as “Complete” at the end of an inspection.
6. Download Settings - configure what will be uploaded to the target project.
1. Disable: Disables the download feature of ITpipes Sync.
2. All: Downloads all inspections from the target database.
   
3. Open inspections: Only downloads inspections that are marked as “Open” on the WO_Status field. This setting is mainly used for AMS integrations.
   
7. Upload and Download Settings:
   
1. Match Handling - How ITpipes Sync handles the assets that match assets on the target database to the source database. The options are Update changes, Discard changes, and Add a new record.
   
2. Overwrite Media - If enabled, ITpipes Sync will replace matching media files.
   
3. Check for the file before copying media: This setting will check to make sure the file exists before copying the media over.
   
4. File Types - What file types that ITpipes Sync will copy over.
   
5. Server Media Path - Location of the media path where the python scripts are located. This is for ITpipes Sync python scripting.
   
8. Error Log - Whether the desire is to have a text file and/or the log to be recorded on the Master Database(target database).
   
3. Adding Projects :
   
1. Select
   
2. Project Alias: Nickname for the project that will be displayed and used to differentiate between projects.
   
3. SQL Server: Name of the Server.
   
4. SQL DB: Name of the database being referenced.
   
5. Username / Password: SQL account with database read and write permissions.
   
6. Select “Connect”. If everything was successful, a display will pop up stating “Connection Successful.” The media path will be automatically populated.
   
1. If a UNC path is not populated. Input the UNC path to the media location. For example \\server\folder\Media_Path\.
   
7. Select “Add Connection” to finish adding the project to ITpipes Host.
   
8. The correct media path needs to be mapped on all the PCs running ITpipes Sync.
   
9. Windows authentication is not recommended. ITpipes suggests that a static user be set up to connect to the SQL database.
   
10. This does not include the name of the database
    
11. An IT person or Database Administrator may be required to get the information to connect to a SQL database.
    
12. Must be able to browse to the location to apply.
    
4. Editing Projects:
   
1. Editing projects allows the modification to the UNC path and enables S3 upload.
   
2. Select the project that needs to be modified.
   
3. Select to edit the project.
   
5. Deleting Projects:
   
1. Deleting projects only removes the link to the project, not the project itself.
   
2. Select the project that is needing to be deleted.
   
3. Select to remove the project link.
   

ITpipes Sync Install, Setup, and Features

ITpipes Sync - Install:

Prerequisites

1. Ensure Sync requirements have been met
2. Proper permissions will need to be set for the “C:\Program Files (x86)\Infrastructure Technologies” directory. Users will need “Modify” access on the directory for ITpipes Sync to function properly.
3. If ITpipes Host is installed on a centralized server
4. Server Name
5. If ITpipes Host is installed locally on the CCTV system
6. Ensure local mode is checked in the Sync Client settings.
   

Note: If located on a domain. ITpipes Host can be installed on a centralized server. ITpipes Sync will still need to be installed on individual computers. If installing or reinstalling, try to match the ITpipes Sync version to the ITpipes Host version. Call ITpipes Tech support for assistance, if desired, in installing or reinstalling ITpipes Sync.

1. If a new install:
   
1. Unzip and run the ITpipes Sync installation program that was downloaded from Google Drive.
   
2. Download the zip file containing the updated files.
   
3. Unblock the downloaded file.
   
1. Right-click the downloaded file.
   
2. Select Properties.
   
3. Select the checkmark next to unblock.
   
4. Select Ok.
   
4. Unzip the downloaded file
   
5. Copy and paste the files inside the unzipped folder to the correct folder location.
1. Client -> C:\Program Files (x86)\Infrastructure Technologies\ITSync Client
   
6. Proceed to setup.
   
2. If updating:
   
1. Download the zip file containing the updated files.
   
2. Unblock the downloaded file.
   
1. Right click the downloaded file.
   
2. Select Properties.
   
3. Select the checkmark next to unblock.
   
4. Select Ok.
   
3. Unzip the downloaded file
   
4. Copy and paste the files inside the unzipped folder to the correct folder location.
1. Client -> C:\Program Files (x86)\Infrastructure Technologies\ITSync Client
   
5. Proceed to setup.

Note: ITpipes Host will be necessary to run ITpipes Sync. If ITpipes Host is installed on a centralized server, ITpipes Sync can be installed by itself. If either: a domain does not exist or ITpipes Host is not installed on a server. ITpipes Host will need to be installed locally.

ITpipes Sync - Setup:

1. Before running ITpipes Sync
1. Ensure Sync requirements have been met
2. Ensure all prerequisites have been met
3. Launch ITpipes Sync from the start menu if the program is not already started. ITpipes Sync will launch and remain open in the system tray in the lower right corner by the system clock.
4. If using ITpipes Mobile, SQL Server will need to be used to set up Sync.
   

To set up the client:

1. Double click or right-click on the ITpipes icon and select “Show scheduler”.
1. To start ITpipes Sync manually go to C:\Program Files (x86)\Infrastructure Technologies\ITSync for the 64-bit version of Windows.
2. Select “Settings”.
1. If ITpipes Host is installed on a centralized server, uncheck “Local mode” and input the name of the server where the ITpipes Host is installed.
2. Refer to Client-Server Name
3. Select “Add” to open the “Add Connections” Window.
4. Select “Project” or “Directory”.
1. Select the folder icon to the right of the path. A navigation window will pop up depending on what option is selected.
2. If Access is selected
1. If Project is selected:
1. A file navigation window will pop up.
2. Navigate to the database that needs to be synced.
3. Select “Open”.
4. Select “Save” to save the connection.
2. If Directory is selected:
1. A directory navigation window will pop up.
2. Navigate to the directory of the projects folder.
3. ITpipes Sync can run from one source ITpipes database to a one target ITpipes database or multiple sources ITpipes databases to one target ITpipes database.
4. You May need to expand to see the icon.
5. Select “Ok”.
6. Select “Save” to save the connection.
3. If SQL Server is selected
1. Enter the SQL Server name.
2. If setting up Sync for ITpipes Mobile on the CCTV system.
3. Enter localhost\sqlexpress for the Server. Do not use the local computer name.
4. Enter the local database project name
5. The database name can be found in ITpipes Mobile.
6. Enter the local SQL Server username and password
7. By default the username and password are “itpipes”
8. Select the target destination database in the dropdown
9. Select “Save”
10. If the credentials are entered incorrectly a message will pop up showing “Error connecting to SQL project. Check credentials.”
11. Select “Save” in the Settings window.

Automatic Sync Settings

1. The following conditions must be met for Sync to run automatically:
1. ITpipes Sync must be running.
2. The system running ITpipes Sync must be able to connect to ITpipes Host.
3. If ITpipes Host is running on a central server, the system will need to be connected to the same network as that server.
4. The system running ITpipes Sync must be unlocked.
5. The previous conditions must occur during the start and stop times shown in ITpipes Host Setup.
6. Automatic Sync while Locked
7. A user with admin rights will allow Sync to run with the computer locked:
1. Exit ITpipes Sync.
2. Open a File Explorer and navigate to C:\Program Files (x86)\Infrastructure Technologies\ITSync Client
3. Right-click ITSyncClient.exe and select Properties
4. Select the Compatibility tab.
5. Check the box to Run this program as an administrator
6. Select OK.
7. Open ITpipes Sync.
   

Performing a Sync:

1. Select “Sync Now” to manually start syncing your source project or projects to the target project.
2. Select “Sync Partial Now” to perform a partial sync. The partial sync will only sync over snapshots and reports.
3. When completed a window will pop up stating that ITpipes
4. Sync has completed the syncing process.
5. If Sync shows ITpipes Host: Server: localhost Locked
6. Select Unlock Server at the bottom of Sync.
   

NOTE: This should NOT be done if an AMS integration is in place (Cityworks, Cartegraph, Lucity, etc)

ITpipes Sync Purge:

1. Select the “Purge” tab.
2. Select the checkbox next to “Enable Purge”.
3. A message box will pop up letting you know that data loss may occur.
4. Select “Yes” to enable the purge feature.
   

Purge Settings:

Note: A help text will show below the settings signifying what each function does. The client will be able to customize the deletion of information to their desire. ITpipes Sync will check to make sure that the media and inspections are in the master project before deleting anything.

1. Days After Last Modification to Wait: This is the number of days ITpipes Sync waits to purge the inspections and/or media.
2. Don’t Purge if Missing Videos: This is an exception that will stop purge from deleting inspections if ITpipes Sync is unable to find the video. Recommended to be enabled.
3. Don’t Purge if Missing Pictures: This is an exception that will stop purge from deleting inspections if ITpipes Sync is unable to find the snapshots. Recommended to be enabled.
4. Delete Video Files: This feature will delete the video files and references from the source database.
5. Delete Picture Files: This feature will delete the snapshot files and references from the source database.
6. Delete PDF Files: This feature will delete the PDF report files and references from the source database.
7. Clean Up Empty Directories: This will go through the source project and delete the empty directories. If there is ANY file in there. The folder will not be deleted.
8. Purge Asset Records with No Inspections: This will delete the asset records if there are no inspections tied to the asset.
9. Purge Inspections Records: This feature will delete the inspection records that meet the criteria.
   

ITpipes Sync Logs

To view the log.

1. Right-click on the “ITpipes” logo to bring up the options for ITpipes Sync.
2. Select “Open Logs” and a folder will open.
3. Sort by modified date to find the newest logs.
4. To manually find the log files go to C:\Program Files (x86)\Infrastructure Technologies\ITSync Client\Logs. If running a 64-bit version of Windows.
5. The taskbar may need to be expanded to see the ITpipes Sync icon.
6. The logs will let you know what projects were synced or if there were any errors.
7. If an AMS Integration is enabled, the log will also show the status regarding open and closed work orders at the very bottom of the ITpipes Sync log.
8. All errors or exceptions will be located under the “Error Log” or the “Exception Log”.
   

Reading the Sync Log File

1. Merge log:
1. This portion of the log is informational items.
2. It is possible to see: Datatype mismatch in field ML.DS_RimtoGrade. This is normal and informational only.
3. There is useful information as well such as: Committing changes to record 4d9160e5-a649-4eed-a69a-85024cafe7a5 in table INSPECTION. Field Inspection_Date set to 8/28/2019 at 9:00:22 AM. This entry lets the user know that there was a modification done to a certain field. In this case, it was the Inspection_Date field.
2. Error log:
1. This portion is one of the two areas that are the cause of concern.
   
1. Below are examples of errors that will show in this area:
2. Disabling UNC file transfers for this session. Tried to copy the file to the UNC path more than 5 times. Windows cannot locate network share to copy the file
1. This means that when the sync software was initiated. But it could not connect to the specified UNC path. Normal troubleshooting is to make sure that the computers do have access to the UNC path that is specified to be the location of the ITpipes Videos.
3. Could not sync file X. Error: Cannot copy file. Ensure that the specified drive letter or UNC share location exists for path: X
1. As this message explains. We would need to verify that the computer that is running sync has access to the drive letter or UNC path.
   
3. Exception log:
1. This portion is the second of the two areas that are cause for concern.
1. Below is an example of an error that will show up:
2. Exception encountered during sync: Execution Timeout Expired. The timeout period elapsed before completion of the operation or the server is not responding.
1. This error is letting the user know that it could not connect to the target database. It attempted but timed out.