GTRI RMS Server Software

GTRI RMS Server Software or Global Threat Reduction Initiative Remote Monitoring Server Software was conceived with an entirely different philosophy from the non-proliferation surveillance systems, like the Gemini Surveillance System, supported by GARS. While there was still the concept of historical review of data, there was much more attention focused on being reactive to events in real-time.  For this reason, I liken the GTRI RMS design to a security or an alarm system instead of a surveillance system. Interestingly, for a variety of reasons, some of the hardware and design ideas for the GTRI RMS were repurposed from previous non-proliferation surveillance systems leading to some challenging trade-offs.

Background

Traditionally, the non-proliferation surveillance systems employed by the IAEA were based on inspections and the use of archival data from the previous two to three months. The archival data consisted of image data and sensor data and was retrieved from the field by inspectors. After the data retrieval, the inspectors would review the data in Vienna to determine if there were any violations by the countries being surveilled and then act accordingly. Due to the highly technical nature of the inspections, the inspectors were often highly paid scientists. Budget concerns dictated an effort to streamline and lower the cost of inspections resulting in the idea of remote monitoring.

Remote monitoring initially was conceived as a way to reduce the travel to sites to retrieve the surveillance data, but it was soon discovered that the infrastructure available for remote communication was not compatible with transmission of large volumes of data. Often the only link to the sites was a phone line and a modem. Because of this, systems were designed to retrieve data weekly and then daily from the sites. Before long the concept of real-time data retrieval was discussed along with when and where such a system would be appropriate. Removal of nuclear material from a power plant is a slow and arduous task and constitutes a treaty violation for the host country, which would most likely necessitate a political solution. Stealing nuclear material from a hospital or lab for a dirty bomb, however, is considered a terrorist act that is best handled by the local police force, where a rapid response to events is essential, making it the ideal market for a real-time remote monitoring system.

Hardware

With the focus on radiation events, it was a necessity that the GTRI RMS incorporate a radiation detector and with the system architect’s background in surveillance, a camera module was also included. The addition of some external triggers for power monitoring, switches and seals completed the system's hardware for which I was to write the software.

Design Challenges and Choices

Initially the overall system consisted of two or three GTRI RMS units connected to an assessment computer running my software, the GTRI RMS Server Software. The GTRI RMS units provided images from up to four cameras, radiation, environmental, and sensor data, with the sensor data consisting of seal and switch status. The camera module acted as the main interface for all communication using a number of UDP and TCP ports. The GTRI RMS Server Software stored the data and provided an intuitive interface to system configuration, system status and alarm notifications.

An important design parameter was the level of data storage required, determined by the perceived value of historical data, both in terms of system health and functionality, and in evaluating the significance of current activity with respect to past events when assessing threat levels. For future-proofing and maximum flexibility, a SQL database was chosen to store the sensor and image data. Because of data constraints, only image meta data and not the actual images were stored in the database.  Actual images were saved in day-based block files.

Since only two or three GTRI RMS Units would be sending data, and for simplicity of design, Microsoft SQL Server (SQL CE) Database was chosen for storage. SQL CE is a compact relational database with all its databases residing in a single .sdf file, making for easy backup and upgrades. Also, since no separate applications needed to be installed to use SQL CE, installation was very straight forward.

The next aspect of the design to be determined was the role of the video feeds in the overall functionality of the system. While recognizing that the radiation detection was the primary security risk being monitored along with switch data, there was still upwards of twelve cameras (3 systems with 4 per system) available for viewing. Because of this, the initial design was very similar to a standard video security system with all the images being displayed. When alarms occurred, the image representing the system would blink in red and the user could click on the image to zoom in and get further data.

And finally the software system was designed around three separate programs: the server service, the management application and the client application. The server service was a standards Windows service consisting of up to 5 services. The management application required administrative privileges and provided configuration of all the system parameters in addition to support level data and utilities. The client application allowed an end user with limited privileges access to system monitoring and alarm notification functions.

Optimizations

During BETA testing of the initial release of the GTRI RMS Server Software it quickly became obvious that a few of the initial choices missed the mark. For example, while video was essential for real-time situational alarm assessment in the GTRI RMS, using it for security was redundant and unnecessarily distracting. Also, once the system was operational and passed qualification, the storage of much of the system and environmental data was deemed not useful and slowed the system performance. To address these issues, the client was redesigned from a video surveillance system to a control panel with status lights. Images could still be displayed when necessary, for example for alarm confirmation, but system status could now be viewed at a glance. The services were also modified to enable selective data storage with the default minimum being the alarm and radiation data, and images downloaded only from the time of the initial connection to the GTRI RMS Unit.

Though the primary design has remained stable, many improvements and optimizations have been added to the software since its release. One of the first areas identified for improvement, was the updating of the SQL CE database. Even though the SQL CE database did not incorporate a separate server, it was felt that with only two GTRI RMS systems reporting, latency would not be an issue.  However upon initial connection to an GTRI RMS that has been in service, up to 30,000 images in the GTRI RMS could be downloaded. On a fast local network the image download speeds outstripped database indexing of the images resulting in data loses. In order to manage this situation changes were needed in the code. First an image cache was added for the downloaded images allowing temporary storage of images on the hard drive prior to indexing and sorting. While this addressed the problem of data loss, it was soon discovered that the SQL CE database access code also needed to be optimized. A multi-threaded scheduler was added that sorted and cataloged data in parallel and combined database writes for batch processing. The scheduler also provided faster response times for alarms since the processing of alarm signals was now independent of the database code.

While batch scheduling of the writes to the database greatly improved performance, the asynchronous nature of the data transmission to the server and the need for rapid response to events by the client demonstrated the latency of the database access to be problematic. This, along with the increase of GTRI RMS units supported by a single assessment computer to as many as 50, necessitated the change away from SQL CE to Microsoft’s SQL Server Express 2014. The tremendous increase in database performance did come with some drawbacks, but they were mitigated somewhat by improvements in the Microsoft SQL Server setup, which supports the use of a configuration file through the command prompt installation and the availability of SQL Server scripts.

Enhancements

Other enhancements to the software were centered on the user experience. The server added the ability to send emails and texts when alarms occurred. Data analysis tools along with rudimentary statistical analysis were incorporated in both the management and client software for assessing radiation data. And, while the primary focus of the GTRI RMS was as a security system, an ability to evaluate the image data was added with functionality that allowed a user to perform a surveillance review.

And finally, to increase the usability of the GTRI RMS Server Software and to facilitate its integration into currently implemented security systems, an interface was added to the server that allowed non-OEM systems to receive security data and alarm notifications. The interface was made secure not only in terms of the data transmission but also in authentication of the recipients of the data. In addition, the interface uses industry standard protocols to minimize the effort needed to develop external clients to receive the data.

GTRI RMS Server Software Version 2.90