Publications – ISOLA https://isola-project.eu EU Horizon 2020 Project Thu, 03 Nov 2022 10:31:40 +0000 en-US hourly 1 https://wordpress.org/?v=5.9.16 https://isola-project.eu/wp-content/uploads/2020/07/cropped-favicon-32x32.png Publications – ISOLA https://isola-project.eu 32 32 Usefulness of ISOLA Pilot Use Cases https://isola-project.eu/usefulness-of-isola-pilot-use-cases/ Thu, 03 Nov 2022 10:27:30 +0000 https://isola-project.eu/?p=3855

The usefulness of ISOLA Pilot Use Cases for passenger shipping companies

ISOLA will develop, integrate, test, deploy, demonstrate and validate a systematic and fully automated security approach by incorporating innovative technologies for sensing, monitoring, data fusion, alarming and reporting real-time during illegal incidents.

The main objectives of ISOLA are:

  • To create strategies and methods in order for a ship to easily integrate solutions regarding passengers and crew safety in the existing ship systems.
  • To propose innovative sensor and visual technologies to support security and safety.
  • To create a complex collaborative system for monitoring and detecting security incidents and events.
  • To create early warning methods for the ship security crew to prevent security issues.
  • To collect incident evidence by exploiting the Augmented Reality features.
  • To allow easy engagement of different authorities in a ship related crisis.
  • To model, classify and easily report a security event.

ANEK, as an end-user, participates in PUC1 – Intoxicated troubles & PUC2 – Detection of a theft incident at a vessel’s shops of a total of five Pilot Use Cases (PUCs). The company has provided information on the procedures currently being followed on its ships and has contributed in recognizing the operational requirements to set up the use cases. 

Based on our engagement in the project so far, it is evident to say that a company operating in passenger shipping can benefit the greatest through the use of ISOLA system since it significantly strengthens the company’s safety and security procedures in terms of:

  • detection and identification of illegal action and suspicious behaviour of intoxicated people (under the influence of alcohol and/or drugs)
  • detection and identification of theft incidents caused by passengers

Enhancing the prevention and/or effective management of potentially dangerous and harmful situations for the passengers, the crew and the ship itself. 

]]>
Crisis classification https://isola-project.eu/crisis-classification/ Mon, 26 Sep 2022 12:58:18 +0000 https://isola-project.eu/?p=3849

Real-time Crisis Classification in the ISOLA System

Security threats across the oceans of the world and to the coastal areas continue to expose people and property at risk, while piracy and armed robbery are considered some of the most hazardous threats. Every year numerous violent attacks result in injuries, loss of lives and psychological traumas, as in many cases pirates proceed to hostage-taking and kidnapping of seafarers. Furthermore, security threats such as hijacking of vessels for ransom, cargo/property theft or damage arise disruptions of maritime transport and trade flows and have a significant financial impact in maritime trade. Hence, the development of new methods is needed to enhance the security of the vessels and to reduce the risk of a successful attack. ISOLA aims at enhancing the security of passenger ships through the development of an intelligent and automated security superintendence system. The National Technical University of Athens (NTUA), as part of the ISOLA Consortium, is developing a smart tool that estimates the risk level, in real time, for potential security incidents and provides a real-time threat classification alert. The classification tool analyses the data gathered from the ship’s legacy systems and various sensors and modules installed on the ship as part of ISOLA. The core scientific principles of the tool are based on Bayesian probabilistic models. The tool is expected to effectively support the decision-making process in terms of the vigilance, surveillance, early threat detection and adequate situation evaluation, during this time-sensitive and extremely stressful circumstances and it is expected to minimize human error as well as the overall response time.

]]>
Crowd Monitoring https://isola-project.eu/crowd-monitoring/ Tue, 06 Sep 2022 13:48:20 +0000 https://isola-project.eu/?p=3843

Crowd monitoring and abnormal behavior detection from visual content

As a concept, crowd monitoring is generally an umbrella term and it might include either individual human behavior analysis or extended to a group of people and estimating number of participants in a specific area. Intelligent crowd-monitoring solutions can actually supply a valuable aim to the usual measures for crowd monitoring by combining state-of-the-art computer vision techniques that excel in both processing speed and precision accuracy. The actual real-time dynamic reporting of the number of passengers or crew members present, as well as the reporting of abnormal actions they might undertake, can be expanded to a variety of scenarios in maritime environment that would otherwise necessitate a large number of people dedicated entirely to perform this monitoring activity. The application, for example, includes, but is not limited to, early reporting of stressful situations or recognizing scenarios when ship’s security personnel should intervene to prevent or save individuals.

For ISOLA, the crowd-monitoring tool developed by CERTH takes input from ship’s surveillance cameras or dedicated drones and calculates the actual number of gathered persons in critical areas. Furthermore, it provides information about certain individual or crowd activities, which are considered harmful under certain conditions. The visual content is analyzed in near-real time, in order to offer the system with a higher degree of conceptual information about numerous individuals’ abnormal behavior, in other words a crowd analysis framework. When a suspicious and violent event occurs, the crowd-monitoring tool will trigger an alert to the ISOLA system.

]]>
Visual analysis for object detection supporting edge-computing solutions https://isola-project.eu/visual-analysis-for-object-detection-supporting-edge-computing-solutions/ Thu, 25 Aug 2022 09:49:35 +0000 https://isola-project.eu/?p=3837

Visual Object Detection

Object detection is one of the most fundamental and difficult problems in computer vision. It aims to discover object instances in real images from a huge number of established categories.  Handcrafted characteristics were initially used to create the majority of the early object detection algorithms. People had no choice but to build complicated feature representations and a range of speed up skills to exhaust the use of limited computing resources due to the lack of effective image representation at the time. The last decade, deep learning has emerged as a powerful tool for learning feature representations directly from data, resulting in significant advances in the field of generic object detection. Convolutional Neural Networks (CNNs) are the most common deep learning models employed in the field of object detection right now. A typical CNN has a hierarchical structure and is made up of several layers that are used to learn data representations with multiple levels of abstraction. The number of convolutional layers and kernels in the CNN model can be increased to improve accuracy. However, increasing the number of convolutional layers demands more computational resources, otherwise it slows down the CNN model’s processing speed. This serious problem is considered in cases, where the object detection needs to be applied locally on embedded devices with lower computational resources in inaccessible areas. The low-power image recognition challenge emphasizes a balance of accuracy, throughput, and power budget. In certain sectors, such as the internet of things (IoT), robots, autonomous driving, and drone-based surveillance, these goals are not just appealing but also necessary.

In the context of the ISOLA project, a number of monitoring-related services will be exploited, which will handle multiple visual data streams from security cameras and visual sensors running as UxV payloads. CERTH will develop an object detection solution based on deep neural networks that will give state of the art accuracy and performance, while taking into account the available computational resources. There are cases, where the object detector will be used for monitoring within the ship and it will have available the resources of powerful and high-end hardware. However, there are additional scenarios, in which this object detector must be used at great distances from the ship, on an embedded device mounted on a UAV. CERTH’s algorithm will be able to return results with increased performance and accuracy, despite the limited processing resources now available. In this way, CERTH will ensure that the ship’s security officers will receive the best situational awareness from all the visual sources available at the minimum time. 

]]>
Chemical dispersion modelling for critical infrastructure https://isola-project.eu/chemical-dispersion-modelling-for-critical-infrastructure/ Tue, 09 Aug 2022 09:18:40 +0000 https://isola-project.eu/?p=3831

HAVAC, the indoor dispersion modelling tool

Monitoring and regulating the air quality in indoor areas such as cruise ships and critical infrastructure is important. The outdoor air can be contaminated with agents such as diesel and car exhaust or Toxic Industrial Chemicals (TICs). These airborne agents can penetrate the building via windows and doors or via the ventilation system.

The main purposes of a Heating, Ventilation, and Air Conditioning (HVAC) system is to help maintain good indoor air quality in a building through adequate ventilation and also to provide thermal comfort. However, this also makes buildings vulnerable to external toxicological threats. Once toxic agents reach the air intakes of the HVAC or once these toxic agents are released somewhere inside the building (either accidentally or even intentionally), the ventilation system can transport them rapidly to various parts of the building. Within each ventilation zone, the agents can be drawn through return ducts and distributed through supply ducts at high flow rates. The indoor dispersion can also be driven by draft through the building as a result of external wind or even by the chimney effect via stairwells or elevator shafts, particularly in winter when temperature differences between inside and outside are large.

The HAVAC model

HAVAC is a computer model, describing the air flow in the building can give insights into the transportation of these agents through the building. Furthermore, once combined with a detection system, the HAVAC model can use the output of the detection system to fine-tune the prediction of how the agent will disperse into the building. That way HAVAC can indicate which rooms in the building are “hotzones” of agent and which rooms are safe. Possible further actions, such as evacuation of people or even decontamination of the specific rooms can be taken, based on the model results.

In the HAVAC model every room in a building as well as every duct in the HVAC system is defined as a zone. The outdoor environment is also defined as a zone. A zone can exchange air with a neighboring zone via flow paths, which represents doors, windows and other leakages. The air flow through these flow paths can be present due to natural causes, such as outdoor wind pressure or as a result of ventilators, which are located in these flow paths (for instance in the HVAC system). As a result, air will flow through the rooms and the ducts (the zones). As a result it is possible to calculate the contaminant concentration in every room.

Capabilities

The most important capabilities of the HAVAC indoor dispersion model are:

  • The model can describe the concentration of agent in every room in the building as a function of time. That way it can identify hotspots of agent.
  • The model can even work with airborne Toxic Industrial Chemicals (TICs) and Chemical or Biological warfare agents.
  • The building structure (floorplan) can easily be implemented inside the model, including rooms, corridors, doors and windows and various floor levels.
  • An HVAC system can also be integrated into this digital building description, with various ventilation rates per room.
  • The outdoor air concentration of agent around the building can be used as input for the model, including the incorporation of the outdoor wind direction and wind speed (important for windows etc.). For ships this capability is especially important while sailing.
  • Agent detector output can be integrated with the model results to generate even better dispersion predictions.
  • The model can be used to indicate which locations (rooms or ventilation ducts) are best to deploy detectors.

]]>
ISOLA article published to Border Security Report https://isola-project.eu/isola-article-published-to-border-security-report/ Mon, 23 May 2022 14:14:33 +0000 https://isola-project.eu/?p=3770

Innovative Solutions to Filling-in the Ships’ Security Capability
Gaps: EU funded project ISOLA

 

A very large percentage of international commerce is using passenger ships as the preferred mean of transportation. Cruise vessels are also more and more used for cruises all around the world. This type of transportation means are more and more targeted by malevolent people or organizations for illegal activities such as Illegal boarding and transportation, unauthorized access to restricted areas and criminal acts on-board, terrorists’ activities, in terms of dangerous material and substances boarding, transportation, misplacement, abuse etc., international crimes like piracy, armed robbery, underwater threats during mooring and more. A number of studies conducted by various maritime organizations and stakeholders reported more than 75% of accidents/incidents of ships worldwide are due to human and organizational errors (International Maritime Organization, 1994). In more recent studies, the majority (75% – 95%, depending on the nature of the incident related to safety, security etc.) of maritime accidents/incidents/non-conformities, resulting in marine casualties, are caused by Human Element’s errors, negligence, or shortfalls (Allianz, 2021). Hence, any attempts to reduce incidents/accidents at sea should concentrate on eliminating human errors on board ships, since this is where the problem is greatest and where the biggest improvements should be made.

In the operational context of security on-board ships, the persons that are called upon to monitor and detect security threats, prevent emergency occurrences and deal with them, are the duly trained and certified vessels’ crews. Crews need to have adequate organizational support to conduct their mission effectively, while their mission capabilities gaps need to be adequately identified and covered. The most common shortfalls (Fig. 1), which cause crews’ mission accomplishment capability gaps and prevent them to act as required, mainly derive from the DOTMLPF (Doctrine, Organization, Training, Materiel, Leadership/Education, Personnel, and Facilities) factor structure analysis (Cellucci, 2008) and are organized in operational, infrastructure/facilities/materiel and personnel shortfalls. Specifically, the Doctrine analysis examines the way the Organization operates by culture and industry’s common practice to see if there is a better way that might solve a capability gap. Sometimes, Shipping Companies happens to suffer a lack of adequate Doctrine and Institutional documentation i.e. Company’s Policies, SOPs, detailed Security Risk Assessments, detailed Emergency Action Plans and relevant Check Lists addressed to ships’ crews. Consequently, ships’ crews are not served with efficient guidance and/or do not have a clearly defined compliance framework with undesired operational effects. Sometimes, security culture and mentality are not thoroughly cultivated throughout the Company and ships’ crews. Hence, the related security provisions, preventive and risk mitigation measures are not always in place causing undesired operational effects. The Organizational analysis examines how the Organization is organized to operate; Company, Ship’s Master, Ship Security Officer, Officer of the Watch, Personnel designated with security duties and other. It examines to see if there is a better organizational structure or capability that can be developed to solve a capability gap. Sometimes, there are organizational issues that prevent operations’ efficiency such as non-clearly defined hierarchy and chain of command, duties and/or authorities overlapping, lack of essential personnel back-up and substitution when required, communication overloading and/or gaps of communication and reporting etc.     The Leadership/Education analysis examines how the Organization prepares its leaders to lead the situation from the Company Security Officer, Master, Ship Security Officer and their overall professional development. Sometimes, people assigned with specific Leadership duties do not have the education, knowledge or experience to conduct their duties effectively. Sometimes, there is no suitable delegation of authority for the sake of smooth operations’ conducting or no delegation of authority at all, causing operations overloading to responsible personnel or confusion to authorities, duties and responsibilities. Additionally, sometimes, there is a lack of resources needed to conduct efficient operations e.g. time, money, people etc. Lack of adequate infrastructure and installations required to facilitate the needs of efficient operations e.g. offices, conference rooms, training facilities and equipment can be considered as a shortfall.  Furthermore, on occasions, there is a lack of adequate operational equipment and systems required to conduct efficient operations, e.g. binoculars, night vision goggles, CCTVs, etc. Finally, there are cases that the existing equipment and systems are not operable and/or not maintained/calibrated e.g. radars, binoculars, night vision goggles, CCTVs, etc. The Personnel analysis examines availability of qualified people for watch keeping, response to incidents and various contingency plans and operations to support a capability gap by restructuring and remedying, while basic needs of personnel such as avoidance of fatigue are taken into consideration. The Training analysis examines how the Organization prepares its personnel to cope with occurrences tactically, from basic training, advanced individual training, various types of team collective training, drills or exercises, and other ways to see if improvement can be made to offset capability gaps. Sometimes, responsible office personnel or crew have not the required qualification standards or adequate training for their position’s duties and responsibilities. In some rare cases there is personnel demonstrating relevant certificates of training without actually having passed through the related training.

Sometimes, the qualified and trained for specific position personnel is not placed at this position, or non-qualified and irrelevantly trained personnel is placed in this position and hence, there is not a skills set match to the position’s needs causing non efficient operations. Under-manning, excessive overtime and/or parallel duties without adequate resting hours, usually cause fatigue. Fatigue is a bad guide for efficient operations and for safety overall. In addition, non-proper manning, under-manning, duties’ duplication, non-skills-set match to the position, usually cause non-efficient operations. Finally, accountability culture results into operations’ efficiency, as responsible persons are always accountable for their actions according to their position’s duties and responsibilities. Lack of accountability culture causes non-efficient operations, non-safety/security acts, near-misses, non-conformities and breaches of security.

Figure 1: The Security Response Shortfalls

However, the cause of the majority of ship-born disasters and major security incidents, which may lead to major safety incidents, is the “Human Factor” and the lack of adequate Situational Awareness. Situational awareness is the ability to always having a good perception of the ship’s surroundings, comprehending what’s happening around and predicting how this will affect the ship, voyage, operations and people on board. It can support in eliminating the mistakes, prevent incidents/accidents, make better and faster decisions, conduct more efficient operations and respond quicker/better to incidents and emergency occurrences.

There is always a need to eliminate mistakes/shortfalls derived from humans and save critical response time in an overwhelming and information-overloaded situation environment and/or during normal routine. All ship managers, according to the legislative/regulative framework, have issued the required policies and all ships have relevant Security Risk Assessments and Ship Security Plans (SSP), including all required emergency procedures guidelines, action plans, and checklists on-board that need to be fully implemented in case of an emergency occurrence. All office personnel and crewmembers must have passed through detailed training, evaluation and certification upon IMO defined Leadership and STCW specific security model courses (i.e. Company Security Officer, Ship Security Officer, Seafarers Designated with Security Duties and Security Awareness courses) respectively. All office personnel and crewmembers must implement on the Operational and the Tactical level all required actions according to the established procedures, their experience and training, in order to prevent and respond to incidents. Moreover, internal and external communication, post-incident reporting and protection of evidences, for incident’s investigation, incident’s route cause analysis, lessons learned and future corrective / preventive actions, are always necessary and mandatory. Operational stress and cross-communication distraction are bad guides during an emergency situation and the likelihood of crews’ partial non-compliance with the policies and emergency procedures’ action plans is getting higher due to that reasons too.

The solution: an innovative ships’ security System and its security domains

European Commission requires the research and development of innovative security solutions, to help out crews to fill-in these security capability gaps. ISOLA project is an EU HORIZON 2020 R&D funded project (Fig.2) (https://isola-project.eu/) aligned on this direction.

Figure 2: ISOLA project

The suggested by the ISOLA project System acts as a Situational Awareness and Emergency Procedures Superintendence Tool, in order to fill the aforementioned capabilities gaps in the ships crew’s mission accomplishment, according to the ISPS Code and all relevant legislative and regulatory requirements. By implementing smart technologies it accommodates the need of the ship’s Master, the Incident’s Manager and Crew, for precise and on-time information, emergency procedure overview and emergency action plans / check-lists fulfillment, while it also relays real-time information to the ship Manager and minimizes cross-communication destruction and the overwhelming operational stress. Additionally, it monitors personnel’s accountability, in terms of receiving and recording the required feedback from the end users upon system’s suggestions acknowledgement, acceptance or rejection, response actions’ implementation, check lists’ fulfillment etc., tracks and keeps incident’s records for reporting, future investigation, root cause analysis, lessons learnt, corrective/preventive action’s needs. Specifically, ISOLA System supports crews in the situational awareness, decision making support / suggestion for action, communication/reporting and protection of evidences domains (Fig.3). The system provides for continuous security surveillance while the ship is in-port, at anchorage or underway, within a State’s territorial waters or in the high seas, during all time, under all weather and environmental conditions, up to an optimum security distance from the ship, having the capability of security threats’ distinction and classification. It is capable to detect breaches of security and potential security threats that satisfy specific criteria of suspicion, to recognize specific acts and acts’ manners. It evaluates, verifies and classifies actual security threats satisfying specific threat escalation criteria, from potential threat to actual threat and notifies security personnel accordingly. It tracks detected threats and continuously updates relevant situational awareness status. It provides suggestions on security personnel’s acts upon potential and/or actual threats, according to Ship Security Plan (SSP), Standard Operational Procedures (SOP) and Security Emergencies Response Plans. It monitors security personnel’s accountability, in terms of receiving and recording the required feedback from the end users upon system’s suggestions’ acknowledgement, acceptance or rejection, suggested response actions’ implementation, check lists’ fulfillment etc. It facilitates real time cross-communication of essential information (to all interested parties) as regards situation escalation and progress updates and supports formal reporting processes’ implementation with suggestions for Reporting (internal: within the ship at the tactical level and external: with the Company and the Authorities at the operational level) according to Policies and SOPs. Finally, the system protects and preserves evidences of the security events. Video footages, audio records, sequence of events data, ship’s data log-book, passengers and crew monitoring data are recorded and kept for future needs of investigation, incidents’ root cause analysis, lessons learnt and preventive actions processes. Ship’s Legacy Systems data available throughout an incident’s timeline are also included.

Figure 3: ISOLA System Support

As the security threats are growing bigger in quantity and severity and more sophisticated, innovative ships security systems, providing for situational awareness, early warning and alarming, decision making support and evidences protection, will be more and more required.

Author: Captain Ioannis Androulakis / AVERSA OOD (ISOLA Project partner) COO

The full report can be found at Border Security Report Mar-Apr 2022 issue (p.14) or download the whole issue here: https://border-security-report.com/back-issues/

]]>