From Video Surveillance to an AI Safety Ecosystem: How to Turn HSE Data into Incident Prevention

The implementation of digital tools in occupational safety is often limited to installing video cameras and basic incident logging. However, the true potential of data is unlocked when moving from simple violation recording to a proactive ecosystem. In his presentation, Alexander Bondarenko, Head of HSE at SSGPO (part of ERG), details the experience of transforming fragmented digital solutions into a unified AI ecosystem that enabled the enterprise to achieve historical peaks in fatality-free operations.

The Evolution of Control: From Drones to Educational Content

Historically, the enterprise averaged five fatal accidents per year. The turning point came with the implementation of adapted global best practices and large-scale digitalization. One of the first steps, implemented back in 2016, was the use of drones to monitor high-risk work. In practice, this tool proved its critical value: in 2021, a drone pilot spotted two workers resting in the shade under a bench where mining operations were underway. The sound of the propellers prompted them to leave the danger zone just 12 seconds before a rockfall occurred.

Today, technical monitoring encompasses over 1,800 stationary cameras and around 500 body-worn video recorders. Rather than sitting as dead weight on servers, this vast stream of video data is converted into educational material. Every month, the HSE department edits short video clips highlighting both errors and examples of safe task execution. This content is broadcast on 200 TV screens in shift assignment rooms before every shift, creating a continuous micro-learning process.

Artificial Intelligence in Behavior Management

The speaker pays special attention to evaluating communication between supervisors and workers. The enterprise has deployed an automated system to analyze safety toolbox talks and behavioral safety dialogues. Foremen record their briefings on voice recorders, after which algorithms evaluate the quality of the conversation according to an approved methodology.

Thanks to automation tools, the supervisor receives structured feedback within just 45 seconds: the system highlights the strengths of the dialogue and areas for improvement. The data is synchronized with dashboards to provide a unified overview across departments.

To prepare junior foremen for conducting safety audits, a specialized electronic coaching simulator was developed. In this system, artificial intelligence role-plays various worker profiles—from cooperative to resistant or fatalistic. The foreman's goal is to structure the dialogue to guide the virtual employee to risk awareness.

Comprehensive Assessment of Driver Fatigue

Personnel fatigue remains a hidden hazard in production that cannot be detected by standard breathalyzers. To address this issue, the team developed a hardware-software module (PAM 60).

The system evaluates human reaction time using a gaming steering wheel and pedals while delivering signals for one minute. Simultaneously, a 4K camera equipped with an LED ring light performs pupillometry—measuring the speed of pupil constriction and recovery under light exposure, as well as detecting eye redness. A key distinction of the system is the elimination of average standards. The module accumulates statistics for each employee and compares current indicators against their personal baseline. If the worker's reaction and pupil parameters deviate from normal, they are not penalized but reassigned to auxiliary duties in the garage, eliminating the risk of traffic accidents on site.

Local AI Tools and Data Protection

Deploying generative models often involves risks of confidential data leakage. To avoid this, the enterprise relied on autonomous, browser-based offline applications and local language models. This allows complex analytics to be automated securely:

• Incident Investigation: The system analyzes audio interviews and case materials, automatically constructs cause trees, builds bow-tie diagrams, and generates presentations.
• Knowledge Verification: A multi-agent system conducts oral examinations on regulatory requirements, asking questions vocally and evaluating the substance of employee responses as a percentage score.
• Standard Operating Procedure (SOP) Development: Based on completed process cards, agents generate full operating procedures in strict compliance with methodological guidelines.

What You Will Learn from This Presentation:

• How to build a pipeline for creating educational video content based on footage from body-worn cameras.
• How to automate the quality assessment of behavioral safety dialogues using neural networks.
• How to use pupillometry and reaction tests for personalized driver fatigue monitoring.
• How to apply AI simulators to train supervisors in complex communications with personnel.
• How to deploy autonomous AI tools for incident investigation without data leakage risks.