Deeraj Nagothu

The rapid advancement in artificial intelligence (AI) and wide deployment of Internet of Video Things (IoVT) enable situation awareness (SAW). The robustness and security of IoVT systems are essential for a sustainable urban environment. While blockchain technology has shown great potential in enabling trust-free and decentralized security mechanisms, directly embedding cryptocurrency oriented blockchain schemes into resource-constrained Internet of Video Things (IoVT) networks at the edge is not feasible. By leveraging Electrical Network Frequency (ENF) signals extracted from multimedia recordings as region-of-recording proofs, this paper proposes EconLedger, an ENF-based consensus mechanism that enables secure and lightweight distributed ledgers for small-scale IoVT edge networks. The proposed consensus mechanism relies on a novel Proof-of-ENF (PoENF) algorithm where a validator is qualified to generate a new block if and only if a proper ENF-containing multimedia signal proof is produced within the current round. The decentralized database (DDB) is adopted in order to guarantee efficiency and resilience of raw ENF proofs on the off-chain storage. A proof-of-concept prototype is developed and tested in a physical IoVT network environment. The experimental results validated the feasibility of the proposed EconLedger to provide a trust-free and partially decentralized security infrastructure for IoVT edge networks.

Large scale Internet of Video Things (IoVT) supports situation awareness for smart cities; however, the rapid development in artificial intelligence (AI) technologies enables fake video/audio streams and doctored images to fool smart city security operators. Authenticating visual/audio feeds becomes essential for safety and security, from which an Electric Network Frequency (ENF) signal collected from the power grid is a prominent authentication mechanism. This paper proposes an ENF-based Video Authentication method using steady Superpixels (EVAS). Video superpixels group the pixels with uniform intensities and textures to eliminate the impacts from the fluctuations in the ENF estimation. An extensive experimental study validated the effectiveness of the EVAS system. Aiming at the environments with interconnected surveillance camera systems at the edge powered by an electricity grid, the proposed EVAS system achieved the design goal of detecting dissimilarities in the image sequences.

Advancements in generative models, such as deepfake, allow users to imitate a targeted person and manipulate online interactions. It has been recognized that disinformation may cause disturbance in society and ruin the foundation of trust. This article presents DeFakePro, a decentralized consensus mechanism-based deepfake detection technique in online video conferencing tools. Leveraging electrical network frequency (ENF), an environmental fingerprint embedded in digital media recording affords a consensus mechanism design called proof-of-ENF (PoENF) algorithm. The similarity in ENF signal fluctuations is utilized in the PoENF algorithm to authenticate the media broadcasted in conferencing tools. By utilizing the video conferencing setup with malicious participants to broadcast deepfake video recordings to other participants, the DeFakePro system verifies the authenticity of the incoming media in both audio and video channels.

With the fast development of Fifth-/Sixth-Generation (5G/6G) communications and the Internet of Video Things (IoVT), a broad range of mega-scale data applications emerge (e.g., all-weather all-time video). These network-based applications highly depend on reliable, secure, and real-time audio and/or video streams (AVSs), which consequently become a target for attackers. While modern Artificial Intelligence (AI) technology is integrated with many multimedia applications to help enhance its applications, the development of General Adversarial Networks (GANs) also leads to deepfake attacks that enable manipulation of audio or video streams to mimic any targeted person. Deepfake attacks are highly disturbing and can mislead the public, raising further challenges in policy, technology, social, and legal aspects. Instead of engaging in an endless AI arms race “fighting fire with fire”, where new Deep Learning (DL) algorithms keep making fake AVS more realistic, this paper proposes a novel approach that tackles the challenging problem of detecting deepfaked AVS data leveraging Electrical Network Frequency (ENF) signals embedded in the AVS data as a fingerprint. Under low Signal-to-Noise Ratio (SNR) conditions, Short-Time Fourier Transform (STFT) and Multiple Signal Classification (MUSIC) spectrum estimation techniques are investigated to detect the Instantaneous Frequency (IF) of interest. For reliable authentication, we enhanced the ENF signal embedded through an artificial power source in a noisy environment using the spectral combination technique and a Robust Filtering Algorithm (RFA). The proposed signal estimation workflow was deployed on a continuous audio/video input for resilience against frame manipulation attacks. A Singular Spectrum Analysis (SSA) approach was selected to minimize the false positive rate of signal correlations. Extensive experimental analysis for a reliable ENF edge-based estimation in deepfaked multimedia recordings is provided to facilitate the need for distinguishing artificially altered media content.

The rapid advancement in Artificial Intelligence (AI) based on large-scale Internet of Things (IoT) networks leads to the proliferation of new smart applications that enable Smart Cities. Since the cryptocurrency Bitcoin, blockchain technology has evolved for extensive applications in various financial services and industrial applications. The increase in research interest from academic and industrial perspectives aims to exploit blockchain technology to enable a decentralized, verifiable, and traceable IoT networks. However, directly integrating cryptocurrency-oriented blockchain technologies into IoT systems faces performance and scalability issues. Splitting the whole blockchain network into multiple independent small-scale consensus networks is promising to overcome performance and scalability problems in heterogeneous IoT networks. In this chapter, following an in-depth review of state-of-the-art solutions for scaling blockchain networks, key design challenges and techniques are identified in terms of epoch randomness generation, network traffic model, and consensus committee configuration. Focusing on scalable and secure random committee selection, this chapter introduces an epoch randomness-enabled consensus committee configuration (ECOM) scheme. A proof-of-concept prototype is implemented and evaluated on a physical network that uses Raspberry Pis to simulate IoT devices. The experimental results show that the proposed ECOM protocol efficiently guarantees unpredictable randomness generation and committee selection under a small-scale byzantine network environment.

Large-scale Internet of Video Things (IoVT) based smart edge surveillance systems (SESS) become essential for smart cities because of the increasing demands on situation awareness (SAW). However, the rapid development in artificial intelligence (AI) technologies enables fake video/audio streams and doctored images to fool smart city security operators. Authenticating visual/audio feeds becomes the frontier defending the safety and security of residents, properties, and critical infrastructures. To date, due to the gap between the compute-intensive video processing technologies and the resource-constraints in the IoVT environments, there is a compelling need for lightweight, real-time, online video feeds authentication technologies. Starting with the background and fundamentals of IoT and video surveillance, this Spotlight Series book provides an overview of visual layer attacks on SESS and well-known video authentication technologies. Then, following an introduction to the Electric Network Frequency (ENF) signals, this book highlights an ENF-based audio/video authentication technology. Along with a detailed discussion on the rationales and principles, the False Frame Injection (FFI) attacks is selected as a case study that illustrates the system implementation and performance evaluation of the ENF-based video feeds authentication technology.