Publications

2024

1. IEEE-CCNC

   Authenticating AI-Generated Social Media Images Using Frequency Domain Analysis

   Nihal Poredi , Deeraj Nagothu, and Yu Chen

   In 2024 IEEE 21st Consumer Communications & Networking Conference (CCNC) , Jan 2024

   ISSN: 2331-9860

   Abs

   Living in the age of social media, it is a daily routine for individuals to post videos, audio, pictures, and text online. In addition, the proliferation of Artificial Intelligence (AI) technology allows customizing multimedia content to meet personal demands. However, the popularity of AI-based text-to-image generators like DeepAI also opens the door to generating images for social media platforms that impersonate unsuspecting users without their permission. While people enjoy high creativity, such “fake” images could enable the propagation of deceptive information that negatively impacts an individual’s personal life and potentially cause public unrest. Therefore, reliable methods to facilitate image authentication are vital to identify and flag them. In this paper, we present AUSOME-2, an upgraded version of our system that AUthenticates SOcial MEdia images (AUSOME) using frequency analysis technologies and machine learning (ML) algorithms. Images from several text-to-image platforms, such as Dall-E 2 and Google Deep Dream, are distinguished from genuine images. Spectral analysis techniques are used to obtain features and fingerprints in the frequency domain. These features enable the ML model to classify AI -generated social media images from genuine ones. The experimental results, on top of a proof-of-concept prototype, showed that the AUSOME-2 system is a promising approach to authenticate images with decent detection accuracy.

2. MDPI

   The Microverse: A Task-Oriented Edge-Scale Metaverse

   Qian Qu , Mohsen Hatami , Ronghua Xu , and 6 more authors

   Future Internet, Feb 2024

   Number: 2 Publisher: Multidisciplinary Digital Publishing Institute

   Abs

   Over the past decade, there has been a remarkable acceleration in the evolution of smart cities and intelligent spaces, driven by breakthroughs in technologies such as the Internet of Things (IoT), edge–fog–cloud computing, and machine learning (ML)/artificial intelligence (AI). As society begins to harness the full potential of these smart environments, the horizon brightens with the promise of an immersive, interconnected 3D world. The forthcoming paradigm shift in how we live, work, and interact owes much to groundbreaking innovations in augmented reality (AR), virtual reality (VR), extended reality (XR), blockchain, and digital twins (DTs). However, realizing the expansive digital vista in our daily lives is challenging. Current limitations include an incomplete integration of pivotal techniques, daunting bandwidth requirements, and the critical need for near-instantaneous data transmission, all impeding the digital VR metaverse from fully manifesting as envisioned by its proponents. This paper seeks to delve deeply into the intricacies of the immersive, interconnected 3D realm, particularly in applications demanding high levels of intelligence. Specifically, this paper introduces the microverse, a task-oriented, edge-scale, pragmatic solution for smart cities. Unlike all-encompassing metaverses, each microverse instance serves a specific task as a manageable digital twin of an individual network slice. Each microverse enables on-site/near-site data processing, information fusion, and real-time decision-making within the edge–fog–cloud computing framework. The microverse concept is verified using smart public safety surveillance (SPSS) for smart communities as a case study, demonstrating its feasibility in practical smart city applications. The aim is to stimulate discussions and inspire fresh ideas in our community, guiding us as we navigate the evolving digital landscape of smart cities to embrace the potential of the metaverse.

2023

1. Springer

   ECOM: Epoch Randomness-Based Consensus Committee Configuration for IoT Blockchains

   Ronghua Xu , Deeraj Nagothu, and Yu Chen

   In Principles and Practice of Blockchains , Feb 2023

   Abs

   The rapid advancement in Artificial Intelligence (AI) based on large-scale Internet of Things (IoT) networks leads to the proliferation of new smart applications that enable Smart Cities. Since the cryptocurrency Bitcoin, blockchain technology has evolved for extensive applications in various financial services and industrial applications. The increase in research interest from academic and industrial perspectives aims to exploit blockchain technology to enable a decentralized, verifiable, and traceable IoT networks. However, directly integrating cryptocurrency-oriented blockchain technologies into IoT systems faces performance and scalability issues. Splitting the whole blockchain network into multiple independent small-scale consensus networks is promising to overcome performance and scalability problems in heterogeneous IoT networks. In this chapter, following an in-depth review of state-of-the-art solutions for scaling blockchain networks, key design challenges and techniques are identified in terms of epoch randomness generation, network traffic model, and consensus committee configuration. Focusing on scalable and secure random committee selection, this chapter introduces an epoch randomness-enabled consensus committee configuration (ECOM) scheme. A proof-of-concept prototype is implemented and evaluated on a physical network that uses Raspberry Pis to simulate IoT devices. The experimental results show that the proposed ECOM protocol efficiently guarantees unpredictable randomness generation and committee selection under a small-scale byzantine network environment.

2. SPIE

   AUSOME: authenticating social media images using frequency analysis

   Nihal Poredi , Deearj Nagothu , and Yu Chen

   In Disruptive Technologies in Information Sciences VII , Jun 2023

   Abs

   Ever since human society entered the age of social media, every user has had a considerable amount of visual content stored online and shared in variant virtual communities. As an efficient information circulation measure, disastrous consequences are possible if the contents of images are tampered with by malicious actors. Specifically, we are witnessing the rapid development of machine learning (ML) based tools like DeepFake apps. They are capable of exploiting images on social media platforms to mimic a potential victim without their knowledge or consent. These content manipulation attacks can lead to the rapid spread of misinformation that may not only mislead friends or family members but also has the potential to cause chaos in public domains. Therefore, robust image authentication is critical to detect and filter off manipulated images. In this paper, we introduce a system that accurately AUthenticates SOcial MEdia images (AUSOME) uploaded to online platforms leveraging spectral analysis and ML. Images from DALL-E 2 are compared with genuine images from the Stanford image dataset. Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT) are used to perform a spectral comparison. Additionally, based on the differences in their frequency response, an ML model is proposed to classify social media images as genuine or AI-generated. Using real-world scenarios, the AUSOME system is evaluated on its detection accuracy. The experimental results are encouraging and they verified the potential of the AUSOME scheme in social media image authentications.

3. SPIE

   DEMA: decentralized electrical network frequency map for social media authentication

   Deeraj Nagothu, Ronghua Xu , and Yu Chen

   In Disruptive Technologies in Information Sciences VII , Jun 2023

   Abs

   The information era has gained a lot of traction due to the abundant digital media contents through technological broadcasting resources. Among the information providers, the social media platform has remained a popular platform for the widespread reach of digital content. Along with accessibility and reach, social media platforms are also a huge venue for spreading misinformation since the data is not curated by trusted authorities. With many malicious participants involved, artificially generated media or strategically altered content could potentially result in affecting the integrity of targeted organizations. Popular content generation tools like DeepFake have allowed perpetrators to create realistic media content by manipulating the targeted subject with a fake identity or actions. Media metadata like time and location-based information are altered to create a false perception of real events. In this work, we propose a Decentralized Electrical Network Frequency (ENF)-based Media Authentication (DEMA) system to verify the metadata information and the digital multimedia integrity. Leveraging the environmental ENF fingerprint captured by digital media recorders, altered media content is detected by exploiting the ENF consistency based on its time and location of recording along with its spatial consistency throughout the captured frames. A decentralized and hierarchical ENF map is created as a reference database for time and location verification. For digital media uploaded to a broadcasting service, the proposed DEMA system correlates the underlying ENF fingerprint with the stored ENF map to authenticate the media metadata. With the media metadata intact, the embedded ENF in the recording is compared with a reference ENF based on the time of recording, and a correlation-based metric is used to evaluate the media authenticity. In case of missing metadata, the frames are divided spatially to compare the ENF consistency throughout the recording.

4. ProQuest

   Lightweight Multimedia Authentication at the Edge Using Environmental Fingerprint

   Deeraj Nagothu

   Jun 2023

   ISBN: 9798380566988

   Abs

   With the fast development of Fifth-/Sixth-Generation (5G/6G) communications and the Internet of Video Things (IoVT), a broad range of mega-scale data applications have emerged. The rapid deployment of IoVT devices in modern smart cities has enabled secure infrastructures with minimal human intervention. Accompanied by the proliferation of multimedia is the increasing number of attacks against the IoVT systems. With increased onboard computational resources and technological advances in machine learning (ML) models, attack vectors and detection techniques have evolved to use ML-based techniques more effectively, resulting in non-equilibrium dynamics. Modern Artificial Intelligence (AI) technology is integrated with many multimedia applications to help enhance its applications. However, the development of General Adversarial Networks (GANs) also led to DeepFake attacks resulting in the misuse of AI. Instead of engaging in an endless AI arms race “fighting fire with fire”, where new Deep Learning (DL) algorithms keep making fake media more realistic and inadvertently causing the spread of misinformation, this dissertation proposes a novel authentication approach leveraging the Electrical Network Frequency (ENF) signals embedded in the multimedia as an environmental fingerprint against visual layer attacks. An ENF operates at a nominal frequency of 60 Hz/50 Hz based on the geographical location and maintains similar fluctuations throughout the power grid. The multimedia recordings can be verified using signal consistency and similarity by leveraging the time-varying nature of the ENF signal collected from both the Audio-Video recordings and simultaneous ground truth ENF. By studying the spectral estimation techniques STFT and MUSIC along with robust frequency enhancement techniques like Weighted Harmonic Combination and Robust Filtering Algorithm (RFA), the optimal ENF estimation workflow for online forgery detection is analyzed and presented. To improve ENF-based authentication performance, frame processing techniques like Selective Superpixel Masking (SSM) to minimize noise from moving subjects and Singular Spectrum Analysis (SSA) to minimize false alarms in detection are introduced. Equipped with reliable and robust algorithms, AI-generated media like DeepFake can be detected leveraging the ENF-based authentication system proposed as DeFakePro. Furthermore, the synchronous random fluctuations of ENF throughout the grid are integrated as part of a distributed consensus mechanism for IoVT devices, where the proposed LEFC system can detect faulty nodes with fake media broadcast in the network. Along with online detection capabilities, the DEMA system exploits the media metadata information to verify the source based on the time and location of recording using the reference ENF database from all interconnects. The experimental results show that the proposed systems designed using ENF as the environmental fingerprint can enable effective detection and authentication measures against visual layer attacks.

5. SPIE

   Authentication of Video Feeds in Smart Edge Surveillance Networks

   Deeraj Nagothu, and Yu Chen

   Jun 2023

   Abs

   Large-scale Internet of Video Things (IoVT) based smart edge surveillance systems (SESS) become essential for smart cities because of the increasing demands on situation awareness (SAW). However, the rapid development in artificial intelligence (AI) technologies enables fake video/audio streams and doctored images to fool smart city security operators. Authenticating visual/audio feeds becomes the frontier defending the safety and security of residents, properties, and critical infrastructures. To date, due to the gap between the compute-intensive video processing technologies and the resource-constraints in the IoVT environments, there is a compelling need for lightweight, real-time, online video feeds authentication technologies. Starting with the background and fundamentals of IoT and video surveillance, this Spotlight Series book provides an overview of visual layer attacks on SESS and well-known video authentication technologies. Then, following an introduction to the Electric Network Frequency (ENF) signals, this book highlights an ENF-based audio/video authentication technology. Along with a detailed discussion on the rationales and principles, the False Frame Injection (FFI) attacks is selected as a case study that illustrates the system implementation and performance evaluation of the ENF-based video feeds authentication technology.

6. IEEE-NAECON

   Decentralized Vehicular Identification and Tracking on Lightweight IoT Edge Nodes

   John Parker , Deeraj Nagothu, and Yu Chen

   In NAECON 2023 - IEEE National Aerospace and Electronics Conference , Aug 2023

   ISSN: 2379-2027

   Abs

   Video Surveillance Systems (VSSs) are among the most investigated and widely adopted systems in smart cities by administrative officials for public safety and by private sector individuals to secure residents, employees, and properties. While many new technologies based on Machine Learning (ML) have enabled systems to be utilized for a diverse set of applications, such as license plate detection and car YMM (Year, Make Model) classification, one of the underlying concerns behind VSSs for both public and private sectors is the cost and resource constraints of the devices themselves. Given that optical character recognition and object detection are already computing and data-intensive activities, both in regards to the training of those ML models, as well as the real-time processing of the data, it is highly desired to create or optimize systems to operate on lightweight Internet of Things (IoT) devices. In this paper, we introduce a lightweight scheme for Decentralized Vehicular Identification (DEVID), which is affordable to inexpensive IoT devices as long as they are capable of managing their inputs and outputs (IO) in a fashion that is not costly in terms of storage or power consumption. While the performance in the experimental study is not satisfactory, the DEVID system possesses the potential to be a cost-friendly lightweight solution for smart cities.

7. IEEE-NAECON

   Application of Electrical Network Frequency as an Entropy Generator in Distributed Systems

   Deeraj Nagothu, Ronghua Xu , Yu Chen , and 2 more authors

   In NAECON 2023 - IEEE National Aerospace and Electronics Conference , Aug 2023

   ISSN: 2379-2027

   Abs

   The development of public information technology (IT) infrastructures using modern edge devices has revolutionized the smart city ecosystem. As the number of devices in the ecosystem exponentially increases, the power grid infrastructure has continued to function as the backbone of smart cities and support the growing requirements for electricity, such as multimedia services. Media authentication technology leveraging the power system frequency, the Electrical Network Frequency (ENF) signal, has gained traction as an environmental fingerprint to counteract visual layer attacks targeted toward multimedia-based edge devices. Due to the unique and similar fluctuation patterns throughout the grid, the ENF signal is reliable in applications like multimedia synchronization, authentication, and source identification. ENF -based systems are inevitably targeted by perpetrators to bypass their detection schemes; however, such attacks are successful only if the ENF fingerprint can be pre-dicted. In this work, we studied the randomization nature of ENF signals and the societal impacts on its fluctuation patterns due to power grid consumption. Leveraging the ENF signal fluctuations, we integrate ENF as a source of randomness for the Random Bit Generator (RBG) system in cryptographic applications. An RBG-based encrypted communication and committee election mechanism is proposed for an enhanced distribution multimedia system. The paper further discusses the potential applications of the ENF -based randomness generator.

2022

1. IEEE-MMSP

   Robustness of Electrical Network Frequency Signals as a Fingerprint for Digital Media Authentication

   Nihal Poredi , Deeraj Nagothu, Yu Chen , and 4 more authors

   In 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) , Sep 2022

   Abs

   Leveraging modern Artificial Intelligence (AI) technology, Deepfake attacks manipulate audio/video streams (AVS) to mimic any targeted person or scenario. Deepfake attacks are highly disturbing, and the misinformation can mislead the public, raising further challenges in policy, technical, social, and legal aspects. Electrical Network Frequency (ENF) signals embedded in AVS data are promising to be utilized as fingerprints to authenticate digital media and timely detect deepfaked audio or video. Meanwhile, the success of ENF-based deepfake detection approaches will be forfeited if attackers can create false ENF fingerprints to fool the detector. In this paper, a thorough experimental study validates the robustness of ENF signals as a fingerprint for digital media authentication. Taking statistical, supervised learning, and deep learning approaches, this work shows that it is infeasible to forecast the future ENF signals based on historical records. While strict theoretical proof is yet to be done, this work experimentally verifies ENF signals as a reliable fingerprint to authenticate digital media.

2. IEEE

   DeFakePro: Decentralized Deepfake Attacks Detection Using ENF Authentication

   Deeraj Nagothu, Ronghua Xu , Yu Chen , and 2 more authors

   IT Professional, Sep 2022

   Abs

   Advancements in generative models, such as deepfake, allow users to imitate a targeted person and manipulate online interactions. It has been recognized that disinformation may cause disturbance in society and ruin the foundation of trust. This article presents DeFakePro, a decentralized consensus mechanism-based deepfake detection technique in online video conferencing tools. Leveraging electrical network frequency (ENF), an environmental fingerprint embedded in digital media recording affords a consensus mechanism design called proof-of-ENF (PoENF) algorithm. The similarity in ENF signal fluctuations is utilized in the PoENF algorithm to authenticate the media broadcasted in conferencing tools. By utilizing the video conferencing setup with malicious participants to broadcast deepfake video recordings to other participants, the DeFakePro system verifies the authenticity of the incoming media in both audio and video channels.

3. MDPI

   Deterring Deepfake Attacks with an Electrical Network Frequency Fingerprints Approach

   Deeraj Nagothu, Ronghua Xu , Yu Chen , and 2 more authors

   Future Internet, May 2022

   Abs

   With the fast development of Fifth-/Sixth-Generation (5G/6G) communications and the Internet of Video Things (IoVT), a broad range of mega-scale data applications emerge (e.g., all-weather all-time video). These network-based applications highly depend on reliable, secure, and real-time audio and/or video streams (AVSs), which consequently become a target for attackers. While modern Artificial Intelligence (AI) technology is integrated with many multimedia applications to help enhance its applications, the development of General Adversarial Networks (GANs) also leads to deepfake attacks that enable manipulation of audio or video streams to mimic any targeted person. Deepfake attacks are highly disturbing and can mislead the public, raising further challenges in policy, technology, social, and legal aspects. Instead of engaging in an endless AI arms race “fighting fire with fire”, where new Deep Learning (DL) algorithms keep making fake AVS more realistic, this paper proposes a novel approach that tackles the challenging problem of detecting deepfaked AVS data leveraging Electrical Network Frequency (ENF) signals embedded in the AVS data as a fingerprint. Under low Signal-to-Noise Ratio (SNR) conditions, Short-Time Fourier Transform (STFT) and Multiple Signal Classification (MUSIC) spectrum estimation techniques are investigated to detect the Instantaneous Frequency (IF) of interest. For reliable authentication, we enhanced the ENF signal embedded through an artificial power source in a noisy environment using the spectral combination technique and a Robust Filtering Algorithm (RFA). The proposed signal estimation workflow was deployed on a continuous audio/video input for resilience against frame manipulation attacks. A Singular Spectrum Analysis (SSA) approach was selected to minimize the false positive rate of signal correlations. Extensive experimental analysis for a reliable ENF edge-based estimation in deepfaked multimedia recordings is provided to facilitate the need for distinguishing artificially altered media content.

4. SPIE

   A Distributed Crawler for IoVT-based Public Safety Surveillance Exploring the Spatio-Temporal Correlation

   Deeraj Nagothu, Daniel Dimock , Adrian Kulesza , and 2 more authors

   In Sensors and Systems for Space Applications XV , Jun 2022

   Abs

   Modern infrastructure development has led to a rise in deployed surveillance cameras to monitor remote locations and widespread infrastructures. In today’s networked surveillance environment, however, human operators are often overwhelmed with the huge amount of visual feeds, which causes poor judgment and delayed response to emergencies. This paper proposes a distributed crawler scheme (DiCrawler) for smart surveillance systems deployed on Internet of Video Things (IoVT). The IoVT camera nodes monitor continuous video input, track the object of interest while preserving privacy, and relay correlative information to targeted nodes for constant monitoring. Each IoVT node monitors the space inside its field of view (FoV) and notifies the neighboring nodes about the objects leaving the FoV and heading in their directions. A smart communication algorithm among IoVT nodes is designed to prevent network bandwidth bottlenecks and preserve computational power. The DiCrawler system can corroborate with human operators and assist with decision-making by raising alarms in case of suspicious behavior. The IoVT network is completely decentralized, using only peer-to-peer (P2P) communication. DiCrawler does not rely on a central server for any computations, preventing a potential bottleneck if hundreds of cameras were connected and constantly uploading data to a server. Each module is also in a compact form factor, making it viable to be mounted on traditional security surveillance cameras. Extensive experimental study on a proof-of-concept prototype validated the effectiveness of the DiCrawler design.

5. IntechOpen

   Evolution of Attacks on Intelligent Surveillance Systems and Effective Detection Techniques

   Deeraj Nagothu, Nihal Poredi , Yu Chen , and 3 more authors

   Aug 2022

   Abs

   Intelligent surveillance systems play an essential role in modern smart cities to enable situational awareness. As part of the critical infrastructure, surveillance systems are often targeted by attackers aiming to compromise the security and safety of smart cities. Manipulating the audio or video channels could create a false perception of captured events and bypass detection. This chapter presents an overview of the attack vectors designed to compromise intelligent surveillance systems and discusses existing detection techniques. With advanced machine learning (ML) models and computing resources, both attack vectors and detection techniques have evolved to use ML-based techniques more effectively, resulting in non-equilibrium dynamics. The current detection techniques vary from training a neural network to detect forgery artifacts to use the intrinsic and extrinsic environmental fingerprints for any manipulations. Therefore, studying the effectiveness of different detection techniques and their reliability against the defined attack vectors is a priority to secure the system and create a plan of action against potential threats.

2021

1. SPIE

   CriPI: An Efficient Critical Pixels Identification Algorithm for Fast One-Pixel Attacks

   Wei Quan , Deeraj Nagothu, Nihal Poredi , and 1 more author

   In Sensors and Systems for Space Applications XIV , Apr 2021

   Abs

   Deep neural networks (DNN) have been studied intensively in recent years, leading to many practical applications. However, there are also concerns about the security problems and vulnerabilities of DNN. Studies on adversarial network development have shown that relatively more minor perturbations can impact the DNN performance and manipulate its outcome. The impacts of adversarial perturbations have led to the development of advanced techniques for generating image-level perturbations. Once embedded in a clean image, these perturbations are not perceptible to human eyes and fool a well-trained deep learning (DL) convolutional neural network (CNN) classifier. This work introduces a new Critical-Pixel Iterative (CriPI) algorithm after a thorough study on critical pixels’ characteristics. The proposed CriPI algorithm can identify the critical pixels and generate one-pixel attack perturbations with a much higher efficiency. Compared to a one-pixel attack benchmark algorithm, the CriPI algorithm significantly reduces the time delay of the attack from seven minutes to one minute with similar success rates.

2. IEEE

   Decentralized Video Input Authentication as an Edge Service for Smart Cities

   Ronghua Xu , Deeraj Nagothu, and Yu Chen

   IEEE Consumer Electronics Magazine, Nov 2021

   Abs

   Situational awareness is essential for a safe and sustainable urban environment. While the wide deployment of Internet of Video Things (IoVT) enables efficient monitoring of the footage of the Smart Cities, it also attracts attackers and abusers. Disinformation injected into the IoVT can mislead the city administrations, policy makers, and emergency responders, and lead to disastrous consequences. It is very challenging to timely authenticate each video stream from pervasively deployed IoVT devices. This article presents a Blockchain Enhanced Video input Authentication (BEVA) scheme as an edge service to fight against the visual layer attacks on smart IoVT systems such as online false video injection and offline video streams tampering attacks. The BEVA scheme leverages the electrical network frequency (ENF) embedded in video recordings as an environmental fingerprint for online false frame detection. Blockchain is integrated to enable a decentralized security networking infrastructure, and it empowers an immutable, traceable, and auditable distributed ledger for ENF-based authentication scheme without relying on a third-party trust authority. This secure-by-design solution efficiently provides trust and secure services in IoVT in modern Smart Cities.

3. MDPI

   EconLedger: A Proof-of-ENF Consensus Based Lightweight Distributed Ledger for IoVT Networks

   Ronghua Xu , Deeraj Nagothu, and Yu Chen

   Future Internet, Oct 2021

   Abs

   The rapid advancement in artificial intelligence (AI) and wide deployment of Internet of Video Things (IoVT) enable situation awareness (SAW). The robustness and security of IoVT systems are essential for a sustainable urban environment. While blockchain technology has shown great potential in enabling trust-free and decentralized security mechanisms, directly embedding cryptocurrency oriented blockchain schemes into resource-constrained Internet of Video Things (IoVT) networks at the edge is not feasible. By leveraging Electrical Network Frequency (ENF) signals extracted from multimedia recordings as region-of-recording proofs, this paper proposes EconLedger, an ENF-based consensus mechanism that enables secure and lightweight distributed ledgers for small-scale IoVT edge networks. The proposed consensus mechanism relies on a novel Proof-of-ENF (PoENF) algorithm where a validator is qualified to generate a new block if and only if a proper ENF-containing multimedia signal proof is produced within the current round. The decentralized database (DDB) is adopted in order to guarantee efficiency and resilience of raw ENF proofs on the off-chain storage. A proof-of-concept prototype is developed and tested in a physical IoVT network environment. The experimental results validated the feasibility of the proposed EconLedger to provide a trust-free and partially decentralized security infrastructure for IoVT edge networks.

4. EAI

   Authenticating Video Feeds Using Electric Network Frequency Estimation at the Edge

   Deeraj Nagothu, Yu Chen , Alexander Aved , and 1 more author

   EAI Endorsed Transactions on Security and Safety, Feb 2021

   Abs

   Large scale Internet of Video Things (IoVT) supports situation awareness for smart cities; however, the rapid development in artificial intelligence (AI) technologies enables fake video/audio streams and doctored images to fool smart city security operators. Authenticating visual/audio feeds becomes essential for safety and security, from which an Electric Network Frequency (ENF) signal collected from the power grid is a prominent authentication mechanism. This paper proposes an ENF-based Video Authentication method using steady Superpixels (EVAS). Video superpixels group the pixels with uniform intensities and textures to eliminate the impacts from the fluctuations in the ENF estimation. An extensive experimental study validated the effectiveness of the EVAS system. Aiming at the environments with interconnected surveillance camera systems at the edge powered by an electricity grid, the proposed EVAS system achieved the design goal of detecting dissimilarities in the image sequences.

5. IEEE

   DeFake: Decentralized ENF-Consensus Based DeepFake Detection in Video Conferencing

   Deeraj Nagothu, Ronghua Xu , Yu Chen , and 2 more authors

   In IEEE 23rd International Workshop on Multimedia Signal Processing , Oct 2021
6. ACM

   Detecting Compromised Edge Smart Cameras Using Lightweight Environmental Fingerprint Consensus

   Deeraj Nagothu, Ronghua Xu , Yu Chen , and 2 more authors

   In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems , Nov 2021

   Abs

   Rapid advances in the Internet of Video Things (IoVT) deployment in modern smart cities has enabled secure infrastructures with minimal human intervention. However, attacks on audio-video inputs affect the reliability of large-scale multimedia surveillance systems as attackers are able to manipulate the perception of live events. For example, Deepfake audio/video attacks and frame duplication attacks can cause significant security breaches. This paper proposes a Lightweight Environmental Fingerprint Consensus based detection of compromised smart cameras in edge surveillance systems (LEFC). LEFC is a partial decentralized authentication mechanism that leverages Electrical Network Frequency (ENF) as an environmental fingerprint and distributed ledger technology (DLT). An ENF signal carries randomly fluctuating spatio-temporal signatures, which enable digital media authentication. With the proposed DLT consensus mechanism named Proof-of-ENF (PoENF) as a backbone, LEFC can estimate and authenticate the media recording and detect byzantine nodes controlled by the perpetrator. The experimental evaluation shows feasibility and effectiveness of proposed LEFC scheme under a distributed byzantine network environment.

2020

1. IntechOpen

   Smart Surveillance for Public Safety Enabled by Edge Computing

   Deeraj Nagothu, Ronghua Xu , Seyed Yahya Nikouei , and 2 more authors

   Edge Computing: Models, technologies and applications, Nov 2020
2. SPIE

   Enabling Continuous Operations for UAVs with an Autonomous Service Network Infrastructure

   Michael Rosenberg , John Henry Burns , Deeraj Nagothu, and 1 more author

   In Sensors and Systems for Space Applications XIII , Apr 2020

   Abs

   One of the major restrictions on the practical applications of unmanned aerial vehicles (UAV) is their incomplete self-sufficiency, which makes continuous operations infeasible without human oversights. The more oversight UAVs require, the less likely they are going to be commercially advantageous when compared to their alternatives. As an autonomous system, how much human interaction is needed to function is one of the best indicators evaluating the limitations and inefficiencies of the UAVs. Popular UAV related research areas, such as path planning and computer vision, have enabled substantial advances in the ability of drones to act on their own. This research is dedicated to in-flight operations, in which there is not much reported effort to tackle the problem from the aspect of the supportive infrastructure. In this paper, an Autonomous Service network infrastructure (AutoServe) is proposed. Aiming at increasing the future autonomy of UAVs, the AutoServe system includes a service-oriented landing platform and a customized communication protocol. This supportive AutoServe infrastructure will autonomize many tasks currently done manually by human operators, such as battery replacement. A proof-of-concept prototype has been built and the simulation experimental study validated the design.

3. MDPI

   BlendSPS: A BLockchain-ENabled Decentralized Smart Public Safety System

   Ronghua Xu , Seyed Yahya Nikouei , Deeraj Nagothu, and 2 more authors

   Smart Cities, Sep 2020

   Abs

   Due to the recent advancements in the Internet of Things (IoT) and Edge-Fog-Cloud Computing technologies, the Smart Public Safety (SPS) system has become a more realistic solution for seamless public safety services that are enabled by integrating machine learning (ML) into heterogeneous edge computing networks. While SPS facilitates convenient exchanges of surveillance data streams among device owners and third-party applications, the existing monolithic service-oriented architecture (SOA) is unable to provide scalable and extensible services in a large-scale heterogeneous network environment. Moreover, traditional security solutions rely on a centralized trusted third-party authority, which not only can be a performance bottleneck or the single point of failure, but it also incurs privacy concerns on improperly use of private information. Inspired by blockchain and microservices technologies, this paper proposed a BLockchain-ENabled Decentralized Smart Public Safety (BlendSPS) system. Leveraging the hybrid blockchain fabric, a microservices based security mechanism is implemented to enable decentralized security architecture, and it supports immutability, auditability, and traceability for secure data sharing and operations among participants of the SPS system. An extensive experimental study verified the feasibility of the proposed BlendSPS that possesses security and privacy proprieties with limited overhead on IoT based edge networks.

2019

1. IEEE-ISC2

   Real-Time Index Authentication for Event-Oriented Surveillance Video Query Using Blockchain

   Seyed Y. Nikouei , Ronghua Xu , Deeraj Nagothu, and 3 more authors

   In 2018 IEEE Int. Smart Cities Conf. ISC2 2018 , Sep 2019

   Abs

   Information from surveillance video is essential for situational awareness (SAW). Nowadays, a prohibitively large amount of surveillance data is being generated continuously by ubiquitously distributed video sensors. It is very challenging to immediately identify the objects of interest or zoom in suspicious actions from thousands of video frames. Making the big data indexable is critical to tackle this problem. It is ideal to generate pattern indexes in a real-Time, on-site manner on the video streaming instead of depending on the batch processing at the cloud centers. The modern edge-fog-cloud computing paradigm allows implementation of time sensitive tasks at the edge of the network. The on-site edge devices collect the information sensed in format of frames and extracts useful features. The near-site fog nodes conduct the contextualization and classification of the features. The remote cloud center is in charge of more data intensive and computing intensive tasks. However, exchanging the index information among devices in different layers raises security concerns where an adversary can capture or tamper with features to mislead the surveillance system. In this paper, a blockchain enabled scheme is proposed to protect the index data through an encrypted secure channel between the edge and fog nodes. It reduces the chance of attacks on the small edge and fog devices. The feasibility of the proposal is validated through intensive experimental analysis.

2. IEEE-WSC

   A Distributed Agent-Based Framework for a Constellation of Drones in a Military Operation

   Alem H. Fitwi , Deeraj Nagothu, Yu Chen , and 1 more author

   In Proc. - Winter Simul. Conf. , Sep 2019

   Abs

   A seamless communication capability is important in military operations. Likewise, enhanced security, increased capacity, and robust communication mechanisms are vital for humanitarian and disaster-response operations. Often, a system of wide-band satellites is employed for real-time exchange of information and over-the-horizon control, but the communications are prone to denial of service (DoS) attacks, and delayed redeployment. Hence, a swarm of drones could be deployed in mission-critical operations in times of urgency for a secured and robust distributed-intercommunication which is essential for survivability and successful completion of missions. In this paper, a distributed-agent-based framework for secure and reliable information exchange between drones in a constellation is proposed. The framework comprises a mechanism for path planning simulation and estimation, a flexible network architecture for improved client-server(C/S) and peer-to-peer (P2P) connectivity, as well as agents for identity authentications and secure communications. The framework has been simulated and verified with results showing promise.

3. MDPI

   Detecting Malicious False Frame Injection Attacks on Surveillance Systems at the Edge Using Electrical Network Frequency Signals

   Deeraj Nagothu, Yu Chen , Erik Blasch , and 2 more authors

   Sensors (Basel)., Sep 2019

   Abs

   Over the past few years, the importance of video surveillance in securing national critical infrastructure has significantly increased, with applications including the detection of failures and anomalies. Accompanied by the proliferation of video is the increasing number of attacks against surveillance systems. Among the attacks, False Frame Injection (FFI) attacks that replay video frames from a previous recording to mask the live feed has the highest impact. While many attempts have been made to detect FFI frames using features from the video feeds, video analysis is computationally too intensive to be deployed on-site for real-time false frame detection. In this paper, we investigated the feasibility of FFI attacks on compromised surveillance systems at the edge and propose an effective technique to detect the injected false video and audio frames by monitoring the surveillance feed using the embedded Electrical Network Frequency (ENF) signals. An ENF operates at a nominal frequency of 60 Hz/50 Hz based on its geographical location and maintains a stable value across the entire power grid interconnection with minor fluctuations. For surveillance system video/audio recordings connected to the power grid, the ENF signals are embedded. The time-varying nature of the ENF component was used as a forensic application for authenticating the surveillance feed. The paper highlights the ENF signal collection from a power grid creating a reference database and ENF extraction from the recordings using conventional short-time Fourier Transform and spectrum detection for robust ENF signal analysis in the presence of noise and interference caused in different harmonics. The experimental results demonstrated the effectiveness of ENF signal detection and/or abnormalities for FFI attacks.

4. IEEE-ISC2

   A Microservice-Enabled Architecture for Smart Surveillance Using Blockchain Technology

   Deeraj Nagothu, Ronghua Xu , Seyed Y. Nikouei , and 1 more author

   In 2018 IEEE Int. Smart Cities Conf. ISC2 2018 , Sep 2019

   Abs

   While the smart surveillance system enhanced by the Internet of Things (IoT) technology becomes an essential part of Smart Cities, it also brings new concerns in security of the data. Compared to the traditional surveillance systems that is built following a monolithic architecture to carry out lower level operations, such as monitoring and recording, the modern surveillance systems are expected to support more scalable and decentralized solutions for advanced video stream analysis at the large volumes of distributed edge devices. In addition, the centralized architecture of the conventional surveillance systems is vulnerable to single point of failure and privacy breach owning to the lack of protection to the surveillance feed. This position paper introduces a novel secure smart surveillance system based on microservices architecture and blockchain technology. Encapsulating the video analysis algorithms as various independent microservices not only isolates the video feed from different sectors, but also improve the system availability and robustness by decentralizing the operations. The blockchain technology securely synchronizes the video analysis databases among microservices across surveillance domains, and provides tamper proof of data in the trustless network environment. Smart contract enabled access authorization strategy prevents any unauthorized user from accessing the microservices and offers a scalable, decentralized and fine-grained access control solution for smart surveillance systems.

5. SPIE

   A Study on Smart Online Frame Forging Attacks against Video Surveillance System

   Deeraj Nagothu, Jacob Schwell , Yu Chen , and 2 more authors

   In Proc. SPIE - Int. Soc. Opt. Eng. , Sep 2019

   Abs

   Video Surveillance Systems (VSS) have become an essential infrastructural element of smart cities by increasing public safety and countering criminal activities. A VSS is normally deployed in a secure network to prevent the access from unauthorized personnel. Compared to traditional systems that continuously record video regardless of the actions in the frame, a smart VSS has the capability of capturing video data upon motion detection or object detection, and then extracts essential information and send to users. This increasing design complexity of the surveillance system, however, also introduces new security vulnerabilities. In this work, a smart, real-time frame duplication attack is investigated. We show the feasibility of forging the video streams in real-time as the camera’s surroundings change. The generated frames are compared constantly and instantly to identify changes in the pixel values that could represent motion detection or changes in light intensities outdoors. An attacker (intruder) can remotely trigger the replay of some previously duplicated video streams manually or automatically, via a special quick response (QR) code or when the face of an intruder appear in the camera field of view. A detection technique is proposed by leveraging the real-time electrical network frequency (ENF) reference database to match with the power grid frequency.

2017

1. LNCS

   iCrawl: A Visual High Interaction Web Crawler

   Deeraj Nagothu, and Andrey Dolgikh

   In Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) , Sep 2017

   Abs

   This paper presents “iCrawl”, a visual high interaction client honeypot system. Web-based cyber-attacks have increased exponentially along with the growth of cloud-based web application technologies. Web browsers provide users with an entry point to these web applications. The iCrawl system is designed to deliver a high interaction honey client that is virtually indistinguishable from a real human-driven client. The system operates by driving an actual web browser in a fashion closely resembling a genuine user’s actions. Unlike most crawlers iCrawl attempts to operate over visual elements on the web page, not code elements. The honeypot system consists of pre-configured decoy virtual machines. Each virtual machine includes spider program, which upon execution automates the process of driving the web browser and crawling the targeted website. It performs browsing by observing the page and simulating human user input through mouse and keyboard activity. The data collected from the crawling is stored in a graph database in the form of nodes and relations. This data captures the context and the changes in system behavior due to interaction with the crawled website. The graph data can be queried and monitored online for structural patterns and anomalies. The iCrawl system is enabling technology for studying sophisticated malicious websites that can avoid detection by the simpler crawlers typically utilized by well-known security companies.

2016

1. ProQuest

   iCrawl: A High Interaction Client Honeypot System

   Deeraj Nagothu

   Dec 2016

   Abs

   This thesis presents “iCrawl”, a high interaction client honeypot system. Web-based cyber-attacks have increased exponentially along with the growth of cloud-based web application technologies. Web browsers provide users with an entry point to these web applications. This makes browsers a valuable target for attackers. The iCrawl system is designed to provide a high interaction honey client that is virtually indistinguishable from a real human driven client. The system operates by driving an actual web browser in a fashion closely resembling a genuine user’s actions. The honeypot system consists of pre-configured decoy virtual machines. Each virtual machine consists of spider programs, which upon execution automate the process of driving the web browser and crawling the targeted website by simulating human user behavior through mouse and keyboard interaction. The data collected from the crawling is stored in a graph database in the form of nodes and relations. This data together in the graph database represents the changes in system behavior due to interaction with the crawled website. The iCrawl system is enabling technology for studying sophisticated malicious websites that are capable of avoiding detection by the simpler crawlers typically utilized by well-known security companies.