NCTA 2023 Abstracts

Full Papers Short Papers
Full Papers
Paper Nr: 27
Title:

MA-VAE: Multi-Head Attention-Based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-Series Applied to Automotive Endurance Powertrain Testing
Authors:

Lucas Correia, Jan-Christoph Goos, Philipp Klein, Thomas Bäck and Anna V. Kononova
Abstract: A clear need for automatic anomaly detection applied to automotive testing has emerged as more and more attention is paid to the data recorded and manual evaluation by humans reaches its capacity. Such real-world data is massive, diverse, multivariate and temporal in nature, therefore requiring modelling of the testee behaviour. We propose a variational autoencoder with multi-head attention (MA-VAE), which, when trained on unlabelled data, not only provides very few false positives but also manages to detect the majority of the anomalies presented. In addition to that, the approach offers a novel way to avoid the bypass phenomenon, an undesirable behaviour investigated in literature. Lastly, the approach also introduces a new method to remap individual windows to a continuous time series. The results are presented in the context of a real-world in-dustrial data set and several experiments are undertaken to further investigate certain aspects of the proposed model. When configured properly, it is 9% of the time wrong when an anomaly is flagged and discovers 67% of the anomalies present. Also, MA-VAE has the potential to perform well with only a fraction of the training and validation subset, however, to extract it, a more sophisticated threshold estimation method is required.
Download
Paper Nr: 50
Title:

Exploring Segnet Architectures for iGPU Embedded Devices
Authors:

Jean-Baptiste Chaudron and Alfonso Mascarenas-Gonzalez
Abstract: Image segmentation is an important topic in computer vision which encompasses a variety of techniques to divide image into multiple areas or sub-regions in order to extract meaningful information. Artificial Neural Networks (ANNs), biologically inspired algorithms, are nowadays widely used to perform such tasks and popular models are usually based on encoder-decoder architectures. Segnet was one of the first proposed model of this kind in the literature and, despite its efficiency, it has several drawbacks for embedded systems especially due to the huge amount of arithmetic operations and memory used in the original version. However, its simple sequential based architecture offers interesting properties for optimization and real-time analysis. In this paper, we deeply investigate how to tune and adapt original Segnet architecture to allow efficient run-time execution on embedded targets equipped with an iGPU. We propose our own implementation design which is experimented and validated on iGPU embedded devices for two state of the art datasets from Unmanned Aerial Vehicle (UAV) applications.
Download
Paper Nr: 51
Title:

Neural Network-Based Approach for Supervised Nonlinear Feature Selection
Authors:

Mamadou Kanouté, Edith Grall-Maës and Pierre Beauseroy
Abstract: In machine learning, the complexity of training a model increases with the size of the considered feature space. To overcome this issue, feature or variable selection methods can be used for selecting a subset of relevant variables. In this paper we start from an approach initially proposed for classification problems based on a neural network with one hidden layer in which a regularization term is incorporated for variable selection and then show its effectiveness for regression problems. As a contribution, we propose an extension of this approach in the multi-output regression framework. Experiments on synthetic data and real data show the effectiveness of this approach in the supervised framework and compared to some methods of the literature.
Download
Paper Nr: 84
Title:

Unsupervised Representation Learning by Quasiconformal Extension
Authors:

Hirokazu Shimauchi
Abstract: In this paper, we introduce a novel unsupervised representation learning method based on quasiconformal extension. It is essential to develop feature representations that significantly improve predictive performance, regardless of whether the approach is implicit or explicit. Quasiconformal extension extends a mapping to a higher dimension with a certain regularity. The method introduced in this study constructs a piecewise linear mapping of real line by leveraging the correspondence between the distribution of individual features and a uniform distribution. Subsequently, a higher-order feature representation is generated through quasiconformal extension, aiming to achieve effective representations. In experiments conducted across ten distinct datasets, our approach enhanced the performance of neural networks, extremely randomized trees, and support vector machines, when the features contained a sufficient level of information necessary for classification.
Download
Paper Nr: 86
Title:

Molecule Builder: Environment for Testing Reinforcement Learning Agents
Authors:

Petr Hyner, Jan Hůla and Mikoláš Janota
Abstract: We present a reinforcement learning environment designed to test agents’ ability to solve problems that can be naturally decomposed using subgoals. This environment is built on top of the PyVGDL game engine and enables to generate problem instances by specifying the dependency structure of subgoals. Its purpose is to enable faster development of Reinforcement Learning algorithms that solve problems by proposing subgoals and then reaching these subgoals.
Download
Paper Nr: 87
Title:

A Comparison Between Seasonal and Non-Seasonal Forecasting Techniques for Energy Demand Time Series in Smart Grids
Authors:

Sabereh T. Rastkar, Danial Zendehdel, Enrico De Santis and Antonello Rizzi
Abstract: Accurate energy consumption forecasting is essential for optimizing resource allocation and ensuring a reliable energy supply. This paper conducts a thorough analysis of energy consumption forecasting using XGBoost, SARIMA, LSTM, and Seasonal-LSTM algorithms. It utilizes two years of hourly electricity demand data from Italy and the PJM region (USA), categorizing algorithms into seasonality and non-seasonality groups. Performance metrics like RMSE, MAE, R 2 , and MSPE are employed. The study underscores the importance of considering seasonality, with SARIMA and Seasonal-LSTM achieving high accuracy in the seasonality group. In the non-seasonality group, XGBoost and LSTM perform competitively. In summary, this research aids in choosing suitable forecasting algorithms for building an Energy Management System for smart energy management in microgrids, considering seasonality and data attributes. These insights can also benefit energy companies in efficient resource management, promoting sustainable energy practices and urban development.
Download
Short Papers
Paper Nr: 28
Title:

CoreSelect: A New Approach to Select Landmarks for Dissimilarity Space Embedding
Authors:

Sylvain Chabanet, Philippe Thomas and Hind B. El-Haouzi
Abstract: This paper studies an application of indefinite proximity learning to the prediction of baskets of products of logs in the sawmill industry. More precisely, it focuses on the usage of the dissimilarity space embedding framework to generate a set of features representing wood logs. According to this framework, data points are represented by a vector of dissimilarity measures toward a set of representative data points named landmarks. This representation can then be used to train any of the large variety of available ML models requiring structured features. However, this framework raises the problem of selecting these landmarks. A new method is proposed to select these landmarks which is compared with four other methods from the literature. Numerical experiments are run to compare these methods on a dataset from the Canadian sawmill industry. The data representations obtained are used to train random forests and neural networks ensemble models. Results demonstrate that both the Partition Around Medoids (PAM) method and the newly proposed CoreSelect methods lead to a small but significant reduction in the mean square error of the predictions.
Download
Paper Nr: 34
Title:

MASD: Malicious Web Session Detection Using ML-Based Classifier
Authors:

Dilek Yılmazer Demirel and Mehmet Tahir Sandıkkaya
Abstract: The development of web applications and services has resulted in an increase in security concerns, especially in identifying malicious web session attacks. Malicious web sessions pose a significant risk to users, potentially resulting in data breaches, illegal access, and other malicious activities. This study presents an innovative technique for detecting malicious web sessions using a machine learning-driven classifier. To examine the features of web sessions, the suggested technique combines an embedding layer and machine learning approaches. Three different datasets were used in the empirical studies to confirm the effectiveness of the approach. They include a unique compilation of Internet banking web request logs, provided by Yap Kredi Teknoloji, as well as the well-known HTTP dataset CSIC 2010 and the publicly accessible WAF dataset. The experimental results are compared to known approaches such as Random Forest, Convolutional Neural Networks (CNN), Support Vector Machines (SVM), Naı̈ve Bayes, Decision Trees, DBSCAN, and Self-Organizing Maps (SOM). The actual findings demonstrate the superiority of the suggested technique, especially when Random Forest is used as the chosen classifier. The attained accuracy rate of 99.17% surpasses the comparison methodologies, highlighting the approach’s ability to efficiently identify and block malicious web sessions.
Download
Paper Nr: 37
Title:

Robust Drone Detection and Classification from Radio Frequency Signals Using Convolutional Neural Networks
Authors:

Stefan Glüge, Matthias Nyfeler, Nicola Ramagnano, Claus Horn and Christof Schüpbach
Abstract: As the number of unmanned aerial vehicles (UAVs) in the sky increases, safety issues have become more pressing. In this paper, we compare the performance of convolutional neural networks (CNNs) using first, 1D in-phase and quadrature (IQ) data and second, 2D spectrogram data for detection and classification of UAVs based on their radio frequency (RF) signals. We focus on the robustness of the models to low signal-to-noise ratios (SNRs), as this is the most relevant aspect for a real-world application. Within an input type, either IQ or spectrogram, we found no significant difference in performance between models, even as model complexity increased. In addition, we found an advantage in favor of the 2D spectrogram representation of the data. While there is basically no performance difference at SNRs ≥ 0 dB, we observed a 100% improvement in balanced accuracy at -12 dB, i.e. 0:842 on the spectrogram data compared to 0:413 on the IQ data for the VGG11 model. Together with an easy-to-use benchmark dataset, our findings can be used to develop better models for robust UAV detection systems.
Download
Paper Nr: 44
Title:

Using Abstraction Graphs to Promote Exploration in Curiosity-Inspired Intrinsic Motivation
Authors:

Mahtab Mohtasham Khani, Kathryn Kasmarik, Shadi Abpeikar and Michael Barlow
Abstract: This paper proposes a new approach to modelling IM using abstraction graphs to create a novel curiosity model. An abstraction graph can model the topology of a physical environment in a manner that is not captured by typical curiosity models that use self-organising or adaptive resonance theory networks. We hypothesise that this can permit more systematic exploration and skill development in a motivated reinforcement learning setting. To evaluate the proposed model, we have compared our agent’s behaviour with an existing curiosity model in the same environment and the same configuration. For this evaluation, we examined each agent’s behaviour at different life stages and characterised the exploration and attention focus of each agent in each life stage. We concluded the best uses of each approach.
Download
Paper Nr: 52
Title:

Testing Variants of LSTM Networks for a Production Forecasting Problem
Authors:

Nouf Alkaabi, Sid Shakya and Rabeb Mizouni
Abstract: Forecasting the production of essential items such as food is one of the issues that many retail authorities encounter frequently. A well-planned supply chain will prevent an under- and an oversupply. By forecasting behaviors and trends using historical data and other accessible parameters, AI-driven demand forecasting techniques can address this problem. Earlier work has focused on the traditional Machine Learning (ML) models, such as Auto-Regression (AR), Auto-regressive Integrated Moving Average (ARIMA), and Long Short-Term Memory (LSTM) for forecasting production. A thorough experimental analysis demonstrates that various models can perform better in various datasets. However, with additional hyper-parameters that may be further tweaked to increase accuracy, the LSTM technique is typically the most adaptable. In this work, we explore the possibility of incorporating additional non-sequential features with the view of increasing the accuracy of the forecast. For this, the month of production, temperature, and the number of rainy days are considered as additional static non-sequential features. There are various ways such static features can be incorporated in a sequential model such as LSTM. In this work, two variants are built, and their performances for the problem of food production forecasting are compared.
Download
Paper Nr: 75
Title:

Development of Kendo Motion Prediction System for VR Kendo Training System
Authors:

Yuki Saigo, Sho Yokota, Akihiro Matsumoto, Daisuke Chugo, Satoshi Muramatsu and Hiroshi Hashimoto
Abstract: In this study, we developed and evaluated a system within the system to predict the user’s Kendo (Japanese fencing) motions which is the function of the VR Kendo system that enables easy Kendo training at home or in similar settings. We utilized markerless motion capture and machine learning based on recurrent neural networks (RNN) to learn and predict kendo motions. As a result, the proposed system successfully predicted Kendo motions as it started with high accuracy.
Download
Paper Nr: 76
Title:

Multiple Additive Neural Networks: A Novel Approach to Continuous Learning in Regression and Classification
Authors:

Janis Mohr, Basile Tousside, Marco Schmidt and Jörg Frochte
Abstract: Gradient Boosting is one of the leading techniques for the regression and classification of structured data. Recent adaptations and implementations use decision trees as base learners. In this work, a new method based on the original approach of Gradient Boosting was adapted to nearly shallow neural networks as base learners. The proposed method supports a new architecture-based approach for continuous learning and utilises strong heuristics against overfitting. Therefore, the method that we call Multiple Additive Neural Networks (MANN) is robust and achieves high accuracy. As shown by our experiments, MANN obtains more accurate predictions on well-known datasets than Extreme Gradient Boosting (XGB), while also being less prone to overfitting and less dependent on the selection of the hyperparameters learn rate and iterations.
Download
Paper Nr: 21
Title:

Application of the Flocking Method for Spatial Analysis of Brain Activity in Optogenetics Datasets
Authors:

Margarita Zaleshina and Alexander Zaleshin
Abstract: This work introduces a new approach for spatial analysis of assumed dynamics of neuronal activity in mouse brain images obtained by light-sheet fluorescence microscopy methods (LSM). In calculations we used flocking algorithms based on neuronal activity distributions from slice to slice with a time delay that occurs during scanning. We applied GDAL Tools and LF Tools in QGIS for topological processing of multi-page TIFF files with LSM datasets. As a result, we identified localizations of sites with small movements of group neuronal activity passing in the same locations (with retaining localization) from slice to slice. An important advantage of this result is the ability to reveal locations with pronounced neuronal activity in a sequence of several adjacent slices, as well as to identify set of sites with interslice activity.
Download
Paper Nr: 49
Title:

Feature Extraction Methods for Neural Networks in the Classification of Structural Health Anomalies
Authors:

Natasha Hamilton, Jim Harkin, Liam McDaid, Junxiu Liu and Eoghan Furey
Abstract: Failure of large complex structures such as buildings and bridges can have monumental repercussions such as human mortality, environmental destruction and economic consequences. It is therefore paramount that detection of structural damage or anomalies are identified and managed early. This highlights the need to develop automated Structural Health Monitoring (SHM) systems that can continuously allow the safety status of structures to be determined, even in the worst and most isolated conditions, to ultimately help prevent destruction and save lives. Signal processing is a crucial step to detecting structural anomalies and recent work demonstrates the opportunities for neural networks, however the encoding of data for SHM requires the extraction of features due to often, noisy data. This paper focuses on feature extraction methods for artificial neural networks (ANNs) and spiking neural networks (SNNs) and aims to identify bespoke features which enable SNNs to encode data and perform the classification of anomalies. Results show that extraction of particular features in large real-world applications improve the classification accuracy of SNNs.
Download
Paper Nr: 78
Title:

Automatic Emoticons Insertion System Based on Acoustic Information of User Voice: 1st Report on Data Model for Emotion Estimation Using Machine Learning
Authors:

Ryo Senuma, Sho Yokota, Akihiro Matsumoto, Daisuke Chugo, Satoshi Muramatsu and Hiroshi Hashimoto
Abstract: In social media, text information has a problem that it is difficult to convey the nuances and emotions to the other people. Moreover, manual texting is a time consuming task. Therefore, this research proposes a system that creates text information by voice input from the acoustic information, and automatically insert emoticons matching the user’s emotion. The proposed system is employed based on the eight basic emotions of Plutchik’s Wheel of Emotions. Two types of data: natural utterances voice and acting voice was applied to the SVM (Support Vector Machine) method in the experiment to estimate emotions. The result shows that the accuracy of natural utterances voice and acting voice are 30% and 70%, respectively.
Download
Paper Nr: 82
Title:

The Opaque Nature of Intelligence and the Pursuit of Explainable AI
Authors:

Sarah L. Thomson, Niki van Stein, Daan van den Berg and Cees van Leeuwen
Abstract: When artificial intelligence is used for making decisions, people are more likely to accept those decisions if they can be made intelligible to the public. This understanding has led to the emerging field of explainable artificial intelligence. We review how research on explainable artificial intelligence is being conducted and discuss the limitations of current approaches. In addition to technical limitations, there is the huge problem that human decision-making is not entirely transparent either. We conclude with our position that the opacity of intelligent decision-making may be intrinsic, and with the larger question of whether we really need explanations for trusting inherently complex and large intelligent systems — artificial or otherwise.
Download