CSIS 690 LU Computer Science Research Methods Discussion Responses
Post 1:
Veto Bishop
Research Design
A very fundamental tool in scientific research is methodology (Apuke, 2017). Researchers have the following three choices when it comes to designing their methodology: quantitative, qualitative, or hybrid. But the choice that is made is inextricably linked to the research design. The research design is the blueprint from the entire study. The design is the structure, strategy, and plan on how the research question will be answered (Akhtar, 2016). To answer my research question, I will be employing the quantitative approach. This approach has been chosen because the model will be evaluated against mathematical metrics. It is a possibility that the output could be evaluated using humans, but time does not permit such an approach. So, I could have opted for a hybrid approach. However, for the sake of the time allotted the quantitative approach will be employed. The remaining section will explain each step of my research plan. It will start by discussing the dataset that will be used.
Dataset
To conduct the experiments in picture caption creation, a variety of datasets are available. I will use the Flickr30k Dataset for English Caption Generation. For Spanish, MS-COCO-ES will be used which is created from the original MS-COCO dataset. This subset is composed of 20,000 captions of 4,000 images (5 captions per image) in machine-translated Spanish and original English that have been used along with the main subset to carry out. The size of the photos, the number of captions per image, the structure of the captions, and the number of images all vary amongst the designed datasets.
Methodology
The image captions (word sequences) are encoded using a recurrent neural network in the proposed model, while the image characteristics are encoded using a convolutional neural network. A text feature decoder then receives the encoded image and guesses the caption word by word. The model used attention to concentrate on the most crucial element of the image as it came up with each phrase for the caption. Figure 1 depicts the architecture of the suggested model, which principally uses the encoder-decoder mechanism.
Figure 1
Model Overview
Encoder: Image Features
In the first stage, I pre-process the raw images using transfer learning and a CNN-based network that has already been trained. This procedure takes input photos and generates encoded image vectors that capture the key elements of the input image. I utilized EfficientNetB3 (Tan & Le, 2019) and the pre-trained VGG16 (Simonyan & Zisserman, 2014) to extract picture features. ImageNet dataset will be used for training. In the past, neural networks with lots of layers have been effective at recognizing patterns. In addition, these models are difficult to optimize and suffer from overfitting. Residual CNNs are made up of numerous layers connected to one another. These relationships govern how identity mapping will be done. Compared to EfficientNetB3, VGG16 is simpler and contains 16 layers. Simple to optimize, EfficientNetB3 performs better as network depth grows. We used only the encoded image characteristics generated by the hidden layers and deleted the final output layer of the pre-trained models, which provided an output for image classification.
Justification for EfficientNetB3
ElMOUFIDI & Amoun (2021), in their researched showed that the use of EfficientNetB3 pre-trained network and used pre-processing improved network performance. EfficientNetB3 is a current, strong, and lucrative approach proposed by scaling three variables like resolution, depth, and width (Parthiban & Kamarasan 2022; Khan et al., 2021).
Justification for VGG16
NK. Simonyan and A. Zisserman in their paper reported that VGG16 model achieved a test data accuracy of 92.77%, with ImageNet, for dataset that a total of 14 million images, with a total number of classes of 1000 (Mascarenhas & Agarwal, 2021). VGG 16 is core base of CNN, and eliminates the high computational cost associated with CNN (Deepa & Chokkalingam, 2022).
Encoder: Word Features
With Tensorflow, I tokenize the sentences and retrieve the top 25,000 words tokens. The tokens are then transmitted via an RNN based on Gated Recurrent Units (GRU) and an Embedding layer with an embedding size of 256. GRU was created by Cho et al. (2014) and has been effectively applied to sequence creation and machine translation. In a conventional RNN, the update gate and reset gate are employed to address the vanishing gradient issue. The behavior of the cell is controlled by these two gates. The GRU model’s central memory cell houses data about each time step. The vectors responsible for choosing which data should be transmitted to the output are the update and reset gates. These two gates have been taught to keep data from earlier time steps without wasting it and to throw away data that isn’t relevant to the forecast. For the same dataset, GRU trains model 29.29 percent more quickly than LSTM. In terms of performance, GRU outperforms LSTM when dealing with long texts and small datasets but falls short in other situations.
Attention
According to the research article "Neural Machine Translation by Jointly Learning to Align and Translate," I used Bahnadau Attention ( Bahdanau et al., 2014) in our experiments. While EfficientNetB3’s attention features shape is 100, VGG16’s is 49. The final hidden layer of the pre-trained model provides a context vector, which is provided to attention and used as an input by GRU to produce an image description. This design performs better than conventional CNN and RNN architectures that use Long Short-Term Memory (LSTM) as a decoder.
The encoded picture and concealed state from the previous timestep are delivered by the Decoder to the Attention module at each keyframe. It produces an Attention Score that assigns a weight to each pixel in the image and instructs the model where to direct its attention to produce the associated word. For instance, if the intended result is A boy is kicking the ball, the boy’s pixels in the image are highlighted when the word "boy" is generated, while the pixels of the ball are highlighted when the word "ball" is generated.
Decoder: Description Generation
The Description generator is a simple decoder with a Dense layer and ReLU activation (You et al., 2016). The output of the picture feature encoder is sent into the dense layer, which also considers attention weights. Each word in the vocabulary is given a softmax prediction by the dense layer, which then selects the word with the highest likelihood of appearing next in the sequence. I will feed these encoded picture attributes into our Image Caption model instead of the raw images. Additionally passed in are the desired captions for each encoded image. By decoding visual information, the model tries to predict captions that are complementary to the goal captions. To predict the following word during the training phase, we employ the Teacher forcing method. This method passes the target word as the decoder’s subsequent input. Up until a final token is generated, this process is repeated.
Comparing Methodology with previous work
You et al. (2016) created a frame that obtained both top-down and bottom-up features of an uploaded image. You et al. (2016) used an input attention model for image captioning. The framework assigned a score to each detected attribute ( You et al., 2016). In their model the visual features were also fed into an RNN for caption generation. For their dataset, the Flickr30k and MS-COCO were also used to evaluate the performance of the model. You et al. (2016) used the following evaluation metrics METEOR, ROUGE-L and CIDEr.
Wang et al. (2016) in their work on developing an image caption model used Flickr8K, Flickr30K and MSCOCO datasets. Their model was evaluated using BLEU, METERTOR, and CIDEr .The reference mode was pre-trained using VGG16.
Evaluation Metric Justification
The results of the model will be evaluated using automatic evaluation metrics BLEU and ROUGE. BLEU (Papineni et al., 2002) evaluates by calculating the precision of n-grams in a translated sentence as compared to a reference sentence. Although BLEU is not appropriate for code summarization (Roy et al., 2021) it is applicable for this project as our caption will be only one sentence long. BLEU is popular and easy to use for projects with shore duration (Sai et al., 2022).
ROUGE (Lin, 2004) calculates the count of several overlapping units such as n-grams, word pairs, and sequences. ROUGE is like BLEU in that it counts n-grams, but it is precision based ((Sai et al., 2022).
Table 1
Use case description – Generate Caption for image
Figure 2
Activity Diagram
Figure 3
Class Diagram
Reference
Akhtar, D. M. I. (2016). Research design. Research Design (February 1, 2016).
Apuke, O. D. (2017). Quantitative research methods: A synopsis approach. Kuwait Chapter of Arabian Journal of Business and Management Review, 33(5471), 1-8.
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Deepa, N., & Chokkalingam, S. P. (2022). Optimization of VGG16 utilizing the Arithmetic Optimization Algorithm for early detection of Alzheimer’s disease. Biomedical Signal Processing and Control, 74, 103455.
ElMOUFIDI, A., & Amoun, H. (2021). EfficientNetB3 Architecture for Diabetic Retinopathy Assessment using Fundus Images.
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360.
Khan, I. U., Aslam, N., Anwar, T., Aljameel, S. S., Ullah, M., Khan, R., ... & Akhtar, N. (2021). Remote Diagnosis and Triaging Model for Skin Cancer Using EfficientNet and Extreme Gradient Boosting. Complexity, 2021.
Lin, C. Y. (2004, July). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out (pp. 74-81).
Mascarenhas, S., & Agarwal, M. (2021, November). A comparison between VGG16, VGG19 and ResNet50 architecture frameworks for Image Classification. In 2021 International Conference on Disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON) (Vol. 1, pp. 96-99). IEEE.
Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002, July). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics (pp. 311-318).
Parthiban, K., & Kamarasan, M. (2022, January). EfficientNet with Optimal Wavelet Neural Network for DR Detection and Grading. In 2022 4th International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 1081-1086). IEEE.
Roy, D., Fakhoury, S., & Arnaoudova, V. (2021, August). Reassessing automatic evaluation metrics for code summarization tasks. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 1105-1116).
Sai, A. B., Mohankumar, A. K., & Khapra, M. M. (2022). A survey of evaluation metrics used for NLG systems. ACM Computing Surveys (CSUR), 55(2), 1-39.
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
Tan, M., & Le, Q. (2019, May). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR.
Wang, C., Yang, H., Bartz, C., & Meinel, C. (2016, October). Image captioning with deep bidirectional LSTMs. In Proceedings of the 24th ACM international conference on Multimedia (pp. 988-997).
You, Q., Jin, H., Wang, Z., Fang, C., & Luo, J. (2016). Image captioning with semantic attention. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4651-4659).
Post 2:
Landon Smalley
Nov 17 at 9:58pm
Manage Discussion Entry
Aircrew scheduling is crucial not only to civil aviation but also to military pilot training. With a shortage of Remotely Piloted Aircraft (RPA) Pilots, there is a significant push to increase training efficiency in Formal Training Units (FTUs) to meet the ever-rising demand for pilots. This research aims to create a Java program with which a scheduler and a Squadron Director of Operations (DO) can run simulations to determine how to prioritize scheduling crews best to allow them to complete Initial Qualification Training (IQT) in as close to 80 days training days as possible.
To test the efficacy of the schedule processes, the author will conduct a quantitative study of the program’s output. This study involves numerical data the program will output during its execution (Bui, 2020, p. 12). The program will run several simulations to gather a significant amount of data. These simulations resemble a Monte Carlo simulation. Monte Carlo simulations can determine algorithm efficiencies and output errors (De Palma et al., 2019, p. 294).
Monte Carlo simulations are beneficial in experiments that possess uncertainty. In this program, the two variables that significantly affect the output are the probability of crews passing their training events and weather conditions being out of limits to fly. When crews fail an event, they will not be able to progress to their next event before passing the failed event. When weather conditions exceed allowable limits, flying will be canceled for the day, and some of the events will be pushed to simulators if possible. When there is no training occurring, the FTU loses significant time. Rahman et al. (2020) analyze how charging time affects an electrical grid, and they analyze their results with a Monte Carlo simulation. Just as the time is uncertain in Rahman et al.’s study, some time constraints are uncertain in the FTU’s operations. The measurements in this study will show how uncertainty can affect the overall training time of the FTU. Similar to a study by Kabiri et al. (2022), this study will conduct several Monte Carlo simulations and analyze the output to determine which prioritization approach is the most promising.
This program will determine how to schedule crews and events in a way that results in an effective and robust schedule. If a schedule is robust, it can handle unforeseen circumstances with little effect on the original schedule. The results of the Monte Carlo simulations will provide data that can be measured to compare three different approaches. The output of each simulation will be the number of days it takes for classes to finish the program. There will be at least three classes in the program at a time, each consisting of ten crews. Each crew has a Pilot and a Sensor Operator.
In a study of proactive surgery scheduling, Aissaoui et al. (2020) compared the results of Monte Carlo simulations to compare the robustness of different schedules (p. 8). In this study, I plan to analyze the results similarly by comparing the simulations’ results. The result of each simulation will show how many training days each class will complete the simulated training. Important values will be the highest number, lowest number, and the average number of days to completion. These values will represent the worst-case, best-case, and typical scenarios, respectively.
The program will be available for use by the FTU’s Schedulers, each squadron’s DO, and the Registrars. The Schedulers and DOs have the ability to run the simulations, while the DO can view the output. The only action available to the Registrars is the ability to add or remove aircrews. Figure 1 below depicts a use case diagram associated with the proposed system.
Figure 1. Use case diagram of the proposed system depicting the possible actions of three actors.
The program consists of six classes, illustrated below in Figure 2. ReaperScheduler.java is the main class that runs the program. InstructorCrew.java and StudentCrew.java are classes related to the different types of crews utilized in the program. GroundContrlStation.java and Simulator.java both extend TrainingDevices.java. A significant difference between the CCSs and Simulators is that some of the GCSs have to ability to transform into Simulators, then back to a flight-line GCS. This is very useful on weather days when flights face the threat of cancelation.
Here is an explanation of some of ReaperScheduler.java’s variables. This explanation will be further expanded in future discussion boards as well.
Integer simulationCount is the number of iterations of the simulation that the user wishes to run. A higher count of simulations should render a more accurate result than a test of fewer simulations. String[][] coursesByDay is a multi-dimensional Array that consists of training events broken up by each training day. For instance, one day might consist of three academic lessons, while another has only two. Each day of flying will only consist of one event, accomplished in either a Simulator or a GCS. Date[] downDays is an Array of dates where no training is allowed. This Array includes weekends, holidays, and pre-determined “down days.”
ArrayList StudentCrewList consists of all of the student-type crews. Each StudentCrew will have a crew and class number, the next event to be accomplished, their priority in training, and whether they have a day off. If a crew fails a training event, determined by the pass/fail probability in the simulation, the crew will be granted a day off and will not be off of the schedule for that day. ArrayList InstructorCrewList consists of every Instructor pairing.
Figure 2. Class diagram of the proposed system.
The program allows schedulers and squadron leadership to run simulations with various variables. Figure 3 shows a case description of the user setting up and running a schedule test. The user can set the number of Ground Control Stations (GCS) and Simulators available by the squadron. Next, the user can select the probability of weather cancelations based on the date and the pass/fail rate of events based on training trends. The weather constraint can be further optimized by utilizing historical weather data.
Figure 3. Use case description of the steps required when running the program's simulations.
When the user opens the program, they will be prompted to provide inputs regarding the number of training devices – GCSs and Simulators – available, the possibility of weather cancelations, the average pass/fail rate of events, and how many times to run each simulation before storing the data into a database for future reference. The program will run three different simulations before storing their results. Suppose the program struggles to conduct every test before storing the data. In that case, the author can manipulate the program to store the data of each simulation before moving on to the next simulation. Figure 4 shows this process.
Figure 4. Activity diagram depicting the actions required to run the program's simulations.
References
Aissaoui, N. O., Khlif, H. H., & Zeghal, F. M. (2020). Integrated proactive surgery scheduling in private healthcare facilities. Computers & Industrial Engineering, 148, 1-14. https://doi.org/10.1016/j.cie.2020.106686
Bui, Y. N. (2020). How to write a master's thesis (3rd ed.) [VitalSource]. SAGE Publications.
De Palma, B., Erba, M., Mantovani, L., & Mosco, N. (2019). A Python program for the implementation of the t-method for Monte Carlo simulations. Computer Physics Communications, 234, 294-301. https://doi.org/10.1016/j.cpc.2018.07.004
Kabiri, N. N., Emami, S., & Safaei, A. S. (2022). Simulation–optimization approach for the multi-objective production and distribution planning problem in the supply chain: Using nsga-ii and monte carlo simulation. Soft Computing, 26(17), 8661-8687. https://doi.org/10.1007/s00500-022-07152-2
Rahman, M. M., Al-ammar, E. A., Das, H. S., & Ko, W. (2020). Comprehensive impact analysis of electric vehicle charging scheduling on load-duration curve. Computers & Electrical Engineering, 85, 1-14. https://doi.org/10.1016/j.compeleceng.2020.106673
Post 3:
Stanley Glover
Research Methodology
In the realm of information technology and cyber security, there are a variety of different technologies utilized in order to prevent cyber attacks. A popularly implemented solution used by individuals and companies is the use of Intrusion detection software. The goal of this software is to actively monitor network/host activity and detect cyber attacks before they happen. One of the most commonly utilized intrusion detection software is the open-source project SNORT(Bhosale,2015) .SNORT is an intrusion detection software that allows the user to configure a set of rules to define malicious activity on the network and analyze packet traffic that may be harmful. All of these actions conducted by the SNORT software are then formatted into an alert, which provides details on the threat. The aim of this research thesis is to develop a software plugin that will provide simple containment capabilities that can be automatically executed based on the alerts provided by SNORT.
Research Method
The research method that will be pursued in regard to this research question will be implementing simulation and experimentation. The nature of this research question is centered around the development of a new piece of software that acts in conjunction with a previously developed IDS tool. The approach that will be taken to validate this research question is going to utilize simulation and experimentation methodology. This combination of research methods has been utilized extensively in IDS research.
Three instances of this method being utilized in other similar research is as follows;
Autonomous rule generation and security event correlation
In a research study title “hybrid-based approach of snort automatic rule generator and security event correlation” the goal of the researcher Ebrima Jaw was to develop a software plugin that allows for the automation of rule setting and correlation of packet data in a raw format. The key functionality of the plugin consists of collecting the unformatted packets from the IDS, and bundling the information together before bundling the events into manageable categories. This is all to be done without prior knowledge being imputed into the IDS.
In order to validate the functionality of the novel software and ensure that it functioned in the appropriate manner an experimental approach was taken. The researcher proceeded to select a variety of 4 different types of instructions that would be deployed against the system. The intrusions that were selected were as follows; DDOS, NMAP scan, Ping, and DOS attacks(Jaw,2022). As per the nature of the experiment the researcher would launch the same four attacks collecting the number of alerts generated as a control. They would then launch the same attacks with the plugin operational, then collect the number of categorized alerts. The final point made The result of this study showed that the plugin limited the number of alerts received by the IDS viewer and also increased the efficiency of correlation cyber events.
2. IDS quality of service configuration and parallel technology
In the research paper titled “Improving Networks IDS performance through QOS and Parallel technology” the author Waleed Bul'ajoul utilized an experimental approach to conducting this research. The goal of this research is to performance of the SNORT IDS in a high-traffic network. The experiments conducted by the researcher aim to expose the inefficiencies in the IDS and provide a more viable solution that is less prone to packet drop.(Bul'ajoul,2014)
The proposed solution within the research was to develop a software solution that allows for the IDS to be configured using QoS and parallels technology, with the hypothesis that this new software will allow better performance when on higher traffic networks. In order to test the hypothesis, Bul'ajoul utilized a network stress testing tool along with a DDOS tool to simulate an attack on high traffic network. Although the research’s results were given in more of a statistical format the overall methodologies used were based on simulating situations and testing the function of the software configuration.
3. Faster string matching for intrusion detection or exceeding the speed of Snort
This research paper by researcher C.J. Coit focuses on developing new additions to the algorithm that SNORT IDS operates on in order to make its detection time more effective and faster. The author states that the problem with this IDS is that it relies on exact string matching, which inhibits the speed of the detection process. The method that is used in this research is along the same lines as the prior research methods. It takes the approach of running multiple simulations and experiments in order to find out the result of the algorithm modifications.
In regard to this thesis, the research methodology that will be implemented will consist of the following actions. The first is step is to develop and configure the IDS within a virtual host environment. Once the host is configured, from a different machine a set of predetermined attacks will be sent to the machine operating the IDS. This will ensure that there is a control in which the alerts are untampered and the priority level can be recorded.
Once a control has been established and the priority level of each of the attacks is recorded, the novel plugin will be activated on that same machine. Since the plugin is designed to take responsive pre-configured options based on the priority level, the results can be hypothesized ahead of time.
References
Bul'ajoul, W., James, A., & Pannu, M. (2015). Improving network intrusion detection system performance through quality of service configuration and parallel technology. Journal of Computer and System Sciences, 81(6), 981-999.
Bhosale, D. A., & Mane, V. M. (2015, October). Comparative study and analysis of network intrusion detection tools. In 2015 International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT) (pp. 312-315). IEEE.
Coit, C. J., Staniford, S., & McAlerney, J. (2001, June). Towards faster string matching for intrusion detection or exceeding the speed of snort. In Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01 (Vol. 1, pp. 367-373). IEEE.
Jaw, E., & Wang, X. (2022). A novel hybrid-based approach of snort automatic rule generator and security event correlation (SARG-SEC). PeerJ Computer Science, 8, e900.