Malware Presence Detection using Wireshark Analysis

Introduction

Malware is malicious software that communicates with external systems to perform unauthorized activities such as data exfiltration, remote control, or network scanning. Network traffic analysis is an effective method to identify malware presence without executing the malicious program. In this work, Wireshark is used to analyze a packet capture file and identify abnormal communication patterns that indicate malware activity.

Objectives

To analyze network traffic using Wireshark for identifying malware activity
To observe abnormal communication patterns such as repeated connection attempts and failed handshakes
To confirm malware presence through packet-level analysis of suspicious traffic

Malware Source

Malware traffic was obtained from Malware-Traffic-Analysis.net:
https://www.malware-traffic-analysis.net/training-exercises.html

PCAP file Description

The selected packet capture file contains network traffic generated from an infected system communicating with external servers. The capture includes multiple TCP connection attempts, retransmissions, and abnormal traffic patterns. This dataset is widely used for security analysis and provides realistic malware communication behavior.

Architecture of Work

The analysis involves an infected host communicating with multiple external servers over the internet. Wireshark is used to capture and analyze this traffic to identify abnormal communication patterns.

Figure: Architecture representing the workflow of malware traffic analysis using Wireshark.

Procedure

The malware packet capture file was downloaded from Malware-Traffic-Analysis.net.
The file was opened in Wireshark for analysis.
Various filters such as TCP SYN packets, retransmissions, and reset packets were applied.
Abnormal patterns like repeated connection attempts, failed handshakes, and communication with multiple external IPs were observed.
Screenshots were captured as evidence of malware presence.

Proof 1

The analysis begins with TCP SYN packets, which are used to initiate new TCP connections. A noticeably high number of SYN packets from the same source host indicates repeated automated connection attempts. Such behavior is uncommon in regular browsing activity and often suggests malware trying to contact external systems.

Proof 2

After observing SYN packets, the source IP addresses were analyzed. The same internal host repeatedly generated multiple requests in a short period of time. This consistency suggests that one compromised machine was responsible for the suspicious traffic rather than multiple users.

Proof 3

The infected system was then found communicating with several external destination IP addresses. Legitimate traffic generally follows predictable destinations, whereas malware often attempts multiple servers for command-and-control, downloads, or fallback communication.

Proof 4

Frequent retransmission packets were visible in the capture. Retransmissions occur when responses are not received properly, causing the sender to resend packets. This indicates unstable or blocked communication attempts commonly seen in malicious outbound traffic.

Proof 5

Reset packets were also identified during the analysis. These packets abruptly terminate active sessions and often appear when connections are rejected or closed unexpectedly. Their presence supports the finding of abnormal and unsuccessful communication.

Proof 6

Several TCP sessions showed incomplete handshakes, where SYN packets were sent but proper completion did not occur. This suggests repeated attempts to establish connections that were either blocked or unreachable, which is consistent with malware behavior.

Proof 7

A dense cluster of packets was observed within short intervals. Human-generated traffic is usually more varied, whereas malware scripts can rapidly generate multiple packets in bursts. This high packet density indicates automated network activity.

Proof 8

Traffic leaving the local network toward internet destinations was analyzed next. Unexpected outbound communication from a host may indicate malware attempting to receive commands, exfiltrate data, or maintain persistence with remote servers.

Proof 9

Conversation statistics revealed several simultaneous sessions between the host and different destinations. This shows that the system was actively maintaining or attempting multiple network relationships, strengthening the suspicion of malicious automation.

Proof 10

Endpoint statistics showed repeated communication between the same internal machine and several remote endpoints. This confirms that the observed suspicious behavior was concentrated around one infected host.

Proof 11

Some packets had relatively large frame sizes compared to normal control packets. Large packets may contain transferred files, payloads, or additional suspicious data being exchanged with external systems.

Proof 12

Traffic was observed on ports other than commonly used web ports such as 80 and 443. Malware often uses unusual or less-monitored ports to avoid simple detection mechanisms.

Proof 13

Multiple attempts toward specific destination ports were recorded. This repeated targeting suggests that the malware was programmed to contact particular services or applications.

Proof 14

Short bursts of packets appeared repeatedly throughout the capture. Burst-based traffic is a common sign of automated scripts executing scheduled communication attempts.

Proof 15

The same packet pattern was repeated again later in the trace. Repetition indicates programmed behavior, which strongly differs from random human network usage.

Proof 16

Different combinations of TCP flags were visible during the sessions. Irregular or excessive flag usage can indicate unstable sessions, forced retries, or suspicious connection handling.

Proof 17

TCP stream inspection showed structured communication rather than isolated random packets. This suggests intentional exchanges between the host and remote systems.

Proof 18

Packet timestamps showed inconsistent delays between transmissions. Malware commonly waits and retries periodically, producing irregular timing gaps instead of smooth user-driven traffic.

Proof 19

Repeated retry attempts were visible across the packet capture. This persistence indicates that the host continuously tried to establish communication despite failures.

Proof 20

When all observations are combined, the traffic clearly demonstrates repeated connection attempts, failed sessions, multiple destinations, automated timing patterns, and suspicious outbound communication. These characteristics strongly confirm malware presence in the analyzed packet capture.

Effects of Malware

Unauthorized communication with external servers
Increased network traffic due to automated processes
Failed or incomplete connections
Use of unusual ports and protocols
System instability due to repeated network activity

New Findings from Analysis

The analysis revealed automated malicious behavior such as repeated SYN requests, failed handshakes, retransmissions, irregular traffic bursts, and communication with multiple external IPs, clearly differing from normal traffic patterns. Based on these findings, it is recommended to continuously monitor suspicious outbound traffic, configure firewall and IDS rules, perform regular malware scans, and use packet analysis tools like Wireshark for early threat detection.

Use of AI

ChatGPT
Gemini

These AI tools were used for report structuring, packet analysis assistance, and document refinement.

Conclusion

The analysis confirms the presence of malware through abnormal traffic patterns such as repeated connection attempts, retransmissions, and communication with multiple external systems. Unlike normal traffic, malware communication is irregular and does not always follow standard TCP behavior. This demonstrates that Wireshark can effectively be used to identify malicious activity through packet-level analysis.

Github Repository Link

https://github.com/Anush65/malware-analysis

YouTube Video Link

https://youtu.be/z5FdF1m6IoM

References

Malware-Traffic-Analysis.net
https://www.malware-traffic-analysis.net/training-exercises.html

Wireshark Official Documentation
https://www.wireshark.org/docs/

Acknowledgement

I sincerely thank the School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology Chennai, for offering the theory and laboratory courses during the Winter Semester 2025–2026 with an industry-standard syllabus.

I express my gratitude to Dr. T. Subbulakshmi, Professor, SCOPE, VIT Chennai, for her guidance and support throughout this course and for providing this valuable opportunity.

I thank Gerald Combs, founder of Wireshark and recipient of the ACM Software System Award (2018), for developing an excellent tool for packet capture and traffic analysis.

I also thank Bradley Duncan, creator of Malware-Traffic-Analysis.net, for providing high-quality malware traffic datasets and educational blogs that help students learn malware behavior safely without executing malicious software.

I appreciate my peers and friends, for their suggestions and support during the completion of this assignment.

I am especially grateful to my special friend Darshan S, who guided me during the initial stages and helped me throughout the learning process, making it easier to understand complex concepts.

I express sincere gratitude to my parents, family members, and well-wishers for their encouragement and continuous support.

Finally, I would like to acknowledge all the books, websites and other resources that I referred to during the course of this work. Their contributions have been invaluable in enhancing my knowledge and understanding.

Mr. Anush V, II year B.Tech. CSE student, School of Computer Science and Engineering , VIT Chennai