Ikeriri network service co., ltd. Packet capture company - Sharkfest ...

PA-3 Debugging Wireless with WiresharkIncluding Large Trace Files, AirPcap &Cascade PilotMegumi Takeshita, ikeriri network serviceIkeriri network service co., ltdhttp://www.ikeriri.ne.jp1

Ikeriri network service co., ltd.Packet capture companyIkeriri network service co., ltdhttp://www.ikeriri.ne.jp3Training at JGSDF• Consulting• Reselling• Debugging• Investigating• TrainingPacketCapture

Reseller of Riverbed Technology,Metageek, OSCIUM, Dualcomm etc.• Ikeriri is one of the reseller of Packet capture /analysis products in Japan• Riverbed Technology’s AirPcap, TurboCap, Pilot• Metageek Wi-Spy and Chanalyzer• OSCIUM products• Dualcomm productsetc.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp4

Comparison pcap files• We should capture comparison pcap files fordebugging because there may be clues !• Using boundary value analysis and equivalencepartitioning, collect comparison pcaps.• Some cases we can easily find the problem, keys,and the answer only watching 2 pattern of pcaps.• Frame color,Expert info is easiestIkeriri network service co., ltdhttp://www.ikeriri.ne.jp7

Gathering information and making tableCreate plan ofExperiment8• Hearing the customer in deep,address (MAC,IP) port (TCP,UDP)log message, how to ? How many ?• Host type, OS, Software version*Android is difficult ( many variation)iOS ( iPhone and iPad ) is simpleWindows 7/8 may be in same result• Frequency is also importantIkeriri network service co., ltdhttp://www.ikeriri.ne.jpTest captureprocedurethe iterationnumbertest kind, typestest configuration

Standards and protocol and sequence• Standards, protocol helps usdebugging, using documents,White Papers in IEEE, RFCsin IANA and other sites• Sequence diagram is verymuch hint for debug forchecking and comparingIkeriri network service co., ltdhttp://www.ikeriri.ne.jp9

Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp10

11Before capturing• Clear browser cache for capturing all communicationpacket.• DNS cache is also clear if you need to get DNS queryresponsepacket• Disable or turn off Windows firewall and personalfirewall etc.• Stop and exit software and service of sending packetlike VPN(keep alive), UPnP(SSDP discovery), iTunes• Record Date, IP address, tcp port and MAC address forinspecting later.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp

Tips1 redirecting information12• Executing ipconfig and getmac command andredirecting help us inspecting laterIkeriri network service co., ltdhttp://www.ikeriri.ne.jp

TIPS2: netstat –a and netstat -bIkeriri network service co., ltdhttp://www.ikeriri.ne.jp13• Show tcp/udp connections using netstat,and I recommend piping and find matching (LISTEN)netstat –b tells bind application to socket.

TIPS3• Please check your NIC status ( including Error andDiscard frames ) using netstat –e command.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp14

Check settings in NIC• Today almost NICs offload tcp, udp/ip function.• Almost NICs support Gigabit Ethernet and carrierextension ( over 1500MTU ex. 9kb MTU)• Wireshark read pcap stream from WinPcap• Please check offload settings in propertiesin NIC ( from device manager)• Also please check MTU setting too.(Jumbo frame or MTU)Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp15

Use Windows Search Index• To add extension of cap and pcap,set type as clear text search,We can search pcap/cap files like Google !off course in multibytes ( in Japanese )• Control panel -> index option / folder option16

Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp17

Capturing many interface in one timeCheck multiple interface and capture• In case of checking many interface in the same time,now check multiple interface and start capture.• Trace file is combined with multiple interface• For example upstream/downstream from router,client/server and so on.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp18

USB Debugging• We can capture USB frames using Linux• VMware environment also worksIkeriri network service co., ltdhttp://www.ikeriri.ne.jp19

Using display filter• Protocol.field.value style• Easiest way is taking use of actual header field(right click and show submenu and set/prepare filter )• Condition of multiple format &&(AND) ||(OR)parameter value can be compared ( gt ge / lt le )• Automatic complication will help you to create• Contains keywordhttp.request.url contains ikeririIkeriri network service co., ltdhttp://www.ikeriri.ne.jp20

Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp23

Capturing PING(ICMP) packet• Start capturing, then test ping command• Source / destination IP and MAC address is necessary forcommunication under TCP/IP• ARP request / response loop make address resolution.• ARP result is remembered and cached for 120 seconds ineach PCs• ICMP echo request / response loop check layer 3 connectivity.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp24

IP trace file analysis• Check identification field of IP headersame Identification number means re-send packet,fragmentation, and security problem.• TTL field is the hint of hop counts ( always the nodeuses 128/64 )• Check DF/MF bit and offset field in IP header.• Compare IP length field and MTU size.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp25

ping a.b.c.d –l 1500 -f• originalEthernet II(14)IP(20)ICMP(8)Message1500• Fragment1Ethernet II(14)IP(20)DF= MF=offset=ICMP(8)Message1472• Fragment2Ethernet II(14)IP(20)DF= MF=offset=Message28Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp26

Count packet size (MTU1500)ICMP -28Ethernet II(14)IP(20)ICMP(8)Message1472(MTU=1500)• ping IP –l size ※-f fragment disabledTCP HTTP and many protocols -40Ethernet II(14)IP(20)TCP(20)UDP VOIP and video transmission -28Ethernet II(14)IP(20)UDP(8)Segment sizeMSS=1460Datagram size1472(MTU=1500)Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp27

PPPoE Header and MTU size accordingto Japanese ISPs• NTT east fletsMTU 1454Bytes MSS 1414Bytes• NTT west flets premiumMTU 1438Bytes MSS 1398Bytes• GRE + IPsec (transport mode) 1440 BytesGRE + IPsec (tunneling mode) 1420 Byte• UDP(NAT Traversal)IP(20) UDP(8) , PPPoE, PPP headerIkeriri network service co., ltdhttp://www.ikeriri.ne.jp

(C) いけりり★ネットワークサービスhttp://www.ikeriri.ne.jp29

tfgen• For checking TCP vs. UDPit is very usefulIkeriri network service co., ltdhttp://www.ikeriri.ne.jp30

Create IO graph and visualize• Compare TCP ( connection oriented ) and UDP(connectionless protocol ) and visualize.• Lets use IO graph function and filter packet by protocol• Set X axis to seconds and Y axis to bit/tick (means bps)Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp31

Check streams TCP/UDP• Wireshark set stream ID (tcp.stream) in each TCPconnection automatically.• Filter by tcp stream number and colorize conversation.• Check bytes using “Follow TCP Stream”• UDP stream is also analyzing by “Follow UDP Stream”Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp32

Export function is very good for HTTP• We can restore HTTP data from WEB communicationpcap/pcapng files by File>Export>Object>HTTP• HTTP statistics is importantthe count value means Web application performance1 image map vs. 100 gif fileIkeriri network service co., ltdhttp://www.ikeriri.ne.jp33

FlowGraph gives you a new look of debugging• Statistics>FlowGraph and maximize the screen• Display filter is very good ways to create good visualization.• If you need to follow TCP, set graph to TCP graph.• Compare behavior with RFC and standardsIkeriri network service co., ltdhttp://www.ikeriri.ne.jp34

Trend analysis BASICTopN style, and drilled down in details1. Create TOPN list table of Endpoint and filtered2. Create Nother list table of Conversation3. Then create protocol hierarchy and check streamIkeriri network service co., ltdhttp://www.ikeriri.ne.jp35

Utilize IO graph in two ways• Set packets to Y axis to create ERROR graphHistogram style is good for Frequency graph• Set bit to Y axis to create BPS graphline style is good for amount graph.Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp36

Digest Auth SUCCESS/FAILアクセスー 認 証 ( 失 敗 )ー 認 証 ( 成 功 ).pcapng• Digest authentication will be failed when ID/Password mismatch401 Unauthorized• If successIkeriri network service co., ltdhttp://www.ikeriri.ne.jp37

Sample trace• Try to click “Home” and check trace file.• Once called control.cgi and c.1.ae.brightness==0c.1.wb==auto c.1.shade==off c.1.focus==auto c.1.zoom==6040c.1.pan:=-4014 c.1.tilt:=-153value send to the server• Moving picture needs 5Mbpshow about creating IO graph and set Y axisas a bit/tickIkeriri network service co., ltdhttp://www.ikeriri.ne.jp38

(C) いけりり★ネットワークサービスhttp://www.ikeriri.ne.jp39

Use AirPcap and set clear text if possible• Need Jumbo frame or IEEE802.11a/n go NX• We have to capture their own 4 way handshake todecrypt pcap file secured by WPA2-PSK,• Its terrible troublesome to match between the WPA2handshake and the communication packet.• Set free channelin test capture( android 14ch NG)Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp1 年 保 守 なし

Type/Subtype, TX rate, BSSID, CH, RSSI• In Wireless environment, please watch importantfield of IEEE802.11 header, physical (radiotap/PPI)header ( Type/Subtype, TX Rate, BSSID, CH, RSSI)• Many troubles are occurred before Data exchangehostAPProve Req/AckAuthenticationAssociation Req/AckIkeriri network service co., ltdhttp://www.ikeriri.ne.jpData1 年 保 守 なし

42Between deployments and standards• IEEE802.11 and related standards, protocols arenot so punctual and irritate rules ( they are notdescribed in detail and all step, procedure, but justset the summary )• For example WPS is famous and many user usethe PIN or button settings, but the deployments inWireless devices differs a lot• We have to check sequencesin detail for debuggingIkeriri network service co., ltdhttp://www.ikeriri.ne.jp

(C) いけりり★ネットワークサービスhttp://www.ikeriri.ne.jp43

Huge packet case• In old days we use sampling technologieslike SNMP, MRTG, and many flow analysissuch as Cisco NetFlow, sFlow, iFlowIgnored Ignored Ignored May be• But small packet ( 64 bytes – 100 bytes ) may be ignored.Some small packet is important symptom of analysis ( ARP/ TCP SYN / HTTP GET and others )Ikeriri network service co., ltdhttp://www.ikeriri.ne.jp

Debugging Environment• Using TurboCap, MMMM packets received by theapplication NNNN packets accepted by the filter anddumped to disk• To fix, Optimize I/O access flowpacket -> IRQ -> SVC -> driver -> OS• Use 6 cores Xeon-L5640 and 24GB RAM !( power resolve things and no page files )• Stop tcpdump and create program using pcaplibraries in C/C++ ( dumpcap.exe )• Pcap -> standard output -> FIFO -> SQLite• 3 month no problemIkeriri network service co., ltdhttp://www.ikeriri.ne.jp

Driving 250GB pcap file with Pilot• We use 250GB pcap file,huge huge file with CascadePilotPE installed into NotePC• Use view to check macro analysis,and finally check the actual pcapsusing Wireshark• Only, best, easiest way todrive huge pcap fileIkeriri network service co., ltdhttp://www.ikeriri.ne.jp1 年 保 守 なし

QA and DemonstrationIkeriri network service co., ltdhttp://www.ikeriri.ne.jp48

Thank You !!ikeriri network servicehttp://www.ikeriri.ne.jp(C) Ikeriri network servicehttp://www.ikeriri.ne.jp