![Internet Traffic Monitoring and Analysis 홍 원 기 포항공과대학교 컴퓨터공학과 분산처리 및 네트워크관리 연구실 [email_address] http://dpnm.postech.ac.kr/ Tel: 054-279-2244](https://image.slidesharecdn.com/kt-2003-7-4-100512143755-phpapp01/85/internet-traffic-monitoring-and-analysis-1-320.jpg)
![Table of Contents ,[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (20)
Similar a Internet Traffic Monitoring and Analysis
Similar a Internet Traffic Monitoring and Analysis (20)
Más de Information Technology
Más de Information Technology (20)
Último
Último (20)
Internet Traffic Monitoring and Analysis
- 1. Internet Traffic Monitoring and Analysis 홍 원 기 포항공과대학교 컴퓨터공학과 분산처리 및 네트워크관리 연구실 [email_address] http://dpnm.postech.ac.kr/ Tel: 054-279-2244
- 27. MRTG+ Network Sensitive Map (1997)
- 35. WebTrafMon-II Architecture database Traffic analyzer (minutely, hourly, daily, monthly, yearly) probe network point promiscuous mode packet capture hash log format and save into DB user distributed environment request response packet header information log file log format port information port information make short term, long term traffic data minutely minutely hourly, daily, monthly, yearly statistics network traffic data analyzer Flow generator
- 46. NG-MON - Deployment at POSTECH http://ngmon.postech.ac.kr Packet Capture Flow Generator Flow Store Analyzer Presenter 141.223.182. 40 EnterFLEX at Computer Center Flow Store 141.223. 182.[31,32,33,34] POSTECH Computer Center 141.223.182. 38 EnterFLEX at Computer Center 141.223.182. 37 EnterFLEX at Computer Center 141.223.182. 36 EnterFLEX at Computer Center INTERNET 1Gbps Optical link NetOptics 1Gbps Optical Splitter Packet Capture Flow Generator Packet Capture Flow Generator Packet Capture Flow Generator POSTECH Gigabit Campus Network Router Router
- 47. NG-MON - Host Data Received Minute View
- 48. NG-MON - Host Data Exchanged Minute View
- 49. NG-MON - Detailed Subnet Data Sent Minute View
- 50. NG-MON - Application Protocol Minute View
- 51. NG-MON - Time Series Minute View
Notas del editor
- Abstract Most Internet networking devices are now equipped with a Web server for providing Web-based element management so that an administrator may take advantage of this enhanced and powerful management interface. On the other hand, for network management, an administrator normally buys and deploys SNMP-based network management platform to be customized to his network. Each management scheme has mutually exclusive advantages; consequently, two schemes coexist in the real world. This results in both a high development cost and a dual management interface for administrator. We propose an embedded Web server (EWS)-based network management architecture as an alternative to an SNMP based network management and to leverage on already existing embedded web server. We extend EWS-based element management architecture to the network management architecture. Our proposed architecture uses HTTP as a communication protocol with management information and operation encoding. Further we designed a management system on the basis of our proposed architecture that supports basic management functions.
- ------------------------------------------------------------------------------------------------------------------------------------------------------------- [http://http://www.caida.org/outreach/metricswg/faq.xml] CAIDA Outreach Network Measurement FAQ 2.1. Why should I measure my network's behaviour? If you don't measure it, you have no objective record or benchmark of how it behaves. This could make it difficult to judge whether changes in the network have improved its performance, or degraded it. If you are buying Internet connectivity from an ISP you need to understand the kind of service being offered, and you need to measure the actual performance so as to verify that you're getting what you pay for. ------------------------------------------------------------------------------------------------------------------------------------------------------------- [KRNET Tutorial] http://dpnm.postech.ac.kr/webboard/ Internet Traffic Monitoring & Analysis User’s Needs * Monitor the performance experienced by one ’ s application - Why is the web page download so slow? - Why is my multicast video stream jerky? * Check if level of service meets one ’ s need - Do I have enough b/w? * Check if one experiences intrusions and attacks - Is someone attacking me? Service provider ’ s needs * Monitor the current level of activity * Enforce SLAs(service level agreements) * Detect faults and failures * Engineer the network for better performance * Plan for future capacity * Feedback to customers -----------------------------------------------------------------------------------------------------------------------
- ------------------------------------------------------------------------------------------------------------------------------------------------------------- [http://http://www.caida.org/outreach/metricswg/faq.xml] CAIDA Outreach Network Measurement FAQ 2.1. Why should I measure my network's behaviour? If you don't measure it, you have no objective record or benchmark of how it behaves. This could make it difficult to judge whether changes in the network have improved its performance, or degraded it. If you are buying Internet connectivity from an ISP you need to understand the kind of service being offered, and you need to measure the actual performance so as to verify that you're getting what you pay for. ------------------------------------------------------------------------------------------------------------------------------------------------------------- [KRNET Tutorial] http://dpnm.postech.ac.kr/webboard/ Internet Traffic Monitoring & Analysis User’s Needs * Monitor the performance experienced by one ’ s application - Why is the web page download so slow? - Why is my multicast video stream jerky? * Check if level of service meets one ’ s need - Do I have enough b/w? * Check if one experiences intrusions and attacks - Is someone attacking me? Service provider ’ s needs * Monitor the current level of activity * Enforce SLAs(service level agreements) * Detect faults and failures * Engineer the network for better performance * Plan for future capacity * Feedback to customers -----------------------------------------------------------------------------------------------------------------------
- ------------------------------------------------------------------------------------------------------------------------------------------------------------- [http://http://www.caida.org/outreach/metricswg/faq.xml] CAIDA Outreach Network Measurement FAQ 2.1. Why should I measure my network's behaviour? If you don't measure it, you have no objective record or benchmark of how it behaves. This could make it difficult to judge whether changes in the network have improved its performance, or degraded it. If you are buying Internet connectivity from an ISP you need to understand the kind of service being offered, and you need to measure the actual performance so as to verify that you're getting what you pay for. ------------------------------------------------------------------------------------------------------------------------------------------------------------- [KRNET Tutorial] http://dpnm.postech.ac.kr/webboard/ Internet Traffic Monitoring & Analysis User’s Needs * Monitor the performance experienced by one ’ s application - Why is the web page download so slow? - Why is my multicast video stream jerky? * Check if level of service meets one ’ s need - Do I have enough b/w? * Check if one experiences intrusions and attacks - Is someone attacking me? Service provider ’ s needs * Monitor the current level of activity * Enforce SLAs(service level agreements) * Detect faults and failures * Engineer the network for better performance * Plan for future capacity * Feedback to customers -----------------------------------------------------------------------------------------------------------------------
- ------------------------------------------------------------------------------------------------------------------------------------------------------------- [http://http://www.caida.org/outreach/metricswg/faq.xml] CAIDA Outreach Network Measurement FAQ 2.1. Why should I measure my network's behaviour? If you don't measure it, you have no objective record or benchmark of how it behaves. This could make it difficult to judge whether changes in the network have improved its performance, or degraded it. If you are buying Internet connectivity from an ISP you need to understand the kind of service being offered, and you need to measure the actual performance so as to verify that you're getting what you pay for. ------------------------------------------------------------------------------------------------------------------------------------------------------------- [KRNET Tutorial] http://dpnm.postech.ac.kr/webboard/ Internet Traffic Monitoring & Analysis User’s Needs * Monitor the performance experienced by one ’ s application - Why is the web page download so slow? - Why is my multicast video stream jerky? * Check if level of service meets one ’ s need - Do I have enough b/w? * Check if one experiences intrusions and attacks - Is someone attacking me? Service provider ’ s needs * Monitor the current level of activity * Enforce SLAs(service level agreements) * Detect faults and failures * Engineer the network for better performance * Plan for future capacity * Feedback to customers -----------------------------------------------------------------------------------------------------------------------
- ------------------------------------------------------------------------------------------------------------------------------------------------------------- [http://http://www.caida.org/outreach/metricswg/faq.xml] CAIDA Outreach Network Measurement FAQ 2.1. Why should I measure my network's behaviour? If you don't measure it, you have no objective record or benchmark of how it behaves. This could make it difficult to judge whether changes in the network have improved its performance, or degraded it. If you are buying Internet connectivity from an ISP you need to understand the kind of service being offered, and you need to measure the actual performance so as to verify that you're getting what you pay for. ------------------------------------------------------------------------------------------------------------------------------------------------------------- [KRNET Tutorial] http://dpnm.postech.ac.kr/webboard/ Internet Traffic Monitoring & Analysis User’s Needs * Monitor the performance experienced by one’s application - Why is the web page download so slow? - Why is my multicast video stream jerky? * Check if level of service meets one’s need - Do I have enough b/w? * Check if one experiences intrusions and attacks - Is someone attacking me? Service provider’s needs * Monitor the current level of activity * Enforce SLAs(service level agreements) * Detect faults and failures * Engineer the network for better performance * Plan for future capacity * Feedback to customers -----------------------------------------------------------------------------------------------------------------------
- ------------------------------------------------------------------------------------------------------------------------------------------------------------- [http://http://www.caida.org/outreach/metricswg/faq.xml] CAIDA Outreach Network Measurement FAQ 2.1. Why should I measure my network's behaviour? If you don't measure it, you have no objective record or benchmark of how it behaves. This could make it difficult to judge whether changes in the network have improved its performance, or degraded it. If you are buying Internet connectivity from an ISP you need to understand the kind of service being offered, and you need to measure the actual performance so as to verify that you're getting what you pay for. ------------------------------------------------------------------------------------------------------------------------------------------------------------- [KRNET Tutorial] http://dpnm.postech.ac.kr/webboard/ Internet Traffic Monitoring & Analysis User’s Needs * Monitor the performance experienced by one ’ s application - Why is the web page download so slow? - Why is my multicast video stream jerky? * Check if level of service meets one ’ s need - Do I have enough b/w? * Check if one experiences intrusions and attacks - Is someone attacking me? Service provider ’ s needs * Monitor the current level of activity * Enforce SLAs(service level agreements) * Detect faults and failures * Engineer the network for better performance * Plan for future capacity * Feedback to customers -----------------------------------------------------------------------------------------------------------------------
- To monitor high speed network such 10Gpbs link, the NG-MON should consider these 5 significant requirements. The first one, as stated, NG-MON needs distributed, load-balancing architecture. To distribute the processing load , we should divide monitoring and analysis task into several functional units, and we also need an efficient load sharing mechanism within each phase. For load distribution method , we considered the pipeline and parallel methods. The second is lossless packet capture . NG-MON should capture all packets without a loss to provide all the required information to various analysis applications. The fourth one is, to reduce processing load , flow based analysis is essential. by the flow-based analysis, NG-MON can aggregate packet information into flows for efficient processing. Also, limited storage at each phase should be considered. By the consideration of these requirements we designed the architecture of NG-MON.
- This is an overall architecture of NG-MON design. The key feature in our design is an pipelined distribution and load balancing technique. Whole tasks are divided into 5 phases like this. Packet capture, Flow Generation, Flow Store, Traffic Analysis and Presentation phase. The entire raw packets are captured in the Packet Capture phase. And packet header information extracted from raw packets are delivered to the second phase: Flow Generation phase, The flow information is generated in this Flow Generation phase. the flow information is stored in the Flow Store phase. Traffic Analyzer queries to Flow Store and store analyzed data, provide them to Presenter. Load distribution mechanism used in each phase will be explained in the following slides in detail.
- This slide shows the first phase of our NG-MON design: packet capture phase. Large bulk traffic on the network links is distributed over probe systems and sent to next phase, Flow Generation. In the distribution of raw packets we can use one of these methods. First one is by using splitting function provided by an optical splitter. And Using mirroring functions provided by network devices is the second one. These probe systems captures incoming packets and extract packet header information form layered headers of each raw packet, then push into the export buffer-queues by packet header’s 5-tuple based hashing. Each probe system maintain the same number of buffer queues corresponding to the number of flow generators. If a buffer queue becomes full , probe constructs packet header messages then export to next phase. The raw packets with the same color indicates that they belong to the same flow. As you can see, packets which belong to the same flow put together into the same packet header messages. ( 5-tuple : src & dst address, protocol number, src & dst port number )
- This and next slides shows the second phases of our NG-MON design. In this phase, packet headers are compressed into flows. For the distribution of packet header information, we used 5-tuple based hashing and buffer queue for each flow generator. Therefore the packet header information of potentially the same flow get delivered to the same flow generator. There can’t be the case that same flow is generated in different flow generator at a certain moment. Flow generators simply generate flow messages from incoming packet header messages, then exports these to next phase, flow store.
- This slide shows the third phase of our NG-MON architecture: Flow Store phase The main role of Flow Store phase is to store flow information and handle the request from analyzer: those are write operation and read operation . For the load distribution and efficient processing , we considered a method that prevent write operations from occurring with read operations at the same time in a single flow store system. In order to do this, the destination address of flow messages should be changed over to Flow Store sequentially depending on the time slot changes. While one or more flow stores are inserting flow data, the other flow stores are queried by the traffic analyzers. As you can see here , at the time slot t1, Flow Store 1 only receives flow messages and the other Flow Stores are processing queries from Analyzers. Before the time slot changes from t1 to t2, queries to Flow Store 2 should be finished. Then the time slot becomes t2, flow messages will go into the Flow Store 2, and queries to Flow Store 1 will be started. In our earlier work , we realized that one of the bottleneck of the monitoring process is a huge storage space required. So, Flow Store keeps flow information for only several time slots, and then discard them when they are finished an analysis by traffic analyzers. Therefore, flow store only requires a small and fixed amount of disk space. Flow store provides traffic information to support various analysis applications and provide an analysis API to analyzers.
- This slide shows the fourth and fifth phases of our NG-MON architecture. These two phases are tightly coupled according to the analysis purpose; such as Traffic Throughput Analysis, Usage-based billing analysis, DDOS and DOS attack analysis, such like that. Analyzer extracts information from Flow Stores and can perform application specific analysis . Separate analyzer is needed for each application. we separated the presenter from traffic analyzer, because more than one systems tend to be allocated in the traffic analysis phase.
- In this summer We implemented a prototype of NG-MON and deployed our system in our campus backbone network. In the implementation, we used Net Optics’ Gigabit Fiber Optic tap to split the traffic and used GE Card to get it. The hardware configuration we used are, P-III 800MHz, 256 Mbytes memory, 20Gbytes HD. And we developed our system on Redhat Linux 7.2 OS. And used C language with pcap library in Packet Capture phase. In the Flow Store, we used MySQL Database to store flows. Presenter uses PHP with jpgraph library to present the analysis result through the web.
- This and other two slides show some selected screen shots of our prototype implementation. Our analyzer shows various throughput information according to the HOST, SUBNET, and PROTOCOL. This screen shot shows the throughput of host received in one minute, and total throughput changes by the time in one hour is illustrated in the form of line graph.
- This and other two slides show some selected screen shots of our prototype implementation. Our analyzer shows various throughput information according to the HOST, SUBNET, and PROTOCOL. This screen shot shows the throughput of host received in one minute, and total throughput changes by the time in one hour is illustrated in the form of line graph.
- This is a detailed subnet data sent view in a certain minute.
- Left one is an application protocol view in a certain minute. And right one is Time series graph of the throughput at each protocol layer during a certain hour.
- Left one is an application protocol view in a certain minute. And right one is Time series graph of the throughput at each protocol layer during a certain hour.