Este documento describe Poptrie, una estructura de datos trie comprimido con conteo de población para búsqueda rápida y escalable de tablas de rutas IP en software. Poptrie reduce el tamaño de memoria y mejora el rendimiento mediante compresión de punteros con conteo de población y agregación de rutas. Evaluaciones exhaustivas muestran que Poptrie es 4-578% más rápido que otras tecnologías de vanguardia para tablas de rutas IPv4 y IPv6 públicas y privadas, incluidas tablas futuras
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
The Linux Block Layer - Built for Fast StorageKernel TLV
The arrival of flash storage introduced a radical change in performance profiles of direct attached devices. At the time, it was obvious that Linux I/O stack needed to be redesigned in order to support devices capable of millions of IOPs, and with extremely low latency.
In this talk we revisit the changes the Linux block layer in the
last decade or so, that made it what it is today - a performant, scalable, robust and NUMA-aware subsystem. In addition, we cover the new NVMe over Fabrics support in Linux.
Sagi Grimberg
Sagi is Principal Architect and co-founder at LightBits Labs.
It describes the MMC storage device driver functionality in Linux Kernel and it's role. It explains different type of storage devices available and how they are handled from MMC driver point of view. It describes eMMC (internal storage) device and SD (external storage) devices in details and SD protocol used for communicating with these devices in Linux.
Virtual File System in Linux Kernel
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
The Linux Block Layer - Built for Fast StorageKernel TLV
The arrival of flash storage introduced a radical change in performance profiles of direct attached devices. At the time, it was obvious that Linux I/O stack needed to be redesigned in order to support devices capable of millions of IOPs, and with extremely low latency.
In this talk we revisit the changes the Linux block layer in the
last decade or so, that made it what it is today - a performant, scalable, robust and NUMA-aware subsystem. In addition, we cover the new NVMe over Fabrics support in Linux.
Sagi Grimberg
Sagi is Principal Architect and co-founder at LightBits Labs.
It describes the MMC storage device driver functionality in Linux Kernel and it's role. It explains different type of storage devices available and how they are handled from MMC driver point of view. It describes eMMC (internal storage) device and SD (external storage) devices in details and SD protocol used for communicating with these devices in Linux.
Virtual File System in Linux Kernel
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Linux Memory Management with CMA (Contiguous Memory Allocator)Pankaj Suryawanshi
Fundamentals of Linux Memory Management and CMA (Contiguous Memory Allocator) In Linux.
Virtual Memory, Physical Memory, Swap Space, DMA, IOMMU, Paging, Segmentation, TLB, Hugepages, Ion google memory manager
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Netronome's half-day tutorial on host data plane acceleration at ACM SIGCOMM 2018 introduced attendees to models for host data plane acceleration and provided an in-depth understanding of SmartNIC deployment models at hyperscale cloud vendors and telecom service providers.
Presenter Bios
Jakub Kicinski is a long term Linux kernel contributor, who has been leading the kernel team at Netronome for the last two years. Jakub’s major contributions include the creation of BPF hardware offload mechanisms in the kernel and bpftool user space utility, as well as work on the Linux kernel side of OVS offload.
David Beckett is a Software Engineer at Netronome with a strong technical background of computer networks including academic research with DDoS. David has expertise in the areas of Linux architecture and computer programming. David has a Masters Degree in Electrical, Electronic Engineering at Queen’s University Belfast and continues as a PhD student studying Emerging Application Layer DDoS threats.
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
LinuxCon 2015 Linux Kernel Networking WalkthroughThomas Graf
This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
#MENT: Online & Mobile Trends - India & Worldwide by Gurmit Singh, MD Yahoo I...tiemumbai
Gurmit Singh, MD - Yahoo India shares his views about the Online & Mobile Trends in India and around the World. Also a part of the presentation shows the transformation & evolution of Yahoo.
This session was part of MENT (Media & Entertainment Network by TiE Mumbai). MENT is an initiative, by TiE Mumbai, towards enabling and empowering entrepreneurs in the media & entertainment industry across Mumbai & Pune. It aims to provide a platform for entrepreneurs, founders and CXOs in the industry to interact with peers, solve common founder challenges and facilitate curated networking.
Linux Memory Management with CMA (Contiguous Memory Allocator)Pankaj Suryawanshi
Fundamentals of Linux Memory Management and CMA (Contiguous Memory Allocator) In Linux.
Virtual Memory, Physical Memory, Swap Space, DMA, IOMMU, Paging, Segmentation, TLB, Hugepages, Ion google memory manager
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Netronome's half-day tutorial on host data plane acceleration at ACM SIGCOMM 2018 introduced attendees to models for host data plane acceleration and provided an in-depth understanding of SmartNIC deployment models at hyperscale cloud vendors and telecom service providers.
Presenter Bios
Jakub Kicinski is a long term Linux kernel contributor, who has been leading the kernel team at Netronome for the last two years. Jakub’s major contributions include the creation of BPF hardware offload mechanisms in the kernel and bpftool user space utility, as well as work on the Linux kernel side of OVS offload.
David Beckett is a Software Engineer at Netronome with a strong technical background of computer networks including academic research with DDoS. David has expertise in the areas of Linux architecture and computer programming. David has a Masters Degree in Electrical, Electronic Engineering at Queen’s University Belfast and continues as a PhD student studying Emerging Application Layer DDoS threats.
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
LinuxCon 2015 Linux Kernel Networking WalkthroughThomas Graf
This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
#MENT: Online & Mobile Trends - India & Worldwide by Gurmit Singh, MD Yahoo I...tiemumbai
Gurmit Singh, MD - Yahoo India shares his views about the Online & Mobile Trends in India and around the World. Also a part of the presentation shows the transformation & evolution of Yahoo.
This session was part of MENT (Media & Entertainment Network by TiE Mumbai). MENT is an initiative, by TiE Mumbai, towards enabling and empowering entrepreneurs in the media & entertainment industry across Mumbai & Pune. It aims to provide a platform for entrepreneurs, founders and CXOs in the industry to interact with peers, solve common founder challenges and facilitate curated networking.
Software libre y modelos de programación en la investigación con supercomputa...Andrés Gómez
Presentación hecha en el II Congreso de Software Libre para Educación en julio 2013 en donde se presentan los resultados de una encuesta realizada a los usuarios del CESGA sobre las necesidades computacionales y las herramientas de programación utilizadas.
Klink-2: integrating multiple web sources to generate semantic topic networksFrancesco Osborne
ISWC 2015 research paper: http://oro.open.ac.uk/43793/1/ISWC2015_CR.pdf
Abstract:
The amount of scholarly data available on the web is steadily increasing, enabling different types of analytics which can provide important insights into the research activity. In order to make sense of and explore this large-scale body of knowledge we need an accurate, comprehensive and up-to-date ontology of research topics. Unfortunately, human crafted classifications do not satisfy these criteria, as they evolve too slowly and tend to be too coarse-grained. Current automated methods for generating ontologies of research areas also present a number of limitations, such as: i) they do not consider the rich amount of indirect statistical and semantic relationships, which can help to understand the relation between two topics – e.g., the fact that two research areas are associated with a similar set of venues or technologies; ii) they do not distinguish between different kinds of hierarchical relationships; and iii) they are not able to handle effectively ambiguous topics characterized by a noisy set of relationships. In this paper we present Klink-2, a novel approach which improves on our earlier work on automatic generation of semantic topic networks and addresses the aforementioned limitations by taking advantage of a variety of knowledge sources available on the web. In particular, Klink-2 analyses networks of research entities (including papers, authors, venues, and technologies) to infer three kinds of semantic relationships between topics. It also identifies ambiguous keywords (e.g., “ontology”) and separates them into the appropriate distinct topics – e.g., “ontology/philosophy” vs. “ontology/semantic web”. Our experimental evaluation shows that the ability of Klink-2 to integrate a high number of data sources and to generate topics with accurate contextual meaning yields significant improvements over other algorithms in terms of both precision and recall.
Quality is a mainstream issue for Websites and Web-based systems. Analysts, designers and developers can choose from a variety of models and methods to design and evaluate quality Websites. However, our experience, both as users and professionals, is too often disappointing. Besides usability, the most known of the factors contributing to a Website’s quality, there are a large numbers of other features to take into account. Moreover, these features go beyond the scope of Computer Science and Information Systems. A systemic and systematic approach to Website quality evaluation has to take into account (1) the role of the Website for the company or organization and (2) a collection of requirements arising from the usually large number of different users of the Website. This presentation illustrates the application of an approach in which a Website’s quality goals are described in terms of the elimination of quality gaps. The quality gaps are derived from a highly successful model for the quality of services widely applied in Economics and Management. The paper focuses on the need for a flexible evaluation process starting from the identification of the quality gaps. The result is a modular process including an initial set of guidelines extracted from projects run on more than fifteen years for Websites in diverse sectors.
Situación de las iniciativas de Open Data internacionales (y algunas recomen...Oscar Corcho
Presentación sobre iniciativas de Open Data Internacionales y nacionales, realizada en el contexto del Curso de Verano de la Universidad de Extremadura "BigData y Machine Learning junto a fuentes de datos abiertos para especializar el sector agroganadero", el 25/09/2018
Las TIC y su aporte al Desarrollo de la Sociedad de la InformacionProGobernabilidad Perú
Material utlizado para el Seminario Internacional de Tecnologías de Información y Gobierno Electrónico -SITIGE 2015 realizado los días 11, 12 y 13 de febrero de 2015.
Expositor: Richard Gagne, especialista canadiense en Tecnologías de Información de ProGobernabilidad
En los últimos años Esri está apostando por transformar ArcGIS en una plataforma abierta e interoperable, de esta forma cualquiera que lo desee podrá no sólo extender la funcionalidad de la plataforma y sus componentes si no también consumir y generar algunos de los formatos creados por Esri. En este seminario hablaremos de los seis pilares en los que se sostiene la “Open Vision” de Esri: estándares, datos y código abierto, interoperabilidad de datos y productos, APIs y especificaciones abiertas.
Así pues, a lo largo del seminario mostraremos qué normas y estándares soporta Esri, qué producto se debe emplear para sea conforme a estándares, qué componentes podemos extender y cómo hacerlo, etc. Además de éstas, también responderemos a otras preguntas relacionadas con geoservicios, estándares, OGC o INSPIRE. Esri-leaftlet, KoopJS, GIS-Tools for Hadoop, Esri Geometry API for Java, LERC, ArcGIS Editor for OpenStreetMap.
Presentació a càrrec de Maria Isabel Gandia, cap de Comunicacions del CSUC, mostrada a la 17a edició de la reunió ESNOG/GORE celebrada els dies 12 i 13 de maig al CSUC.
Similar a Poptrie: A Compressed Trie with Population Count for Fast and Scalable Software IP Routing Table Lookup (19)
control de emisiones de gases contaminantes.pptxjesusbellido2
en el siguiente documento s epodra apreciar los gases que emiten los vehiculos y sus consecuencias tambien se podra apreciar las normas euro cino y las normas euro seis
Los emprendimientos socio productivos generan bienes y servicios en los territorios, con el propósito de que los procesos de producción activen al mercado y facilite el desarrollo personal mediante la integración social de los agentes sociales excluidos.
36. LockVfree)Update)Procedure
Internal node
1 1 0 0
vector base0
N[2048] N[2049] N[2050]
31290 0 0 1
leafvec base1
2048
N
L
1
L[3129]
base1
1024
Node array (X) Top of the marked internal nodes
Parent to be replaced
Figure 6: The lock-free update procedure to th
internal node.
Updated prefix
Updated leaves
Replace these
internal nodes
Top of the marked triangles