This document provides an overview of kernel debugging on Solaris systems using the modular debugger Mdb and dynamic tracing framework DTrace. It discusses debugging live kernels with Mdb, analyzing system crash dumps with Mdb, and using DTrace to monitor the kernel at runtime by enabling probes published by different providers. The document outlines the key tools, techniques, and challenges involved in kernel debugging and crash analysis on Solaris.
Falco is an open source runtime security monitor for containers that detects anomalous activity using rules. It builds on Sysdig by instrumenting the kernel and collecting system calls and events. Falco rules define suspicious behaviors and integrate signals from the kernel, containers, and Kubernetes. Falco detects threats by matching patterns in real time and alerts on suspicious activity, helping operators enforce policies and spot abnormal behavior.
This document summarizes how to deploy a full Prometheus monitoring stack using Juju charms. It first introduces Juju and how it can orchestrate services. It then describes how Prometheus and Telegraf are deployed as charms and related to collect metrics from other services like HAProxy, MediaWiki, MariaDB and Memcached. The Prometheus and Telegraf charms, along with other supporting charms, are open source and available for deploying a complete monitoring stack.
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg
This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
Patching: answers to questions you probably were afraid to ask about oracle s...DATA SECURITY SOLUTIONS
This document discusses Oracle database security updates and patching. It begins by addressing common questions and concerns about patching, such as whether patching is needed if behind a firewall, how often to apply proactive bugfixes, and potential issues like downtime and breakage. It then covers the different types of proactive patches, how to apply them to reduce downtime, and resources for further information. The conclusion emphasizes the importance of reading the patch README for installation instructions and details.
The document discusses various topics related to Rails deployment in the enterprise, including recommendations to use Ruby on Rails, MongoDB, and deployment tools like Capistrano and Puppet. It also covers some performance considerations and pitfalls to be aware of when using Rails.
Web technologies are evolving at such a frenetic pace that it becomes almost mandatory to learn on your own. A lot of us still depend on other people to do this learning for us, and we tend to use their answers to solve our everyday problems.
Inconsistent implementations, rapidly evolving specs, questionable performance impacts and maintenance implications mean we cannot always depend on others for answers but must involve ourselves actively in the process of developing specifications for new Web technologies. But how do we go about it?
There are some simple rituals we can all do, which can have us be better-informed and also better inform the people and groups who are most directly involved in the development of new Web technologies.
This document provides an overview of kernel debugging on Solaris systems using the modular debugger Mdb and dynamic tracing framework DTrace. It discusses debugging live kernels with Mdb, analyzing system crash dumps with Mdb, and using DTrace to monitor the kernel at runtime by enabling probes published by different providers. The document outlines the key tools, techniques, and challenges involved in kernel debugging and crash analysis on Solaris.
Falco is an open source runtime security monitor for containers that detects anomalous activity using rules. It builds on Sysdig by instrumenting the kernel and collecting system calls and events. Falco rules define suspicious behaviors and integrate signals from the kernel, containers, and Kubernetes. Falco detects threats by matching patterns in real time and alerts on suspicious activity, helping operators enforce policies and spot abnormal behavior.
This document summarizes how to deploy a full Prometheus monitoring stack using Juju charms. It first introduces Juju and how it can orchestrate services. It then describes how Prometheus and Telegraf are deployed as charms and related to collect metrics from other services like HAProxy, MediaWiki, MariaDB and Memcached. The Prometheus and Telegraf charms, along with other supporting charms, are open source and available for deploying a complete monitoring stack.
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg
This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
Patching: answers to questions you probably were afraid to ask about oracle s...DATA SECURITY SOLUTIONS
This document discusses Oracle database security updates and patching. It begins by addressing common questions and concerns about patching, such as whether patching is needed if behind a firewall, how often to apply proactive bugfixes, and potential issues like downtime and breakage. It then covers the different types of proactive patches, how to apply them to reduce downtime, and resources for further information. The conclusion emphasizes the importance of reading the patch README for installation instructions and details.
The document discusses various topics related to Rails deployment in the enterprise, including recommendations to use Ruby on Rails, MongoDB, and deployment tools like Capistrano and Puppet. It also covers some performance considerations and pitfalls to be aware of when using Rails.
Web technologies are evolving at such a frenetic pace that it becomes almost mandatory to learn on your own. A lot of us still depend on other people to do this learning for us, and we tend to use their answers to solve our everyday problems.
Inconsistent implementations, rapidly evolving specs, questionable performance impacts and maintenance implications mean we cannot always depend on others for answers but must involve ourselves actively in the process of developing specifications for new Web technologies. But how do we go about it?
There are some simple rituals we can all do, which can have us be better-informed and also better inform the people and groups who are most directly involved in the development of new Web technologies.
Talk for YOW! by Brendan Gregg. "Systems performance studies the performance of computing systems, including all physical components and the full software stack to help you find performance wins for your application and kernel. However, most of us are not performance or kernel engineers, and have limited time to study this topic. This talk summarizes the topic for everyone, touring six important areas: observability tools, methodologies, benchmarking, profiling, tracing, and tuning. Included are recipes for Linux performance analysis and tuning (using vmstat, mpstat, iostat, etc), overviews of complex areas including profiling (perf_events) and tracing (ftrace, bcc/BPF, and bpftrace/BPF), advice about what is and isn't important to learn, and case studies to see how it is applied. This talk is aimed at everyone: developers, operations, sysadmins, etc, and in any environment running Linux, bare metal or the cloud.
"
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
bcc/BPF tools - Strategy, current tools, future challengesIO Visor Project
Brendan Gregg discusses the current state and future potential of BPF and BCC tools for observability in Linux. He outlines 18 areas where BPF support has progressed and 16 areas still needing work. Gregg also discusses challenges like dynamic tracing stability, overhead, ease of coding, and developing visualizations. He proposes finishing ports of his old DTrace tools and links to resources on BPF, BCC, and flame graphs.
This document summarizes a presentation about tuning parallel code on Solaris. It discusses:
1) Using tools like DTrace, prstat, and vmstat to analyze performance issues like thread scheduling and I/O problems in parallel applications on Solaris.
2) Two examples of using DTrace to analyze thread scheduling and troubleshoot I/O performance problems in a virtualized Windows server.
3) How the examples demonstrated using DTrace to identify unbalanced thread scheduling and discover that a domain controller was disabling disk write caching, slowing performance.
This document contains the slides from a presentation given by WonoKaerun at the Indonesian Security Conference 2011 in Palembang. The presentation introduces rootkits and techniques for hiding malware at the kernel level on Linux systems. It covers topics like loadable kernel modules, interrupt descriptor table hooking, virtual file system hacking, page fault handler hijacking, debugging register abuse, and kernel instrumentation patching. The goal is to evade detection by security solutions by gaining control of the kernel before anti-rootkit defenses can activate. Throughout, the document emphasizes the cat-and-mouse nature of offensive and defensive security research.
Performance Wins with BPF: Getting StartedBrendan Gregg
Keynote by Brendan Gregg for the eBPF summit, 2020. How to get started finding performance wins using the BPF (eBPF) technology. This short talk covers the quickest and easiest way to find performance wins using BPF observability tools on Linux.
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
Linux kernel tracing superpowers in the cloudAndrea Righi
The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
Here is a bpftrace program to measure scheduler latency for ICMP echo requests:
#!/usr/local/bin/bpftrace
kprobe:icmp_send {
@start[tid] = nsecs;
}
kprobe:__netif_receive_skb_core {
@diff[tid] = hist(nsecs - @start[tid]);
delete(@start[tid]);
}
END {
print(@diff);
clear(@diff);
}
This traces the time between the icmp_send kernel function (when the packet is queued for transmit) and the __netif_receive_skb_core function (when the response packet is received). The
OSSNA 2017 Performance Analysis Superpowers with Linux BPFBrendan Gregg
Talk by Brendan Gregg for OSSNA 2017. "Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will be a dive deep on these new tracing, observability, and debugging capabilities, which sooner or later will be available to everyone who uses Linux. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
This document discusses Brendan Gregg's opinions on various tracing tools including sysdig, perf, ftrace, eBPF, bpftrace, and BPF perf tools. It provides a table comparing the scope, capability, and ease of use of these tools. It then gives an example of using BPF perf tools to analyze readahead performance. Finally, it outlines desired additions to tracing capabilities and BPF helpers as well as challenges in areas like function tracing without frame pointers.
Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
Performance Analysis Tools for Linux Kernellcplcp1
Perf is a collection of Linux kernel tools for performance monitoring and profiling. It provides sampling and profiling of the system to analyze performance bottlenecks. Perf supports hardware events from the CPU performance counters, software events from the kernel, and tracepoint events from the kernel and loaded modules. It offers tools like perf record to sample events and store them, perf report to analyze stored samples, and perf trace to trace system events in real-time.
Kernel Recipes 2017: Using Linux perf at NetflixBrendan Gregg
This document discusses using the Linux perf profiling tool at Netflix. It begins with an overview of why Netflix needs Linux profiling to understand CPU usage quickly and completely. It then provides an introduction to the perf tool, covering its basic workflow and commands. The document discusses profiling CPU usage with perf, including potential issues like JIT runtimes and missing symbols. It provides several examples of perf commands for listing, counting, and recording events. The overall summary is that perf allows Netflix to quickly and accurately profile CPU usage across the entire software stack, from applications to libraries to the kernel, to optimize performance.
BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Mechanisms and tools of development and monitoring in Linux Kernel
A brief presentation about the tools and mechanisms about development for the Linux Kernel to help understanding of what is required.
Tools that create an execution profile or provide instrumentation through static or dynamic methods, the Linux Kernel code, will be presented.
Also discussed will be the GDB debugger and how through a remote virtual serial connection to a virtual machine it can be used to debug a live Kernel and Linux Kernel modules. Also demonstrated will be how a deeper understanding of the code can be attained by attaching the memory locations used by the Kernel module to the GDB session.
Lastly, some of the Kernel execution contexts, such as, interrupts, deferrable work, context, etc. are presented.
Spying on the Linux kernel for fun and profitAndrea Righi
Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
This document provides an overview of performance analysis tools for Linux systems. It describes Brendan Gregg's background and work analyzing performance at Netflix. It then discusses different types of tools, including observability tools to monitor systems, benchmarking tools to test performance, and tuning tools to optimize systems. A number of command line monitoring tools are outlined, such as vmstat, iostat, mpstat, and netstat, as well as more advanced tools like strace and tcpdump.
The document discusses Linux/Unix interview questions and answers. It covers topics such as the GRUB bootloader, the Linux boot process, user profile files, changing the default runlevel, displaying user information with the finger command, inode numbers, increasing disk read performance, password expiration times, locking user passwords, default shells, user attributes defined in /etc/login.defs, changing the system's authentication method, modifying file attributes with chattr, network interface configuration files, changing network interface settings, the DNS configuration file, exporting NFS directories, checking open ports, soft vs hard links, setting expired passwords, restricting file insertion, displaying or killing processes accessing files/folders, killing all processes for a user, daily system analysis reports
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
Talk for YOW! by Brendan Gregg. "Systems performance studies the performance of computing systems, including all physical components and the full software stack to help you find performance wins for your application and kernel. However, most of us are not performance or kernel engineers, and have limited time to study this topic. This talk summarizes the topic for everyone, touring six important areas: observability tools, methodologies, benchmarking, profiling, tracing, and tuning. Included are recipes for Linux performance analysis and tuning (using vmstat, mpstat, iostat, etc), overviews of complex areas including profiling (perf_events) and tracing (ftrace, bcc/BPF, and bpftrace/BPF), advice about what is and isn't important to learn, and case studies to see how it is applied. This talk is aimed at everyone: developers, operations, sysadmins, etc, and in any environment running Linux, bare metal or the cloud.
"
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
bcc/BPF tools - Strategy, current tools, future challengesIO Visor Project
Brendan Gregg discusses the current state and future potential of BPF and BCC tools for observability in Linux. He outlines 18 areas where BPF support has progressed and 16 areas still needing work. Gregg also discusses challenges like dynamic tracing stability, overhead, ease of coding, and developing visualizations. He proposes finishing ports of his old DTrace tools and links to resources on BPF, BCC, and flame graphs.
This document summarizes a presentation about tuning parallel code on Solaris. It discusses:
1) Using tools like DTrace, prstat, and vmstat to analyze performance issues like thread scheduling and I/O problems in parallel applications on Solaris.
2) Two examples of using DTrace to analyze thread scheduling and troubleshoot I/O performance problems in a virtualized Windows server.
3) How the examples demonstrated using DTrace to identify unbalanced thread scheduling and discover that a domain controller was disabling disk write caching, slowing performance.
This document contains the slides from a presentation given by WonoKaerun at the Indonesian Security Conference 2011 in Palembang. The presentation introduces rootkits and techniques for hiding malware at the kernel level on Linux systems. It covers topics like loadable kernel modules, interrupt descriptor table hooking, virtual file system hacking, page fault handler hijacking, debugging register abuse, and kernel instrumentation patching. The goal is to evade detection by security solutions by gaining control of the kernel before anti-rootkit defenses can activate. Throughout, the document emphasizes the cat-and-mouse nature of offensive and defensive security research.
Performance Wins with BPF: Getting StartedBrendan Gregg
Keynote by Brendan Gregg for the eBPF summit, 2020. How to get started finding performance wins using the BPF (eBPF) technology. This short talk covers the quickest and easiest way to find performance wins using BPF observability tools on Linux.
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
Linux kernel tracing superpowers in the cloudAndrea Righi
The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
Here is a bpftrace program to measure scheduler latency for ICMP echo requests:
#!/usr/local/bin/bpftrace
kprobe:icmp_send {
@start[tid] = nsecs;
}
kprobe:__netif_receive_skb_core {
@diff[tid] = hist(nsecs - @start[tid]);
delete(@start[tid]);
}
END {
print(@diff);
clear(@diff);
}
This traces the time between the icmp_send kernel function (when the packet is queued for transmit) and the __netif_receive_skb_core function (when the response packet is received). The
OSSNA 2017 Performance Analysis Superpowers with Linux BPFBrendan Gregg
Talk by Brendan Gregg for OSSNA 2017. "Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will be a dive deep on these new tracing, observability, and debugging capabilities, which sooner or later will be available to everyone who uses Linux. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
This document discusses Brendan Gregg's opinions on various tracing tools including sysdig, perf, ftrace, eBPF, bpftrace, and BPF perf tools. It provides a table comparing the scope, capability, and ease of use of these tools. It then gives an example of using BPF perf tools to analyze readahead performance. Finally, it outlines desired additions to tracing capabilities and BPF helpers as well as challenges in areas like function tracing without frame pointers.
Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
Performance Analysis Tools for Linux Kernellcplcp1
Perf is a collection of Linux kernel tools for performance monitoring and profiling. It provides sampling and profiling of the system to analyze performance bottlenecks. Perf supports hardware events from the CPU performance counters, software events from the kernel, and tracepoint events from the kernel and loaded modules. It offers tools like perf record to sample events and store them, perf report to analyze stored samples, and perf trace to trace system events in real-time.
Kernel Recipes 2017: Using Linux perf at NetflixBrendan Gregg
This document discusses using the Linux perf profiling tool at Netflix. It begins with an overview of why Netflix needs Linux profiling to understand CPU usage quickly and completely. It then provides an introduction to the perf tool, covering its basic workflow and commands. The document discusses profiling CPU usage with perf, including potential issues like JIT runtimes and missing symbols. It provides several examples of perf commands for listing, counting, and recording events. The overall summary is that perf allows Netflix to quickly and accurately profile CPU usage across the entire software stack, from applications to libraries to the kernel, to optimize performance.
BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Mechanisms and tools of development and monitoring in Linux Kernel
A brief presentation about the tools and mechanisms about development for the Linux Kernel to help understanding of what is required.
Tools that create an execution profile or provide instrumentation through static or dynamic methods, the Linux Kernel code, will be presented.
Also discussed will be the GDB debugger and how through a remote virtual serial connection to a virtual machine it can be used to debug a live Kernel and Linux Kernel modules. Also demonstrated will be how a deeper understanding of the code can be attained by attaching the memory locations used by the Kernel module to the GDB session.
Lastly, some of the Kernel execution contexts, such as, interrupts, deferrable work, context, etc. are presented.
Spying on the Linux kernel for fun and profitAndrea Righi
Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
This document provides an overview of performance analysis tools for Linux systems. It describes Brendan Gregg's background and work analyzing performance at Netflix. It then discusses different types of tools, including observability tools to monitor systems, benchmarking tools to test performance, and tuning tools to optimize systems. A number of command line monitoring tools are outlined, such as vmstat, iostat, mpstat, and netstat, as well as more advanced tools like strace and tcpdump.
The document discusses Linux/Unix interview questions and answers. It covers topics such as the GRUB bootloader, the Linux boot process, user profile files, changing the default runlevel, displaying user information with the finger command, inode numbers, increasing disk read performance, password expiration times, locking user passwords, default shells, user attributes defined in /etc/login.defs, changing the system's authentication method, modifying file attributes with chattr, network interface configuration files, changing network interface settings, the DNS configuration file, exporting NFS directories, checking open ports, soft vs hard links, setting expired passwords, restricting file insertion, displaying or killing processes accessing files/folders, killing all processes for a user, daily system analysis reports
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
You have a system with an advanced programmatic tracer: do you know what to do with it? Brendan has used numerous tracers in production environments, and has published hundreds of tracing-based tools. In this talk he will share tips and know-how for creating CLI tracing tools and GUI visualizations, to solve real problems effectively. Programmatic tracing is an amazing superpower, and this talk will show you how to wield it!
Video: https://www.youtube.com/watch?v=uibLwoVKjec . Talk by Brendan Gregg for Sysdig CCWFS 2016. Abstract:
"You have a system with an advanced programmatic tracer: do you know what to do with it? Brendan has used numerous tracers in production environments, and has published hundreds of tracing-based tools. In this talk he will share tips and know-how for creating CLI tracing tools and GUI visualizations, to solve real problems effectively. Programmatic tracing is an amazing superpower, and this talk will show you how to wield it!"
This document provides an overview of shell scripting, cron, and atd for automating tasks in Linux. It discusses using shell scripts to execute a series of commands from a file, as well as topics like shebangs, variables, and conditionals. Cron allows scheduling commands to run on a time-based schedule using crontab files, while atd can run one-off commands at a specific time. The document provides examples and explanations of common cron and atd utilities.
This document discusses Docker concepts and implementation in Chinese. It covers Linux kernel namespaces, seccomp, cgroups, LXC, and Docker. Namespaces isolate processes and resources between containers. Cgroups control resource limits and prioritization. LXC provides containerization tools while Docker builds on these concepts and provides an easy-to-use interface for containers. The document also provides examples of using namespaces, cgroups, LXC, and building Docker images.
This document provides solutions to common Linux commands and tasks. It covers topics such as environment setting, hardware and system specifications, file editing and compression, networking, performance monitoring, package management with RPM, and multimedia. Solutions are provided for tasks like changing the startup runlevel, monitoring swap size, editing files, getting the network IP and registering the hostname, and burning discs.
This document provides an introduction to basic Linux commands. It begins with an outline covering Unix systems, file transfer methods, basic commands like ls, mv and cp, users and permissions, learning about your system, and handy commands for bioinformatics. Specific commands and concepts are explained over 6 sections, with examples of how to use commands like grep, cut, sort and awk for bioinformatics tasks. The document encourages practicing these skills and provides additional resources for learning Bash oneliners. It concludes by posing a question about counting genes from a Drosophila genome annotation file.
Docker and friends at Linux Days 2014 in Praguetomasbart
Docker allows deploying applications easily across various environments by packaging them along with their dependencies into standardized units called containers. It provides isolation and security while allowing higher density and lower overhead than virtual machines. Core OS and Mesos both integrate with Docker to deploy containers on clusters of machines for scalability and high availability.
This document provides an overview of shell scripting, cron, and atd for scheduling tasks on Linux systems. It discusses using shell scripts to automate commands, the cron daemon for executing commands on a schedule, and atd for running one-time jobs. Key points covered include using the shebang to directly execute shell scripts, crontab syntax for defining schedules, and commands like crond, at, atq and atrm for managing cron and at jobs. The document is intended as a learning resource for system administration tasks using scheduling utilities in Linux.
The document is a reference guide for Unix/Linux commands and tasks useful for system administration and advanced users. It contains over 20 sections covering topics like the system, processes, file system, networking, encryption, version control and programming. Each section provides concise explanations of relevant commands and how to perform common tasks in that area. The reader is expected to have a working knowledge of the Unix environment.
Introduction to Docker (as presented at December 2013 Global Hackathon)Jérôme Petazzoni
Not on board of the Docker ship yet? This presentation will get you up to speed, and explain everything you want to know about Linux Containers and Docker, including the new features of the latest 0.7 version (which brings support for all Linux distros and kernels).
Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek Jana treek
The document is a reference guide for Unix/Linux commands, organized into sections covering topics such as the system, processes, file system, network, and programming. It provides concise explanations of commands and tasks for advanced users, with the goal of being a practical toolbox reference. Sections include commands for viewing hardware and software information, monitoring system performance and activity, managing users and groups, and configuring process limits.
CONFidence 2017: Escaping the (sand)box: The promises and pitfalls of modern ...PROIDEA
Users of modern Linux containerization technologies are frequently at loss with what kind of security guarantees are delivered by tools they use. Typical questions range from Can these be used to isolate software with known security shortcomings and rich history of security vulnerabilities? to even Can I used such technique to isolate user-generated and potentially hostile assembler payloads?
Modern Linux OS code-base as well as independent authors provide a plethora of options for those who desire to make sure that their computational loads are solidly confined. Potential users can choose from solutions ranging from Docker-like confinement projects, through Xen hypervisors, seccomp-bpf and ptrace-based sandboxes, to isolation frameworks based on hardware virtualization (e.g. KVM).
The talk will discuss available today techniques, with focus on (frequently overstated) promises regarding their strength. In the end, as they say: “Many speed bumps don’t make a wall
Slackware Demystified provides an overview of the Slackware Linux distribution. It discusses Slackware's philosophy of keeping things simple and sticking close to upstream. It describes Slackware's init system, configuration files, package structure, and community support. The presentation emphasizes Slackware's minimalist approach and encourages learning through documentation rather than abstracted interfaces.
This document provides an introduction to Linux, including its history and architecture. It describes Linux's origins from Unix in the 1960s and the development of the Linux kernel by Linus Torvalds in 1991. It outlines the key components of a Linux system, including the kernel, shell, file system, processes, networking, and desktop environments. It also discusses booting a Linux system and provides resources for learning more about Linux distributions and building your own operating system.
Linux Server Deep Dives (DrupalCon Amsterdam)Amin Astaneh
Over the past few years the Linux kernel has gained features that allow us to learn more about what's really happening on our servers and the applications that run on them.
This talk will explore how these new features, particularly perf_events and ebpf, enable us to answer questions about what a Drupal site is doing in real time beyond what the standard logs, server performance tools, and even strace will reveal. Attendees will be provided a brief introduction to example uses of these tools to diagnose performance problems.
This talk is intended for attendees that are familiar with Linux, the command line, and have used host observability tools in the past (top, netstat, etc).
This document discusses Linux accounting and monitoring user activity. It begins with an overview of the yum and dnf package managers and how they can be used to install the psacct or acct packages for monitoring user activity. It then covers various commands provided by psacct/acct like ac, lastcomm, sa to view user login times, previously executed commands, and account activity summaries. The document also provides overviews of Kerberos for authentication, LDAP for user information storage, and lists some common system utility commands.
Similar a Linux Capabilities - eng - v2.1.5, compact (20)
Consistent toolbox talks are critical for maintaining workplace safety, as they provide regular opportunities to address specific hazards and reinforce safe practices.
These brief, focused sessions ensure that safety is a continual conversation rather than a one-time event, which helps keep safety protocols fresh in employees' minds. Studies have shown that shorter, more frequent training sessions are more effective for retention and behavior change compared to longer, infrequent sessions.
Engaging workers regularly, toolbox talks promote a culture of safety, empower employees to voice concerns, and ultimately reduce the likelihood of accidents and injuries on site.
The traditional method of conducting safety talks with paper documents and lengthy meetings is not only time-consuming but also less effective. Manual tracking of attendance and compliance is prone to errors and inconsistencies, leading to gaps in safety communication and potential non-compliance with OSHA regulations. Switching to a digital solution like Safelyio offers significant advantages.
Safelyio automates the delivery and documentation of safety talks, ensuring consistency and accessibility. The microlearning approach breaks down complex safety protocols into manageable, bite-sized pieces, making it easier for employees to absorb and retain information.
This method minimizes disruptions to work schedules, eliminates the hassle of paperwork, and ensures that all safety communications are tracked and recorded accurately. Ultimately, using a digital platform like Safelyio enhances engagement, compliance, and overall safety performance on site. https://safelyio.com/
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...XfilesPro
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
Boost Your Savings with These Money Management AppsJhone kinadey
A money management app can transform your financial life by tracking expenses, creating budgets, and setting financial goals. These apps offer features like real-time expense tracking, bill reminders, and personalized insights to help you save and manage money effectively. With a user-friendly interface, they simplify financial planning, making it easier to stay on top of your finances and achieve long-term financial stability.
Penify - Let AI do the Documentation, you write the Code.KrishnaveniMohan1
Penify automates the software documentation process for Git repositories. Every time a code modification is merged into "main", Penify uses a Large Language Model to generate documentation for the updated code. This automation covers multiple documentation layers, including InCode Documentation, API Documentation, Architectural Documentation, and PR documentation, each designed to improve different aspects of the development process. By taking over the entire documentation process, Penify tackles the common problem of documentation becoming outdated as the code evolves.
https://www.penify.dev/
Stork Product Overview: An AI-Powered Autonomous Delivery FleetVince Scalabrino
Imagine a world where instead of blue and brown trucks dropping parcels on our porches, a buzzing drove of drones delivered our goods. Now imagine those drones are controlled by 3 purpose-built AI designed to ensure all packages were delivered as quickly and as economically as possible That's what Stork is all about.
14 th Edition of International conference on computer visionShulagnaSarkar2
About the event
14th Edition of International conference on computer vision
Computer conferences organized by ScienceFather group. ScienceFather takes the privilege to invite speakers participants students delegates and exhibitors from across the globe to its International Conference on computer conferences to be held in the Various Beautiful cites of the world. computer conferences are a discussion of common Inventions-related issues and additionally trade information share proof thoughts and insight into advanced developments in the science inventions service system. New technology may create many materials and devices with a vast range of applications such as in Science medicine electronics biomaterials energy production and consumer products.
Nomination are Open!! Don't Miss it
Visit: computer.scifat.com
Award Nomination: https://x-i.me/ishnom
Conference Submission: https://x-i.me/anicon
For Enquiry: Computer@scifat.com
Superpower Your Apache Kafka Applications Development with Complementary Open...Paul Brebner
Kafka Summit talk (Bangalore, India, May 2, 2024, https://events.bizzabo.com/573863/agenda/session/1300469 )
Many Apache Kafka use cases take advantage of Kafka’s ability to integrate multiple heterogeneous systems for stream processing and real-time machine learning scenarios. But Kafka also exists in a rich ecosystem of related but complementary stream processing technologies and tools, particularly from the open-source community. In this talk, we’ll take you on a tour of a selection of complementary tools that can make Kafka even more powerful. We’ll focus on tools for stream processing and querying, streaming machine learning, stream visibility and observation, stream meta-data, stream visualisation, stream development including testing and the use of Generative AI and LLMs, and stream performance and scalability. By the end you will have a good idea of the types of Kafka “superhero” tools that exist, which are my favourites (and what superpowers they have), and how they combine to save your Kafka applications development universe from swamploads of data stagnation monsters!
What to do when you have a perfect model for your software but you are constrained by an imperfect business model?
This talk explores the challenges of bringing modelling rigour to the business and strategy levels, and talking to your non-technical counterparts in the process.
Transforming Product Development using OnePlan To Boost Efficiency and Innova...OnePlan Solutions
Ready to overcome challenges and drive innovation in your organization? Join us in our upcoming webinar where we discuss how to combat resource limitations, scope creep, and the difficulties of aligning your projects with strategic goals. Discover how OnePlan can revolutionize your product development processes, helping your team to innovate faster, manage resources more effectively, and deliver exceptional results.
Building API data products on top of your real-time data infrastructureconfluent
This talk and live demonstration will examine how Confluent and Gravitee.io integrate to unlock value from streaming data through API products.
You will learn how data owners and API providers can document, secure data products on top of Confluent brokers, including schema validation, topic routing and message filtering.
You will also see how data and API consumers can discover and subscribe to products in a developer portal, as well as how they can integrate with Confluent topics through protocols like REST, Websockets, Server-sent Events and Webhooks.
Whether you want to monetize your real-time data, enable new integrations with partners, or provide self-service access to topics through various protocols, this webinar is for you!
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...The Third Creative Media
"Navigating Invideo: A Comprehensive Guide" is an essential resource for anyone looking to master Invideo, an AI-powered video creation tool. This guide provides step-by-step instructions, helpful tips, and comparisons with other AI video creators. Whether you're a beginner or an experienced video editor, you'll find valuable insights to enhance your video projects and bring your creative ideas to life.
The Comprehensive Guide to Validating Audio-Visual Performances.pdfkalichargn70th171
Ensuring the optimal performance of your audio-visual (AV) equipment is crucial for delivering exceptional experiences. AV performance validation is a critical process that verifies the quality and functionality of your AV setup. Whether you're a content creator, a business conducting webinars, or a homeowner creating a home theater, validating your AV performance is essential.
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...Luigi Fugaro
Vector databases are redefining data handling, enabling semantic searches across text, images, and audio encoded as vectors.
Redis OM for Java simplifies this innovative approach, making it accessible even for those new to vector data.
This presentation explores the cutting-edge features of vector search and semantic caching in Java, highlighting the Redis OM library through a demonstration application.
Redis OM has evolved to embrace the transformative world of vector database technology, now supporting Redis vector search and seamless integration with OpenAI, Hugging Face, LangChain, and LlamaIndex. This talk highlights the latest advancements in Redis OM, focusing on how it simplifies the complex process of vector indexing, data modeling, and querying for AI-powered applications. We will explore the new capabilities of Redis OM, including intuitive vector search interfaces and semantic caching, which reduce the overhead of large language model (LLM) calls.
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...kalichargn70th171
Visual testing plays a vital role in ensuring that software products meet the aesthetic requirements specified by clients in functional and non-functional specifications. In today's highly competitive digital landscape, users expect a seamless and visually appealing online experience. Visual testing, also known as automated UI testing or visual regression testing, verifies the accuracy of the visual elements that users interact with.
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
Linux Capabilities - eng - v2.1.5, compact
1. Linux Capabilites
Table of Contents
Title Pages Title Pages
Table of Contents 1 Extracting and... 4
Copyright Notice 1 System Configuration 2
LC: what they are... 3 Capability Assignment 10
The ping case 4 Distro Status 7
What LC for ping? 3 Issues? 2
Managing LCs 4 Aknowledgments 1
More examples 10
3. Linux Capabilites:
what they are and do 1-3
● Started as an implementation of POSIX:
1003.1e (API), "Protection, Audit and
Control Interfaces"
1003.2c (Shell and Utilities), "Protection
and Control Interfaces"
● Sometimes called Linux POSIX Capabilities
● 1003.1e and 1003.2c were last revised in 1997
● In 1999 they were set as withdrawn drafts
● Linux has thus gone it's own way
4. Linux Capabilites:
what they are and do 2-3
Advantages:
● They allow delegation of super-user rights to
unprivileged processes, like suid bit
● They are recorded on the filesystem, like suid bit
● They do not work on a “everything or nothing”
basis, unlike suid
In short:
● they allow a process to do some selected things
with root rights, while not anything else
5. Linux Capabilites:
what they are and do 3-3
Disadvantages:
● Commands need code to be made LC-conscious
● Some FS do not support LC (NFS v.3, though squashfs,
tmpfs and f2fs now do):
[root@debian ~]# getcap /mnt/nfs/nfsserver/dumpdates
Failed to get capabilities of file `/mnt/nfs/nfsserver/dumpdates'
(Operation not supported)
[root@debian ~]#
● LCs are not dropped like SUID
The newer libcap-ng libs makes this easy
6. TThhee ppiinngg ccaassee 11--44
● A classic example: ping
● It needs superuser rights to send echo-request
ICMP packets
● You surely do not want to let anyone be able to
produce arbitrary ICMP packets!
● This is the traditional approach:
7. TThhee ppiinngg ccaassee 22--44
Starting situation:
[alessandro@ubuntu ~]$ lsb_release drc
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty
[alessandro@ubuntu ~]$ ll /bin/ping
rwsrxrx
1 root root 44178 mag 7 23:51 /bin/ping
[alessandro@ubuntu ~]$ ping c
3 routeadd.
net
PING routeadd.
net (195.182.210.166) 56(84) bytes of data.
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=1 ttl=55 time=46.8 ms
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=2 ttl=55 time=46.6 ms
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=3 ttl=55 time=45.5 ms
routeadd.
net ping statistics 3
packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 45.549/46.329/46.829/0.558 ms
[alessandro@ubuntu ~]$
8. TThhee ppiinngg ccaassee 33--44
● According to standards, the process rights are
those of the user who launched the exec
Not of the owner of the executable file!
● Unless the suid bit is set, that is.
rwsrxrx
1 root root 44178 mag 7 23:51 /bin/ping
9. TThhee ppiinngg ccaassee 44--44
Let's “sabotage” ping:
[root@ubuntu ~]# chmod us
/bin/ping
[root@ubuntu ~]# ll /bin/ping
rwxrxrx
1 root root 44178 mag 7 23:51 /bin/ping
[root@ubuntu ~]#
The aftermath:
[alessandro@ubuntu ~]$ ping c
3 routeadd.
net
ping: icmp open socket: Operation not permitted
[alessandro@ubuntu ~]$
If not otherwise stated, Ubuntu and Fedora it's the same.
10. WWhhaatt LLCC ffoorr ppiinngg?? 11--33
● We want both the power and the safety
● We want to be able to open ICMP sockets
without assuming full root privileges!
● How do Linux Capabilities help us out?
● RTFM, of course!
➢ man capabilities(7)
➢ Most up-to-date list & info:
linux/include/uapi/linux/capability.h
11. WWhhaatt LLCC ffoorr ppiinngg?? 22--33
We get many capabilities to choose from:
AUDIT_CONTROL AUDIT_WRITE BLOCK_SUSPEND CHOWN DAC_OVERRIDE DAC_READ_SEARCHC
AP_LEASE
FOWNER FSETID IPC_LOCK IPC_OWNER KILL LEASE
LINUX_IMMUTABLE MAC_ADMIN MAC_OVERRIDE MKNOD NET_ADMIN NET_BIND_SERVICE
This is what we need now
NET_RAW SETGID SETFCAP SETPCAP SETUID SYS_ADMIN
SYS_BOOT SYS_CHROOT SYS_MODULE SYS_NICE SYS_PACCT SYS_PTRACE
SYS_RAWIO SYS_RESOURCE SYS_TIME SYS_TTY_CONFIG SYSLOG WAKE_ALARM
12. WWhhaatt LLCC ffoorr ppiinngg?? 22--33
We get many capabilities to choose from:
AUDIT_CONTROL AUDIT_WRITE BLOCK_SUSPEND CHOWN DAC_OVERRIDE DAC_READ_SEARCHC
AP_LEASE
Leading CAP_ omitted everywhere
FOWNER FSETID IPC_LOCK IPC_OWNER KILL LEASE
LINUX_IMMUTABLE MAC_ADMIN MAC_OVERRIDE MKNOD NET_ADMIN NET_BIND_SERVICE
This is what we need now
NET_RAW SETGID SETFCAP SETPCAP SETUID SYS_ADMIN
SYS_BOOT SYS_CHROOT SYS_MODULE SYS_NICE SYS_PACCT SYS_PTRACE
SYS_RAWIO SYS_RESOURCE SYS_TIME SYS_TTY_CONFIG SYSLOG WAKE_ALARM
13. WWhhaatt LLCC ffoorr ppiinngg?? 33--33
Man page says:
CAP_NET_RAW
* use RAW and PACKET sockets;
* bind to any address for transparent proxying.
Sounds like what we need.
How shall we apply it?
14. MMaannaaggiinngg LLCCss 11--44
Capabilities can be handled whith commands:
➢ getcap, display file capabilities
➢ setcap, set file capabilities
➢ getpcaps, display process capabilities
➢ capsh, capability shell wrapper
Each capability can have these sets associated:
➢ p, process is permitted the capability
➢ e, capability is marked as effective
➢ i, capability is inherited after an execve()
15. MMaannaaggiinngg LLCCss 11--44
Capabilities can be handled whith commands:
➢ getcap, display file capabilities
➢ setcap, set file capabilities
➢ getpcaps, display process capabilities
➢ capsh, capability shell wrapper
Each capability can have these sets associated:
➢ p, process is permitted the capability
➢ e, capability is marked as effective
➢ i, capability is inherited after an execve()
This has become a bit
17. MMaannaaggiinngg LLCCss 22--44
Let's start using them:
[root@ubuntu ~]# chmod us
/bin/ping
[root@ubuntu ~]# ll /bin/ping
rwxrxrx
1 root root 44168 mag 7 23:51 /bin/ping*
[root@ubuntu ~]# getcap /bin/ping
[root@ubuntu ~]# setcap CAP_NET_RAW=ep /bin/ping
[root@ubuntu ~]# getcap /bin/ping
/bin/ping = cap_net_raw+ep
[root@ubuntu ~]#
Can use this notation
with setcap, too
18. MMaannaaggiinngg LLCCss 33--44
Does it work?
[alessandro@ubuntu ~]$ ping c
3 routeadd.
net
PING routeadd.
net (195.182.210.166) 56(84) bytes of data.
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=1 ttl=55 time=30.7 ms
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=2 ttl=55 time=30.9 ms
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=3 ttl=55 time=31.3 ms
routeadd.
net ping statistics 3
packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 30.709/30.988/31.350/0.304 ms
[alessandro@ubuntu ~]$
19. MMaannaaggiinngg LLCCss 44--44
Just a note: ping capabilities in Fedora 20:
[root@fedora ~]# getcap /bin/ping
/bin/ping = cap_net_admin,cap_net_raw+ep
[root@fedora ~]#
CAP_NET_ADMIN is for packet tagging:
[alessandro@fedora ~]$ ping c1
m
123 routeadd.
net
PING routeadd.
net (195.182.210.166) 56(84) bytes of data.
Warning: Failed to set mark 123
64 bytes from routeadd.
net (195.182.210.166): icmp_seq=1 ttl=53 time=45.8 ms
routeadd.
net ping statistics 1
packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 45.802/45.802/45.802/0.000 ms
[alessandro@fedora ~]$
20. MMoorree eexxaammpplleess 11--1111
Everyone knows of extended POSIX attributes,
right?
RRiigghhtt??
If you don't, you ought to.
● LC are useful, even necessary to make use of
EPA.
21. MMoorree eexxaammpplleess 22--1111
EPAs let you, among the rest, set files that:
a:
can only have data be appended to
i:
are immutable (cannot overwrite or erase)
S:
do synchronous writes on the media
D:
synchronous writes on directories
... (there can be up to 19)
Figuring out what filesystem supports which EPA
and with not is a matter of guesswork...
22. MMoorree eexxaammpplleess 33--1111
There is one catch, though:
● chattr does not work for ordinary users:
[alessandro@fedora ~]$ chattr +i .bashrc
chattr: Operation not permitted while setting flags on
.bashrc
[alessandro@fedora ~]$
24. MMoorree eexxaammpplleess 55--1111
But...
[alessandro@fedora ~]$ lsattr ~games/.profile
e/
opt/games/.profile
[alessandro@fedora ~]$ chattr +i ~games/.profile
[alessandro@fedora ~]$ lsattr ~games/.profile
ie/
usr/games/.profile
[alessandro@fedora ~]$
Which is normal for superuser rights.
We definitely do not want this to happen.
25. MMoorree eexxaammpplleess 66--1111
This is the right way to do it:
[root@fedora ~]# chmod us
/bin/chattr
[root@fedora ~]# ll /bin/chattr
rwxrxrx
1 root root 10736 Aug 3 11:36 /bin/chattr
[root@fedora ~]# setcap CAP_LINUX_IMMUTABLE=ep /bin/chattr
[root@fedora ~]# getcap /bin/chattr
/bin/chattr = cap_linux_immutable+ep
[root@fedora ~]#
26. MMoorree eexxaammpplleess 77--1111
Which causes the right things to happen:
[alessandro@fedora ~]$ lsattr .bashrc
e.
bashrc
[alessandro@fedora ~]$ chattr +i .bashrc
[alessandro@fedora ~]$ lsattr .bashrc
ie.
bashrc
[alessandro@fedora ~]$ chattr +i ~games/.profile
chattr: Permission denied while setting flags on
/usr/games/.profile
[alessandro@fedora ~]$
28. MMoorree eexxaammpplleess 88--1111
●DAC = Discretionary Access Control
● It's the classic Unix “who gets to do what”
decision mechanism
● In the filesystem it shows as the rwx file rights
● Linux Capabilities can mess them up as well! :-)
29. MMoorree eexxaammpplleess 99--1111
Let's beep the beeper! (In an xterm)
[alessandro@fedora ~]$ beep f
2000 l
100
Could not open /dev/tty0 or /dev/vc/0 for writing
open: No such file or directory
[alessandro@fedora ~]$
Man page says:
By default beep is not installed with the suid bit set, because that
would just be zany. On the other hand, if you do make it suid
root, all your problems with beep bailing on ioctl calls will
magically vanish, which is pleasant, and the only reason not to is
that any suid program is a potential security hole. Conveniently,
beep is very short, so auditing it is pretty straightforward.
30. MMoorree eexxaammpplleess 1100--1111
Let's try using LC instead. The error we got was:
Could not open /dev/tty0 or /dev/vc/0 for writing
That is, it's a DAC issue.
[root@fedora ~]# setcap CAP_DAC_OVERRIDE=pe /bin/beep
[root@fedora ~]#
[alessandro@fedora ~]$ beep f
2000 l
100
ioctl: Operation not permitted
ioctl: Operation not permitted
[alessandro@fedora ~]$
CAP_SYS_TTY_CONFIG
Use vhangup(2); employ various privileged ioctl(2) operations
on virtual terminals.
From
beep(1):
33. Extracting and Decoding LC info
1-4
What LCs are active now?
[root@ubuntu ~]# cp /bin/bash /tmp/
[root@ubuntu ~]# setcap CAP_KILL,CAP_DAC_OVERRIDE+epi /tmp/bash
[root@ubuntu ~]# getcap /tmp/bash
/tmp/bash = cap_dac_override,cap_kill+eip
[root@ubuntu ~]#
Inheritance is set
[alessandro@ubuntu ~]$ /tmp/bash l
[alessandro@ubuntu ~]$ getpcaps $$
Capabilities for `15400': = cap_dac_override,cap_kill+ep
[alessandro@ubuntu ~]$ grep ^Cap /proc/$$/status
CapInh: 0000000000000000
CapPrm: 0000000000000022
CapEff: 0000000000000022
CapBnd: 0000001fffffffff
[alessandro@ubuntu ~]$
Inheritance is missing!
34. Extracting and Decoding LC info
2-4
Decoding Hex LC bitmap:
[alessandro@ubuntu ~]$ grep ^Cap /proc/$$/status
CapInh: 0000000000000000
CapPrm: 0000000000000022
CapEff: 0000000000000022
CapBnd: 0000001fffffffff
[alessandro@ubuntu ~]$ capsh decode=
0000000000000000
0x0000000000000000=
[alessandro@ubuntu ~]$ capsh decode=
0000000000000022
0x0000000000000022=cap_dac_override,cap_kill
[alessandro@ubuntu ~]$
Still represented as a set
35. Extracting and Decoding LC info
3-4
0000001fffffffff = Capability Bounding Set
1 = allowable cap (kept if present)
0 = masked cap (or no LC associated to bit)
Bits are mapped one-to-one to a LC
0000001fffffffff is a 64-bit mask
● It's been a 64 bit mask since libcap vers. 2.03
● Before it was 32-bit
36. Extracting and Decoding LC info
4-4
Number of bit-mapped capabilities is consistent
with zero-based value stored in
/proc/sys/kernel/cap_last_cap
Kernel 3.14.19:1fffffffff → 1+9*4=37
/proc/sys/kernel/cap_last_cap = 36
Kernel 3.16.3: 3fffffffff → 2+9*4=38
/proc/sys/kernel/cap_last_cap = 37
37. SSyysstteemm CCoonnffiigguurraattiioonn 11--22
capability.conf1: users who inherit LCs
[root@ubuntu ~]# cat /etc/security/capability.conf
cap_kill alessandro
none *
[root@ubuntu ~]#
Changes are effective after a new login:
Ubuntu 14.04.1 LTS ubuntu tty1
If missing all users get all
available capabilities!
CAP_KILL is now inherited
ubuntu login: alessandro
Password:
[alessandro@ubuntu ~]$ getpcaps $$
Capabilities for `8586': = cap_kill+i
[alessandro@ubuntu ~]$
38. SSyysstteemm CCoonnffiigguurraattiioonn 22--22
Child process of new shell now does inherit LCs:
[alessandro@ubuntu ~]$ getpcaps $$
Capabilities for `16332': = cap_kill+i
[alessandro@ubuntu ~]$ getcap /tmp/bash
/tmp/bash = cap_dac_override,cap_kill+eip
[alessandro@ubuntu ~]$ /tmp/bash l
[alessandro@ubuntu ~]$ getpcaps $$
Capabilities for `16849': = cap_kill+eip cap_dac_override+ep
[alessandro@ubuntu ~]$
Cap is in config file Cap is not in config file
1) Ubuntu package libpam-cap is required. Fedora has package libcap,
but this setup does not work (PAM failure: System error)
39. Capability aassssiiggnnmmeenntt 11--1100
2.2.11 2.6.25 optional
2.6.33 built-in
System-wide bounding-set Per-thread bounding-set
● Before 2.6.25 Bounding Set was system-wide
● Beginning 2.6.25 it's per-thread
● Can be dropped by process with CAP_SETPCAP
via prctl(2) PR_CAPBSET_DROP operation
● Inherited at fork(2), preserved on execve(2)
● Optional until 2.6.33, then built-in
40. Capability aassssiiggnnmmeenntt 22--1100
N New P Previous F Capab. set to the File
e Effective
capability p Permitted capability i Inherited capability
CBS Capability Bounding Set1
New capabilities used to be computed this way:
Np (Pi∪Fi)∩(Fp∪CBS)
Ne Fe∪Np
Ni Pi
1) Mask of permitted capabilities retained after an execve(2). No effect on inherited caps.
From kernel 2.6.25, limits inherited capabilities that can be added to self even if they are permitted.
41. Capability aassssiiggnnmmeenntt 33--1100
N New P Previous F Capab. set to the File
e Effective
capability p Permitted capability i Inherited capability
CBS Capability Bounding Set1
Now they are computed this way:
Np (Pi∪Fi)∩(Fp∪CBS)
Ne Fe ? Np : 02
Ni Pi
2) Not and AND any more, because Effective set is a bit, no longer a set (one bit per capability) as it
used to be.
42. Capability aassssiiggnnmmeenntt 44--1100
What does the Capability Bounding Set do?
1) On an execve(2), it ANDs out thread's
permitted caps from file's permitted caps.
2) Limits thread's inheritable caps that can be
added with a capset(2) (kernel >= 2.6.25).
a)Filtered capability can still be permitted
b)Inheritable cap can still be set if it's in
file's inheritable set.
43. Capability aassssiiggnnmmeenntt 55--1100
Possibile security risk: undesired permitted caps.
Consider the situation:
1) Parent process has CAP_X Inh, not in CBS
2) CAP_X cannot be in Perm set
a) But it is retained in Inh set by children
3) Process can still exec child with CAP_X in
both Perm and Inh sets!
Dropping CBS can instill a false sense of security!
Alone it's not sufficient!
44. Capability aassssiiggnnmmeenntt 66--1100
Like SUID, shell scripts' LC are not honored:
[alessandro@ubuntu ~]$ cat ~/bin/test_cap.sh
#!/bin/dash
getpcaps $$
grep ^Cap /proc/$$/status
[alessandro@ubuntu ~]$ getcap ~/bin/test_cap.sh
/home/alessandro/bin/test_cap.sh = cap_linux_immutable+ep
[alessandro@ubuntu ~]$ test_cap.sh
Capabilities for `9093': =
Not honored
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
Not honored
CapBnd: 0000001fffffffff
[alessandro@ubuntu ~]$
45. Capability aassssiiggnnmmeenntt 77--1100
Only binary executables' LC are honored:
[alessandro@ubuntu ~]$ cat ~/bin/test2_cap.sh
#!/tmp/dash
getpcaps $$
grep ^Cap /proc/$$/status
[alessandro@ubuntu ~]$ getcap ~/bin/test2_cap.sh /tmp/dash
/tmp/dash = cap_linux_immutable+ep
[alessandro@ubuntu ~]$ test2_cap.sh
Capabilities for `9218': = cap_linux_immutable+ep
CapInh: 0000000000000000
CapPrm: 0000000000000200
CapEff: 0000000000000200
Now honored Now honored
CapBnd: 0000001fffffffff
[alessandro@ubuntu ~]$
47. Capability aassssiiggnnmmeenntt 99--1100
CBS reduction with capsh. Initial state:
[alessandro@ubuntu ~]$ getcap $(which capsh ping)
/sbin/capsh = cap_setpcap+p
/bin/ping = cap_net_raw+p
[alessandro@ubuntu ~]$ getpcaps $$
Capabilities for `7842': =
[alessandro@ubuntu ~]$ ping qc1
routeadd.
net
PING routeadd.
net (195.182.210.166) 56(84) bytes of data.
routeadd.
net ping statistics 1
packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 27.718/27.718/27.718/0.000 ms
[alessandro@ubuntu ~]$
48. CCaappaabbiilliittyy aassssiiggnnmmeenntt 1100--1100
Running new shell with lowered CBS via capsh:
[alessandro@ubuntu ~]$ capsh drop=
cap_net_raw [
alessandro@ubuntu ~]$ grep ^Cap /proc/$$/status
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000001fffffdfff
[alessandro@ubuntu ~]$ capsh decode=
2000
0x0000000000002000=cap_net_raw
[alessandro@ubuntu ~]$ ping qc1
routeadd.
net
ping: icmp open socket: Operation not permitted
[alessandro@ubuntu ~]$
49. DDiissttrroo ssttaattuuss 11--77
The plan is to replace as many SUID/SGID
executables with capabilities as possible:
● Fedora: Last update: 2011-04-05, completion:
100% (many, but not all, SUID taken off)
● Ubuntu: WIP (Last update: 2011-09-27)
Not everything SUID could be ported to Linux
Capabilities (yet)
51. DDiissttrroo ssttaattuuss 33--77
Ubuntu 14.04.1 (Trusty Tahr): a few more (also in
a non-standard, XFCE install)
# find / /usr xdev
type
f perm
/111 perm
/4000 user
root | wc l
29
#
They used to be 34 in 13.10 (Saucy Salamander)
52. DDiissttrroo ssttaattuuss 44--77
Let's see one case where LC do not work:
[alessandro@fedora ~]$ ll /usr/bin/at
rwsrxrx
1 root root 53208 5 dic 12.34 /usr/bin/at
[alessandro@fedora ~]$ getcap /usr/bin/at
[alessandro@fedora ~]$
Let's strip off the SUID bit and any cap:
[root@fedora ~]# chmod us
/usr/bin/at
[root@fedora ~]# setcap r
/usr/bin/at
[root@fedora ~]#
[alessandro@fedora ~]$ at f
~/bin/at.sh now+5min
cannot set egid: Operation not permitted
[alessandro@fedora ~]$
53. DDiissttrroo ssttaattuuss 55--77
Let's try to make it happy:
[root@fedora ~]# setcap CAP_SETGID+ep /usr/bin/at
[root@fedora ~]#
No joy:
[alessandro@fedora ~]$ at f
~/bin/at.sh now+5min
cannot set euid: Operation not permitted
[alessandro@fedora ~]$
[root@fedora ~]# setcap CAP_SETUID,CAP_SETGID+ep /usr/bin/at
[root@fedora ~]#
[alessandro@fedora ~]$ at f
~/bin/at.sh now+5min
PAM failure: System error
[alessandro@fedora ~]$ :(
54. DDiissttrroo ssttaattuuss 66--77
Log says:
[alessandro@fedora ~]$ tail 1
/var/log/secure
Feb 23 10:08:43 fedora at: PAM audit_log_acct_message()
failed: Operation not permitted
[alessandro@fedora ~]$
Game over, for now.
55. DDiissttrroo ssttaattuuss 77--77
Ubuntu has taken a different approach:
● it uses the same at/atd implementation (by
Thomas Koenig, ig25@rz.uni-karlsruhe.de)
● it runs the daemon as user daemon
● the client command is SUID and SGUI
daemon:daemon, allowing it to:
write into /var/spool/cron/atjobs/.SEQ
send atd signals
Old school, simple, effective, safe.
56. IIssssuueess?? 11--22
«Why are not more people aware of and using
capabilities? I believe that poor
documentation is the primary reason. For
example, Fedora 10 is missing the man pages
for getpcaps, capsh and pam_cap, and
the Fedora Security Guide does not even
mention capabilities»
(Finnbarr P. Murphy, May 28th, 2009 )
Every bit as true today!
57. IIssssuueess?? 22--22
●Online distro docs stopped years ago
● Fedora Security Guide in same condition in
19.1, and is absent in 20 documentation
➢ At least man pages are there
➢ SELinux only future?
● Behaviour of LC changing over time
➢ Same commands, capabilities and sets yield
different result from 2.6.24 with no errors
● libcap-ng should make programming easier
58. AAkknnoowwlleeddggmmeennttss 11--11
● The Linux man-pages Project people (
https://www.kernel.org/doc/man-pages/)
● Finnbarr P. Murphy (blog post and references,
useful though outdated)
● And kernel developers, of course!
Andrew G. Morgan <morgan@kernel.org>
Alexander Kjeldaas <astor@guardian.no> with help
from Aleph1, Roland Buresund and Andrew Main.