LCE13: Introduction to OpenEmbedded/Yocto Cross-Compile Environment

OpenEmbedded and Yocto
introduction
Nicolas Dechesne <nicolas.dechesne@linaro.org>
Riku Voipio <riku.voipio@linaro.org>
Trevor Woerner <trevor.woerner@linaro.org>
LCE13, Dublin

2
Agenda
● Intro to OpenEmbedded / Yocto
● Overview of Yocto Project, organization and releases
● Yocto Project vs. DIY
● Introduction to OpenEmbedded concepts (recipes,
packages, images, layers, configuration, SDK, …)
● OpenEmbedded/Yocto and Linaro
● 'OpenEmbedded/Yocto Lab' (this afternoon, 3PM)
● Beginners practical introduction
● Guide you through a 'qemuarm' console image build
● Q&A

3
What is the Yocto Project?
It provides all the basic bits that every Linux product needs, pre-integrated and
tested to let you spend more time on differentiators that matter to you!

4
The Yocto Project
● Open source project, hosted by the Linux Foundation.
● Not the “new kid on the block”, OpenEmbedded has been around since
2003!
● A collaboration project, an “umbrella” project. A collection of projects that
make up the “Yocto Project”
● Bitbake
● OpenEmbedded-Core
● Poky
● Application Development Toolkit
● And even more...
● It helps you build a 'tailored' Linux distribution/product for your exact needs.
● Support for ARM, PPC, MIPS, x86

5
Yocto Project organization
● Hierarchical technical leadership:
● Richard Purdie as the project architect
● Sub system maintainers
● Somehow similar to Linux kernel technical leadership
● Yocto Project Advisory Board for administrative leadership
● community, finance, infrastructure, advocacy
● From linaro: TI, Enea, Huawei
● Open to all contributors
● Yocto Compliance program
● Yocto project participants
● Yocto project compatible

6
Yocto/OE releases
● 1 new release ~6 months
● Yocto Project releases in sync with OE, bitbake projects
releases
● Yocto Project release is the aggregation of oe-core, bitbake and
Poky layers/tools/documentation.
● Stable releases
● v1.4 (aka dylan), v1.3 (aka danny)
● No new feature, only bug fixes
● Development / master branch
● New features (systemd, wayland, Qt5, build engine improvements, ...)
● Components version updates
● Does not 'work' every single day

7
Yocto/OpenEmbedded vs DIY
● Spend less time on things that bring no value to your business:
● OE provides and maintains thousands of up-to-date recipes
● Build an entire Linux system from sources in less than 1 hour, using a validated set of
packages (toolchain, busybox, libc, init system...)
● Built in support for package management
● Predictable and reproducible builds
● Provides set of standard tools and build guidelines.
● What you build for a product can be reused for other products.
● autobuilder 'bot' for continuous integration
● Minimal dependencies on host, build as standard user (no root)
● Still, you have all the flexibility you need to
● patch as needed any OE component for your needs/product
● Add any unsupported component, even proprietary bits
● Fully configure the system, and make it exactly the way you want.

8
OpenEmbedded metadata
● OpenEmbedded does not contain components source code, only their metadata
● Bitbake is the build engine, a metadata parser, written in Python
● Recipes
● Component (.bb, .bbappend)
● Image (.bb)
● Class (.bbclass)
● Configuration files: behavior largely controlled by variables
● bitbake.conf
● <machine>.conf
● <distro>.conf
● local.conf, site.conf
● bblayers.conf
● Patches

10
OpenEmbedded packages
● A recipe generates 1 or more binary packages
● Each recipe build is split into 'tasks'
● Tasks are scheduled by bitbake based on parallelism and dependencies
● fetch, unpack, patch, configure, build, install, package
● All packages are built from scratch with same compiler configuration (CPU,
optimizations, tuning, …) according to the underlying target CPU.
● Standard packages are created managed automatically (most of the time)
● Binaries are stripped automatically, and debug binaries are placed in -dbg
packages
● Development packages with .h , .so, .a, .pc, …
● ...
● Packages dependencies
● build depends (DEPENDS) are mandatory and explicit in recipes
● Run time depends (RDEPENDS) are semi-automatic (shlibdeps, pcdeps)

11
OpenEmbedded images
● Images are standard recipe (e.g. .bb file), with a different 'task set'
● Images are expected to be 'deployed' on target
● Most standard image format are supported (ext2/3, tarball, jffs2, cramfs, ramdisk,
…)
● Support for image generation 'post scripting'
● Packages are assembled into the final root file system to make up images
● Image recipe lists all components/applications required in the image
● RDEPENDS used to pull all the needed libraries, ...
● Multiple images can be built from the same 'build', by assembling different set of
packages
● OpenEmbedded has (optional) support for package management in the target
image: .rpm, .ipk, .deb.
● Can be used to implement a binary package based distro (with package feed)
● However OE always uses packages 'internally' to build images.

12
OpenEmbedded layers
● Couple of years ago, OpenEmbedded used to be a monolithic repo with thousands of
recipes, and became impractical to maintain and use
● Layer concept introduced
● Layers contain additional metadata (recipes, machine, distro, …)
● Oe-core is the base layer for the most fundamental components
● layers provide additional features: meta-systemd, meta-gnome, …
● Layers can be inter-dependent
● Layers can be hosted in different 'trees' (git)
● All layers must work with oe-core
● Layers can add, modify content/beahvior, and/or alter, extend metadata from other
layers.
● Generally, each BSP or distro has its own layer(s)
● meta-linaro, https://git.linaro.org/gitweb?p=openembedded/meta-linaro.git
● OpenEmbedded metadata on line index: http://layers.openembedded.org/

13
Poky
● Poky is a reference distro implementation.
● Minimal console only image
● Full Linux system with GUI using “sato”
● Generally used as a starting point (or an example) for building one's own
(commercial) distro: Angstrom, ELDK, Mentor Linux, Wind River Linux, ENEA
Linux, ...
● “it is often just a starting point, not an end goal”
● Built with bitbake and these layers: oe-core, meta-yocto, meta-yocto-bsp
(and yes, the name is confusing!)
● Used to test/validate OpenEmbedded by Yocto Project developers
● “Poky-tiny” is a variant that provides size optimized image.
● “Poky-bleeding” is a variant that uses bleeding edge software version

14
SDK
● Application Development Toolkit (ADT) provides custom built,
cross development SDK tailored for the image being built
● Cross toolchain
● Matching sysroot
● Eclipse IDE Yocto plugin
● Support SDK for x86 and/or x86-64 hosts
● ADT can be installed on 'OEM', customer hosts without the
need to rebuild the entire image packages
● “bitbake meta-toolchain” : includes only the minimal cross toolchain and
libs
● “bitbake -c populate_sdk <image>” : includes all the development
headers, libs, … for every package included in the image. e.g. build SDK
that matches the target image

15
Anatomy of a build folder
● Lots of space … ~20 Gb
● Entirely contained in '<build>' folder
● Sources used are in '<build>/downloads'
● Builds are in '<build>/tmp/work/<arch>'
● 'deploy' has
● Ready to use images
● Packages
● license information
● Buildhistory records information about content of
images, and package build after build (with diffs)
● 'flat' rootfs also available in image <work> folder
● INHERIT += “rm_work” cleans up each <work> folder
upon successful build

17
Background
● Originally we started using OE to bootstrap
Aarch64 support
● Also used by Toolchain WG to validate compiler
● Extended interest by networking WG and others
● Integrated to our CI loop

18
Quickstart
● Git clone git://git.linaro.org/openembedded/jenkins-setup.git
● $ sudo bash pre-build-root-install-dependencies.sh
● $ init-and-build.sh linaro-image-minimal
● Have a nice cup
● of coffee!

19
Show me the code
● git.l.org/openembedded/jenkins-setup.git
● Setup and build script as used by integration
● git.l.org/openembedded/manifest.git
● Repo manifest, describes used git repositores
● git.l.org/openembedded/meta-linaro.git
● Contains Linaro, toolchain and aarch64 layers
● Branches for head and stable (dylan)
● Poky support being worked on

20
Continuous integration
● Jenkins: builds – LAVA runtime testing
● https://ci.linaro.org/jenkins/job/openembedded-armv8-rootfs/
● Building the minimal, lamp and Java Images
● https://git.linaro.org/gitweb?p=qa/test-
definitions.git;a=tree;f=openembedded
● Test definitions for LAVA
● http://validation.linaro.org/lava-server/dashboard/image-
reports/minimal-armv8
● Test results in LAVA
● Similar pages for LAMP and JAVA images

21
Future plans
● Switching to Poky/Yocto or stable OE
● Changes in OE head disruptive
● Repo manifest tagging for releases
● For reproducibility
● CI integration of ARMv7 and big endian
● Bitbake world -k for toolchain validation
● Try to keep things simple instead of supporting every possible
usecase.

22
Resources
● https://www.yoctoproject.org/
● http://openembedded.org/wiki/Main_Page
● http://www.slideshare.net/rossburton/the-yocto-project
● https://www.yoctoproject.org/documentation/current
● http://layers.openembedded.org/layerindex/
● http://www.linuxfoundation.org/news-media/blogs/browse/2013/05/new-paper-how-engineering-
leaders-can-use-yocto-project-solve
● http://www.linuxfoundation.org/news-media/announcements/2010/10/linux-foundation-and-
consumer-electronics-linux-forum-merge
● http://www.linuxfoundation.org/news-media/blogs/browse/2010/10/while-apple-debates-open-vs-
integrated-we-want-best-both-worlds

25
Demystify OE/Yocto
build through a practical exercise
● Build a minimal console image for qemuarm
● Run the image in QEMU
● Explore and understand the OE build directory
structure
● Look at some basic OE developers tasks
● Build/run an application with SDK

26
Practical information
● Source and build cache
● “repo” tool
● Sample of recipe
● Lab instructions are here:
● Part1: https://collaborate.linaro.org/x/NYBm
● Part2: https://collaborate.linaro.org/x/q4Bm
● Part3: https://collaborate.linaro.org/x/zIBm
● Ask questions, we are here to answer them!

27
Interesting things to do...
● Explore configuration files
● Checkout some recipes
● Explore 'WORKDIR' and 'deploy' folders (image and package
feed)
● Customize your image
● Add a new recipe, create your own layer
● Exercise with .bbappend
● Create a patch, and apply in recipe
● Use/understand bitbake 'tasks' and experiment typical
developer workflow (build, debug, edit, build, debug, ...)

28
Sources mirror, build cache
● OpenEmbedded 'trees' contain no source code
● By design, everything is built from scratch.
● Source code for all recipes is fetched during the build
● 'fetchall' target let you download all source first
● PREMIRROR let you configure OE to use an alternative mirror before going to the Internet
(can be local/intranet)
● OE implements “shared state code” (aka sstate) to support incremental builds
● OE uses hash/signature on each 'task' input variables to decide if task needs to be rebuilt.
● Prebuilt components are stored in “sstate” cache, which can be local or http, and can be
shared across multiple users.
● Even after deleting the entire working folder, a rebuild is 'instantaneous'
● We prepared 'tarballs' for 'download' and 'sstate' to make this workshop more
efficient, hopefully you've downloaded everything already
● http://people.linaro.org/~trevor.woerner/LCE2013/
● Otherwise we have a couple of HDDs

29
“repo” tool
● Implemented initially by the Android team to manage Android source repositories
● Open source tool: https://code.google.com/p/git-repo/
● Wrapper on top of git
● Manage 'forest' of git trees
● Support multiple 'product' branches and easy switch from one checkout to another
● Trees metadata stored in 'XML manifest' file
https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.txt
● Manifest can checkout:
● Latest commit on branch (development)
● Fix commit/tag (release)
● All Linaro OE projects are moving to “repo”
● Repo can (optionally) be used with Gerrit code review

30
Manifest sample
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote name="linaro" fetch="git://git.linaro.org" />
<remote name="oe" fetch="git://git.openembedded.org" />
<remote name="beagleboard" fetch="git://github.com/beagleboard" />
<default remote="oe" revision="master" sync-j="2" />
<project name="openembedded-core" />
<project name="meta-openembedded" />
<project path="openembedded-core/bitbake" name="bitbake" />
<project path="meta-linaro" name="openembedded/meta-linaro" remote="linaro" />
<project name="meta-beagleboard" remote="beagleboard"/>
</manifest>

31
How to use “repo”
# install repo wrapper (once on each machine)
$ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > /tmp/repo
$ chmod a+x /tmp/repo
$ sudo mv /tmp/repo /usr/local/bin/
# initialize the repo, and download the manifest
$ mkdir <mydir> && cd <mydir>
$ repo init -u <manifest.git> -b <branch>
# download and checkout all projects
$ repo sync
# build, edit, commit, debug, …
# update my working copy
$ repo sync
# switch to another branch/product
$ repo init -b <my other branch>
$ repo sync
commands:
list
status
info
diff
grep

32
Hello World!
DESCRIPTION = "Simple helloworld application"
SECTION = "examples"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
PR = "r0"
SRC_URI = "file://helloworld.c"
S = "${WORKDIR}"
do_compile() {
${CC} helloworld.c -o helloworld
}
do_install() {
install -d ${D}${bindir}
install -m 0755 helloworld ${D}${bindir}
}
Packages:
helloworld
helloworld-dev
helloworld-dbg

33
smartmontools
# meta-oe/recipes-extended/smartmontools/smartmontools_6.1.bb
SECTION = "console/utils"
DESCRIPTION = "Control and monitor storage systems using S.M.A.R.T."
HOMEPAGE = "http://smartmontools.sourceforge.net/"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
SRC_URI = "${SOURCEFORGE_MIRROR}/smartmontools/smartmontools-${PV}.tar.gz"
SRC_URI[md5sum] = "83a3a681f8183ed858392d550ae1cca6"
SRC_URI[sha256sum] = "a9003b8bccc82682f658ce76d70edb1842411e51dc56d4cd6b56618da1d9ce07"
inherit autotools
http://cgit.openembedded.org/meta-openembedded/commit/?id=8fec906a4584f9cbc467fb0e406bb6a705848b64

34
Using layer to alter recipes
$ more meta-linaro/recipes-core/busybox/busybox_1.20.2.bbappend
SRC_URI += "file://linaro.cfg"
$ more meta-aarch64/recipes-sato/webkit/webkit-gtk_1.8.3.bbappend
SRC_URI_append = "file://aarch64.patch”
● “<packagename>_<version>.bbappend” files can be used in layers
● Any variable from the original recipe can be modified
● Very neat for patching! Avoid duplication, fragmentation.
● “bitbake-layers show-appends”

36
Tuning compiler
● CPU compiler options, FPU and ABI can be fully tuned from the machine
configuration file
● Each 'architecture' gets a set of pre-defined 'tunes'
● For ARM : VFP vs neon, Thumb vs ARM instructions, Little vs big endian, Calling
conventions (hardfp vs softfp, …)
TUNEVALID[neon] = "Enable Neon SIMD accelerator unit."
TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "neon", " -mfpu=neon", "" ,d)}"
ARMPKGSFX_FPU .= "${@bb.utils.contains("TUNE_FEATURES", "neon", "-neon", "" ,d)}"
TUNEVALID[vfp] = "Enable Vector Floating Point (vfp) unit."
ARMPKGSFX_FPU .= "${@bb.utils.contains("TUNE_FEATURES", "vfp", "-vfp", "" ,d)}"
TUNEVALID[callconvention-hard] = "Enable EABI hard float call convention, requires VFP."
TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard",
" -mfloat-abi=hard", " -mfloat-abi=softfp", d), "" ,d)}"
ARMPKGSFX_EABI .= "${@bb.utils.contains("TUNE_FEATURES", [ "callconvention-hard", "vfp" ], "hf", "", d)}"

37
Layer configuration file
# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "5"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= "
${TOPDIR}/openembedded-core/meta
${TOPDIR}/meta-openembedded/toolchain-layer
${TOPDIR}/meta-openembedded/meta-multimedia
${TOPDIR}/meta-openembedded/meta-oe
${TOPDIR}/meta-linaro/meta-linaro
${TOPDIR}/meta-linaro/meta-linaro-toolchain
${TOPDIR}/meta-qt5
"
BBLAYERS_NON_REMOVABLE ?= "
${TOPDIR}/openembedded-core/meta
"

38
Machine configuration file
#@TYPE: Machine
#@NAME: common_pc
#@DESCRIPTION: Machine configuration for running a common x86
PREFERRED_PROVIDER_virtual/xserver ?= "xserver-xorg"
PREFERRED_PROVIDER_virtual/libgl ?= "mesa"
PREFERRED_PROVIDER_virtual/libgles1 ?= "mesa"
PREFERRED_PROVIDER_virtual/libgles2 ?= "mesa"
require conf/machine/include/qemu.inc
require conf/machine/include/tune-i586.inc
KERNEL_IMAGETYPE = "bzImage"
SERIAL_CONSOLE = "115200 ttyS0"
XSERVER = "xserver-xorg
mesa-driver-swrast
xf86-input-vmmouse
xf86-input-keyboard
xf86-input-evdev
xf86-video-vmware"
MACHINE_FEATURES += "x86"
MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "v86d"
qemux86.conf

39
User local configuration file
# Parallelism Options
#BB_NUMBER_THREADS = "4"
#PARALLEL_MAKE = "-j 4"
MACHINE = "qemuarm"
DISTRO = “poky”
PACKAGE_CLASSES ?= "package_ipk"
# "dbg-pkgs" - add -dbg packages for all installed packages
# (adds symbol information for debugging/profiling)
# "tools-testapps" - add useful testing tools (ts_print, aplay, arecord etc.)
# "debug-tweaks" - make an image suitable for development
# e.g. ssh root access has a blank password
EXTRA_IMAGE_FEATURES = "debug-tweaks"
# - 'buildstats' collect build statistics
# - 'image-prelink' in order to prelink the filesystem image
USER_CLASSES ?= "buildstats image-prelink"
#SSTATE_MIRRORS ?= "
#file://.* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH n
#file://.* file:///some/local/dir/sstate/PATH"

40
Yocto Project Branding/Compliance
● Compliance controls the usage of the Yocto project name, logo
and brand.
● Yocto Project Participants
● Open source project, non profit, small business
● Committed to promoting and contributing to Yocto Projects (oe-core,
bitbake, yocto layers)
● Yocto Project Compatible
● Products, BSP, layers maintained by an open source, non profit, or Yocto
Project member organisation
● Compatible with a specific release of Yocto
● Contribute all Yocto Projects patches upstream
● All Yocto recommendations/processes implemented

LCE13: Introduction to OpenEmbedded/Yocto Cross-Compile Environment

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (6)

Más de Linaro

Más de Linaro (20)

Último

Último (20)

LCE13: Introduction to OpenEmbedded/Yocto Cross-Compile Environment