Distro Recipes 2013 : Make Debian and compiler agnostic

Make Debian compiler agnostic :
A rebuild with Clang

Sylvestre Ledru – sylvestre@debian.org

Disclamer !

April, 04 2013 Make Debian compiler agnostic
Sylvestre Ledru

Current status :
All C, C++, Objective-C sources are being built
with gcc for all supported Debian arches and
Kernel.

Sylvestre Ledru

gcc is THE FLOSS compiler for the last 25 years
Used for (pretty much) everywhere or anything

Still growing and maintained (cf gcc 4.8)

Sylvestre Ledru

Why a new compiler in Debian ?

Sylvestre Ledru

Other compilers can find programming errors that
gcc could not find

Sylvestre Ledru

Code built by many compilers is more likely to
be more strictly correct and more portable then
code only built with gcc

Sylvestre Ledru

Some compilers can have advantages on some
archs (ex : clang on ARM)

Sylvestre Ledru

Provide some interesting side projects (lldb, scan-
build, ASAN, libc++, etc)

Sylvestre Ledru

As we were able to do with decoupling Linux
from Debian with kFreeBSD and the HURD, we're
aiming to decouple gcc in Debian.

Open the door to other compilers or analyzers

Sylvestre Ledru

LLVM/Clang

Sylvestre Ledru

Started as an academic project
Versatile platform for compilation and virtual
machine

Designed originally for the investigation of
dynamic compilation techniques for static and
dynamic languages

Sylvestre Ledru

Developed initially as a base for a next major
GCC release
Source:
http://gcc.gnu.org/ml/gcc/2005-11/msg00888.html

Sylvestre Ledru

Sponsored by Apple since 2005 to replace gcc
(GPL vs BSD)

Has now a strong and diverse community
(academics, individuals and corporates)

Many universities/research centers are basing
their research on LLVM

Sylvestre Ledru

Sylvestre Ledru

Clang

C, C++ & Objective-C compiler.
(no Fortran)
Based on LLVM

Default compiler for Mac OS X (Xcode)/iOS [1]
and FreeBSD [2]

Sources:
[1] https://developer.apple.com/technologies/tools/
[2] http://lists.freebsd.org/pipermail/freebsd-stable/2012-May/067486.html

Sylvestre Ledru

Some advantages :

More recent base code (ie less legacy code)

Strong interest of hardware manufactors (ARM,
MIPS, Intel, Nvidia, etc)

Supposed to be faster to build code than gcc

Accept the same arguments as gcc

Sylvestre Ledru

Example
Full build of Scilab (doc, essential tests)
~24 minutes gcc
~22 minutes clang

Sylvestre Ledru

Some advantages (bis)
More intelligent detections
– foo.c – $ gcc -Wall -Werror foo.c ; echo $?

int main() { 0

unsigned int i = 0; $ clang -Werror foo.c
foo.c:3:17: error: comparison of
return i < 0;
unsigned expression < 0 is always
} false
[-Werror,-Wtautological-compare]
return i < 0;
~^~
1 error generated.

Sylvestre Ledru

Side effect
=> Brings (friendly) competition in the free
compiler world.

Sylvestre Ledru

http://gcc.gnu.org/wiki/ClangDiagnosticsComparison

Sylvestre Ledru

http://clang.llvm.org/comparison.html#gcc

Sylvestre Ledru

Rebuild of Debian using Clang

Sylvestre Ledru

Crappy method :

VERSION=4.7
cd /usr/bin
rm g++-$VERSION gcc-$VERSION cpp-$VERSION
ln -s clang++ g++-$VERSION
ln -s clang gcc-$VERSION
ln -s clang cpp-$VERSION
cd -

Sylvestre Ledru

Testing the rebuild of the package under amd64.

NOT the performances (build time or execution)
nor the execution of the binaries

Sylvestre Ledru

Full results published:
http://clang.debian.net/

Done on the cloud-qa - EC2 (Amazon cloud)
Thanks to Lucas Nussbaum

Sylvestre Ledru

Rebuild with clang 3.0
February 28, 2012

15658 packages built : 1381 (8.8 %) failed.

Sylvestre Ledru

June 23, 2012


Sylvestre Ledru

January 28, 2013


Sylvestre Ledru

Why these differences between 3.0 vs 3.1/3.2?

Sylvestre Ledru

Some information about -Wall & -Werror :

-Wall enables many warnings

-Werror transforms Warning to Error
int main() { $ gcc -Wall -Werror foo.c && echo $?
unsigned int i = 0; 0
return i < 0;
} $ clang -Wall -Werror foo.c && echo $?
foo.c:3:14: error: comparison of unsigned expression < 0 is always false
[-Werror,-Wtautological-compare]
return i < 0;
~^~
1 error generated.

Sylvestre Ledru

-Werror & unused args
40 occurrences

Clang detects unused argument.

clang --param ssp-buffer-size=4 clang: error: argument unused
-Werror foo.c during compilation: '--param ssp-
buffer-size=4'
And generates a normal warning …
96 occurrences
Which becomes an error with -Werror

Sylvestre Ledru

Security check introduced in clang 3.1
9 occurences

$ gcc -Werror -c foo.c && echo $?
#include <stdio.h>
0
void foo(void) { $ clang -Werror -c foo.c && echo $?
char buffer[1024]; foo.c:5:23: error: use of '%n' in
sprintf(buffer, "%n", 2); format string discouraged
(potentially insecure) [-Werror,-
}
Wformat-security]
sprintf(buffer, "%n", 2);
~^
1 error generated.

Sylvestre Ledru

Some of the most common errors

Sylvestre Ledru

Unsupported options
50 occurrences
$ gcc -O9 foo.c && echo $?
0

$ clang -O9 foo.c
error: invalid value '9' in '-O9'

Record by libdbi-drivers with -O20 o/

Sylvestre Ledru

Different default behavior
133 occurrences
– noreturn.c – $ gcc -c noreturn.c; echo $?

int foo(void) { 0
# -Wall shows it as warning
return;
$ clang -c noreturn.c
}
noreturn.c:2:2: error: non-void
function 'foo' should return a value
[-Wreturn-type]
return;
^
1 error generated.

Sylvestre Ledru

Different default behavior (bis)
15 occurrences
$ gcc -c returninvoid.c; echo $?
– returninvoid.c –
returninvoid.c: In function ‘foo’:
void foo(void) {
returninvoid.c:2:2: warning: ‘return’ with a
return 42; value, in function returning void [enabled
by default]
}
0
$ clang -c returninvoid.c
returninvoid.c:2:2: error: void function
'foo' should not return a value
[-Wreturn-type]
return 42;
^ ~~
1 error generated.

Sylvestre Ledru

Different understanding of the C++ standard
– mailboxField.cpp – $ g++ -c mailboxField.cpp && echo $?
class address {
0
protected:
static int parseNext(int a); $ clang++ -c mailboxField.cpp
};
mailboxField.cpp:17:22: error: 'parseNext' is a
class mailbox : public address { protected member of 'address'
friend class mailboxField; address::parseNext(a);
}; ^
class mailboxField {
mailboxField.cpp:4:16: note: declared protected
void parse(int a) { here
address::parseNext(a); static int parseNext(int a);
// will work with: ^
// mailbox::parseNext(a);
}
References:
}; http://llvm.org/bugs/show_bug.cgi?id=6840
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52136
Sylvestre Ledru

Different set of warnings with -Wall
Plenty of occurences echo $?
$ gcc -Wall -Werror -c foo.cpp &&
– plop.c – 0

void foo() { $ clang -Wall -Werror -c foo.cpp
foo.cpp:3:13: error: equality comparison with extraneous
int a=1; parentheses

if ((a == 1)) { [-Werror,-Wparentheses-equality]
if ((a == 1)) {
return; ~~^~~~
foo.cpp:3:13: note: remove extraneous parentheses around the
} comparison to

} silence this warning
if ((a == 1)) {
~ ^ ~
foo.cpp:3:13: note: use '=' to turn this equality comparison into an
assignment
if ((a == 1)) {
^~
=
1 error generated.
Sylvestre Ledru

gcc extensions which won't be supported
25 occurences
– foo.cpp – $ g++ -c foo.cpp; echo $?

#include <vector> 0
$ clang++ -c foo.cpp
void foo() {
foo.cpp:4:29: error: variable length
int N=2;
array of non-POD element type
std::vector<int> best[2][N]; 'std::vector<int>'
} std::vector<int> best[2][N];
^
1 error generated.

Sylvestre Ledru

gcc accepts stuff which should not
34 occurences
$ g++ -c foo.cpp; echo $?
– foo.cpp –
0
// Uncomment this line will fix the issue.
$ clang++ -c foo.cpp
// template<typename Value_t>
foo.cpp:6:5: error: call to function 'b' that is neither
// void b(Value_t value)
visible in the template
template<typename Value_t> definition nor found by argument-dependent lookup
void a(Value_t value) {
b(value);
b(value);
^
}
foo.cpp:15:5: note: in instantiation of function template
specialization
'a<int>' requested here
template<typename Value_t>
void b(Value_t value) { a(y);

} ^
foo.cpp:9:33: note: 'b' should be declared prior to the call site
template<typename Value_t> void b(Value_t value)
void foo(int y) {
^
a(y);
1 error generated.
}

Sylvestre Ledru

Last rebuild proved that clang is now ready

Remaining problems are upstream

Some work started to report bugs upstream

Sylvestre Ledru

Status

Sylvestre Ledru

Packages LLVM-toolchain containing
LLVM + Clang + compiler-rt + polly + lldb

version 3.2 & 3.3 waiting in Debian NEW

Sylvestre Ledru

Build results integrated on the Debian
Package Tracking System

By Paul Wise

Sylvestre Ledru

Version of pbuilder/cowbuilder with a hook to
change the compiler
By Junich Uekawa
Exemple :

sudo CHOOSE_COMPILER=clang cowbuilder --build yourpackage_1.00-1.dsc

Sylvestre Ledru

Interest by upstreams (Google, Intel & Apple)

Publication of Debian (wheezy, unstable) and
Ubuntu (Quantal, Precise, Raring) of Nightly
packages of LLVM, Clang, Compiler-rt, Polly &
LLDB

http://llvm.org/apt
http://blog.llvm.org/2013/04/llvm-debianubuntu-nightly-packages.html

Sylvestre Ledru

Future

Sylvestre Ledru

Discussions to move the build as a new port
on http://www.debian-ports.org/

Various potential issues (not really new arch, mix
of gcc/clang-built packages, etc)

Sylvestre Ledru

Hack the Debian tools to :

Force dpkg to export CC=/usr/bin/cc,
●

CXX=/usr/bin/c++ and OBJC=/usr/bin/objc

● Check for hardcoded CC=gcc in debian/rules

● Set the /usr/bin/{cc,c++,objc} alternatives to
{clang, clang++}

Sylvestre Ledru

Update the debian policy to include something
like :
Hardcoded usage of CC or CXX (for example, CC=gcc) should be
avoid and documented if necessary.
Debian build tools must respect the CC and CXX variables if
provided. If not, they shall default to /usr/bin/cc and /usr/bin/c++

See :
http://lists.debian.org/debian-devel/2012/08/msg00783.html

Sylvestre Ledru

Add a lintian warning like

W: yourpackage: Hardcoded call to gcc/g++.
Use /usr/bin/cc or /usr/bin/c++ instead

Sylvestre Ledru

Potential the rebuild of Debian with :

●clang+plugin. Ex : polly : cache-locality
optimisation auto-parallelism and vectorization, etc

● address sanitizer (ASAN)

● scan-build : static C/C++ analyzer

● Intel compilers
Sylvestre Ledru

Any questions ? Remarks ?
Troll ? (+1)

Sylvestre Ledru

Distro Recipes 2013 : Make Debian and compiler agnostic

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Distro Recipes 2013 : Make Debian and compiler agnostic

Similar to Distro Recipes 2013 : Make Debian and compiler agnostic (20)

More from Anne Nicolas

More from Anne Nicolas (20)

Recently uploaded

Recently uploaded (20)

Distro Recipes 2013 : Make Debian and compiler agnostic