by 許立文 (lwhsu)

1. Python on FreeBSD
Li-Wen Hsu <lwhsu@{lwhsu,FreeBSD}.org>
PyCon Taiwan 2012

2. Who is this ↘

3. lwhsu:
2006 開始接觸 Python
2007 得到 FreeBSD ports commit bit
2008 Sysadm & Instructor at CS.NCTU
2009 國軍 online
2010~ 菜鳥碼農+修電腦的
另外也是管 {ftp,www,cvsup,anoncvs,
portsnap,update}.tw.freebsd.org 的

4. </開場白>

5. Python
on
FreeBSD

6. What is
FreeBSD?

7. The FreeBSD Project
• FreeBSD is an advanced operating system
derived from BSD, the version of UNIX®
developed at the University of California,
Berkeley.
o http://www.freebsd.org/about.html

8. Who’s using FreeBSD?
• FreeBSD is used in the devices
from the world’s big companies:
o Apple
o Blue Coat
o Cisco
o F5 Networks
o Juniper
o NetApp
o Sony’s PlayStation 3

9. Who’s also using FreeBSD?
• FreeBSD is also used to power
some of the busiest sites on
Internet:
o Apache
o Netflix
o Pixnet
o WhatsApp
o Yahoo!
o Yandex

10. More?
• Projects based on FreeBSD
o PC-BSD
o m0n0wall
o psSense
o FreeNAS
o Debian GNU/kFreeBSD
o Gentoo/FreeBSD

11. FreeBSD: The Good Parts
• BSD License • TrustedBSD MAC
• GEOM Framework
• Netgraph • TrustedBSD Audit
• Jail • LLVM & Clang
• ZFS • Ports system
• DTrace • And more…
http://www.freebsd.org/features.html

12. </FreeBSD>
http://wiki.freebsd.org/Myths

13. <FreeBSD+Python>

14. Origin
• /usr/ports/lang/python/Makefile:
# New ports collection makefile for: python
# Date created: 08 August 1995
# Whom: jkh
"Director of Engineering of Unix Technologies" at Apple

15. python@FreeBSD.org
• Mailing list freebsd-python@FreeBSD.org
o FreeBSD-specific Python issues
o Founded in January 2004
o http://lists.freebsd.org/mailman/listinfo/freebsd-python
o Maintaining python and python related ports, except py-qt and
py-gtk
• gnome@FreeBSD.org
• kde@FreeBSD.org

16. python@FreeBSD.org
• Wiki
o http://wiki.freebsd.org/Python
o Who has the “Hat”
o Action items
o Contact information
• IRC
o #freebsd-python on FreeNode

17. Friends of
python@FreeBSD.org
• zope@FreeBSD.org
o Zope for FreeBSD -- porting and maintaining
o http://lists.freebsd.org/mailman/listinfo/freebsd-zope
o Founded in April 2012
• M_FBSD.python at bbs.cs.nctu.edu.tw

18. python@FreeBSD.org
• People
o alexbl (alumni) o perky (alumni)
o clsung (Taiwan) o rm
o lwhsu (Taiwan) o sunpoet (Taiwan)
o miwi o wen
o nivit o yzlin (Taiwan)

19. Install Python on FreeBSD
• Via ports system
o cd /usr/ports/lang/python
o make install clean

20. Install Python on FreeBSD
• Ports options

21. Compiling...
http://xkcd.com/303/

22. import antigravity
http://xkcd.com/353/

23. Install Python on FreeBSD
• Via package system
o pkg_add -r python

24. Install Python on FreeBSD
• Via package system (advanced)
o setenv PACKAGEROOT http://ftp.tw.freebsd.org
o setenv PACKAGESITE
http://ftp.tw.freebsd.org/pub/FreeBSD/ports/`un
ame -p`/packages-`uname -r | cut -d - -f 1,2 | tr
"[:upper:]" "[:lower:]"`/Latest/
o pkg_add -r python

25. Install Python on FreeBSD
• In the (on so far) future:
• Pkgng
o pkg install python

26. Wait, I have a question!
• Q: Why compile Python myself?
• A:
o Optimize for your system
o Customize build options
o Easier to hack into Python in the system

27. Wait, I have a question!
• Q: Do I really need to compile myself
on *every* machine I want to deploy?
• A: Build once, install everywhere.
o cd /usr/ports/lang/python
o make package
• /usr/ports/packages/lang/python-2.7,2.tbz
o install
• pkg_add python-2.7,2.tbz

28. Run Python on FreeBSD
• In /usr/ports • Also in /usr/ports
o lang/python o lang/jython
• (meta package) o lang/pypy
o lang/python24
o lang/ironpython
o lang/python25
o lang/python26
o devel/ipython
o lang/python27
o lang/python31
o lang/python32

29. Run many Python
versions on FreeBSD
• /usr/local/bin/python
o A symbolic link to default version
o PYTHON_DEFAULT_VERSION
• Can be manually defined in /etc/make.conf
• Possible values: python{2.4,2.5,2.6,2.7,3.1,3.2}
• You can install as many different
of Python versions you want
o just install via /usr/ports/lang/python*
o First installed Python version will be the default

30. Run many Python
versions on FreeBSD
• Install modules for different python
versions
o cd /usr/ports/foo/py-bar
o make PYTHON_VERSION=python2.6 install clean
o This is still work in progress, there might be some
minor errors.
• For modules install scripts in bin/

31. Wait, I have a question!
• Q: there is XXX, why do I need ports to
install python modules?
o (XXX=distutils, setuptools, distribute, distutils2, pip)
• A:
o For better integration with system
o Do more complex deploy
o Ex: install web applications like reviewboard, trac
• A system only consists with python and python module
• Also apache/nginx, mod_wsgi, etc.

32. Develop with Python on
FreeBSD
• virtualenv + Jenkins
o devel/py-virtualenv
o devel/jenkins
• Use virtualenv in jenkins, make sure
clean dependency.
• Have a job to build a virtualenv that is
usable by other jobs.

33. Develop with Python on
FreeBSD
• ZFS + jail
• Jail
o Light weight virtual machine
o /etc/rc.conf
• jail_enable="YES”
• ZFS
o zpool create tank ad1
o zfs create tank/jail1

34. Develop with Python on
FreeBSD
• Create a new jail
o bsdinstall jail /tank/jail1
o freebsd-update -b /tank/jail1 fetch install
o Modify /etc/rc.conf
• jail_jail1_rootdir="/tank/jail1”
• jail_jail1_hostname="jail1”
• jail_jail1_ip="192.168.xxx.xxx”
• jail_jail1_interface="lo0"
o /etc/rc.d/jail start jail1

35. Develop with Python on
FreeBSD
• Clone a jail
o zfs snapshot tank/jail1@milestone1
o zfs clone tank/jail1@milestone1 tank/jail2
o Modify /etc/rc.conf
o /etc/rc.d/jail start jail2
o Another development environment is created!

36. Run your Python Apps on
FreeBSD
• Nothing special or different with
other Unix-like platform

37. Run your Python Web
Apps
• GIL problem
<VirtualHost *:80>
DocumentRoot "/home/www/webroot"
ServerName python.example.com
WSGIProcessGroup pythonexample
WSGIDaemonProcess pythonexample processes=16
threads=128 maximum-requests=1024
display-name=%{GROUP}
</VirtualHost>

38. Make Python
module port

39. FreeBSD Ports
• Each ``port’’ contains any patches
necessary to make the original application
source code compile and run on FreeBSD.
Installing an application is as simple as
typing make in the port directory.
o http://www.freebsd.org/ports/

40. Inside a port
• ls -1 /usr/ports/devel/py-coverage
Makefile
distinfo
pkg-descr
pkg-plist

41. Create a new Python port
• Use porttools to help:
o cd /usr/ports/ports-mgmt/porttools
o make install clean
• Create a new port
o cd /your/workspace
o port create py-newport

42. Makefile
# New ports collection makefile for: py-coverage
# Date created: Jun. 22, 2009
# Whom: Li-Wen Hsu <lwhsu@FreeBSD.org>
#
# $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $
#
PORTNAME= coverage
PORTVERSION= 3.5.2
CATEGORIES= devel python
MASTER_SITES= CHEESESHOP
PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
MAINTAINER= lwhsu@FreeBSD.org
COMMENT= Code coverage measurement for Python
LICENSE= BSD
USE_PYTHON= yes
USE_PYDISTUTILS= easy_install
PYEASYINSTALL_ARCHDEP= yes
.include <bsd.port.mk>

43. Makefile
# New ports collection makefile for: py-coverage
# Date created: Jun. 22, 2009
# Whom: Li-Wen Hsu <lwhsu@FreeBSD.org>
#
# $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11
lwhsu Exp $
#

44. Makefile
PORTNAME= coverage
PORTVERSION= 3.5.2
CATEGORIES= devel python
MASTER_SITES= CHEESESHOP
PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}

45. Makefile
LICENSE= BSD
USE_PYTHON= yes # 2.6 2.6+ -2.7 3.1+
USE_PYDISTUTILS= easy_install
PYEASYINSTALL_ARCHDEP= yes
.include <bsd.port.mk>

46. distinfo
SHA256 (coverage-3.5.2.tar.gz) =
b9e0dc65e42236a3c9b8978701bc9e7298ed7e24d39f99
ea2ba4b945d537fe55
SIZE (coverage-3.5.2.tar.gz) = 115497

47. pkg-descr
Coverage measures code coverage, typically during test execution.
It uses the code analysis tools and tracing hooks provided in the
Python standard library to determine which lines are executable,
and which have been executed.
WWW: http://nedbatchelder.com/code/coverage

48. pkg-plist
bin/coverage
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/PKG-INFO
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/SOURCES.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/dependency_links.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/entry_points.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/native_libs.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/not-zip-safe
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/top_level.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.py
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyc
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyo
[...]
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.py
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyc
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyo
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/htmlfiles
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%

49. Before submitting
• Check
o Use portlint (ports-mgmt/portlint)
o Command:portint –abct
• Test
o Use porttools (ports-mgmt/porttools)
o Command: port test

50. Before submitting
• More test
o Use Tinderbox (ports-mgmt/tinderbox)
o A comprehensive way to test ports
• And builds packages!

52. Wait, I have a question!
• Q: Setting up a tinderbox is too complex
• A: Use RedPorts http://redports.org/

53. Submit!
• In your port directory
• Command:
o port submit
• Quit editor:
o s)end,
e)dit or
a)bort? s

54. After submitting
• A PR (problem report) is created
• A committer will handle it
• A new port is in the ports tree!
o See on http://freshports.org/foo/bar
• To update your ports tree:
o portsnap fetch update
• Install the new port:
o cd /usr/ports/foo/bar; make install clean

55. Future Work of
python@FreeBSD.org
• Remove 2.4 and 2.5 in the ports tree
o 2012-06-30
• Handle packages with pip/distribute
• Debug versions of lang/python*
• Install only unzipped eggs
• Truly modules for multiple python
versions co-exist

56. Q&A