1. STRING ENCODINGS

2. I’m still
@malditogeek

3. We’re hiring!

4. STRING ENCODINGS

7. FIRST THERE WAS...
ASCII

9. BUT WHAT ABOUT
THE 8TH BIT?

10. UH OH!

11. BUT MY
LANGUAGE
HAS 2000
CHARACTERS

12. WELCOME TO MULTIBYTE
ENCODINGS

13. UNICODE

14. CHARACTER SETS
vs
ENCODINGS

15. CHARACTER SETS
U+0061 = 97 = a

16. ENCODINGS
" abc ".encode("UTF-32BE")
"x00x00x00ax00x00x00bx00x00x00c"

17. UTF-8

18. RUBY 1.8
STRINGS ARE JUST
BYTE ARRAYS!

19. $KCODE=’u’

20. RUBY 1.9

21. M17N

22. STRING EXAMPLES
e = "é"
e.encoding.name # => UTF-8
e.size # => 1
e.bytesize # => 2

23. FORCING ENCODINGS
x.encoding.name #=> ISO-8859-1
x.bytesize #=> 6
x.valid_encoding? #=> true
x.force_encoding("UTF-8")
x.encoding.name #=> UTF-8
x.bytesize #=> 6
x.valid_encoding? #=> false
x =~ /x/ #=>
invalid byte sequence in UTF-8

24. TRANSCODING
x.encoding.name #=> ISO-8859-1
x.bytesize #=> 6
x.valid_encoding? #=> true
x.encode!("UTF-8")
x.encoding.name #=> UTF-8
x.bytesize #=> 12
x.valid_encoding? #=> true

25. COMPATIBILITY
if Encoding.compatible?(ascii_string, utf8_string)
new_string = ascii_string + utf8_string
new_string.encoding.name #=> UTF-8
end

26. ITERATION
x.bytes
x.each_byte {|b| puts b }
x.codepoints
x.each_codepoint {|c| puts c }
x.chars
x.each_char {|c| puts c }
x.lines
x.each_line {|l| puts l }

27. SOURCE ENCODING
# encoding: UTF-8

28. INTERNAL ENCODING
> ruby -E :UTF-8
Encoding.default_internal = 'UTF-8'

29. EXTERNAL ENCODING
> ruby -E UTF-8:
Encoding.default_external = 'UTF-8'
File.open("file.txt", "w:UTF-8")

30. WHAT DO?
USE MAGIC COMMENTS
DECLARE IO ENCODINGS
CONVERT BEFORE COMPARISONS
UNICODE-UTILS GEM

31. C’EST FINI

32. We’re hiring!

33. Thanks!
http://blog.grayproductions.net/
articles/understanding_m17n