This document discusses TensorFlow and provides information about: 1. Major TensorFlow releases from 2016-2018 and new features introduced each year like TensorFlow Serving, Keras API, TensorFlow Lite, etc. 2. Common TensorFlow operations that have changed names or behaviors between TensorFlow 1.x and 2.x like tf.variable, tf.global_variables_initializer, and Session.run. 3. How to distribute training across multiple GPUs and CPUs using strategies like MirroredStrategy in TensorFlow 2.x.

1. 2 2
2 2 2
A D
9 01 9! 1/ . 2 1

7. • l io … a ~ nt
• 1 ()
S m s f… dX e h
S c
• C ()+ ()/
S io PMC rd yk
• 5 K L ()-
S 7 M K K S T bwp u bwp
S NJJ KM L
ü C NFM L K3F (). 7 8 M ()/ 18 5 ()0
• M
S F 1C D K 26,4

8. • h S~g vk m e uny tb
• .GG D ld a T U ( ) (o N
• x
O 5MI GF 5/ DAC FG
ü t 0 V P jis F IA AF AF U
O v
ü 1AFJL
ü /,/ .5 5 D tc 2, .5 A A tc

9. • ) (/i ­ h
0 D MM TS S DF 4FD ) (,
0 ( D T CUT S
0 . PULL F UFSTS
0 ) G KS f () ­
ü 2 TST BP l v ! (,
0 F S 6L cq 7 T8UC
• ~
3 MPLFTF M EFL P T T P H
4 ST CUTFE T B H
3 7 C LF SUPP T
• F S 6L >F H
l s d
• A 1 D MP LF (
l y x ~ b
• F BS 1 >UPP T ( )
a repw 1 F BS
• 5BHF 5XFDUT (
TF BDT F M EF
nu ep k TI
nr g
• F S 6L 4BTB (
m u lth o b

10. 2016
2017
⏤ TensorFlow Serving
⏤ Keras API
⏤ Eager Execution
⏤ TensorFlow Lite
⏤ XLA
⏤ OpenCL w/ OpenCompute
⏤ Distributed TensorFlow
⏤ Multi GPU support
⏤ Mobile TensorFlow
⏤ TensorFlow Datasets
⏤ SKLearn (contrib)
⏤ TensorFlow Slim
⏤ SyntaxNet
⏤ DRAGNN
⏤ TFLearn (contrib)
⏤ TensorFlow TimeSeries
2018
⏤ TensorFlow Hub
⏤ Swift for TensorFlow
⏤ TensorFlow Models
⏤ ROCm TensorFlow (Test)
⏤ TensorRT integration

11. • 3
L IA GE EG
ü ) G IAED
ü -3 GI AD A
L D 3
• -3
L ) 1 A A ➜ 11
L D) ) ➜ EC I 33 ➜
C PU Oe
• 3 I G ? D
L dS X c ) .
L -EE? MV
• ? 3
L NM a! TR b(

12. • + +
: + +)
: A P
(+
• a (
: CA C U
a NP b
: aI NPb
c
: D G

13. • I K1G P 0 EH K (
R t l[ EH K s fo T a V r Tix .74
• I K1G P 9 K EID c w
• I K1G P K .74 ) (
• I K1G P / / / 9 (
• I K1G P D K AN E I ( S ( , ) vj u
• I K1G P F ( , )
• 9PE G IDN D K I K1G P , ( , S ( US nv
• I K1G P 3N US b
• I K8
R 4/4. 27 mg e ]

14. •
•
Servables
Loaders
Managers
•
/
RPC (gRPC ) REST API

16. • . 8 (
. 8 (
. 8 (
1 2 . 8 (
.( ) 1
. 8 ( 1
• 2 . 8 (
1
.( 1
. 8 . 8
1 (

17. •
•
TransformExtract Load

18. •
•
Repositories

19. •
•
TensorFlow
Hub

20. • ah e n P
• (F g
22 2 8 A .28
8 TiD c p )
•
ah o
l . Id . I
ü pip install tensorflowjs
8 12 0 83 j

21. • l
9 U
9 0, Ge as U y­ m
9 b P v
ü 1 1 !
• s
9 2 ,
ü Ge AA ? X
ü Ge ­: J v
9 2 2
ü U y­
ü WL !

22. • P
• O
•
A
A S
ID B F A K
P

28. • T
n w o n
A
•
w T e D I
u FD b
•
• s s A
r ­ b
• b
1 51 8 0/ 0 5 2 9 iP A l
d
t b

29. ( 1 ) 2
• 2 3 e
A ­
A r I o P y e / 8 / E F lG
• 038 r w
A r ha u
•
A 7/ u r s
A n g ,/1 t
A
•
A / 8 N U
A mI
A a
A / 8 T·

30. ) ( ) 2
• o
o
• h E G a
@defun A o
ü : @py_func o d
FunctionDefA p g
FunctionDef e
• o c

31. ( ) 23
• , A 0 9 7 87 h p d s G
E c Pr
E 7 87 h ia Ty b a h
E
ü n F w gr
ü n 27 / C p . 87 h p u l
• 1 A9 p 67 8 Pr
E 1 A9 e o 27 / C e
E , 7 A : A: o 27 / Cp u
E A 77 t

32. ( ) 4 2
•
@autograph.convert()
def my_dynamic_rnn(rnn_cell, input_data, initial_state, se
q_len):
outputs = tf.TensorArray(tf.float32, input_data.shape[0])
state = initial_state
max_seq_len = tf.reduce_max(seq_len)
for i in tf.range(max_seq_len):
new_output, new_state = rnn_cell(input_data[i], state)
output = tf.where(i < seq_len, new_output, tf.zeros_lik
e(new_output))
state = tf.where(i < sequence_length, new_state, state)
outputs = outputs.write(i, output)
return tf.transpose(outputs.stack(), [1, 0, 2]), state
def tf__my_dynamic_rnn(rnn_cell, input_data, initial_state, sequence_length):
try:
with tf.name_scope('my_dynamic_rnn'):
outputs = tf.TensorArray(tf.float32, ag__.get_item(input_data.shape,
0, opts=ag__.GetItemOpts(element_dtype=None)))
state = initial_state
max_sequence_length = tf.reduce_max(sequence_length)
def extra_test(state_1, outputs_1):
with tf.name_scope('extra_test'):
return True
def loop_body(loop_vars, state_1, outputs_1):
with tf.name_scope('loop_body'):
i = loop_vars
new_output, new_state = ag__.converted_call(rnn_cell, True, False,
False, {}, ag__.get_item(input_data, i, opts=ag__.GetItemOpts
(element_dtype=None)), state_1)
output = tf.where(tf.less(i, sequence_length), new_output, tf.
zeros(new_output.shape))
state_1 = tf.where(tf.less(i, sequence_length), new_state, state_1)
outputs_1 = outputs_1.write(i, output)
return state_1, outputs_1
state, outputs = ag__.for_stmt(tf.range(max_sequence_length),
extra_test, loop_body, (state, outputs))
return tf.transpose(outputs.stack(), ag__.new_list([1, 0, 2])), state
except:
ag__.rewrite_graph_construction_error(ag_source_map__)

33. • n o A
a o A hr ➜ ho
• a
tf.io tf.nn tf.math
t m ,(
• r
), G :p : o
dim ➜ axis, keep_prob ➜ rate

34. 2
• Keras ( https://keras.io )
High-level API —
“Deep learning accessible to everyone”
•
2017 2
TensorFlow 1.2 contributor
TensorFlow 1.4
•
“Simplified workflow for TensorFlow users, more powerful features to Keras users”
Keras (keras. tf.keras.)
TensorFlow Keras
tf.keras.
• !

35. () / : 1 2
• ( F
) 1 ,
ü tf.variable
) 1 , :
ü tf.global_variables_initializer() tf.get_global_step() :
) T ,
) , 1 , ? ,

36. ( ) / 2 : 2
•
tf.global_variables_initializer() tf.get_global_step()
V R
• ) (, R
tf.assign(x,100) ➜ x.assign(100)
•
K R

37. ( ) 1 2
• Session.run() e
: DF o Ghu o
: t a dP A s e r p y c
•
: T ) r o n ,
: , r P wl (
# TensorFlow 1.X
outputs = session.run(f(placeholder), feed_dict={placeholder: input})
# TensorFlow 2.0
outputs = f(input)

38. ) (
•
print ➜ tf.print
assert ➜ tf.Assert
for/while ➜ tf.while_loop (break and continue are supported)
if ➜ tf.cond
for _ in dataset ➜ dataset.reduce

39. ( ) 1 2
• + A A 1A A
F e + A A 2 DT bo
lM rRc g n
F tf.distribute
• Ua
F g -
F g -
F P
• 1A A d S
F . 1A A
ü )
ü rR
F 2 1A A (
• i b G m A A
F A C ) 0 1A A (
ü P d . 1A A
F A 1 C 1A A (

40. ) ( 2
•
strategy = tf.distribute.MirroredStrategy()
config = tf.estimator.RunConfig(
train_distribute=strategy, eval_distribute=strategy)
regressor = tf.estimator.LinearRegressor(
feature_columns=[tf.feature_column.numeric_column('feats')],
optimizer='SGD',
config=config)
def input_fn():
return tf.data.Dataset.from_tensors(({"feats":[1.]}, [1.]))
.repeat(10000).batch(10)
regressor.train(input_fn=input_fn, steps=10)
regressor.evaluate(input_fn=input_fn, steps=10)

41. ( ) 23
•
train_dataset = tf.data.Dataset(...)
eval_dataset = tf.data.Dataset(...)
model = tf.keras.applications.ResNet50()
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
model.compile(loss="categorical_crossentropy", optimizer=optimizer)
model.fit(train_dataset, epochs=10)
model.evaluate(eval_dataset)

42. ) 4 2 (
• /
train_dataset = tf.data.Dataset(...)
eval_dataset = tf.data.Dataset(...)
model = tf.keras.applications.ResNet50()
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
strategy = tf.contrib.distribute.MirroredStrategy()
model.compile(loss="categorical_crossentropy", optimizer=optimizer,
distribute=strategy)
model.fit(train_dataset, epochs=10)
model.evaluate(eval_dataset)

43. . . 2
• tf.contrib S i a
:F a
ü R tf.contrib.slim
a
L Ti
• R
tf.contrib.slim
tf.contrib.tflearn
tf.contrib.tensorrt e

44. ) 1 ( 2
• 5: 0 . 5// 5: 0 . /GDD CL /G FC CGFI C N
P d V P d 2 9 Rwn~ c
P .DD A DD
P 81 ~ 5// ( ( U
ü tf.contrib.nccl
ü d V sr wn~ V g
P 8 FIG 1DG ( - 8 9 b Xv 2 9
• 5: 0 . 8 FIG 78
P a ~ c
ü tf.contrib.tensorrt
P 8 FIG 1DG y~ l ~ o (
P 5: 0 . :GD t­ wn 8 FIG /G i h e m
ü : R 7 I5 ) T u
P 8 FIG 1DG (- GF C wn G 8 FIG 1DG w

45. ( ) 2
• 2 0.0 5 A 5 rv
oS c g et T
ü tf.contrib.tensorrt
5 A/ bg rv ) g o F
2 0.0 C7 na 5 A A V iM c d
ü I 2 C( l ) N
• 5 A/ b 2 0.0 5 A 5
9 CA 8a 9 A R M 5 A/ a s
7D 1 w o
5 A/ AD w o

46. ) ( 23
• /81 0 m
U NDL0 Xa Vg
U 3 - . m 3HIH M OHP DF
U 06D DL+ 06D / M HC N m 0 e s
ü TDL 2 A0
ü DML + 0MOD h w PRD r
U 0 m ()-Y ) n
ü NDL0 ( Xa
ü z m DLPMOE MR OMC ) )
U 3 ), Xa l rf m
• DLPMO3 MR
U 9HFG c
U iy LHFG t o jp

47. • K
CL : M D K
K
C D 8 3

48. 2 1
• 0 .
). (2 i F
d !
). (2 . 1. f
ü F ea_ F
T X g T X f
tf_upgrade_v2 --infile tf_example-v1.py --outfile tf_example-v2.py
'tensorflow/tools/compatibility/testdata/test_file_v1_12.py' Line 65
--------------------------------------------------------------------------------
Added keyword 'input' to reordered function 'tf.argmax'
Renamed keyword argument from 'dimension' to 'axis'
Old: tf.argmax([[1, 3, 2]], dimension=0))
~~~~~~~~~~
New: tf.argmax(input=[[1, 3, 2]], axis=0))

49. • 1. . . 2
1. .
1. . F 1. . X T

50. • 29 + 29121
b P a
+ 2 ? 9 9 3 8
dT im f S F
• 29 + , 0 0
a
p x S
• 29 29
f S h
S n dT e n
­
• , 9 ➜ 3
E
y
ts 9v
l
ü 9 d f a 3 +v
w r o
ü 3 g 29 + r o

51. !
B : / / CA!
/ :
/ :
/ :
/ :
:.
@@@ /
@@@ / : /
/ / : /
/ : /
@@@ :@