Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Sun sparc enterprise t5120 and t5220 servers technical presentation
1. Sun SPARC Enterprise
T5120 and T5220
Rack Server TTT
Peter A. Wilson
Technical Marketing
Sun Microsystems, Inc.
1
2. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to Sun Fire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 2
3. CMT Processor Strategy
First True System on a Chip
• Scaling performance with frequency is
a dead-end
• Multi-core/Multi-thread processors can
exploit application parallelism
• Able to Aggressively attack memory
latency
• Provide rich availability feature set
• Integrate networking, I/O and security
capability
• Virtualize the architecture
Sun Internal and Approved Partners Only 3
4. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to Sun Fire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 4
5. UltraSPARC T2: • Up to 8 cores @1.2GHz or 1.4GHz
• Up to 64 threads per CPU
TrueSystem On a Chip • Up to 16 FB-DIMMs, 4 memory
controllers
> Up to 64GB memory (4GB DIMMs)
FB DIMM FB DIMM FB DIMM FB DIMM
> 2.5x memory BW of UltraSPARC T1
FB DIMM FB DIMM FB DIMM FB DIMM • 8 x fully pipelined Floating Point
units / core, 1 per core
MCU MCU MCU MCU • Dual 10Gbit Ethernet and PCI-E
L2$ L2$ L2$ L2$ L2$ L2$ L2$ L2$
integrated onto chip
Full Cross Bar
• 4MB L2 (8 banks) 16 way
C0 C1 C2 C3 C4 C5 C6 C7
associative
FPU FPU FPU FPU FPU FPU FPU FPU • Enhanced MAU/Security co-
MAU MAU MAU MAU MAU MAU MAU MAU
DMA Sub-System
processor per core
> DES, 3DES, AES, RC4, SHA1,
NIU Sys I/F
SHA256, MD5, RSA to 2048 key,
PCI-E
(E-net+) Buffer Switch Core ECC,CRC32
Power ~95W • Advanced Power saving features
2x 10GE Ethernet <1.5W/thread x8 @2.5GHz
4GBytes/s bi-directional • 65nm process technology
Sun Internal and Approved Partners Only 5
6. UltraSPARC T2 Processor
L2 Data L2 Data
Bank 0 Bank 4
L2B0 SPARC SPARC SPARC SPARC L2B4
L2 Data Core 0 Core 1 Core 5 Core 4 L2 Data
Bank 1 Bank 5
L2B1 L2B5
L2 L2 L2 L2
MCU0 MCU2
NCU
TAG0 TAG1 TAG5 TAG4 FSR
EFU
MCU1 MCU3
SIO
SII
L2B3 CCX L2B7
FSR CCU
L2 Data L2 Data
Bank 3 Bank 7
L2 L2 L2 L2
L2B2 L2B6
TAG2 TAG3 TAG7 TAG6
L2 Data L2 Data
Bank 2 Bank 6
SPARC SPARC SPARC SPARC
DMU
Core 2 Core 3 Core 7 Core 6 RDP TDS
PEU
RTX
PSR ESR FSR MAC
For a video tour with Rick Hetherington, go to http://www.youtube.com/watch?v=B8DNexYcsa4
Sun Internal and Approved Partners Only 6
7. UltraSPARC T2 Core Block Diagram
• IFU – Instruction Fetch Unit
> 16 KB I$, 32B lines, 8-way SA
TLU IFU > 64-entry fully-associative ITLB
• EXU0/1 – Integer Execution Units
> 4 threads share each unit
> Executes one integer
EXU0 EXU1 instruction/cycle
• LSU – Load/Store Unit
MMU/ > 8KB D$, 16B lines, 4-way SA
HWTW > 128-entry fully-associative
DTLB
• FGU – Floating/Graphics Unit
SPU FGU LSU • SPU – Stream Processing Unit
> Cryptographic acceleration
• TLU – Trap Logic Unit
> Updates machine state, handles
exceptions and interrupts
Gasket
• MMU – Memory Management Unit
> Hardware tablewalk (HWTW)
xbar/L2 > 8KB, 64KB, 4MB, 256MB pages
Sun Internal and Approved Partners Only 7
8. Floating Point Unit
• Each Core has its own Floating Point Unit. This removes
completely the 2% FP restriction of T1.
• Only a 6 cycle penalty to the FPU
• Fully-pipelined (except divide/sqrt)
> Divide/sqrt in parallel with add or multiply operations of
Data
other threads
• Full VIS 2.0 implementation
• FPU performs integer multiply, divide, population count
• Multiplier enhancements for modular arithmetic operations
in the Crypto Unit
• Performance is impressive...testing and tuning still
happening
Sun Internal and Approved Partners Only 8
9. Niagara 2 Crypto Performance
• Supports ten most common ciphers and
Comment
secure hashing functionsCycles/
Iterations/ Block
Block (bytes) Block GB/S
2 rounds/cycle > RC4, DES, 3DES, AES-128, AES0192, AES-256, MDS, SHA-1,
2 rounds/cycle SHA-256
16 rounds 8 8/9+ 12.0
3 overlapped cycles 48 rounds 8 25/24 4.0
Data • On-chip crypto accelerators and built-in 10GbE vs.
4 words into 44 words 1 1 37683 12.0
accelerator cards
6 words into 52 words - - 32 -
with UltraSPARC T2
8 words into 60 words - - 37 -
New Ciphers
>
2 column over two cycles <36% crypto performance 41
- - -
2 column over two cycles 10 rounds 16 30/30 6.36
>
2 column over two cycles
>23% CPU overhead
12 rounds 16 36/36 5.28
2 cycles per step
• Leading cryptography performance
2 cycles per step
14 rounds
80 steps
16
64
42/42
160/160
4.56
4.80
2 cycles per step > Over 30x greater RSA1024 performance than 2-socket
64 steps 64 128/128 6.00 IBM p510
32 bits per cycle 64 steps 64 128/128 6.00
• Best SSL Performance
1 4 1 48.00
> 2.2x better performace/watt than 4-socket HP DL585G2
> 1.74x better performance/watt than 2-socket HP DL380G5
> 15.6x better AES128 performance than crypto acclerator card
Sun Internal and Approved Partners Only 9
10. FB-DIMM Memory Interface
Feature UltraSPARC T1 UltraSPARC T2
Require Pins 1000 440
Available Bandwidth 23GBytes/second 50Gbytes/second
Data Up to 64x DIMM slots
Memory Capacity up to 16x DIMM slots
T5x20 uses 16 DIMM slots
Memory Controllers 4 4
DIMMS/Channel/Controller 02/02/04 08/02/04
Memory Latency 96ns 126ns
DIMM speeds 400MHz 667MHz
DIMM Power 7watts/DIMM 12.5Watts/DIMM
Sun Internal and Approved Partners Only 10
11. 10GbE Network Interface Unit
• Fully Integrated Network Interface Unit
> Eliminates PCI-E chipset latencies
> 30% better performance vs PCI-E NIC
– Mainly through reduced latency
• Two 10 GbE ports (XAUI)
• 16 Rx and 16 Tx DMA channels for virtualization
Data
• Packet classification
> Incoming flows are classified at layer 2 ,3 and 4 and put
into the Rx DMA buffer according to the classification
rules that matched the flow
• nxge device driver, source base shared with Atlas
• Performance estimates, testing/tuning ongoing
– 1 port TX/RX = about 9.4Gbps in each direction with about
only 40-50% CPU utilisation
– 2 port TX/RX = 12GbpsTX, 18GbpsRX with 70-80% CPU
utilisation (still 20 threads available for application)
Sun Internal and Approved Partners Only 11
12. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 12
13. Sun SPARC Enterprise T5x20 Extreme Rackmount Density
UltraSPARC T2 Platform • 1/2U chassis, 28” depth
• Up to 8 cores @ 1.2 - 1.4GHz
> Up to 64 threads
• 16 FB-DIMM memory slots
> 64GB of memory, (4GB DIMMs)
High Reliability
• 4 to 16 hot plug SAS 2.5” disk
• Optimized for best price / drive (RAID 0,1)
performance & performance / watt • N+N Redundant, hot-swappable
PSUs and Fans. DC PSU option
• Scale-Out of web & network for 1U systems only
infrastructure Expandable
• Extension to T1000/T2000 • 3/6 PCI-E expansion slots
(including 2 x 10 GBE I/0 slots)
product line • 4 x 1GbE Ethernet as standard
• 4 USB ports
Available Now
Sun Internal Only 13
14. Sun Fire T1000 / T5120 Differences
Feature Sun Fire T1000 SPARC Enterprise T5120
Form Factor 1U, 18¾” deep 1U, 28” deep
UltraSPARC T1 UltraSPARC T2
CPU
1.0 GHz, 32 threads 1.2/1.4 GHz 64 threads
DDR2 32GB Max FB-DIMM 64GB MAX
Memory
8x slots 16x slots
Network 4x GbE 4x GbE + 2x 10GbE
1x 3.5” SATA, non-hot-swap
Internal Storage Up to 8 x 2.5” SAS, hot-swap
OR 2x 2.5” SAS
Removable Media None 1x DVD-ROM
Serial 1x RS-232, No USB 1x RS-232, 4x USB
PCI Express slots 1x (low profile) 3x(low profile)
1x 300W AC 2x 650W AC & DC option
Power Supply
Non-Redundant Redundant/Hot-Swap
Fans Non-Hot-Swap Redundant/Hot-Swap
Sun Internal and Approved Partners Only 14
15. Sun Fire T2000 / T5220 Differences
Feature Sun Fire T2000 SPARC Enterprise T5220
Form Factor 2U, 24” deep 2U, 28” deep
UltraSPARC T1 UltraSPARC T2
CPU
1.0 - 1.4GHz 32 threads 1.2/1.4 GHz 64 threads
DDR2, 64GB MAX FB-DIMM, 64GB MAX
Memory
16x Slots 16x Slots
Network(10/100/1000) 4x GbE 4x GbE + 2x 10GbE
Internal Storage 4x 2.5” SAS, hot-swap Up to 16 x 2.5” SAS, hot-swap
Removable Media 1x DVD-ROM 1x DVD-ROM
Serial 1x RS-232, 4x USB 1x RS-232, 4x USB
PCI Express slots 3x(low profile) 6x(low profile)
PCI-X slots 2 x (low profile) None
2x 450W AC 2x 750W AC
Power Supply
Redundant/Hot-Swap Redundant/Hot-Swap
Sun Internal and Approved Partners Only 15
16. Front Panel Dual USB 2.0 ports
Locator LED
/Button DVD-ROM Disk Drive Map
Fault LED Top Fault LED
Status LED DISK 0 DISK 1 DISK 2 DISK 3 PSU Fault LED
Power Button Temp Fault LED
Serial Number label
DISK 1 DISK 3 DISK 5 DVD-ROM
DISK 0 DISK 2 DISK 4 DISK 6 DISK 7
Locator LED
/Button
Fault LED Top Fault LED
Status LED PSU Fault LED
Power Button Temp Fault LED
DISK 1 DISK 3 DISK 5 DVD-ROM
Disk Drive Map
Serial Number
DISK 0 DISK 2 DISK 4 DISK 6 DISK 7
Dual USB 2.0 ports
DISK 3 DISK 7 DISK 12 DISK 15
DISK 2 DISK 6 DISK 11 DISK 14
DISK 1 DISK 5 DISK 9 DVD-ROM
DISK 0 DISK 4 DISK 8 DISK 10 DISK 13
Sun Internal and Approved Partners Only 16
17. Rear Panel
PCI-E Slot 0/XAUI 0 PCI-E Slot 1/XAUI 1 PCI-E Slot 2
x8 Slot/x4 Electrical x8 Slot/x4 Electrical x8 Slot/x8 Electrical
PCI/XAUI 0 PCI/XAUI 1 PCI 2
PSU PSU 0 PSU 1
Status
LED's
Chassis
Status LED's Quad Gigabit Dual POSIX DB-9
and Locator Button Ethernet Ports USB 2.0 Serial Port
SP SP Ports
Serial Network
Port Port
PSU 1
PCI 3 PCI 4 PCI 5
PCI/XAUI 0 PCI/XAUI 1 PCI 2
PSU 0
Chassis
PSU Status LED's Quad Gigabit POSIX DB-9
Status Dual
and Locator Button Ethernet Ports USB 2.0 Serial Port
LED's SP SP Ports
Serial Network
Port Port
Sun Internal and Approved Partners Only 17
18. Architecture Block Diagram - T5120
16 x FB-DIMM 4 Memory Controllers, 2 Channels, 2 DIMMS
(DDR2-667)
Disk Chassis
1RU
SSI
x8 FPGA
10GbE
PCI-E 10GbE
x4 LSI x4 Switch x8 MPC885
SAS links x4 1068E ILOM
PLX 8533 Service
DVD x4
x4 Processor
PCI-E
Switch
PCI-E
USB 2.0 PCI-E x1 Switch PLX 8533
to IDE x8
2.0
to 10GbE
USB PLX 8517 x4 SerDes
BCM8704
USB 2.0 x4 x4
Hub 10GbE
USB Intel Intel Cu PHY
2.0
Dual Dual BCMxxxx
GbE GbE XFP
Front Panel
10GbE
0 1 2 3 Fibre
Plugin
USB Quad GbE PCI-E PCI-E PCI-E Serial Network POSIX
Rear Panel Connectors x16 x8 x8 Mgt Mgt Serial DB-9
Sun Internal and Approved Partners Only 18
19. Architecture Block Diagram - T5220
16 x FB-DIMM 4 Memory Controllers, 2 Channels, 2 DIMMS
(DDR2-667)
Disk Chassis
1RU 2RU/8
SSI
x8 FPGA
10GbE
x4 PCI-E 10GbE
LSI x4 Switch x8 MPC885
SAS links x4 1068E ILOM
PLX 8533 Service
DVD x4
x4 Processor
PCI-E
x8 Switch
PCI-E
USB 2.0 PCI-E x1 Switch PLX 8533
to IDE x8
2.0
to 10GbE
USB PLX 8517 x4 x4 x4 SerDes
BCM8704
USB 2.0 x4 x4
Hub 2RU Only 10GbE
USB Intel Intel Cu PHY
2.0
Dual Dual BCMxxxx
GbE GbE XFP
Front Panel
10GbE
0 1 2 3 Fibre
Plugin
USB Quad GbE PCI-E PCI-E PCI-E PCI-E PCI-E PCI-E Serial Network POSIX
Rear Panel Connectors x16 x8 x8 x8 x8 x8 Mgt Mgt Serial DB-9
Sun Internal and Approved Partners Only 19
20. Internal View (T5120) Fault Remind System Configuration
Button Card (SCC)EEPROM
Service Processor
FB1/FM0
FB-DIMMs PCI-E Slot 2
x16(m)/x8(e)
FB-DIMMs
FB0/FM1
FB1/FM1
PCI-E Slot 1
UltraSPARC x8(m)/x4(e)
or
T2 XAUI 1
Disk
Chassis FB-DIMMs
FB1/FM2
PCI-E Slot 0
x8(m)/x4(e)
FB-DIMMs or
XAUI 0
PSU1
Power Power
Board Supplies
PSU0
SAS x4 cable Power BUS Bars Chassis Intrusion Reed
to 4 Fan Assemblies are required Switch X4 SAS Connector 0 (cable attached)
Disk Chassis RED = +12Volts (magnet on underside of lid) X4 SAS Connector 1 (not used in 1RU)
(chassis can take up to 8) BLUE = Ground
Sun Internal and Approved Partners Only 20
21. Internal View (T5220) Fault Remind System Configuration
Button EEPROM
Service Processor
PCI-E Slot 2
FB0/FM0
FB-DIMMs x8(m)/x8(e)
x16(m)/x8(e)
FB-DIMMs
PCI-E Slot 1
UltraSPARC x8(m)/x4(e)
T2 x8(m)/x4(e)
FB0/FM1
or
XAUI 1
Disk
Chassis FB-DIMMs
PCI-E Slot 0
x8(m)/x4(e)
FB-DIMMs x8(m)/x4(e)
or
XAUI 0
FB0/FM2
Power PSU's
Board Power Vertically
Supplies Stacked
Two Power BUS Bars Chassis Intrusion Reed
SAS x4 cables 3 Fan Assemblies are required Switch X4 SAS Connector 0 (cable attached)
to RED = +12Volts (magnet on underside of lid) X4 SAS Connector 1 (cable attached)
(chassis can take up to 6) BLUE = Ground
Disk Chassis
Sun Internal and Approved Partners Only 21
22. Memory Physical Layout
Chassis Chassis
Front Rear
Channel1/DIMM 1/Branch 1 3
Channel1/DIMM 0/Branch 1 2
Channel0/DIMM 1/Branch 1 3
Memory Population Rules
Channel0/DIMM 0/Branch 1 1 1. All Branches must have at least one
DIMM populated
Channel1/DIMM 1/Branch 0 3
2 - min config is 4 x 1GB DIMMS
Channel1/DIMM 0/Branch 0
Channel0/DIMM 1/Branch 0 3 2. Populate Branches starting with the
nearest slot to the CPU first
Channel0/DIMM 0/Branch 0 1
- See “1” in the diagram
C0D0B0, C0D0B1, C0D0B2, C0D0B3
UltraSPARC 3. All DIMMS must be the same size
T2 4. In Branch all DIMMS must be indentical
(SUN Part No.)
- Upgrading from 4 to 8 DIMM
configs means moving DIMMS to
Channel0/DIMM 0/Branch 2 1 ensure sameness
Channel0/DIMM 1/Branch 2 3 5. 12 DIMM configurations are NOT
Channel1/DIMM 0/Branch 2 2 supported.
Channel1/DIMM 1/Branch 2 3 6. Only 1GB, 2GB and 4GB DIMMS are
1 supported
Channel0/DIMM 0/Branch 3
Channel0/DIMM 1/Branch 3 3 7. Memory is FB-DIMM DDR2-667, there
is no clocking down of memory in any
Channel1/DIMM 0/Branch 3 2 config.
Channel1/DIMM 1/Branch 3 3
Sun Internal and Approved Partners Only 22
23. Memory Configuration Rules
• Populate at least 4 DIMMS at a time
• First
> Fill DIMM0 in Channel 0 in each Branch
• Second
> Fill DIMM0 in Channel 1 in each Branch
• Third
> Fill remaining DIMM slots.
• All DIMMS must be the same size(i.e all 1GB, 2GB or 4GB, no
mixing)
• In a Branch all DIMMS must have the same SUN Part No.
• 12 DIMM configurations are NOT supported.
• No clocking down of memory in any config.
Sun Internal and Approved Partners Only 23
25. Network Ports and Options
XAUI 0 XAUI 1
Device NET 0 NET 1 NET 2 NET 3 XAUI 0 XAUI 1
Driver e1000g0 e1000g1 e1000g2 e1000g3 nxge0 nxge1
PCI Device PCI 0/0/1/0/2 PCI 0/0/1/0/3 NIU@80/0 NUI@80/1
NOTES
NET 0 and NET 1 share a common Intel Ophir chip, as do NET 2 and NET 3.
XAUI Adapter cards are required to take advantage of the built-in 10GbE networking
There are two variations, Fibre and Copper
Fibre XAUI cards take one of three current 10GbE transceivers
Short Reach(SR), Long Reach(LR) and Long Reach Multimode(LRM)
Copper versions of the XAUI card are expected Post-RR
XAUI Cards consume the same physical space in the chassis as a PCI card, so are mutually
exclusive with PCI cards in the same location
Sun Internal and Approved Partners Only 25
26. 10GbE XAUI Option Cards
• Interface to the outside world for the Onboard 2 x 10GbE interfaces
from the UltraSPARC T2
• Cards each consume a PCI-E slot if installed
• Attach to dedicated XAUI sockets on special risers
• 10Gbe XAUI Card
– SESY7XA1Z
> Requires Fibre Tranceivers
– 10GbE Short Reach(SR)
– SESY7XT1Z
– 10GbE Long Reach (LR)
– SESY7XT2Z
– 10GbE Long Reach multimode (LRM)
– part No. TBD
• Copper connector XAUI versions to follow post-RR
Sun Internal and Approved Partners Only 26
27. PCI Riser Cards
• Various Riser Cards are used
• Risers should not be moved between slots
• All slots accommodate low Profile PCI-Express cards
• T5120 – 25 Watts per card, 75 Watts Max (3 Slots)
• T5220 – 25 Watts per card, 102 Watts Max (6 Slots)
> Riser Card Mechanical and Electrical widths for T5120 and T5220
T5120 Slot 0 Slot 1 Slot 2
Only Slot x8/XAUI(m) x4(e) x8/XAUI(m) x4(e) x16(m) x8(e)
T5220 Slot 1 Slot 1 Slot 2
Top Slot x8(m) x4(e) x8(m) x4(e) x8(m) x8(e)
Sun Internal and Approved Partners Only 27
28. Disk Cage
• Disk chassis is a FRU
– 4 screws and disconnect cables
• Contains DVD-ROM/Dual USB module as a
sub-FRU/CRU
– Standard in all default configs
– remove disk 3/7 to access release latch, pre-RR
• Non-DVD module available as X-Option Remove Disk and locate
– Still provides Dual USB ports latch beneath DVD
Press up and pull DVD out
• T5120 will be available initially with only a 4
disk chassis, 8 disk in qualification
> Only uses one or two, of the x4 SAS links
• T5220 will be available initially with only an 8
disk chassis, 16 disk in qualification
> Uses both of the x4 SAS link cables
Sun Internal and Approved Partners Only 28
29. Disk Cage - T5120
• Chassis accommodates up to 4 or 8 SAS or SATA 2.5”
drives
– No mixing in a chassis
– No chassis changes needed
• Disk Chassis requires only a single cable
Fan Paddle x4 SAS cable
• Connects to Fan Paddle board with press fit connector Board press-
fit connector
and connector
• Mounting ensures fan to disk vibration isolation
Chassis
Screws
Chassis
Screws
Locator Tabs
'T' head
locator bolts
Sun Internal and Approved Partners Only 29
30. Disk Cage - T5220
• Accommodates up to 8 or 16 SAS or SATA
2.5” drives
• Requires both SAS cables be connected
• Two x4 SAS connector on chassis
• Even 16 disks provides lots of airflow
Paddle Board
press-fit
• Cage and Fan install ensures vibration connector x4 SAS cables
isolation and connectors
Chassis
Screws
Chassis
Screws
Locator Tabs
'T' head
locator bolts
Sun Internal and Approved Partners Only 30
31. Disk Drives
• Disk drives are all 2.5” form factor
• Chassis support both SAS and SATA (post-RR)
• SAS Drives
– 73GB @ 10K RPM
– 146GB @ 10K RPM
Improved
– 73GB @ 15K RPM (post-RR) Airflow above
and below
disks
– 146GB @ 15K RPM (post-RR)
• SATA Drives
– 200GB @ 5400 RPM (soon after RR)
– 300GB @ 5400 RPM (H1CY08) Disk LED's
Ready to
Remove
– 500GB @ 7200 RPM (end CY08) Fault
Status
• New mounting bracket “Marlin”
> Allows better cooling in denser chassis
Sun Internal and Approved Partners Only 31
32. Disk Controller Options
• T5x20 has embedded LSI1068E on the motherboard
– JBOD up to 8 disks
– Controls internal disks only
– RAID 0 (striping) and RAID 1(mirroring)
– Hot-plug supported
– Hot-swap in non-RAID configs is done with 'cfgadm'
• Optional Extended RAID functionality cards (available Q4CY07)
> 'Cougar' - SGXPCIESAS-R-INT-Z
– 8 port SAS RAID 0,1,1E,5,5EE,6,10,50,60 (depending on No. disks available)
– Adaptec Card and Firmware with Intel SunRiseLake chipset
– Battery Backed Write Cache (BBWC) of 256MB
– BBWC battery at least 72 hours (much longer when new/fully charged)
– Only for internal disks, replaces cables to route to standard disk chassis
> 'Prometheus' – SGXPCIESAS-R-EXT-Z
– External port version of 'Cougar', for external disk arrays only (no internal ports)
Sun Internal and Approved Partners Only 32
33. Fans and Cooling
• T5120
> Requires 4 Fan modules
FBD1
FBD1
FBD0
FBD0
FM3
PSU's
FM3
FM3
FM3
> Two cooling groups
– Group 1 – N2 cooling
Disk Chassis
Disk Chassis
FBD1
FBD0
FBD1
FBD0
FM2
FM2
FM2
FM2
– FANBD0/FM1
– FANBD1/FM1
FBD1
FBD0
FBD1
FBD0
FM1
Motherboard
FM1
FM1
FM1
– Group 2 – FB-DIMM cooling
FBD1
FBD0
FBD1
FBD0
– FANBD1/FM0
FM0
FM0
FM0
FM0
– FANBD1/FM2
– Blank Module in FANBD1/FM3 Back Front
• T5220 PSU's
FBD1
FBD0
FBD1
FBD0
FM2
FM2
> Requires 3 Fan Modules
Disk Chassis
Disk Chassis
> Single group for N2 and FB-DIMM
FBD1
FBD0
FBD1
FBD0
FM1
FM1
– FANBD0/FM0 Motherboard
– FANBD0/FM1
– FANBD0/FM2
FBD1
FBD0
FBD1
FBD0
FM0
FM0
FM0
FM0
• Higher disk densities may require Internal andfans Partners Only
Sun
more Approved 33
34. Fans modules and boards
• Fans are speed controlled by the Service
Processor
• Each fan has an Amber FAULT LED
– if lit the fan module is faulty
• 1RU modules
Fan Fault
– 2 Fans at 15K RPM Max. Amber LED
Lit if fan
• 2RU modules faulty
– 2 Fans at 12K RPM Max
• Monitored through ILOM/IPMI/Web
Vibration
Absorbing
Silicon
Dampers
Sun Internal and Approved Partners Only 34
35. Power Supplies
• PSU's are high efficiency
• Output 12V (and 3.3v standby) PSU LED's
Ready to
• Most DC-DC conversion done in MBD Fault
Remove
• Shared with Doradi/o, Tucani/a AC
Present
• Integrated Fan
• T5120
> Side-by-Side
> AC 650Watts
> DC 660 Watts option coming post-RR
> DC or AC, no sharing
• T5220
> Vertically Stacked
> 750Watts (may require the 1050Watt Psu for the 16 disk chassis, pending quals)
> No DC option
> BEWARE... Same form factor as SunFire X4200 PSU's ... DO NOT MIX
Sun Internal and Approved Partners Only 35
36. Rack Mounting Options
• Two Rack mounting options are available..
• Standard Rack Mount (4 post)SESY9RK1Z
> Same as Galaxy servers, T2000, etc..
> Standard configs contain this kit
– ATO : SESY9RK1Z, X-ATO : SESY9RK1Z
• Tool-less Rack Kit (4 post)
> Optional kit, purchased seperately
> Snaps into most 3rd party Racks
> Requires no tools to fit or remove
– ATO : SESY9RK2Z, X-ATO : SESX9RK2Z
Sun Internal and Approved Partners Only 36
37. Solaris for Sun SPARC Enterprise T5x20
• T5x20 servers will require Solaris10 8/07 (update 4)
• This incorporates several UltraSPARC T1/T2
optimisations for CMT servers in general
• Also includes some specific T5x20 optimisations
> UltraSPARC T2 Crypto optimisations
> nxge driver for 10GbE devices
> UltraSPARC T2 performance counters
> Processor grouping, page coloring and default sizing and
bcopy/memcpy
Sun Internal and Approved Partners Only 37
38. Solaris FMA for T5x20
• Now aware of CPU Core oflining
• PCIe Fabric diagnostics improved
• Crypto Unit offlining
• Network interface Units (10GbE)
• POST memory CE tolerance improved
• FRU identity info improved in fault reports (part/ser #)
• Diagnostic Engines (DE)
– Existing DE have been extended for T5x20
– New Eversholt DE used for HostBridge, PCIe and NUI diagnostics
• Limitations...
– Can't isolate Uncorrectable Errors to a single DIMM
– PCIe riser cards not in topology
– XAUI card and XFP not in the topology
– SouthBridge chip is notand Approved Partners Only this time (DVD, USB,...)
Sun Internal
diagnosed at 38
39. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 39
40. ILOM Features at a Glance
• Hardware Control, Inventory, Monitoring
• Host Configuration and Administration
• SP Configuration and Administration
• Logging, Auditing and Alerting
• Authentication :
– LDAP, RADIUS, Active Directory, Local Users
• Services :
– SSH, HTTP(S), NTP, Syslog, SMTP
• Out-of-Band
– Via Serial port, or dedicated Ethernet port
• In-Band
– Via Host OS, firmware upload, Explorer, Syslog
Sun Internal and Approved Partners Only 40
41. ILOM – Industry Standard Manageability
• ILOM enables Sun systems to be easily integrated into
third party management infrastructures
• Key functionality
> IPMI 2.0
> SNMP V1, V2c, V3
> SSH 2.0
> LDAP, RADIUS & MS Active Directory authentication
> Browser UI, SMASH DMTF CLP + ALOM style CLI
• NOTE : No RKVMS functionality like Galaxy ILOM
> No Java plugin required
• Browsers tested include
– Mozilla 1.7 on Solaris 10, IE 6.0 on W2K Server,
Firefox 1.0 and 1.5 on W2K server
Sun Internal and Approved Partners Only 41
42. ILOM – Industry Standard Manageability
• Default user is 'root' with password 'changeme'
> Use for ILOM WEB GUI access and SSH access and
IPMI
• The ALOM 'admin' user is not defined by default, create
this user with an ALOM CLI
• Lost 'root' password and ILOM config is reset using
Motherboard Jumper J6801
• Note... 'sunservice' user shares the password of the
'root' user ONLY FOR SERVICE ACCESS*
• Passwords must be 8 to 16 chars (used to be max 8)
• Max of 10 users allowed (used to be 16)
Sun Internal and Approved Partners Only 42
43. ILOM Common Tasks
• Setting up an ALOM 'admin' user from the ILOM CLI
> SUNSP00144F4641BD login: root
Password: changeme
Waiting for daemons to initialize...
Daemons ready
Sun(TM) Integrated Lights Out Manager
Version 2.0.4.0
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Warning: password is set to factory default.
->create /SP/users/admin role=Administrator cli_mode=alom
Creating user...
Enter new password: ********
Enter new password again: ********
Created /SP/users/admin2
-> exit
SUNSP00144F4641BD login: admin
Password: ********
Waiting for daemons to initialize...
Daemons ready
Sun(TM) Integrated Lights Out Manager
Version 2.0.4.0
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
sc>
Sun Internal and Approved Partners Only 43
44. ILOM Common Tasks
• Setting up ILOM IP configuration from the ILOM CLI
> SUNSP00144F4641BD login: root
Password: changeme
Waiting for daemons to initialize...
Daemons ready
Sun(TM) Integrated Lights Out Manager
Version 2.0.4.0
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Warning: password is set to factory default.
-> set /SP/network state=enabled
Set ’state’ to ’enabled’
-> set /SP/network pendingipaddress=10.8.31.163
Set ’pendingipaddress’ to ’10.8.31.163’
-> set /SP/network pendingipdiscovery=static
Set ’pendingipdiscovery’ to ’static’
-> set /SP/network pendingipnetmask=255.255.252.0 Static IP
Set ’pendingipnetmask’ to ’255.255.252.0’ configuration
-> set /SP/network pendingipgateway=10.8.31.248
Set ’pendingipgateway’ to ’10.8.31.248’
-> set /SP/network commitpending=true
Set ’commitpending’ to ’true’
->
-> set/SP/pendingipdiscovery=dhcp
Set ’pendingipdiscovery’ to ’dhcp’
-> set /SP/network commitpending=true
Set ’commitpending’ to ’true’ Dynamic IP
-> (DHCP)
configuration
Sun Internal and Approved Partners Only 44
45. ILOM IPMI functionality
• IPMI is a widely used management protocol for x86 servers
• ... But its new for SPARC boxes...
• Download the required tools from
> The Solaris Supplemental CD (an older binary package)
> Latest is available at http://ipmitool.sourceforge.net
– Binary packages (usually older versions) are available for Solaris x86 and
SPARC, as well as Linux, Windows, Mac...etc
> There is a Blueprint available from Sun also (mainlyx64 centric, but applies to
SPARC now too)
– http://www.sun.com/blueprints/0107/820-1011.pdf
-bash-ventus2$ /opt/ipmitool/bin/ipmitool -U root -P changeme -H 10.10.10.10 sdr
/MB/V_+1V0_VDD | 1.01 Volts | ok
/MB/V_+3V3_STBY | 3.36 Volts | ok Get the
/MB/V_+3V3_MAIN | 3.36 Volts | ok Sensor Data
/MB/V_+5V0_VCC | 4.97 Volts | ok Repository
/MB/V_+12V0_MAIN | 12.16 Volts | ok
..
..
-bash-ventus2$ /opt/ipmitool/bin/ipmitool -U root -P changeme -H 10.10.10.10 sunoem
sbled set /SYS/LOCATE on Set the System
/SYS/LOCATE | ON Locator LED
-bash-ventus2$ flashing
Sun Internal and Approved Partners Only 45
46. ILOM Common Tasks
• Logging into the ILOM Web GUI
• Point Web Browser at the IP address
of the ILOM
> Will automatically redirect to a
secure HTTPS port
> Best to disable any Web
Proxies for best performance
> Make sure port 80, 443 are
open on any firewall
• Log in as any defined user, default
Administration user is 'root'
Sun Internal and Approved Partners Only 46
47. ILOM/ALOM Common Tasks
• A Tour of the ILOM Web GUI • GUI provides ways to set up IP address
configurations for ILOM
• Configure additional ILOM/ALOM users
• Access Sensor Readings
• Control System configurations
• Carry out firmware updates.
• etc....
Sun Internal and Approved Partners Only 47
48. ALOM 2.0 vs. 1.4 CLI Differences
• Default CLI appearance is ILOM
– User is 'root'
– Must create an 'admin' Administrator user with the ALOM CLI
• Flashupdate in new ALOM expects a TFTP server not an FTP server
• ALOM IP address changes now active only after running ...
– 'setsc netsc_commit true '
– Previously changes were active only after SC reset
– DHCP is enabled by default
• View firmware versions with sc> showhost version
Sun System Firmware 7.0.0_build_13 2007/08/10 23:44
– 'showhost version '
Host flash versions:
Hypervisor 1.5.0.build_13**PROTOTYPE** 2007/08/10 22:54
OBP 4.27.0.build_04***PROTOTYPE BUILD*** 2007/08/10 13:25
• View FPGA version with
Sun Fire[TM] Huron POST 4.27.0.build_04***PROTOTYPE BUILD***
2007/08/10 13:51
– 'fpga version ' sc> fpga version
FPGA Version: 1.5.0.0
Platform ID
Major revision
: 1
: 5
Minor revision : 0
• Only Admin and Operator roles Lab Debug Version: 0
sc>
are defined Sun Internal and Approved Partners Only 48
49. ALOM backward Compatibilty CLI
• Uses ALOM-CMT 1.2 for a base feature set
• Includes ALOM-CMT 1.4 features...
> break -d
> Host Watchdog Timer
> Bootmode LDOMs awareness
> Forward SP events to Hot/Solaris syslog via inband
channel
Sun Internal and Approved Partners Only 49
50. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 50
51. Logical Domains 1.0.1 Features
Released, aligned with UltraSPARC T2/T5x20
• UltraSPARC T2 Platform Support
> 32 domains at RR, 64 domains following
• I/O domain can reboot while the other domains keep
running
> Dependant domains would wait for IO transactions to complete
• LDoms SNMP MIB
> Modeled after industry standard DMTF CIM (Common Information
Model)
> Running on control domain to enable remote SNMP monitoring
• Domain minimization support
> Minimal/reduced Solaris installation
> Base minimal packages plus optional ones for guest, control, I/O or
service domains
Sun Internal and Approved Partners Only 51
52. Logical Domains 1.0.1 Requirements
• Solaris 10 08/07 (update 4)
• Systems firmware version
– T5x20/Glendale Firmware 7.0.x = SP FW 2.0.4.0+
– T2000/T1000/T6300/Pelton Firmware 6.5.x
• SUNWldm.v LDOMs Manager Package
– Version 1.0.1
• T5x20 System FPGA
– Version 1.5.0+ (Not a concern for Post-RR systems)
Sun Internal and Approved Partners Only 52
53. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 53
54. CRU/FRU List - T5120
• Lots shared with other Jake and Elwood systems
Item Description FRU/CRU
Disk Backplane 4 drive Disk Backplane to support 4 Drives only FRU
Fan Module 1U Dual Fan module, 4 populated in system CRU
Fan Board 1U Fan connector board to accommodate Fan Modules, 2 per system FRU
DVD Assembly DVD Assembly including front USB board FRU
Paddle board 1U Conection between PDB, fan boards and Disk Backplane FRU
PDB 1U Power distribution board, contains master chassis serial number FRU
PSU 1U Power supply (650 Watts) CRU
X8 &XAUI, 1U PCIe Riser PCI Riser assembly for left and centre motherboard slots CRU
x16 1U PCIe Riser PCI Riser for right motherboard slot CRU
Rack mount kit (requires tools) Rack mount kit CRU
Rack Mount Kit (tooless) Rack mount kit, no tools required for installation CRU
CMA Cable management arm CRU
Bus Bars Power Bus Bars between PDB and Motherboard FRU
PDB to MB Ribbon Cable Ribbon cable for control connections between PDB abd Motherboard FRU
Light pipe kit Light pipe assemblies for left and right of disk cage FRU
Motherboard assembly Motherboard assembly, part no varies depending on Core count FRU
DIMMs FBDIMMs (three different part numbers for 1GB, 2GB, 4GB) CRU
XAUI board optical/copper XAUI Adapter card (two different parts for copper/optical) CRU
XFP module XFP GBIC module for use in XAUI cards (three different parts SR, HR, HRMM) CRU
Battery Battery backup for NVRAM of Service Processor CRU
HDD 73GB/146GB SAS Hard Disk drive, different part numbers for capacities CRU
System Configurations PROM System Configuration PROM FRU
Sun Internal and Approved Partners Only 54
55. CRU/FRU List - T5220
• Lots shared with other Jake and Elwood systems
Item Description FRU/CRU
Disk Backplane 8 drive Disk Backplane to support 8 Drives only FRU
Fan Module 2U Dual Fan module, 3 populated in system CRU
Fan Board 2U Fan connector board to accommodate Fan Modules, 2 per system FRU
DVD Assembly DVD Assembly including front USB board FRU
Paddle board 2U Connection between PDB, fan boards and Disk Backplane FRU
PDB 2U Power distribution board, contains master chassis serial number FRU
PSU 2U Power supply (750 Watts) CRU
X8,X8, XAUI, 2U PCIe Riser PCI Riser assembly for left and centre motherboard slots CRU
X8, x16 2U PCIe Riser PCI Riser for right motherboard slot CRU
Rack mount kit (requires tools) Rack mount kit CRU
Rack Mount Kit (tooless) Rack mount kit, no tools required for installation CRU
CMA Cable management arm CRU
Bus Bars Power Bus Bars between PDB and Motherboard FRU
PDB to MB Ribbon Cable Ribbon cable for control connections between PDB abd Motherboard FRU
Light pipe kit Light pipe assemblies for left and right of disk cage FRU
Motherboard assembly Motherboard assembly, part no varies depending on Core count FRU
DIMMs FBDIMMs (three different part numbers for 1GB, 2GB, 4GB) CRU
XAUI board optical/copper XAUI Adapter card (two different parts for copper/optical) CRU
XFP module XFP GBIC module for use in XAUI cards (three different parts SR, HR, HRMM) CRU
Battery Battery backup for NVRAM of Service Processor CRU
HDD 73GB/146GB SAS Hard Disk drive, different part numbers for capacities CRU
System Configurations PROM System Configuration PROM FRU
Sun Internal and Approved Partners Only 55
56. Option Parts List
Part Number Option Description Avialable
SECY9PS11Z AC PSU 1RU (650 Watts) RR
SEDY9PS31Z AC PSU 2RU (750 Watts) RR
SECY9PS21Z DC PSU 1RU (660Watts) post-RR
SESY2A1Z 2GB Memory (2x1GB) RR
SESY2B1Z 4GB memory (2x2GB) RR
SESY2C1Z 8GB Memory (2x4GB) RR
SESY3A11Z 73GB, 10KRpm 2.5” SAS Drive RR
SESY3C11Z 146GB, 10KRpm 2.5” SAS Drive RR
- SATA Drives (200, 300,500GB) post-RR
SESY9DV1Z DVD, 8X RW, Slot Load RR
SESY9RK1Z Slide Rail Kit, requires screwdriver (1RU and 2RU) RR
SESY9RK2Z Slide Rail Kit, Tool-less (1RU and 2RU) RR
Sun Internal and Approved Partners Only 56
57. Option Parts List (cont..)
Part Number Option Description Avialable
SG-PCIE1FC-QF4 Qlogic, 4Gb, PCI-E Single Port (up to 3/6) RR
SG-PCIE2FC-QF4 Qlogic, 4Gb, PCI-E Dual Port (up to 3/6) RR
SG-PCIE1FC-EM4 Emulex, 4Gb, PCI-E Single Port (up to 3/6) RR
SG-PCIE2FC-EM4 Emulex, 4Gb, PCI-E Dual Port (up to 3/6) RR
SG-PCIE2SCSI-U320Z U320 SCSI, PCI-E Dual Port (up to 3/6) RR
SG-PCIE8SAS-E-Z SAS, 8 Port, PCI-E (up to 1) RR
SGXPCIESAS-R-INT-Z SAS, 8 Port, RAID, PCI-E Internal Drive Controller (up to 1) RR+3
SGCPCIESAS-R-EXT-Z SAS, 8 Port, RAID, PCI-E, External Drive Controller (up to 1) RR+3
7280A-2 Northstar Dual port GigE MMF Low profile PCI-E (up to 3/6) RR
7281A-2 Northstar Dual port GigE UTP Low profile PCI-E (up to 3/6) RR
1236A-Z Infiniband HBA, Dual 4x ports, PCI-E (up to 3/6) RR
4447A-X Atlas Quad Port, GigE Copper, PCI-E (up to 3/6) RR
X4240A XVR-300, PCI-E x16/x8 Graphics cards (up to 1) TBD
Sun Internal and Approved Partners Only 57
58. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 58
59. Tuning Out of the Box
• Using Solaris 10 08/07...
> Default pages sizes for
– Initdata Segments is 64kB pages
– Private Anon, Heap, Process Stack, Text, Shared Anon segments are
4MB pages
– ISM and DISM use 256MB pages
– New variables introduced to control these sizes if changes are needed
– max_uheap_lpsize,max_ustack_lpsize, max_privmap_lpsize,
max_shm_lpsize, max_uidata_lpsize, max_utext_lpsize,
max_shm_lpsize
> L2 Cache hashing implemented
– helps randomise accesses to L2 caches and ease thrashing
> Stack biasing implemented
– D-Cache is only 4 Way associative... 8 threads
– Running multiple copies of a binary can lead process stacks aligning in
the cache and to hot lines in the cache
– Stack Bias slews the stack variableOnly processes.
Sun Internal and Approved Partners
for 59
60. Tuning Out of the Box (continued..)
> Processor Groups introduced
– Helps associate CPUs with some shared resources (int/FP pipelines,
Crypto units etc)
– Enables improved multi-level CMT load balancing and affinity policies
> 'bcopy'/'uiomove' Optimizations
– Key kernel operations
– new implementation about 2x faster than previous
• Some optimisations did not make it into S10 8/07
> These will be post RR patches as they become available
> Most fixes are already committed in update 5
> Check patch levels for customers with performance issues
> Also check Firmware versions as this can improve
performance too...
Sun Internal and Approved Partners Only 60
61. Tuning Recommendations
• Less involved than with T2000
• S10 8/07 is more 'automatic'
– OOB optimised page size selection, thread scheduling, block copy, IP
instances for zones, page coloring etc...
• If tuning is necessary remember...
> Improved pipeline results in Single threaded performance around 20% better
> Floating Point is no longer an issue
> Start with T2000 tuning options
> Use updated trapstat, cpustat,
• LDOMs may be an easier way to scale poorly threaded applications
• Solaris Crypto framework is still being tuned
> Great for big packets, working on improving small packets
• A Blueprint is being prepared (Denis Sheahan)
Sun Internal and Approved Partners Only 61
62. Tuning Recommendations (cont..)
• Refer to UltraSPARC T1 applications tuning best practices...
> http://www.sun.com/servers/coolthreads/tnb/applications.jsp
• CoolTools are being updated for UltraSPARC T2
> http://www.sun.com/servers/coolthreads/overview/cooltools.jsp
• Sun Studio 12 is UltraSPARC T1 and T2 aware
> Recompiling without code changes should benefit T1/T2 apps
• Processes are being worked on for LDOMS migrations 1.0 to 1.0.1
> LDOMs configs can benefit from a transition to T5x20 platform
> Make sure appropriate LDOMs tuning and sizing is carried out
• Wider Crypto capability, so ensure code is using PKCS#11 framework
> Ensure that the ncp (SSL, RSA, DSA etc) and n2cp (bulk ciphers),
n2rng(random number generator) drivers are used appropriately
• Sun Internal and Approved Partners Only 62
63. Agenda
• Introduction
• Overview of UltraSPARC T2
• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000
> System Architecture and Tours
> Memory, Networking and IO Expansion
> Disks, Fans and Power Supplies
> Solaris for T5x20
• ILOM 2.0
• LDOMs for T5x20
• CRUs/FRUs and Configuration Options
• Tuning and Performance
• Tools, Information and References
• Q & A ...
Sun Internal and Approved Partners Only 63
64. Tools and Information
• Cool Tools
– http://www.sun.com/servers/coolthreads/overview/cooltools.jsp
• Architecture WhitePaper
– http://www.sun.com/
• SUNWIN Tokens
> Sun Intro : 514416
> Architecture Whitepaper : 512750
> This Technical Presentation: 512749
Sun Internal and Approved Partners Only 64
65. Reference Information
• T5120 and T5220 (Huron) Engineering TOI
– http://ittdev.east.sun.com/TechTalk2/EngineeringTOIs/Huron_Apr07/
• ILOM/ALOM User Guides
– http://www.sun.com/products-n-solutions/hardware/docs/pdf/819-1160-13.pdf
– http://www.sun.com/products-n-solutions/hardware/docs/pdf/819-7981-10.pdf
• Solaris FMA
– http://fma.eng
– http://lightside.sfbay/n2-fma/docs.php
•
Sun Internal and Approved Partners Only 65