From Oredev 2014 http://oredev.org/2014/sessions/nuget-3-0--transitioning-from-odata-to-json-ld The NuGet team has spent most of 2014 working on NuGet 3.0. You’ll get a tour of NuGet 3.0, its features, and the direction we’re headed. But then we’ll get to the fun stuff we can all learn from: the challenges we’ve faced, and how we’re overcoming them. You’ll hear chilling tales about how interfaces, LINQ, and general-purpose server APIs have caused us so much grief. But you’ll also see how we broke a cycle of endless design and started making progress when it seemed like none could be made, allowing us to transition from OData to JSON-LD.

1. NuGet 3.0
Transitioning from OData
to JSON-LD
Jeff Handley
@JeffHandley

2. NuGet 3.0
New UI for Visual Studio
• Lots of new UI features
New client/server API
• Built for nuget.org availability, based on usage patterns
Lots of code refactoring
• Easier maintenance going forward

4. NuGet 3.0 Plans
Aligned with Visual Studio “14” release cycle
• Preview in the next release
• Shipping for VS 2012 and VS 2013
• Not shipping for VS 2010
MSBuild will consume package references
Solution Explorer integration
Details at http://blog.nuget.org

5. Transitioning to the
new client/server API
Design retrospective

6. package repository
remote server local folder
package
metadata from API unpacked into memory
Visual Studio project system
C# F# Windows Phone …
file system
/packages folder project system

7. Guiding principles
BEFORE NUGET 3.0
Bolt onto Visual Studio
Heavy use of abstractions
Comprehensive unit tests
Generic client/server API (OData)

8. Visual Studio bolt-on
Use Visual Studio API
Same result as manually referencing libraries
Integrate with most Visual Studio project systems
Support VS-integrated source control

9. Abstractions
IPackageRepository
• nuget.org
• Other remote servers
• Folder on disk or network
IPackage/IPackageMetadata
• Remote
• In-memory
• On disk
IPackageManager
IProjectSystem
• C#
• F#
• Windows Phone
IFileSystem
• Actual file system
• Project systems
• Source control
LOOSE COUPLING & UNIT TESTING
more than 80 other interfaces

10. Generic client/server API
ANOTHER FORM OF LOOSE COUPLING
Don’t couple the client to a server implementation
Avoid versioning the server to add client features
LINQ programming model for local and remote

11. NuGet.VisualStudio
NuGet.Core
PowerShell
console
package
manager
window
VS IDE
integration
source control
integration
command
extensions
nuget.exe

12. NuGet.VisualStudio
Visual Studio IDE integration
• Output window logging
• Settings (package sources)
• Automatic package restore upon build
• Garbage-collecting packages on VS restart
• Extension update banner
Visual Studio APIs
• Project system integration
• Actual package installation and uninstallation

13. NuGet.Core
• Package authoring (including package analysis rules)
• Package reading (OPC/Zip, data model w/validation, file conventions)
• Semantic Version implementation
• Target framework compatibility
• Dependency resolver
• Local and remote sources (API, HTTP, proxies, authentication)
• NuGet.Config Processing (including hierarchical and extensible features)
• 80+ interfaces

14. http://valleyreport.blogspot.com/2012/01
/revised-dumping-ordinance-allows.html

15. Classes fulfilling multiple interfaces
Project systems are also file systems
• Add/remove files
• Enforces VS project system and source control integration
/packages folder is also a package repository
• GetPackages()
• AddPackage()
• RemovePackage()
Extension methods on interfaces to simulate multiple inheritance
• 17 “extensions” classes that act as base classes
33 “utility” classes to contain business logic

16. Extreme loose coupling
Magic values trigger special handling several layers deep
• FindPackage(id, version: null);
Granular interfaces led to unmanageable parameter counts
• ProjectSystemExtensions.DeleteFiles FxCop
suppressions
Events during operations allowed decoupled reactions
• InstallWalker needs to report progress to the VS UI

17. http://www.flickr.com/photos/bump/2807032202/

18. No change was easy
• 18 classes touched adding HTTP headers for downloads
• Dozens of classes affected to add a new field to nuspec
• Mocks require more work than the actual implementation
• Extension methods are not compatible with mocking
•We became afraid to touch the dependency resolver

19. Big changes needed
Replacing API v2 with API v3

20. SQL Azure
database
Azure
Blob
Storage
Gallery / API v2
Azure web role (3 instances)
EntityFramework
code-first
WCF Data
Services and
ASP.NET MVC
and Razor
OData
Search and
feed
requests
Content Delivery Network (CDN)
Package
downloads

21. Azure
Blob
Storage
Search Service
Azure worker role (3+ instances)
Lucene.NET
Full index loaded into memory
Content Delivery Network (CDN)
Metrics Service
Azure website (3+ instances)
SQL Azure
database
Search
queries
Package
metadata
requests
Package
downloads

22. JSON-LD
http://json-ld.org/
{
"@context": "http://json-ld.org/contexts/person.jsonld",
"@id": "http://dbpedia.org/resource/John_Lennon",
"name": "John Lennon",
"born": "1940-10-09",
"spouse": "http://dbpedia.org/resource/Cynthia_Lennon"
}

23. Creating the v3 client
For the new UI

24. API v2 support
The UI must support both v2 and v3 clients
No clean surface area to re-implement
• UI  install walker  API v2  install walker  API v2
• IQueryable<IPackage> exposed throughout code

26. Infinite surface area
Finite scenarios

27. Scenarios and clients
1. Search
2. Install specific version
3. Install latest version
4. Install with dependencies
5. Install from cache
6. Install into another project
7. Check for updates
8. Update and update all
9. Restore
1. Package manager UI
2. PowerShell console
3. nuget.exe

28. http://www.clayburn.wtf/2011/03/to
p-5-reasons-youre-unemployed.html

29. API v3
search
(feed)
Search Service
Azure worker role (3+ instances)
Lucene.NET
Full Index Loaded into Memory
Gallery / API v2
Azure web role (3+ instances)
Content Delivery Network (CDN)
API v3
Package
Metadata
Package
Downloads
Metrics Service
Azure website (3+ instances)
API v2
OData
SQL Azure
database
Azure
Blob
Storage

30. WCF Data Services to the rescue!?
NuGet
Core
WCF Data
Services
API v2
Server
OData
XML
WCF Data
Services
Entity
Framework
API v3
Services
JSON-LD
Convert JSON-LD to XML

31. http://www.marcioezequiel.com
.br/2013/10/bigode-grosso.html

32. Introducing NuGet.Client
Handle the client/server scenarios
Support API v2 and API v3
Scenario-focused object model
Prevent new dependencies on NuGet.Core

33. Wrapping NuGet.Core
Package Manager UI
NuGet.Core
NuGet.Client

34. Dealing with deep API v2 assumptions
• Implemented v2 interfaces in v3 classes
• Threw NotImplementedException on all members
• Ran a scenario, identified code paths required
• Identified more concrete API v3 requirements
• Discovered data needed for each scenario
• Influenced API v3 JSON-LD views

35. Shrinking NuGet.Core
Package Manager UI
NuGet.Core
Package
Authoring
Package
Reading
Semantic
Versioning
Target FX
Compat
Dependency
Resolver
Local
Package
Sources
NuGet
Config
Processing
Project and
NuGet.Core
NuGet.Client
NuGet.Client File Systems

36. Compromises
API v3 isn’t completely “clean” yet
• It exposes the API v2 interfaces
Chatty
• API v3 uses more HTTP requests than a pure v3 server/client would
• API v2 interop performs more OData requests than we used to
Out of scope
• PowerShell console
• nuget.exe

37. New guiding principles
Our new rules

38. Guiding principle #1
Technology is not a substitute
for understanding our scenarios.

39. Guiding principle #2
Don’t expose IQueryable on
public API.
* this applies to our RESTful services too

40. Guiding principle #3
Every client/server scenario
needs an explicit endpoint.

41. Guiding principle #4
Shipping is more valuable than
code purity.

42. Guiding principle #5
Don’t implement interfaces just
because the members match.

43. Guiding principle #6
Side-effecting event handlers
hurt in unpredictable ways.

44. Guiding principle #7
Avoid business logic in extension
methods. They can’t be mocked.

45. Guiding principle #8
If adding new business logic is
hard, refactor right away.

46. Thanks
Come get NuGet stickers
@jeffhandley | jeffhandley.com | jeff.handley@microsoft.com | blog.nuget.org