SlideShare a Scribd company logo

Marty Weiner, Reddit CTO @ SXSWi 2016 - Everything's Harder At Scale

•
•
D
Diana Chow

The slide deck from Reddit CTO Marty Weiner's talk at SXSW Interactive 2016, "Everything's Harder At Scale".

Technology
1 of 99
Everything’s Harder at Scale Marty Weiner CTO, Reddit u/Mart2D2 @MartyWeiner
Building a Startup is Hard
• Reduce the pain of the first 18 months of scalability hell • Look at how a company scales technically and culturally from idea to BIG at major inflection points • Strategies for thinking about technology and culture • Construct a tech stack recipe you can use Goals • Focus is on engineering • For those who have not experienced small -> BIG • Every startup is different, but in many ways they’re the same • Scope is from idea to solid traction and about 100 employees • Lots in this talk, will put slides up • I’m not being paid by any vendors!
Inflection Points LaunchDeveloping the Idea Trickle :( TRAAAACTION! Becoming a Business
Let’s make a fictional startup!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Idea bubbling around in founder’s head • Work on it in spare time • Doodle in class / at work / in dorm / at coffee shop Day -100 · Idea Forming
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Here’s the idea for our startup: • It’s a social network • Topics organized by community • You can send messages • Max of 140 characters • They’re deleted in 10 seconds after viewing • Pin your interests • And you can order a taxi… Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • It’s called SnapinUberTweetySnoo • snapinubertweetysnoo.com taken :( Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Settle on “Salad Rabbit” • saladrabbit.com was the least bad idea for a URL that’s available Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • No serious competition, or at least you totally can do better! • You (or you and engy buddy) start mocking up an app Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Much Ramen later, you slapped together an iPhone app and servers • Friends and family using it • Get some YC money • You set up Google Analytics and add a few extras like push notifications Day -30 · Demo!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • You officially launch! • It’s on the App Store!!!!! Day 0 · Launch!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Sit there refreshing Google Analytics over and over and over and over….. Day 0 · Launch!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Nobody coming. Well, ok, a few.. Day 10 · Waiting
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Developing the idea and the tech stack. Added some features. • A few people are coming. At least they’re very passionate • Is anything going to happen? Day 11 to 119 · A trickle • Is this working? • Should… should I pivot??
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Day 11 to 119 · A trickle Not supposed to be stealth mode :( Pinterest Page Views / Day
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Not really sure why, but users start pouring in • Featured in Apple’s Best New Apps with “Salad” in the Title • Hijacked the jumbotron at the Superbowl • Hopefully this is not a fluke Day 120 · Something’s happening ?!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Users still pouring in! • You’re running out to talk to VCs • Trying to hire engineers. They’re junior or hackers. • Your tech is a confusing blob of shiny new technologies • Site now timing out consistently. Brace yourself, it’s gonna be a tough 12+ months of scalability hell. Day 140 · Traaaaaction!
Let’s go back in time…
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Founder starts digging deeper • Goal should be, above all, on getting a Minimal Viable Product out there for people to see Day -90 · Fleshing it out • Hypothesis is “People will like what I’m building”
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Day -90 · Fleshing it out Maximize for Velocity, Agility, Creativity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Choose one frontend platform: iPhone, Android, Web Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Choose one frontend platform: iPhone, Android, Web • You’ll need a backend to store data, facilitate communications, etc • Maximizes velocity, agility, creativity • One we can evolve with the number of employees and users • Let’s construct a recipe you can use Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Tons of choice for tech. Databases alone: • Postgres • Redis • Cassandra • HBase • Riak • Neo4j Tech Maturity • CouchDB • Membase • MySQL • MongoDB • VoltDB • … hundred others
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • “We’re building the latest and greatest. I need the latest and greatest.” • No no no no no no… no… no • “This tech magically scales for you” • Until it doesn’t… • “It’s shiny and new. It must be faster or betterer” • Stop Tech Maturity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Tech Maturity Maturity = Blood + Sweat Complexity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Redis Maturity = Blood + Sweat Complexity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business HBase Maturity = Blood + Sweat Complexity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business MySQL Maturity = Blood + Sweat Complexity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Sometimes you must use immature tech, but… • Harder to hire • Harder to get help (yes, all tech has problems at some point) • Stability and performance uncertain, especially at larger scale • Debugging will be harder • Tools weaker • If biggest user, challenges greatly amplified • Most likely you won’t *really* need immature tech for a long time Tech Maturity
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use Amazon Web Services (AWS) for hosting • Evolves and scales with you from tiny to BIG • Competitively priced • Lots of tools to help get you moving (DBs, cache, search, email, DNS, CDN, …) Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use AWS Route53 to register your domain (saladrabbit.com) Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Spin up a tiny EC2 instance for your first API (or web) server • Describes outside interfaces and contains product logic • Use latest Ubuntu • Nginx to serve HTTP • Use Python / Flask / pymysql or Python / Django for backend code • Use Route53 to connect your domain to your tiny instance • Using api.saladrabbit.com • Your EC2 instance storage is EBS (Easy backups!) • Call it something descriptive like api001 Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Create an S3 bucket for serving static assets (js, css, images) • S3 is super durable, very available storage • Use Route53 to connect a subdomain to it (e.g, assets.saladrabbit.com) Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use git + Github for your version control • Get code backup for free! • Good for code reviews, bug reporting when you’re ready for it • Makes deployment easier Day -90 · Fleshing it out
AWS API Route53 S3 api. assets.
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Need a place to store user info, content, follower graphs, etc • We need a database Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Just use MySQL • The boring tech revolution is here • Use AWS Relational Database Service (RDS) to manage it (Tiny instance size) • Conveniently manages MySQL instances • Costs ~1.5x • RDS -> self managed MySQL migration is easy • Don’t have to worry about database operations for early days! • No configs to understand • Backups done for you • No messing with failovers (even has automatic failover) • Don’t be afraid of joins, distinct, count, etc! Day -90 · Fleshing it out
AWS API Route53 S3 api. assets. RDS
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • How does your API box know the RDS endpoint? • Just use simple python config files (no haproxy layers, no zookeeper, …) Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Make your backend easy to debug • Use Python logging to syslog (AWS CloudWatch looks like an interesting option) Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Search • Use AWS CloudSearch Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • That’s probably it • Don’t need to send notifications, emails, etc unless they’re vital to your demo • Don’t optimize anything on the backend unless you have to Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • So far… • We have an app • Architecture that doesn’t limit your velocity, agility, creativity • Won’t scale far, but we can evolve and grow it • Now go show people! Day -90 · Fleshing it out
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Show it to friends, family, r/entrepreneur, people on the train • Hopefully people love it! Though they’ll hate the name… • Take feedback, thank people for it, ignore the naysayers Day -30 · Demo!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Focus on last critical product needs • Tighten up the frontend experience • Add asynchronous background processing • For sending email, sharing to Facebook / Twitter, processing images, etc… • Use PinLater for background processing • Push Notifs (AWS SNS), SMS (Twilio), Email (AWS SES) • Add Google Analytics to frontend Day -29 to -1 · Prep for Launch
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Tell everybody! • Connect to Google Analytics and hit refresh over and over again Day 0 · Laaaaunch!
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Stay focused on product (no need to sacrifice velocity, agility, creativity) • Round out product and operations Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • If necessary, add other platforms Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Add StatsD to measure timings and counts live (only use _upper* timings!) Day 1 to 119 · A Trickle * Long story, see appendix statsd_connection.incr(“event.user_signup”) statsd_connection.timing(“event.mysql_queries.get_user_feed”, time_ms)
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Add StatsD timings to all interface points • All API endpoints • All MySQL calls • All calls to other services • All background tasks • Make a dashboard Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Make it faster to launch / config boxes with IT Automation • Puppet, Chef (AWS OpsWorks) Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Set up CloudWatch or Pingdom to notify of downtime • Will save you sleep Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Add more API servers • Use AWS ELB to distribute load • Make it fast to bring one API server up Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Set up development and staging environments • Make Route53 connections (e.g., martydev.saladrabbit.com) • Make it easy to deploy new code to production or staging • Use Teletraan Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • So far… • iPhone app talking to your API + logic servers through Route53 and ELB • Database (MySQL) • Static storage (S3) • Search (Cloudwatch), Virality, Background Task Processing (PinLater) • Visibility (Google Analytics, Statsd, AWS CloudWatch) • Deployability (staging, dev, Teletraan, AWS OpsWorks) Day 1 to 119 · A Trickle
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Ok, people coming. Whoa. Day 120 · Something Happening
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Not a fluke… even more are coming. Winter is here. Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Site begins to timeout or queues start backing up • Profile first! (Check StatsD) • Probably your poor database (will bet money) Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • NOW begin to trade off velocity, agility, creativity for site availability • Don’t worry, we’ll get them back Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • If it is your database that’s causing problems… • Start increasing instance sizes (Add RAM and especially go to SSD) • Short downtime in RDS • Begin optimizing your queries and reducing load on MySQL • Check slowlog and profile (SHOW FULL PROCESSLIST) for worst queries • Add indexes • Start caching • Start normalizing all queries and remove joins, distincts, etc Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Caching • Create Memcache or Redis servers behind McRouter or use Elasticache • Check cache -> go to db -> put back in cache • Hunt for common queries you can cache Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Normalize your data • Object tables are only used for ID -> data lookups • e.g., users table (noun) • Add primary key + auto_increment on ID (this generates IDs for you!) • Mapping tables are only used for From ID -> To ID lookups • e.g., users_said_comments (noun_activeverb_noun) • Add sequence column if you need order and your ID does not support order • Add indexes: Unique on From,To and Index on To Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Normalize SELECT text FROM comments WHERE user_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s SELECT text FROM comments WHERE listing_id=%(listing_id)s SELECT comment_id FROM listing_contains_comments WHERE listing_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Normalize SELECT text FROM comments WHERE user_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s SELECT text FROM comments WHERE listing_id=%(listing_id)s SELECT comment_id FROM listing_contains_comments WHERE listing_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Optimizing Feeds SELECT text FROM comments LEFT JOIN followers ON comments.user_id=followers.followee_id WHERE followers.follower_id=%(user_id)s ORDER BY create_date DESC OFFSET 0,20 SELECT followee_id FROM users_followed_by_users WHERE follower_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id IN %(followee_ids)s ORDER BY sequence DESC OFFSET 0,20 SELECT text FROM comments WHERE id IN %(comment_ids)s
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Optimizing Feeds SELECT followee_id FROM users_followed_by_users WHERE follower_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id IN %(followee_ids)s ORDER BY sequence DESC OFFSET 0,20 SELECT text FROM comments WHERE id IN %(comment_ids)s [Aync after comment] SELECT follower_id FROM users_followed_by_users WHERE followee_id=%(user_id)s Put Comment in Redis follower_to_comments [On Read] comment_ids = redis_get_follower_to_comments(user_id) SELECT text FROM comments WHERE id IN %(comment_ids)s
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Optimizing Feeds SELECT followee_id FROM users_followed_by_users WHERE follower_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id IN %(followee_ids)s ORDER BY sequence DESC OFFSET 0,20 SELECT text FROM comments WHERE id IN %(comment_ids)s [Aync after comment] SELECT follower_id FROM users_followed_by_users WHERE followee_id=%(user_id)s Put Comment in Redis follower_to_comments [On Read] comment_ids = get_follower_to_comments(user_id) SELECT text FROM comments WHERE id IN %(comment_ids)s
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • MySQL doesn’t magically scale out of the box • When to Shard? • Increased instance sizes to big SSD servers • And you’ve moved as many tables to other instances as possible • And you are reading from read slaves • How to Shard? • Click here or Google for “Sharding Pinterest” • Google for “Sharding & IDs at Instagram” Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Logging data and operations • Log all page views (and some info on what was seen) • Log all major events (create, update, delete, click offsite, etc) • Eventually log all service calls • Why? • Will save your butt some day (when you bork your database) • Beginning of really understanding your users and growth Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use Kafka and Secor to get events from backend into AWS S3 • Use AWS EMR to start digging into data • Pinball or AWS SWF help you run queries automatically • Transcode MySQL backups nighty into S3 so EMR can use them • Near future: data science, A/B experiments • Not yet: machine learning Day 140 · Traction
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Getting good at operations • Scale (more) under control • Have data and some goals + metrics • Growing company size (10+) • Foosball table Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • New Eng Goal: Build The Dream Machinetm • Team and tech that can build any idea faster and better with each year • Tech even drives creativity • Start to regain velocity, agility, and creativity Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • You’ve probably built up some tech debt and have a nasty monolithic codebase • Pinboard (Pinterest), R2 (Reddit), Obidos (Amazon) • Begin to pay down tech debt • Align teams for success • Move to a service oriented architecture Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Engineering management • Don’t “promote” best engineer to manager • Lose best coder / doer, made a First Time Manager (FTM) • Do have discussions with engineers about their careers • Management should not be a promotion • Give engies and managers equal path to impact and comp • Engineers who have never managed should not step in lightly • Give 6 month path Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Technically Strong Jerk (TSJ) • Appear at many companies at 10+ employees • 2x faster than others, maybe leading an important initiative • … but make life hell for everybody else (1x -> .5x) 1 tst x 2 strength + 9 engineers x .5 strength = 6.5 effort 0 tst x 2 strength + 9 engineers x 1 strength = 9.0 effort Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Motivation = ?
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Motivation = Autonomy + Mastery + Purpose - Daniel Pink
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Services, assets, and libraries • Components that help product ideas materialize faster and better • Marry your tech and culture • Focus teams on identifying their customer and making them happy • Task them with building the best solution in the world. Go! • Predictable timelines and behavior => Trust Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • A good service, asset, library… • Is easy to start using • Helps anybody in the company develop their ideas faster and better • Is useful in many contexts • Is consistently available with consistent latency at appropriate scales • Enforces boundaries (e.g., interface to credit card data) • Is scalable • Is coded cleanly, encourages testing • Supplies super easy monitoring, debugging, and alerting • Think: Legos Month 12+ · Build Your Business
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • A great service, asset, library… • Drives creativity • Think: Legos Month 12+ · Build Your Business
Unity Asset Store
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Build a good service framework • Look at Reddit’s Baseplate for design, tech choices (still in heavy dev) • Use Thrift for communication • Use ZooKeeper for service discovery • Build an asset store Month 12+ · Build Your Business
AWS API Route53, ELB S3 api. assets. MySQL CloudFrontWeb ZooKeeper, Thrift Data Services Redis Memcache Search Email PinLater Kafka S3 SparkSecor EMR Other Services Arch
MySQL Data Services Redis Memcache Search Email PinLater Kafka S3 Spark Secor EMR Platform Eng Product Eng Web, M Web Android Native iOS Native Frontend Frameworks Backend Services Growth Monetization Your Strategic Pillars Org Structure API ELB Route 53 Feeds ML Terrapin … …
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Salad Rabbit able to prototype and ship great ideas quickly • High functioning creative team that can focus on hard problems • Able to try experiments on massive audiences • Hiring the best people in the world • Able to change the world for the better Now
Turns out, many things do get better with scale…
Ask Me Anything Marty Weiner CTO, Reddit u/Mart2D2 @MartyWeiner
Reddit SnooSpotting: Austin where: Weather Up 1808 E. Cesar Chavez when: 4:00pm - 7:00pm
Appendix
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • For researching EC2 instance types, http://ec2instances.info is the best place ever • For managing Redis, here’s a tip Useful Tidbits
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • StatsD has several percentile options for showing your timing data • upper{50,95,99}, upper, lower{50,95,99}, lower, mean{50,95,99}, mean • upper means highest timing, upper99 means the 99%ile timing • Can be confusing because I would think upper99 would mean lowest 1%… • I don’t know why you would use mean{50,95,99} • Makes no statistical sense (am I missing something?) Tidbits about StatsD
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • StatsD has several percentile options for showing your timing data • These timings measure the percentile over a few seconds • No reason to not just go with the max (upper), since it’s over just a few seconds • You can aggregate max’s to get other percentiles over longer time periods • E.g., 95%ile of latency over 1 day • You cannot (statistically) aggregate anything but max (95%ile of 95%ile is nonsensical) • I don’t know why you would ever want lower percentiles • At least not for latency… • Does not tell you how good you can be Tidbits about StatsD
Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • May get stuck at 80 employees for a surprisingly long time • Not sure anybody understands why this happens… • Cultures usually struggle at 80 to 120 employees • Communications break down => Loss of trust • Goals might be unclear • Comp / HR discussions beginning • Some startup folks not comfortable • Management decisions must be made (can cause tension) • Things often smoother at 140 employees • Things tough again at 200 • … Growth tidbits

Recommended

How to go from structureless to structured without losing your vibe
How to go from structureless to structured without losing your vibeHow to go from structureless to structured without losing your vibe
How to go from structureless to structured without losing your vibeCamille Fournier
 
Surviving a Hackathon and Beyond
Surviving a Hackathon and BeyondSurviving a Hackathon and Beyond
Surviving a Hackathon and Beyondimoneytech
 
The 5 Biggest Productivity Blockers (And How to Fix Them)
The 5 Biggest Productivity Blockers (And How to Fix Them)The 5 Biggest Productivity Blockers (And How to Fix Them)
The 5 Biggest Productivity Blockers (And How to Fix Them)Atlassian
 
Building a Culture of Success on Open Principles
Building a Culture of Success on Open PrinciplesBuilding a Culture of Success on Open Principles
Building a Culture of Success on Open PrinciplesAtlassian
 
The Tools we Use 27 June 2017
The Tools we Use 27 June 2017The Tools we Use 27 June 2017
The Tools we Use 27 June 2017David Swaddle
 
Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...
Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...
Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...Michael Wolfe
 
E-learning and instructional design toolbox
E-learning and instructional design toolboxE-learning and instructional design toolbox
E-learning and instructional design toolboxDavid Swaddle
 
Cultivating Lean Startup Teams When People Don't Know What Lean Is
Cultivating Lean Startup Teams When People Don't Know What Lean IsCultivating Lean Startup Teams When People Don't Know What Lean Is
Cultivating Lean Startup Teams When People Don't Know What Lean IsEmily Holmes
 

Recommended

How to go from structureless to structured without losing your vibe
How to go from structureless to structured without losing your vibeHow to go from structureless to structured without losing your vibe
How to go from structureless to structured without losing your vibeCamille Fournier
 
Surviving a Hackathon and Beyond
Surviving a Hackathon and BeyondSurviving a Hackathon and Beyond
Surviving a Hackathon and Beyondimoneytech
 
The 5 Biggest Productivity Blockers (And How to Fix Them)
The 5 Biggest Productivity Blockers (And How to Fix Them)The 5 Biggest Productivity Blockers (And How to Fix Them)
The 5 Biggest Productivity Blockers (And How to Fix Them)Atlassian
 
Building a Culture of Success on Open Principles
Building a Culture of Success on Open PrinciplesBuilding a Culture of Success on Open Principles
Building a Culture of Success on Open PrinciplesAtlassian
 
The Tools we Use 27 June 2017
The Tools we Use 27 June 2017The Tools we Use 27 June 2017
The Tools we Use 27 June 2017David Swaddle
 
Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...
Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...
Michael Wolfe about Startups at ESADE: "Quitting, sucking, and failing your w...Michael Wolfe
 
E-learning and instructional design toolbox
E-learning and instructional design toolboxE-learning and instructional design toolbox
E-learning and instructional design toolboxDavid Swaddle
 
Cultivating Lean Startup Teams When People Don't Know What Lean Is
Cultivating Lean Startup Teams When People Don't Know What Lean IsCultivating Lean Startup Teams When People Don't Know What Lean Is
Cultivating Lean Startup Teams When People Don't Know What Lean IsEmily Holmes
 
12 Take Aways - Managing the Unmanageable
12 Take Aways - Managing the Unmanageable12 Take Aways - Managing the Unmanageable
12 Take Aways - Managing the UnmanageableRon Lichty
 
Mobile Learning Matters
Mobile Learning MattersMobile Learning Matters
Mobile Learning MattersDavid Swaddle
 
Bootstrapping a-devops-matter
Bootstrapping a-devops-matterBootstrapping a-devops-matter
Bootstrapping a-devops-matterSkills Matter
 
Do you want to be a manager (are you sure)
Do you want to be a manager (are you sure)Do you want to be a manager (are you sure)
Do you want to be a manager (are you sure)Ron Lichty
 
Managing Using Intuition and Rules of Thumb 050113
Managing Using Intuition and Rules of Thumb 050113Managing Using Intuition and Rules of Thumb 050113
Managing Using Intuition and Rules of Thumb 050113MWMantle
 
Graphs are Eating the World
Graphs are Eating the WorldGraphs are Eating the World
Graphs are Eating the WorldAll Things Open
 
Crash Course - managing software people and teams (sfelc, 10.26.16)
Crash Course - managing software people and teams (sfelc, 10.26.16)Crash Course - managing software people and teams (sfelc, 10.26.16)
Crash Course - managing software people and teams (sfelc, 10.26.16)Ron Lichty
 
LAST 2013 - Use games to innovate
LAST 2013 - Use games to innovateLAST 2013 - Use games to innovate
LAST 2013 - Use games to innovateantonrossouw
 
Business, STEM, Entrepreneurship: We all need each other!
Business, STEM, Entrepreneurship: We all need each other!Business, STEM, Entrepreneurship: We all need each other!
Business, STEM, Entrepreneurship: We all need each other!Shashi Jain
 
Vittorio Viarengo, ViVi software
Vittorio Viarengo, ViVi softwareVittorio Viarengo, ViVi software
Vittorio Viarengo, ViVi softwareNicolò Borghi
 
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)Ron Lichty
 
Energize 2013 slides
Energize 2013 slidesEnergize 2013 slides
Energize 2013 slidesNorris Krueger
 
Money, Process, and Culture- Tech 20/20 June, 2012
Money, Process, and Culture- Tech 20/20 June, 2012Money, Process, and Culture- Tech 20/20 June, 2012
Money, Process, and Culture- Tech 20/20 June, 2012Adrian Carr
 
Scrum: From the Classroom to the Workplace :: FCUP 2018
Scrum: From the Classroom to the Workplace :: FCUP 2018Scrum: From the Classroom to the Workplace :: FCUP 2018
Scrum: From the Classroom to the Workplace :: FCUP 2018Pedro Gustavo Torres
 
What your customers REALLY think: Incorporating usability testing into agile
What your customers REALLY think: Incorporating usability testing into agileWhat your customers REALLY think: Incorporating usability testing into agile
What your customers REALLY think: Incorporating usability testing into agilePhil Barrett
 
You Can't Buy Agile
You Can't Buy AgileYou Can't Buy Agile
You Can't Buy AgileRTigger
 
Insights From the Lean Startup Conference 2016
Insights From the Lean Startup Conference 2016Insights From the Lean Startup Conference 2016
Insights From the Lean Startup Conference 2016Jeffrey Tobias
 
Teamwork - making your dream team come true
Teamwork - making your dream team come trueTeamwork - making your dream team come true
Teamwork - making your dream team come trueRon Lichty
 
Would you use this? UX South Africa 2016
Would you use this? UX South Africa 2016Would you use this? UX South Africa 2016
Would you use this? UX South Africa 2016Phil Barrett
 
Steve Huffman - Lessons learned while at reddit.com
Steve Huffman - Lessons learned while at reddit.comSteve Huffman - Lessons learned while at reddit.com
Steve Huffman - Lessons learned while at reddit.comCarsonified Team
 
Defense
DefenseDefense
DefenseTawfiq Shah
 
Python RESTful webservices with Python: Flask and Django solutions
Python RESTful webservices with Python: Flask and Django solutionsPython RESTful webservices with Python: Flask and Django solutions
Python RESTful webservices with Python: Flask and Django solutionsSolution4Future
 

More Related Content

What's hot

12 Take Aways - Managing the Unmanageable
12 Take Aways - Managing the Unmanageable12 Take Aways - Managing the Unmanageable
12 Take Aways - Managing the UnmanageableRon Lichty
 
Mobile Learning Matters
Mobile Learning MattersMobile Learning Matters
Mobile Learning MattersDavid Swaddle
 
Bootstrapping a-devops-matter
Bootstrapping a-devops-matterBootstrapping a-devops-matter
Bootstrapping a-devops-matterSkills Matter
 
Do you want to be a manager (are you sure)
Do you want to be a manager (are you sure)Do you want to be a manager (are you sure)
Do you want to be a manager (are you sure)Ron Lichty
 
Managing Using Intuition and Rules of Thumb 050113
Managing Using Intuition and Rules of Thumb 050113Managing Using Intuition and Rules of Thumb 050113
Managing Using Intuition and Rules of Thumb 050113MWMantle
 
Graphs are Eating the World
Graphs are Eating the WorldGraphs are Eating the World
Graphs are Eating the WorldAll Things Open
 
Crash Course - managing software people and teams (sfelc, 10.26.16)
Crash Course - managing software people and teams (sfelc, 10.26.16)Crash Course - managing software people and teams (sfelc, 10.26.16)
Crash Course - managing software people and teams (sfelc, 10.26.16)Ron Lichty
 
LAST 2013 - Use games to innovate
LAST 2013 - Use games to innovateLAST 2013 - Use games to innovate
LAST 2013 - Use games to innovateantonrossouw
 
Business, STEM, Entrepreneurship: We all need each other!
Business, STEM, Entrepreneurship: We all need each other!Business, STEM, Entrepreneurship: We all need each other!
Business, STEM, Entrepreneurship: We all need each other!Shashi Jain
 
Vittorio Viarengo, ViVi software
Vittorio Viarengo, ViVi softwareVittorio Viarengo, ViVi software
Vittorio Viarengo, ViVi softwareNicolò Borghi
 
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)Ron Lichty
 
Money, Process, and Culture- Tech 20/20 June, 2012
Money, Process, and Culture- Tech 20/20 June, 2012Money, Process, and Culture- Tech 20/20 June, 2012
Money, Process, and Culture- Tech 20/20 June, 2012Adrian Carr
 
Scrum: From the Classroom to the Workplace :: FCUP 2018
Scrum: From the Classroom to the Workplace :: FCUP 2018Scrum: From the Classroom to the Workplace :: FCUP 2018
Scrum: From the Classroom to the Workplace :: FCUP 2018Pedro Gustavo Torres
 
What your customers REALLY think: Incorporating usability testing into agile
What your customers REALLY think: Incorporating usability testing into agileWhat your customers REALLY think: Incorporating usability testing into agile
What your customers REALLY think: Incorporating usability testing into agilePhil Barrett
 
You Can't Buy Agile
You Can't Buy AgileYou Can't Buy Agile
You Can't Buy AgileRTigger
 
Insights From the Lean Startup Conference 2016
Insights From the Lean Startup Conference 2016Insights From the Lean Startup Conference 2016
Insights From the Lean Startup Conference 2016Jeffrey Tobias
 
Teamwork - making your dream team come true
Teamwork - making your dream team come trueTeamwork - making your dream team come true
Teamwork - making your dream team come trueRon Lichty
 
Would you use this? UX South Africa 2016
Would you use this? UX South Africa 2016Would you use this? UX South Africa 2016
Would you use this? UX South Africa 2016Phil Barrett
 

What's hot (19)

12 Take Aways - Managing the Unmanageable
12 Take Aways - Managing the Unmanageable12 Take Aways - Managing the Unmanageable
12 Take Aways - Managing the Unmanageable
 
Mobile Learning Matters
Mobile Learning MattersMobile Learning Matters
Mobile Learning Matters
 
Bootstrapping a-devops-matter
Bootstrapping a-devops-matterBootstrapping a-devops-matter
Bootstrapping a-devops-matter
 
Do you want to be a manager (are you sure)
Do you want to be a manager (are you sure)Do you want to be a manager (are you sure)
Do you want to be a manager (are you sure)
 
Managing Using Intuition and Rules of Thumb 050113
Managing Using Intuition and Rules of Thumb 050113Managing Using Intuition and Rules of Thumb 050113
Managing Using Intuition and Rules of Thumb 050113
 
Graphs are Eating the World
Graphs are Eating the WorldGraphs are Eating the World
Graphs are Eating the World
 
Crash Course - managing software people and teams (sfelc, 10.26.16)
Crash Course - managing software people and teams (sfelc, 10.26.16)Crash Course - managing software people and teams (sfelc, 10.26.16)
Crash Course - managing software people and teams (sfelc, 10.26.16)
 
LAST 2013 - Use games to innovate
LAST 2013 - Use games to innovateLAST 2013 - Use games to innovate
LAST 2013 - Use games to innovate
 
Business, STEM, Entrepreneurship: We all need each other!
Business, STEM, Entrepreneurship: We all need each other!Business, STEM, Entrepreneurship: We all need each other!
Business, STEM, Entrepreneurship: We all need each other!
 
Vittorio Viarengo, ViVi software
Vittorio Viarengo, ViVi softwareVittorio Viarengo, ViVi software
Vittorio Viarengo, ViVi software
 
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
 
Energize 2013 slides
Energize 2013 slidesEnergize 2013 slides
Energize 2013 slides
 
Money, Process, and Culture- Tech 20/20 June, 2012
Money, Process, and Culture- Tech 20/20 June, 2012Money, Process, and Culture- Tech 20/20 June, 2012
Money, Process, and Culture- Tech 20/20 June, 2012
 
Scrum: From the Classroom to the Workplace :: FCUP 2018
Scrum: From the Classroom to the Workplace :: FCUP 2018Scrum: From the Classroom to the Workplace :: FCUP 2018
Scrum: From the Classroom to the Workplace :: FCUP 2018
 
What your customers REALLY think: Incorporating usability testing into agile
What your customers REALLY think: Incorporating usability testing into agileWhat your customers REALLY think: Incorporating usability testing into agile
What your customers REALLY think: Incorporating usability testing into agile
 
You Can't Buy Agile
You Can't Buy AgileYou Can't Buy Agile
You Can't Buy Agile
 
Insights From the Lean Startup Conference 2016
Insights From the Lean Startup Conference 2016Insights From the Lean Startup Conference 2016
Insights From the Lean Startup Conference 2016
 
Teamwork - making your dream team come true
Teamwork - making your dream team come trueTeamwork - making your dream team come true
Teamwork - making your dream team come true
 
Would you use this? UX South Africa 2016
Would you use this? UX South Africa 2016Would you use this? UX South Africa 2016
Would you use this? UX South Africa 2016
 

Viewers also liked

Steve Huffman - Lessons learned while at reddit.com
Steve Huffman - Lessons learned while at reddit.comSteve Huffman - Lessons learned while at reddit.com
Steve Huffman - Lessons learned while at reddit.comCarsonified Team
 
Python RESTful webservices with Python: Flask and Django solutions
Python RESTful webservices with Python: Flask and Django solutionsPython RESTful webservices with Python: Flask and Django solutions
Python RESTful webservices with Python: Flask and Django solutionsSolution4Future
 
Holy Crap! You Can Get Fired For Social Media Posts?
Holy Crap! You Can Get Fired For Social Media Posts?Holy Crap! You Can Get Fired For Social Media Posts?
Holy Crap! You Can Get Fired For Social Media Posts?Eric T. Tung
 
How to build a great coding culture
How to build a great coding cultureHow to build a great coding culture
How to build a great coding cultureMark Halvorson
 
Final venture outlook 2016
Final venture outlook 2016Final venture outlook 2016
Final venture outlook 2016Mark Suster
 
10 (Drunken) Facts about St. Patrick's Day
10 (Drunken) Facts about St. Patrick's Day 10 (Drunken) Facts about St. Patrick's Day
10 (Drunken) Facts about St. Patrick's Day Ethos3
 
SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!
SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!
SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!Eric Jackson
 
DocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360M
DocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360MDocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360M
DocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360MDocSend
 
The Mini-Guide to Presentation Practice
The Mini-Guide to Presentation PracticeThe Mini-Guide to Presentation Practice
The Mini-Guide to Presentation PracticeEthos3
 
How to Lead With Your Presentation
How to Lead With Your PresentationHow to Lead With Your Presentation
How to Lead With Your PresentationEthos3
 
The Weird History of Valentine's Day
The Weird History of Valentine's DayThe Weird History of Valentine's Day
The Weird History of Valentine's DayEthos3
 
Design in Tech Report 2015
Design in Tech Report 2015Design in Tech Report 2015
Design in Tech Report 2015John Maeda
 
2016 A-Z Culture Glossary
2016 A-Z Culture Glossary2016 A-Z Culture Glossary
2016 A-Z Culture Glossarysparks & honey
 
Blitzscaling Session 1: Household Stage
Blitzscaling Session 1: Household StageBlitzscaling Session 1: Household Stage
Blitzscaling Session 1: Household StageGreylock Partners
 
Activate Tech and Media Outlook 2016
Activate Tech and Media Outlook 2016Activate Tech and Media Outlook 2016
Activate Tech and Media Outlook 2016Activate
 
The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.
The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.
The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.Andrew Griffiths Enterprises
 
A Product Manager's Job
A Product Manager's JobA Product Manager's Job
A Product Manager's Jobjoshelman
 

Viewers also liked (20)

Steve Huffman - Lessons learned while at reddit.com
Steve Huffman - Lessons learned while at reddit.comSteve Huffman - Lessons learned while at reddit.com
Steve Huffman - Lessons learned while at reddit.com
 
Defense
DefenseDefense
Defense
 
Python RESTful webservices with Python: Flask and Django solutions
Python RESTful webservices with Python: Flask and Django solutionsPython RESTful webservices with Python: Flask and Django solutions
Python RESTful webservices with Python: Flask and Django solutions
 
Holy Crap! You Can Get Fired For Social Media Posts?
Holy Crap! You Can Get Fired For Social Media Posts?Holy Crap! You Can Get Fired For Social Media Posts?
Holy Crap! You Can Get Fired For Social Media Posts?
 
How to build a great coding culture
How to build a great coding cultureHow to build a great coding culture
How to build a great coding culture
 
Final venture outlook 2016
Final venture outlook 2016Final venture outlook 2016
Final venture outlook 2016
 
10 (Drunken) Facts about St. Patrick's Day
10 (Drunken) Facts about St. Patrick's Day 10 (Drunken) Facts about St. Patrick's Day
10 (Drunken) Facts about St. Patrick's Day
 
SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!
SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!
SpringOwl's 99 Page Presentation On How To Best Turnaround Yahoo!
 
DocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360M
DocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360MDocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360M
DocSend Fundraising Research: What we Learned from 200 Startups Who Raised $360M
 
The Mini-Guide to Presentation Practice
The Mini-Guide to Presentation PracticeThe Mini-Guide to Presentation Practice
The Mini-Guide to Presentation Practice
 
How to Lead With Your Presentation
How to Lead With Your PresentationHow to Lead With Your Presentation
How to Lead With Your Presentation
 
The Weird History of Valentine's Day
The Weird History of Valentine's DayThe Weird History of Valentine's Day
The Weird History of Valentine's Day
 
Design in Tech Report 2015
Design in Tech Report 2015Design in Tech Report 2015
Design in Tech Report 2015
 
2016 A-Z Culture Glossary
2016 A-Z Culture Glossary2016 A-Z Culture Glossary
2016 A-Z Culture Glossary
 
Blitzscaling Session 1: Household Stage
Blitzscaling Session 1: Household StageBlitzscaling Session 1: Household Stage
Blitzscaling Session 1: Household Stage
 
Activate Tech and Media Outlook 2016
Activate Tech and Media Outlook 2016Activate Tech and Media Outlook 2016
Activate Tech and Media Outlook 2016
 
State of Startups 2015
State of Startups 2015State of Startups 2015
State of Startups 2015
 
The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.
The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.
The Ultimate Productivity Webinar - with Andrew Griffiths and Sam Harrop.
 
A Product Manager's Job
A Product Manager's JobA Product Manager's Job
A Product Manager's Job
 
Work Rules!
Work Rules!Work Rules!
Work Rules!
 

Similar to Marty Weiner, Reddit CTO @ SXSWi 2016 - Everything's Harder At Scale

Fast prototypes and customer development for start ups
Fast prototypes and customer development for start upsFast prototypes and customer development for start ups
Fast prototypes and customer development for start upsSerdar Temiz
 
CTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, ViadeoCTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, ViadeoFrance Digitale
 
Life in the tech trenches (2015)
Life in the tech trenches (2015)Life in the tech trenches (2015)
Life in the tech trenches (2015)Julien SIMON
 
Customer Development Fast Protyping
Customer Development Fast ProtypingCustomer Development Fast Protyping
Customer Development Fast ProtypingSerdar Temiz
 
Velocity Conference NYC 2014 - Real World DevOps
Velocity Conference NYC 2014 - Real World DevOpsVelocity Conference NYC 2014 - Real World DevOps
Velocity Conference NYC 2014 - Real World DevOpsRodrigo Campos
 
Emulating Gang Culture to Accelerate In-House SEO
Emulating Gang Culture to Accelerate In-House SEOEmulating Gang Culture to Accelerate In-House SEO
Emulating Gang Culture to Accelerate In-House SEOsimondance
 
Project Management 101 - Wordcamp TO 05112011
Project Management 101 - Wordcamp TO 05112011Project Management 101 - Wordcamp TO 05112011
Project Management 101 - Wordcamp TO 05112011Liesl Barrell
 
DevDay 2013 - Building Startups and Minimum Viable Products
DevDay 2013 - Building Startups and Minimum Viable ProductsDevDay 2013 - Building Startups and Minimum Viable Products
DevDay 2013 - Building Startups and Minimum Viable ProductsBen Hall
 
VC Evolution: Geeks Got Next
VC Evolution: Geeks Got NextVC Evolution: Geeks Got Next
VC Evolution: Geeks Got NextDave McClure
 
Startup Life: the practical guide
Startup Life: the practical guideStartup Life: the practical guide
Startup Life: the practical guideLinda Liukas
 
Recruiting Great Engineers in Six Easy Steps
Recruiting Great Engineers in Six Easy StepsRecruiting Great Engineers in Six Easy Steps
Recruiting Great Engineers in Six Easy StepsAleksandr Yampolskiy
 
CYCLES Course (1): Course Introduction
CYCLES Course (1): Course Introduction CYCLES Course (1): Course Introduction
CYCLES Course (1): Course Introduction Bryan Cassady
 
Android Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and PatternsAndroid Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and Patternsgdgut
 
Design thinking in startups slideshare version
Design thinking in startups slideshare versionDesign thinking in startups slideshare version
Design thinking in startups slideshare versionesegalico
 
Software Developer Career Unplugged - GeeCon 2013
Software Developer Career Unplugged - GeeCon 2013Software Developer Career Unplugged - GeeCon 2013
Software Developer Career Unplugged - GeeCon 2013Wojciech Seliga
 
Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...
Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...
Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...Acquia
 
Let's bring the teams back together
Let's bring the teams back togetherLet's bring the teams back together
Let's bring the teams back togetherKris Buytaert
 
The Hardest Thing To Get Right (GeekWire Startup Day 2015)
The Hardest Thing To Get Right (GeekWire Startup Day 2015)The Hardest Thing To Get Right (GeekWire Startup Day 2015)
The Hardest Thing To Get Right (GeekWire Startup Day 2015)Jordan Ritter
 
How to Pitch a VC (Shanghai, May 2012)
How to Pitch a VC (Shanghai, May 2012)How to Pitch a VC (Shanghai, May 2012)
How to Pitch a VC (Shanghai, May 2012)Dave McClure
 
The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S...
The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S... The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S...
The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S...Seattle Interactive Conference
 

Similar to Marty Weiner, Reddit CTO @ SXSWi 2016 - Everything's Harder At Scale (20)

Fast prototypes and customer development for start ups
Fast prototypes and customer development for start upsFast prototypes and customer development for start ups
Fast prototypes and customer development for start ups
 
CTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, ViadeoCTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, Viadeo
 
Life in the tech trenches (2015)
Life in the tech trenches (2015)Life in the tech trenches (2015)
Life in the tech trenches (2015)
 
Customer Development Fast Protyping
Customer Development Fast ProtypingCustomer Development Fast Protyping
Customer Development Fast Protyping
 
Velocity Conference NYC 2014 - Real World DevOps
Velocity Conference NYC 2014 - Real World DevOpsVelocity Conference NYC 2014 - Real World DevOps
Velocity Conference NYC 2014 - Real World DevOps
 
Emulating Gang Culture to Accelerate In-House SEO
Emulating Gang Culture to Accelerate In-House SEOEmulating Gang Culture to Accelerate In-House SEO
Emulating Gang Culture to Accelerate In-House SEO
 
Project Management 101 - Wordcamp TO 05112011
Project Management 101 - Wordcamp TO 05112011Project Management 101 - Wordcamp TO 05112011
Project Management 101 - Wordcamp TO 05112011
 
DevDay 2013 - Building Startups and Minimum Viable Products
DevDay 2013 - Building Startups and Minimum Viable ProductsDevDay 2013 - Building Startups and Minimum Viable Products
DevDay 2013 - Building Startups and Minimum Viable Products
 
VC Evolution: Geeks Got Next
VC Evolution: Geeks Got NextVC Evolution: Geeks Got Next
VC Evolution: Geeks Got Next
 
Startup Life: the practical guide
Startup Life: the practical guideStartup Life: the practical guide
Startup Life: the practical guide
 
Recruiting Great Engineers in Six Easy Steps
Recruiting Great Engineers in Six Easy StepsRecruiting Great Engineers in Six Easy Steps
Recruiting Great Engineers in Six Easy Steps
 
CYCLES Course (1): Course Introduction
CYCLES Course (1): Course Introduction CYCLES Course (1): Course Introduction
CYCLES Course (1): Course Introduction
 
Android Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and PatternsAndroid Developer Skills, Techniques, and Patterns
Android Developer Skills, Techniques, and Patterns
 
Design thinking in startups slideshare version
Design thinking in startups slideshare versionDesign thinking in startups slideshare version
Design thinking in startups slideshare version
 
Software Developer Career Unplugged - GeeCon 2013
Software Developer Career Unplugged - GeeCon 2013Software Developer Career Unplugged - GeeCon 2013
Software Developer Career Unplugged - GeeCon 2013
 
Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...
Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...
Story of Multnomah County: Migrating from Vignette and Building a Drupal Ecos...
 
Let's bring the teams back together
Let's bring the teams back togetherLet's bring the teams back together
Let's bring the teams back together
 
The Hardest Thing To Get Right (GeekWire Startup Day 2015)
The Hardest Thing To Get Right (GeekWire Startup Day 2015)The Hardest Thing To Get Right (GeekWire Startup Day 2015)
The Hardest Thing To Get Right (GeekWire Startup Day 2015)
 
How to Pitch a VC (Shanghai, May 2012)
How to Pitch a VC (Shanghai, May 2012)How to Pitch a VC (Shanghai, May 2012)
How to Pitch a VC (Shanghai, May 2012)
 
The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S...
The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S... The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S...
The Magic Pitch: What You Can Learn From Crowdfunding to Tell Your Story | S...
 

Recently uploaded

Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 

Recently uploaded (20)

Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 

Marty Weiner, Reddit CTO @ SXSWi 2016 - Everything's Harder At Scale

  • 1. Everything’s Harder at Scale Marty Weiner CTO, Reddit u/Mart2D2 @MartyWeiner
  • 3. • Reduce the pain of the first 18 months of scalability hell • Look at how a company scales technically and culturally from idea to BIG at major inflection points • Strategies for thinking about technology and culture • Construct a tech stack recipe you can use Goals • Focus is on engineering • For those who have not experienced small -> BIG • Every startup is different, but in many ways they’re the same • Scope is from idea to solid traction and about 100 employees • Lots in this talk, will put slides up • I’m not being paid by any vendors!
  • 4. Inflection Points LaunchDeveloping the Idea Trickle :( TRAAAACTION! Becoming a Business
  • 5. Let’s make a fictional startup!
  • 6. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Idea bubbling around in founder’s head • Work on it in spare time • Doodle in class / at work / in dorm / at coffee shop Day -100 · Idea Forming
  • 7.
  • 8.
  • 9.
  • 10. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Here’s the idea for our startup: • It’s a social network • Topics organized by community • You can send messages • Max of 140 characters • They’re deleted in 10 seconds after viewing • Pin your interests • And you can order a taxi… Day -90 · Fleshing it out
  • 11. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • It’s called SnapinUberTweetySnoo • snapinubertweetysnoo.com taken :( Day -90 · Fleshing it out
  • 12. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Settle on “Salad Rabbit” • saladrabbit.com was the least bad idea for a URL that’s available Day -90 · Fleshing it out
  • 13.
  • 14. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • No serious competition, or at least you totally can do better! • You (or you and engy buddy) start mocking up an app Day -90 · Fleshing it out
  • 15. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Much Ramen later, you slapped together an iPhone app and servers • Friends and family using it • Get some YC money • You set up Google Analytics and add a few extras like push notifications Day -30 · Demo!
  • 16. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • You officially launch! • It’s on the App Store!!!!! Day 0 · Launch!
  • 17. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Sit there refreshing Google Analytics over and over and over and over….. Day 0 · Launch!
  • 18. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Nobody coming. Well, ok, a few.. Day 10 · Waiting
  • 19. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Developing the idea and the tech stack. Added some features. • A few people are coming. At least they’re very passionate • Is anything going to happen? Day 11 to 119 · A trickle • Is this working? • Should… should I pivot??
  • 20. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Day 11 to 119 · A trickle Not supposed to be stealth mode :( Pinterest Page Views / Day
  • 21. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Not really sure why, but users start pouring in • Featured in Apple’s Best New Apps with “Salad” in the Title • Hijacked the jumbotron at the Superbowl • Hopefully this is not a fluke Day 120 · Something’s happening ?!
  • 22. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Users still pouring in! • You’re running out to talk to VCs • Trying to hire engineers. They’re junior or hackers. • Your tech is a confusing blob of shiny new technologies • Site now timing out consistently. Brace yourself, it’s gonna be a tough 12+ months of scalability hell. Day 140 · Traaaaaction!
  • 23. Let’s go back in time…
  • 24. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Founder starts digging deeper • Goal should be, above all, on getting a Minimal Viable Product out there for people to see Day -90 · Fleshing it out • Hypothesis is “People will like what I’m building”
  • 25. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Day -90 · Fleshing it out Maximize for Velocity, Agility, Creativity
  • 26. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Choose one frontend platform: iPhone, Android, Web Day -90 · Fleshing it out
  • 27. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Choose one frontend platform: iPhone, Android, Web • You’ll need a backend to store data, facilitate communications, etc • Maximizes velocity, agility, creativity • One we can evolve with the number of employees and users • Let’s construct a recipe you can use Day -90 · Fleshing it out
  • 28. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Tons of choice for tech. Databases alone: • Postgres • Redis • Cassandra • HBase • Riak • Neo4j Tech Maturity • CouchDB • Membase • MySQL • MongoDB • VoltDB • … hundred others
  • 29. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • “We’re building the latest and greatest. I need the latest and greatest.” • No no no no no no… no… no • “This tech magically scales for you” • Until it doesn’t… • “It’s shiny and new. It must be faster or betterer” • Stop Tech Maturity
  • 30. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Tech Maturity Maturity = Blood + Sweat Complexity
  • 31. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Redis Maturity = Blood + Sweat Complexity
  • 32. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business HBase Maturity = Blood + Sweat Complexity
  • 33. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business MySQL Maturity = Blood + Sweat Complexity
  • 34. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Sometimes you must use immature tech, but… • Harder to hire • Harder to get help (yes, all tech has problems at some point) • Stability and performance uncertain, especially at larger scale • Debugging will be harder • Tools weaker • If biggest user, challenges greatly amplified • Most likely you won’t *really* need immature tech for a long time Tech Maturity
  • 35. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use Amazon Web Services (AWS) for hosting • Evolves and scales with you from tiny to BIG • Competitively priced • Lots of tools to help get you moving (DBs, cache, search, email, DNS, CDN, …) Day -90 · Fleshing it out
  • 36. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use AWS Route53 to register your domain (saladrabbit.com) Day -90 · Fleshing it out
  • 37. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Spin up a tiny EC2 instance for your first API (or web) server • Describes outside interfaces and contains product logic • Use latest Ubuntu • Nginx to serve HTTP • Use Python / Flask / pymysql or Python / Django for backend code • Use Route53 to connect your domain to your tiny instance • Using api.saladrabbit.com • Your EC2 instance storage is EBS (Easy backups!) • Call it something descriptive like api001 Day -90 · Fleshing it out
  • 38. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Create an S3 bucket for serving static assets (js, css, images) • S3 is super durable, very available storage • Use Route53 to connect a subdomain to it (e.g, assets.saladrabbit.com) Day -90 · Fleshing it out
  • 39. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use git + Github for your version control • Get code backup for free! • Good for code reviews, bug reporting when you’re ready for it • Makes deployment easier Day -90 · Fleshing it out
  • 41. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Need a place to store user info, content, follower graphs, etc • We need a database Day -90 · Fleshing it out
  • 42. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Just use MySQL • The boring tech revolution is here • Use AWS Relational Database Service (RDS) to manage it (Tiny instance size) • Conveniently manages MySQL instances • Costs ~1.5x • RDS -> self managed MySQL migration is easy • Don’t have to worry about database operations for early days! • No configs to understand • Backups done for you • No messing with failovers (even has automatic failover) • Don’t be afraid of joins, distinct, count, etc! Day -90 · Fleshing it out
  • 44. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • How does your API box know the RDS endpoint? • Just use simple python config files (no haproxy layers, no zookeeper, …) Day -90 · Fleshing it out
  • 45. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Make your backend easy to debug • Use Python logging to syslog (AWS CloudWatch looks like an interesting option) Day -90 · Fleshing it out
  • 46. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Search • Use AWS CloudSearch Day -90 · Fleshing it out
  • 47. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • That’s probably it • Don’t need to send notifications, emails, etc unless they’re vital to your demo • Don’t optimize anything on the backend unless you have to Day -90 · Fleshing it out
  • 48. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • So far… • We have an app • Architecture that doesn’t limit your velocity, agility, creativity • Won’t scale far, but we can evolve and grow it • Now go show people! Day -90 · Fleshing it out
  • 49. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Show it to friends, family, r/entrepreneur, people on the train • Hopefully people love it! Though they’ll hate the name… • Take feedback, thank people for it, ignore the naysayers Day -30 · Demo!
  • 50. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Focus on last critical product needs • Tighten up the frontend experience • Add asynchronous background processing • For sending email, sharing to Facebook / Twitter, processing images, etc… • Use PinLater for background processing • Push Notifs (AWS SNS), SMS (Twilio), Email (AWS SES) • Add Google Analytics to frontend Day -29 to -1 · Prep for Launch
  • 51. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Tell everybody! • Connect to Google Analytics and hit refresh over and over again Day 0 · Laaaaunch!
  • 52. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Stay focused on product (no need to sacrifice velocity, agility, creativity) • Round out product and operations Day 1 to 119 · A Trickle
  • 53. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • If necessary, add other platforms Day 1 to 119 · A Trickle
  • 54. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Add StatsD to measure timings and counts live (only use _upper* timings!) Day 1 to 119 · A Trickle * Long story, see appendix statsd_connection.incr(“event.user_signup”) statsd_connection.timing(“event.mysql_queries.get_user_feed”, time_ms)
  • 55. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Add StatsD timings to all interface points • All API endpoints • All MySQL calls • All calls to other services • All background tasks • Make a dashboard Day 1 to 119 · A Trickle
  • 56. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Make it faster to launch / config boxes with IT Automation • Puppet, Chef (AWS OpsWorks) Day 1 to 119 · A Trickle
  • 57. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Set up CloudWatch or Pingdom to notify of downtime • Will save you sleep Day 1 to 119 · A Trickle
  • 58. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Add more API servers • Use AWS ELB to distribute load • Make it fast to bring one API server up Day 1 to 119 · A Trickle
  • 59. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Set up development and staging environments • Make Route53 connections (e.g., martydev.saladrabbit.com) • Make it easy to deploy new code to production or staging • Use Teletraan Day 1 to 119 · A Trickle
  • 60. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • So far… • iPhone app talking to your API + logic servers through Route53 and ELB • Database (MySQL) • Static storage (S3) • Search (Cloudwatch), Virality, Background Task Processing (PinLater) • Visibility (Google Analytics, Statsd, AWS CloudWatch) • Deployability (staging, dev, Teletraan, AWS OpsWorks) Day 1 to 119 · A Trickle
  • 61. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Ok, people coming. Whoa. Day 120 · Something Happening
  • 62. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Not a fluke… even more are coming. Winter is here. Day 140 · Traction
  • 63. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Site begins to timeout or queues start backing up • Profile first! (Check StatsD) • Probably your poor database (will bet money) Day 140 · Traction
  • 64. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • NOW begin to trade off velocity, agility, creativity for site availability • Don’t worry, we’ll get them back Day 140 · Traction
  • 65. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • If it is your database that’s causing problems… • Start increasing instance sizes (Add RAM and especially go to SSD) • Short downtime in RDS • Begin optimizing your queries and reducing load on MySQL • Check slowlog and profile (SHOW FULL PROCESSLIST) for worst queries • Add indexes • Start caching • Start normalizing all queries and remove joins, distincts, etc Day 140 · Traction
  • 66. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Caching • Create Memcache or Redis servers behind McRouter or use Elasticache • Check cache -> go to db -> put back in cache • Hunt for common queries you can cache Day 140 · Traction
  • 67. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Normalize your data • Object tables are only used for ID -> data lookups • e.g., users table (noun) • Add primary key + auto_increment on ID (this generates IDs for you!) • Mapping tables are only used for From ID -> To ID lookups • e.g., users_said_comments (noun_activeverb_noun) • Add sequence column if you need order and your ID does not support order • Add indexes: Unique on From,To and Index on To Day 140 · Traction
  • 68. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Normalize SELECT text FROM comments WHERE user_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s SELECT text FROM comments WHERE listing_id=%(listing_id)s SELECT comment_id FROM listing_contains_comments WHERE listing_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s
  • 69. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Normalize SELECT text FROM comments WHERE user_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s SELECT text FROM comments WHERE listing_id=%(listing_id)s SELECT comment_id FROM listing_contains_comments WHERE listing_id=%s SELECT text FROM comments WHERE id IN %(comment_ids)s
  • 70. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Optimizing Feeds SELECT text FROM comments LEFT JOIN followers ON comments.user_id=followers.followee_id WHERE followers.follower_id=%(user_id)s ORDER BY create_date DESC OFFSET 0,20 SELECT followee_id FROM users_followed_by_users WHERE follower_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id IN %(followee_ids)s ORDER BY sequence DESC OFFSET 0,20 SELECT text FROM comments WHERE id IN %(comment_ids)s
  • 71. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Optimizing Feeds SELECT followee_id FROM users_followed_by_users WHERE follower_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id IN %(followee_ids)s ORDER BY sequence DESC OFFSET 0,20 SELECT text FROM comments WHERE id IN %(comment_ids)s [Aync after comment] SELECT follower_id FROM users_followed_by_users WHERE followee_id=%(user_id)s Put Comment in Redis follower_to_comments [On Read] comment_ids = redis_get_follower_to_comments(user_id) SELECT text FROM comments WHERE id IN %(comment_ids)s
  • 72. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Optimizing Feeds SELECT followee_id FROM users_followed_by_users WHERE follower_id=%(user_id)s SELECT comment_id FROM users_said_comments WHERE user_id IN %(followee_ids)s ORDER BY sequence DESC OFFSET 0,20 SELECT text FROM comments WHERE id IN %(comment_ids)s [Aync after comment] SELECT follower_id FROM users_followed_by_users WHERE followee_id=%(user_id)s Put Comment in Redis follower_to_comments [On Read] comment_ids = get_follower_to_comments(user_id) SELECT text FROM comments WHERE id IN %(comment_ids)s
  • 73. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • MySQL doesn’t magically scale out of the box • When to Shard? • Increased instance sizes to big SSD servers • And you’ve moved as many tables to other instances as possible • And you are reading from read slaves • How to Shard? • Click here or Google for “Sharding Pinterest” • Google for “Sharding & IDs at Instagram” Day 140 · Traction
  • 74. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Day 140 · Traction
  • 75. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Logging data and operations • Log all page views (and some info on what was seen) • Log all major events (create, update, delete, click offsite, etc) • Eventually log all service calls • Why? • Will save your butt some day (when you bork your database) • Beginning of really understanding your users and growth Day 140 · Traction
  • 76. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Use Kafka and Secor to get events from backend into AWS S3 • Use AWS EMR to start digging into data • Pinball or AWS SWF help you run queries automatically • Transcode MySQL backups nighty into S3 so EMR can use them • Near future: data science, A/B experiments • Not yet: machine learning Day 140 · Traction
  • 77. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Getting good at operations • Scale (more) under control • Have data and some goals + metrics • Growing company size (10+) • Foosball table Month 12+ · Build Your Business
  • 78. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • New Eng Goal: Build The Dream Machinetm • Team and tech that can build any idea faster and better with each year • Tech even drives creativity • Start to regain velocity, agility, and creativity Month 12+ · Build Your Business
  • 79. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • You’ve probably built up some tech debt and have a nasty monolithic codebase • Pinboard (Pinterest), R2 (Reddit), Obidos (Amazon) • Begin to pay down tech debt • Align teams for success • Move to a service oriented architecture Month 12+ · Build Your Business
  • 80. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Engineering management • Don’t “promote” best engineer to manager • Lose best coder / doer, made a First Time Manager (FTM) • Do have discussions with engineers about their careers • Management should not be a promotion • Give engies and managers equal path to impact and comp • Engineers who have never managed should not step in lightly • Give 6 month path Month 12+ · Build Your Business
  • 81. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Technically Strong Jerk (TSJ) • Appear at many companies at 10+ employees • 2x faster than others, maybe leading an important initiative • … but make life hell for everybody else (1x -> .5x) 1 tst x 2 strength + 9 engineers x .5 strength = 6.5 effort 0 tst x 2 strength + 9 engineers x 1 strength = 9.0 effort Month 12+ · Build Your Business
  • 82. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Motivation = ?
  • 83. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business Motivation = Autonomy + Mastery + Purpose - Daniel Pink
  • 84. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Services, assets, and libraries • Components that help product ideas materialize faster and better • Marry your tech and culture • Focus teams on identifying their customer and making them happy • Task them with building the best solution in the world. Go! • Predictable timelines and behavior => Trust Month 12+ · Build Your Business
  • 85. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • A good service, asset, library… • Is easy to start using • Helps anybody in the company develop their ideas faster and better • Is useful in many contexts • Is consistently available with consistent latency at appropriate scales • Enforces boundaries (e.g., interface to credit card data) • Is scalable • Is coded cleanly, encourages testing • Supplies super easy monitoring, debugging, and alerting • Think: Legos Month 12+ · Build Your Business
  • 86. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • A great service, asset, library… • Drives creativity • Think: Legos Month 12+ · Build Your Business
  • 88. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Build a good service framework • Look at Reddit’s Baseplate for design, tech choices (still in heavy dev) • Use Thrift for communication • Use ZooKeeper for service discovery • Build an asset store Month 12+ · Build Your Business
  • 89. AWS API Route53, ELB S3 api. assets. MySQL CloudFrontWeb ZooKeeper, Thrift Data Services Redis Memcache Search Email PinLater Kafka S3 SparkSecor EMR Other Services Arch
  • 90. MySQL Data Services Redis Memcache Search Email PinLater Kafka S3 Spark Secor EMR Platform Eng Product Eng Web, M Web Android Native iOS Native Frontend Frameworks Backend Services Growth Monetization Your Strategic Pillars Org Structure API ELB Route 53 Feeds ML Terrapin … …
  • 91. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • Salad Rabbit able to prototype and ship great ideas quickly • High functioning creative team that can focus on hard problems • Able to try experiments on massive audiences • Hiring the best people in the world • Able to change the world for the better Now
  • 92. Turns out, many things do get better with scale…
  • 93. Ask Me Anything Marty Weiner CTO, Reddit u/Mart2D2 @MartyWeiner
  • 94. Reddit SnooSpotting: Austin where: Weather Up 1808 E. Cesar Chavez when: 4:00pm - 7:00pm
  • 96. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • For researching EC2 instance types, http://ec2instances.info is the best place ever • For managing Redis, here’s a tip Useful Tidbits
  • 97. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • StatsD has several percentile options for showing your timing data • upper{50,95,99}, upper, lower{50,95,99}, lower, mean{50,95,99}, mean • upper means highest timing, upper99 means the 99%ile timing • Can be confusing because I would think upper99 would mean lowest 1%… • I don’t know why you would use mean{50,95,99} • Makes no statistical sense (am I missing something?) Tidbits about StatsD
  • 98. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • StatsD has several percentile options for showing your timing data • These timings measure the percentile over a few seconds • No reason to not just go with the max (upper), since it’s over just a few seconds • You can aggregate max’s to get other percentiles over longer time periods • E.g., 95%ile of latency over 1 day • You cannot (statistically) aggregate anything but max (95%ile of 95%ile is nonsensical) • I don’t know why you would ever want lower percentiles • At least not for latency… • Does not tell you how good you can be Tidbits about StatsD
  • 99. Idea Forming Fleshing It Out Demo! Launch! Trickle :( Something’s Happening ?! TRAAAACTION! Becoming a Business • May get stuck at 80 employees for a surprisingly long time • Not sure anybody understands why this happens… • Cultures usually struggle at 80 to 120 employees • Communications break down => Loss of trust • Goals might be unclear • Comp / HR discussions beginning • Some startup folks not comfortable • Management decisions must be made (can cause tension) • Things often smoother at 140 employees • Things tough again at 200 • … Growth tidbits