Startonomics 2008- Live Blogging

Troy Angrignon

Oct 2, 2008 • 13 min read

Hey all,

I’m at Startonomics today, put on by Dealmaker Media. I’ll be posting updates to this over the course of the day.

Live stream is here:

My notes and editorial comments are in [square brackets.]

These are DRAFT NOTES. Better versions coming after the conference, complete with images.

MORNING SESSION:

Dave McClure, 500 hats

Image representing Dave McClure as depicted in CrunchBase

Image via CrunchBase

Metrics
- ultimately, lead through to sale conversion and total return are the keys.
  - Many people pay attention to uniques at the top of the funnel and $ at the bottom. That’ s okay, but look at the conversion steps INSIDE there.
Be a pirate who says “AARRR” – Acquisition, Activation, Referral, Retention, Revenue
Typical failures:
- business plans are bullshit
- revenue projections are hockety stick delusions
- features don’t matter; usability and MEASURED conversion matters
- Offline PR, “Big-bang launches” are stupid and expensive. In general, offline to online conversion is 100:1 or greater (i.e. it’s very weak.)
- Investors don’t know your business any Better than you do. Their opinions are interesting but don’t change your plan because they have an opinion.
Be Bold. Be Humble.
- 20% inspiration; 80% perspiration
- Be audacious but also creative
- Be humble and be analytically rigorous
What matters?
- passion for problem
- hypothesis of cutosmre lifecycle
- 1 page business model (proiritized list of users and conversion)
- critical, few actionalble metrics and a dashboard of MEASURED user behaviour
- 1 page marketing plan (channels & campaigns * (volume, cost, conv%)
- voleocity of (product execution + cycle time of Testing) * Iteration rate
  - Speed is the advantage, not IP defense
What’s my business model?
- how you get users
- how you drive usage
- How you make money profitably short and long term
which metrics?
- few metrics and you can use them to make decisions
- hypothesize customer lifecycle
1 page business model:
- what TYPES of people use your tool?
  - visitor?
  - contributor?
  - distributor?
- what ACTIONS could they take to help YOU or THEM?
- Look at the TeachStreet dashboard that shows where they are strong and where they are weak (see slide)
What features and why? When are you done? When do you remove them?
- use conversion to guide
- (use evidence-based decision-making, not “I like…” or “he likes…” or “I think…”)
- easy to find, fun, useful, unique features THAT INCREASE CONVERSION are the only things that should go in.
- spend 80% of your time on HYPOTHESIS CORRECTION, not on feature development.
- [This was the key take-away for me. I see most teams focusing on arguing about features, not arguing about their hypothesis and then testing it against market feedback.]
- Most companies are doing 80% feature development with no A/B testing and feedback loop
- The ONLY pupose is to move users from lower to higher value states.
- Features are ways to make that value leap happen faster. That’s it.
Check out KissMetrics for product planning metrics tools
Marketing and sales
- move your marketing from vague to highly rigoroous
- you want to increase volume, decrease cost, and increase conversion rates. That’s it
- design & test multiple marketing channels campaigns
- focus on best peformaing channels and tehme s
- optimize for conversion to target CTAs
- (Look for SLIDE: Example Marketing channels)
One step at a time:
- Make a good product: activation & retention
- Market the product: Acquisition & Referral
  - market once you get your product to an 8/10
- Make Money: Revenue & Profitability
- [I think it’s interesting that he’s still using Build…then market…then sell, and not the sell first, build second model which is safer, following the Dell Model or the Cisco Sale model.]

Dan Olsen, CEO & founder, YourVersion – Designing the DNA of a Killer App

I led product for Intuit and for Friendster
Dan helped Friendster drive massive traffic gains that they were never able to recover fro
How to integrate metrics & optimization into your process
- Plan
  - business objectives
  - product objectives
  - prioritized features list (we use a one page google sheet)
  - requirements for the feature
  - scope of the feature that feeds back to the prioritized features list
  - code
  - test
  - launch
  - track metrics and user feedback to then loop back into biz objectives, product objectives, and product requirements
- Design
- Develop
- Optimize
  - measure metric
  - analyze the metric
  - identify top opportunnities to improve
  - design and develop the enhncement
  - launch the enhancement
  - measure the mtric (repeat
  - The value is in the learning of market, customer, domain, usability
  - The faster you iterate this cycle, the faster you learn
Using metrics to optimize value creation
- for you
  - “given current reality, how do we optimize the busines results subject to our resource constraints?”
  - (grab his slides – missed all of this.)
  - Take-aways:
    - you want to track things over time DAILY
    - you want to track them visually
    - create ratios from primary metrics so that you’re not fooled by volume changes. It also allows you to compare to other sites.
    - (inserrt his slide of sign up page metrics)
  - “using metrics to optimize the equiation of your business”
    - sniper shot – least effort for maximal gain/result
    - [YES!!}
  - optiming metrics for viral loop:
    - active users
    - invite
    - prospective users click
    - registration
    - succeed
    - Lop again
    - (grab this slide)
  - grab the maximum upside potentio of a metric slide. it is KEY here.
    - see third metric (there is no upper boundary of 100% (because it’s how many invitations can they send?)
    - see the “doubling number of invitations sent” slide)
- for your custsomers
  - customer value =
    - satisfies customer needs (It is NOT ABOUT FEATURES)
    - is easy to use
    - has a good price
    - is better than alternatives
  - cust value = customer stisfaction
  - not fuzzy:
    - # of users
    - frequency of use
    - At a bare minimum, track those.
  - understanding user needs and satisfaction:
    - quantitative: site analytics, usage metrics, surveys
    - Qualitative: support emailos, usability sessions (best)
  - Value Proposition:
    - Which benefits are most imporant?
    - How well does it deliver?
    - (grab this slide/table)
  - Analyzing product ideas ROI
    - see related slide/graph
    - this is based on your prioritiezed feature list
  - UI Design iceberg
    - [HA! I have been using this model all along and didn’t realize that asomebody had this awesome graphic.]
    - www.jjg.net – Elements of user experience chart from Jesse James Garrett
  - Put Key conversation actions above the fold.
    - Grab chart from clicktale.com
    - Most people are running at 768 pixels
    - The chrome steals 170 pixels off the top
    - leaving you 600 pixels.
  - Case study: Intuit account signup process redesign
    - we had to ask lots of questions for legal reasons
    - we tracked new and existing signins.
    - we got a 37% improvement in conversion rates
- SUMMARY
  - define success for your customer and your business
- Other resources:
  - slideshare product requirements doc from web 2.0 exp
- CONTACT:
  - dan@yourversion.com
- QUESTIONS
  - long MRD and PRD docs are over;
  - go with light weight wiki docs

Startup Scalability Strategies, Frank Mashraqi

Frank is the expert outside of MySQL;
spending too much is wrong
not worrying aobut iss is also wrongwhat to measure?
nobody knows what cloud computing is; it’s very fuzzy
Agenda
- what to focus on?
  - distribution of data: how is it distributed?
    - how many servers?
    - what percent of data per server?
    - how many connections per shared/ partition?
    - divde the data up into multile databases or shareds
    - MOST People don’t have a clue how to distribute data. They know that they HAVE to do it, they just don’t know how.
      - a client that seaparated data by username first letter found that 15% of usres start userenames with M. So that was a bad distribution strategy
  - utilization
    - how well are your resrouces being utilitzed:
      - IO wait
      - threads
      - CPU
      - queries
      - disk utilization
      - cache utilization
      - disk saturation
      - memory
      - cache hit
      - cache prunes
      - thread thrashing
      - connections per shared
      - swap utilliztion
      - threads running
      - exceeding high or low water marks
      - growth rate
      - connections usage
    - performance:
      - constant access time is the issue
      - if you focus on performance, you may only buy yourself a bit of time, but then growth hits and you hit the same issues and now it’s HARDER to maintain performance.
    - what’s the difference?
      - performance is not high availability which is not scalability
      - scalability is the easiest thing to implement.
      - performance = ability to proces s or execute a task comapred to time and resources used
      - HA = ability to ensure certain degree of operational continuity – expensive!
      - scalability: ability to handle growing amounts of traffic in a graceful manner or ability to be easily enlarged
    - Pick any two of these – not all three.
      - consistency:
        
        choose this if you don’t care about 24×7 or accomodating high traffic
        
        forget about HA and scalability
      - availability: if you need it up 24×7
      - Partition tolerance: great for high-traffic scalable websites;
    - Vertical or Horizontal
      - Vertical: scaling up
      - add resources to a node (more ram or faster processor
      - can be more than the price of the gain (double fast server might cost more than 2x)
    - Horizontal (scaling out)
      - adding more nodes
      - cost efficient
      - increased mgmt complexity
      - more complex prlgramming model
      - throughput and latency between nodes.
      - good for linear scalability
      - considered more complex and harder to do so many people aboid it.
      - This is no longer really true and there are ways to do this from day one.
      - [One of my clients, WorkHabit, is helping their customers do this from DAY ONE – avoiding the vertical model entirely.]
- which way to go?
  - vertical or horizontal
- how to partitiion?
  - make sure that your aplication does not care about partitioning;
  - abstract / isolate that from the application
  - (grab the slide here)
- how to balance?
  - not that difficult if you start with the right foundation
  - use agile methodologies
  - technical debt is expensive (technicla debt is stuff you put off to do “later”)
  - technical mortage is a killer
  - once you have a good foundation layer, you can build on it.
  - there are frameworks for Jave for example like HiveDB that you can build on.
  - Hire the right people who know those frameworks so that you can build correctly from the sstart
- before and after
  - what dto do early
    - lay the right foundation
    - set up the ability shard/partition
    - decouple components
    - effectivevely cache
    - have a plan in place
  - what can wait until after things start to grow?
    - now focus on micro optimization
    - acaquiring and upgrading hardware
    - performance optimization s and OS tuning
    - (get bullet 5)
    - (get bullet 6)
- who to hire?
  - Get the best architect you can find who can help you build the correct foundation
  - find the libraries you can scale on
  - Developing with Hibernate for example, you can move it to Hibernate Charge and then make it scalable very quickly and easily.
  - But if your people don’t know this, then they’ll hurt you.
  - You need people who know how to build for scale from the beginning with the correct frameworks.
- Best practices
  - go asyncronous
  - Synchronous processes KILL scalability – they requrie you to scale VERY expensively
  - go stateless
  - The Best IO is NO IO – serve from memory, not disk
  - decouple as much as possible
  - build using APIs; easy to scale development that way
- how to kill your project
  - focusing too much on performance up front
  - instead, ensure that you are hiring the right people, building the right foundation, the right partitiioning scheme
- can I offload scalability?
  - most clouds are proprietary, that’s not the answer and often you still end up having to do all the hard work of laying the correct foundation anyway.
  - Be cautious with this approach.
- SUMMARY:
  - go horizontal
  - go asycnhronous
  - architect once up front
  - choose two of consistency, availability, and
- RESOURCES
  - http://mashraqi.com/2008/09/startonomics-startup-scalability.html
  - http://mashraqui.com
  - http://twitter.com/mashraqi
  - fmashraqi@yahoo.com
- QUESTIONS:
  - CDN? Play the top vendors – you can get them down 40% off list
  - Generally loook for what suits your budget; Use them for sure to offload content;

Amy Jo Kim, Putting the Fun into Functional

Gaming knowledge can be leveraged in your social media sites;
You want to go after primal response patterns. The most important type of validation is variable reinforcement – i.e. reinforcement that varies by size and duration. Reward people (or rats or other animals) and they will be more addicted.
To drive addiction, apply the following five gaming mechanics (or some of them.)
- COLLECT: Let people “collect stuff” [reminds me of George Carlin’s “stuff and shit” talk]
- POINTS: (many different ways to accrue points and see leaderboards.
  - System Points: assigned by the software
  - Social points: given by other players to each other; eBay’s reputation system fits into this category.
- FEEDBACK: Bejewelled gives you great feedback on how the game is going;
- EXCHANGES: a transaction. This is like the giving of gifts in Facebook. It has a back and forth flow.If you can add an exchange between the person and the software or the person and another person.
  - Explicit: both people are involved in the transaction. I want to be your friend. They have to accept the invitation.
  - Implicit are where there is an opening for others to engage but they don’t have to. Comments and wallposts are great examples of this. Other users don’t have to respond to the comment.
- CUSTOMIZATION: Let people create custom everything; UI; feeeds; blocks of content.
We just built a new game that we’re releasing soon:
- people can upload their photos and create games from them.
- we built awards that people can collect;
- you can achieve points from creating good games, from playing a lot of games, any other dimensions.
- There are muliple ways to achieve leaderboard status.
- [Makes me wonder about Crossfit. We use a leaderboard and even though I’m in good shape, I’m almost never on that damned board! Would I “feel better” if I was hitting a different board? I know that when I used to adventure race, my teams would often win the “team spirit awards'” but not the “top 3 finisher award. And we felt awesome about that because we were recognized for somethihng that was a different dimension of importance. When I was on the administration side of those races, I really enjoyed handing out those awards.]
- The gaming site built in strong communication with the player;
- We let people customize their world…but not entirely. We insert other people into their friend list for example so that they can see things serendipitously.
CONTACT:
- amyjokim@gmail.com
- Amyjokim (Twitter)
- Shufflebrain (Facebook game);www.shufflebrain.com
QUESTION:
- Q: How do you handle different types of audience? Game players vs. non-game players.
- A: Give them the option of opting out of the gaming system.

Introduction: from Dave McClure:

the two biggest lacking areas in Web 2.0 are marketing and design; The limiting factor is NOT engineering ability.
[Yes…totally agree.]

Understanding Designers; Erika Hall, Mule Design; Jeffrey Veen, Small Batch, Inc.

They’re not separate; they’re integrated
- [this is like Rocketbuilder‘s Integrated Marketing & Sales program that suggests that those are unified disciplines that are fully integrated. Very interesting corollary.]
Design is not something you “sprinkle on”
- it isn’t fairy dust
- “jazzing up your site” is useeless;
Start with goals, timeframes, and constraints;
- goal: business goal drives design goal
Roles/Hats:
- there are really a bunch of different roles:
- roles include:
  - Design strategist
  - visual designer <– this is the person who most people think of as “the designer”
  - information architect
  - Interaction designer
    - A failure in interaction design can not be covered over by visual design.
    - Bad ID =
    - You can’t save that with colour and type
  - Sometimes people batch those up into one thing (ID/IA)
  - Writer
- Frameworks are so powerful now, you can design/build/review LIVE;
- documentation is a function of team size and separation;
- If they wrok side by side, they they can iterate quickly
- go for the lightest weight documentation you can afford given your team size and separation;
- Remember that language is part of your design!
  - [I have seen great cases of this where sites changed words from “post to blog” to “ask a question” or from “forum” to “discussion group”
Case study: Measure Map
- [I really liked the visual interface that Jeff discussed.]
Design flaws
- No goals
- subjective (non-qualified users) using “gut feelings”
SUMMARY
- Design is making and documenting decisions about the experience of your audience.
- It required knowledge, skills, and thoughtful feedback.
- It works when you are clear on our expectations, priorities, and goals.

Ted Rheingold, Dogster:

We’re at 3M in revenue this year;
First things first:
- Do people use yoru product right from the beginning?
- Are happy customer creating new happy customers?
- [i.e. don’t waste time on metrics when your overall volume is irrelevant!]
Objectives and Key Results:
- OKRs must be deetermined in advance
- OKR’s will show transfrmative growth (revenue, users, partners, etc.)
- “The CEO often dreams something up and then sets the team to work on something but there are no organizational goals attached.”
Get clear on your Key performance indicators:
- define the metrics that define and measure progress toward organizational goals (OKRs)
- KPIs are often ratios
- KPIs can and will change over time, do conscientiously
- Everything else i sinfo-porn
- “don’t get sucked into the time vortex of looking at all of thiss information.”
“So what?” / Actionability
- Put EVERYTHING through the So What test?
- Don’t do anything rfor 25% gain. Shoot for 10% minimum gains.
- Keep putting effort only towards things that have real results.
- If you have an idea that you can’t
- Focus on REAL results.
- I spent months doubling adsense revenue (from $1500 to $3000) with a 100K burn rate. WHO CARES?
- Stop working on the low-value stuff. Go for 10x gains, not 10% gains.
- [I love this. It follows the Pena model. Go for quantum leaps, not incremental arithmetic growth]
Testing
- It’s like sex education – everybody says they’ll do it. Nobody actually does it.
- Test, test, test
- A/B – test
- YOu don’t understand your customers and they don’t either and can’t tell you.
- try crazy testing methodologies (multivariate testing, click-desnsity, user experience)
- Watch people using your product. It is the only place where you can collect data from people using your site.
  - iIt’s very expensive and complicated and $25K sticker shock.
  - You can do it on the cheap though.
  - Do not test your mother or friends
  - Pay people $50; present them a script; have them look at your site and tell you what they’re thinking.
- [Jay Abraham is a huge fan of testing for absolutely everything that a business does in all dimensions. So does Tim Ferriss.}
SUMMARY
- Be disciplined
- Have a clear line of sight
- Minimize stats review
- Fight info-porn!
- Do not dilute responsibility among too many stakeholders
- test that your data is sound.
Case study: DOGSTER used SEM/campaigns
- We ran a campaign and used the wrong metrics
- Wrote it on a blog entry at startonomics here
- Marketing initiative: Feb 2007
- We went on a massive customer acquisition binge (hooray – we have all these new customers!)
- …and then realized that holy crap, we increased the # of customers but didn’t realize that we drove down the value per customer from $4 to $0.10. We
- [Jay Abraham uses a great model: NumOfCust * ValuePerTransaction * NumberOfAvgTransactionsTotal
- We found out two things:
  - advertisers only care about uniques
  - people just want infomation, not to set up dog page so we moved to a more heavy information centric site;
    - [I like this because it follows the pattern of expanding your TAM (total addressable market) by broadening the possible number of users;]
Favourite resoources
- Web Analytics – Avinash Kaushik
- Advanced Web Metrics – Brian Clifton
- Google Analytics
- Google Trends
- Adwords Keyword tool
- google website optimizer (A/B multivariate)
- AWStats (fee serever side webstats)
- CrazyEgg (click tracking heat maps)
- Kissmetrics (social space analytics
- silverback (experience testing)
Question: what about partnerships?
- Answer: be cautious with partnerships. They can suck a lot of time out of your business.
- Weigh their requests carefully againsta all other investments on the same ROI basis as we discussed above.

Notes from Dave:

For building MINT, we did a lot of landing page and A/B testing before we did product dvelopment.
[This fits well with the Tim Ferriss model of advertise first to test the buy rates, and then build only the products that get radical uptake.]

LUNCH

Sign up for more like this.