Coffee Notes (May 25, 2010)

Attendees

  • Gregarious N (Lil’Grams)
  • Hiten S (Kissmetrics)
  • Maurice W (Pay4Tweet)
  • Dan B (Fanosaurus)
  • Dennis B (Agilist)
  • Ryan B. (Liv)
  • Patrick M (Refinery CMS)
  • Augustine (AE Net)
  • Richard K (mtranslation.com)
  • Mike L. (Android Gaming Middleware)
  • Stuart A. (Architect / Academy of Art Collab Platform)
  • Tommy P
  • Lydia S
  • Steve (CloudFab)
  • Nick (CloudFab)

Why Multi-Variant Testing? (Patrick M)

  • Simple process and hoping to test variations of the copy to get people to sign up (Patrick M)
  • Google’s tool is called Web Site Optimizer
  • Have to be careful since there’s often unintended consequences - make sure you’re getting what you want when you measure (Dennis)
  • How do you construct the analysis to understand your results (Stuart)

When is the right time to start AB Testing? (Gregarious N)

  • You can start it as early as possible, we do it all the time (Ryan B)
  • Need to curb your enthusiasm with the results, make sure you’ve found significance (Gregarious N)
  • How many people does your finding actually apply to? (Lydia S)
  • 1 complain equals 25 actual complaints (Steve)
  • Find the holes in what you’re doing first, then your learning will be more effective (Hiten S) 

What’s a good conversion rate? (Hiten S)

  • What’s your conversion rate?
  • Benchmark against yourself so you understand what you’re trying to achieve
  • Don’t jump the gun (Tommy P)
  • If you can’t drive traffic easily, try to establish your baseline first (Hiten S)

How do you optimize first-user experience? (Hiten S)

  • Find people who have dropped off
  • Use that to generate the AB Test since you’re working from learning
  • Throw out your assumption for the right process first

How do you message people to get them back?

  • Make sure when messaging your users that it doesn’t seem like a mass email
  • We tried a mass mailing and go 1 response (Nick)
  • Followed up with individual messages and we got tons of lengthy responses (Steve)

Experiment in Capturing Feedback (Stuart)

  • Instrumented their app with a small bit of acceptance testing for the UI
  • Green checkbox means they were successful
  • Red checkbox means they missed a spot
  • Small note field for optional comments
  • Larger customers are often scare of this, however, small implementations could be more approachable
  • Got more and more participation as people used it more
  • Mass collaboration and consumer insights
  • Having an option for feedback is always better than not having the option (Maurice)
  • Much less annoying to the user than sending them email

How do you bring out the outliers? (Dennis)

  • Track small points to see if they’re happy or not
  • Use it as early warning system
  • Once you know who they are you can engage them further

On Pittsburgh

  • City is definitely not as involved as it could be
  • Project Olympus is helping to fund early ideas
  • MBAs seem more likely to stay than the engineers

On 3D Printing

  • Prototyping and proof of concept
  • Marketplace for finding the production of physical goods
  • Quality and turnaround is often more important than price
  • Invisalign is likely the most successful, visible use case
  • Done in america often since labor is not the expensive part
  • Very razor-blade industry, selling the supplies are much more expensive
  • Expensive to do samples since materials can be expensive
  • Doing PR via blogs and direct sales to industrial designers
  • Time sensitivity is what helps prevent customers from circumventing the service
  • “we can’t afford to shoot for the big dream now, you gotta have money to reach break-even” (Steve)