The Hypothesis Driven Innovation Lab - Exposing our Assumptions
BresicWhitney is a lifestyle property group serving modern buyers across inner Sydney, Australia. They saw an opportunity to shake up the real estate industry in Australia. They had an idea that would give buyers the information they needed - instantly. Partnering with us, we designed and tested a document delivery system.
The webapp gives potential buyers access to building condition reports amongst others, free of the standard $500 cost, at any time online or via their mobile device. Something no other competitor is offering their buyers. The platform then closes the loop by sending property agents a report notifying them of the download - giving them an opportunity to get in touch with the potential buyer.
Here, I outline the process from the definition of a problem through to the output of a solution in what appears to be a linear fashion. This is misleading, in that we employed an approach which allowed us to move backwards and forwards through the process at anytime. The key was to ensure we were learning, then delivering value from that learning at all times. This included research, design, and technology build, but also applied to the way in which we engaged the client and progressed the project in step with the relationship.
The Situation
The real estate market in Australia is dominated by realestate.com.au. A property listing site that effectively holds a monopoly over buyers, sellers and real estate agents as the only significant market place to buy and sell property. REA offer an excellent service, with a broad reach - BresicWhitney offer the same excellent service, for a niche.
Forces in Australia’s intense love of property ownership and a culture of auctions over private treaty as the method of sale, make competition fierce. The sheer demand puts a strain on the relationship between buyers and agents, and has resulted in a less-than-optimal reputation for most realtors nationwide.
BresicWhitney have been grappling with this issue for some time and realised that if they don’t change the way they look at the business they will fade into irrelevance. When staring down the barrel of your own demise, you are prompted to really get to know yourself, and what value you bring to your clients. In this case, BresicWhitney consider buyers and sellers to be their clients, not just another vendor on their books.
The commonly practiced way to run an open-house (a viewing of the property for sale open to interested buyers) is for the agent to stand at the door and almost demand that the potential buyers provide their name and contact information - kind of like a gate-keeper. Most people just go with it and provide their details, but they generally don’t like it. It’s not a nice experience. This interaction immediately set’s up the agent for an adversarial relationship with any buyer; which is really not good for business. Further, it provides a full day of follow up phone calls that the agent will need to do on Monday usually, since almost all open-homes are conducted on Saturdays.
A new vision emerged. More focussed on empowering the buyer. This was when we were asked to join the conversation. The BresicWhitney stakeholders understood the importance of changing the way they operated, and had articulated a vision which was an ideal situation for success. A clear vision of the kind of interactions they needed to remain relevant in the market and industry, but with flexibility to explore different ways to support that vision. Conversely a vague vision, with its infinite flexibility, would have been much harder to define ways to deliver value.
Beginning the Engagement in an Agile Way
We met with the BresicWhitney team for a discovery session to understand the journey up to that point, and began to define what the engagement would look like. Whilst not being mature in agile practices, BresicWhitney did value the benefits and were open to taking our lead - provided we could demonstrate value.
Our proposal consisted of a set timeline, an estimated team of four Thoughtworkers, and some value based deliverables. We set out for a specified amount of time, with a goal to validate (or reject) a set of hypotheses, and to also to explore the problem space a bit ourselves.
There were no deliverables in the traditional sense and no set processes we would follow. We would however learn about the problem, confirm or deny the things we thought we knew, and define opportunities for potential solutions. We used a selection of the many items in our toolkit, some of those would be technology, many of them not.
We were allowed to go ahead on a toolkit and a promise because we were able to articulate the importance of eliminating risks as early as possible. It was agreed that this timeboxed project would focus on that very task: identifying and eliminating risks.
Research
We set aside the hypotheses and aimed to learn as much about the key actors in the buying relationship as we could. This was important in that it allowed us to follow what we saw and learnt rather than simply following assumptions or common beliefs about the customers. One issue that presented was the Burden of Knowledge which meant that even though we wanted to ignore the hypotheses presented by the client, it was difficult to do so once we are aware of them. We dealt with this by engaging a researcher to observe buyer behaviors separate from us to ensure that the findings would not be tainted by our knowledge of the client’s problem hypotheses.
The research team consisted of two designers, plus the additional researcher addressing the burden of knowledge problem.
Our research began with qualitative methods aimed at understanding the kinds of experiences and journeys buyers and agents take when going through the process. We conducted interviews with agents and buyers, aiming to understand their tasks and activities over the course of their week. This was captured as conversations, rough sketches, and notes for later review. We augmented our research with a series of phone interviews with recent buyers who were both successful and unsuccessful in their property search.
We also began to craft the “as-is” journey map which provided the rich material to begin thinking about potential solutions. We needed to really understand the problem in order to create the right solution.
We followed this up with a contextual inquiry to better understand customers direct experience, and to unearth insights that they may not have been aware of themselves. This included observations at private showings and open-homes on a Saturday. It’s important to point out that there are two types of empathy at play here. Cognitive Empathy and Emotional Empathy.
Cognitive Empathy is the kind of information we gathered when discussing the different views, needs, and opinions of buyers with the key stakeholders. This is important as it helps to foster the idea that there are points of view other than your own, and that we ought to seek them out.
Emotional Empathy is the kind of understanding you get by feeling it yourself. Through observing, or discussion you can truly feel what it would be like to experience life as that person. This is often the most radical in its effect on executives and as a source of insight.
The output of the various observations and conversations was expressed as an Empathy Map. This guided us further in our discovery and is a helpful resource when attempting to articulate problem statements we were trying to solve.
Using some of the empathy maps and insights generated, along with some of the hypotheses put forth by the client, we were able to articulate the core problems, and if solved would have an impact. They were:
- Buyers did not have confidence in buying guides, as they were an estimate by the agent of how much the property would sell for.
- Buyers we reluctant to purchase property inspection reports at around $500 per report because they could not be sure the property would be within financial reach at the auction (due to lack of confidence in the buying guide).
- Buyers didn’t have trust in their interactions with agents, and asked for more transparency.
To further validate these problem areas we conducted an email survey to 4000 recently engaged buyers who had a range of experiences with BresicWhitney. Our questions for this kind of qualitative research were more directed, and the goal was to confirm at scale whether we were on the right track with our problem insights.
With our new findings, we consolidated and refined the “as-is” journey maps to ensure it reflected the situation accurately. We then tested our understanding by playing this back to the stakeholders, something we also did at the end of our conversations with buyers.
Testing is not something you do at the end, it is an integral part along the way to ensure that the the problem you are planning to solve is valid. It’s vital that the design team properly understands the constraints, amongst other things.
Design
So how do we build something awesome?
But we still had to test our idea with the buyers, and this is where the Innovation Lab is most obvious. Really the Innovation process started way back when we had some ideas which we explored with buyers, but this is where it starts to take form.
The Innovation Lab is not a specific place, it’s more of a way to describe the critical activities that allow us rapidly validate assumptions of all kinds.
With a limited amount of time - and a high level of confidence that we were solving the right problem - we proceeded to developing a solution. We were however, acutely aware that we were in fact testing two things; the problem and the solution.
We agreed to work on the following hypotheses:
#1 Providing potential buyers with information will focus the conversation on buying.
#2 If we provide information early in the process, buyers will prefer / seek out BresicWhitney.
We sketched rough interface ideas at the feature level on Post-Its. We were essentially generating material to work with at a more abstract level, comparing and evaluating whether these elements could begin to solve the problems at hand. The material generated blurred the lines between the interface, and a vision of the journey map. With iteration and refinement the ideas evolved, some died, and others emerged that solved the more basic of the problems experienced by the buyers.
The Prototypes
Prototyping in HTML can be quick. No need for big upfront design, just sketches that define the core feature proposed to solve the problem. We needed a way to execute a meaningful handshake with the buyer and provide them with the relevant property reports that would remove the problem of price guide accuracy. We had to do it in a way that allowed them access in the first place.
Agents' needs could not be forgotten at this point also, meaning we needed to include a way for agents to identify who has downloaded (this is a legal compliance issue as well), and to provide meaningful reports and alerts. We used email to send short notifications, a good example of making best use of simple yet effective standardised technology.
Technology was a huge enabler in this phase, as was the team skillset which allowed a solution to be designed using technology as the fabric of the design process, not just the method for delivery.
The platforms chosen provided us with the ideal working conditions and material to visualise ideas, test them with real users, and iterate quickly at scale. An advantage over using paper prototypes.
Equally important was having someone on the team who had these skills. The UX Designer on the team was also able to write front end code, which is an excellent anthropomorphic example of the concept of using digital artefacts as the material for exploration and design.
Quick and dirty sketches moved straight into an HTML prototype. Our view was that given the low frequency of access to buyers, a few times a week at best, we could make use of HTML as a prototyping tool rather than using paper sketches. Prototyping on paper is a fantastic method if you have regular contact with the end user to test each rapid iteration on. In our case we did not have this luxury.
We were now at the end of the first week, with only two weeks and two testing cycles left before the project ended. I should point out that our HTML quickly evolved into a very small node.js app, which was just as fast and flexible to work with as we had an experienced developer on it.
It felt like we wanted some kind of document delivery system, but what exactly would this look like? There were existing solutions out there that do the basics such as Hightail, Dropbox, and even Google Drive. If we were designing in a bubble, we could choose any of these, and essentially build nothing… alas, this is not a useful solution for the client or the buyer.
Both the buyer and the agent needed a solution that would integrate into their existing processes and systems.
Drawing on a number of concepts in the evolving practice of Service Design, we spent more time researching and understanding a new user group; Operations Staff. This group were responsible for managing a number of processes including document management, website content management, and marketing. Our solution was drifting in this direction and it was important we understood all the constraints and opportunities.
Ironically, as important as understanding all the systems and enterprise integrations required are, the key to a successful prototype is to remain free from too many constraints. We needed to ensure we could easily define what we are testing. We resisted trying to solve everything, and focused on validating the problem and the solution as a concept before tackling it as a product integration issue. Our prototype did offer integration into the users’ everyday processes, and faked the enterprise integration component.
The Technology
It’s interesting to point out that we have already talked about technology as part of the design process, and it is definitely true that technology was a strategic enabler for the design process as much as it would become the method of delivery.
Using digital artefacts as the fabric of the design process affords a number of advantages, and comes with a number of challenges.
The good:
- Working with the real, tangible elements increases the designer’s ability to visualise solutions, and to perceive the impacts good and bad
- It’s easy to work at scale (scale for testing purposes)
- It’s easy to work in a distributed team, or audience
- User feedback is closer to what you would get for a finished product (this can also work against you)
- It’s easy to release changes at high frequency
The bad:
- It relies on having a technical method of deployment at the time of concept
- User feedback can be skewed by interface noise such as disliking a colour, or having issues with the text on a particular button
- It’s slower to make changes
- There is a trade off between speed and validity of response, especially if attempting quantitative validation
The Technology Stack
Software: Starting in HTML, we quickly jumped into creating a responsive node.js app, which we used during prototyping. This still met our needs as a lean piece of software and a material for the design process.
Heroku scalable web hosting: an ideal solution for prototyping using code that provides a reliable stack but is scalable based on load required. The free account is ideal for this kind of project.
Magnum CI: The Continuous Integration product of choice for test and release automation, which allowed us to push out new releases as required without having to manage the details.
Mixpanel: This provided user based analytics which allowed us to track user behaviors and learn in real time how users were interacting with the prototype during and after our session with them.
Email: Still worthy of a mention, email is a key piece of technology which when used correctly can be an excellent product.
Lessons Learned
We started with learning goals focussing on who was interested in which properties - we could infer this by the types of documents downloaded. The analytics platform provided this information very well, but it pays to be very clear about what you want to learn before hand. It’s easy to miss a key interaction that would confirm what you really need to know, but without it you’re still just working on hunches. We were able to track exactly who downloaded which document and when, and from what location. This information was important to compare and validate the app against the current manual process.
Unexpected Learning
We were surprised by a number of things. Agents had described interactions with buyers the way they saw them, which were filled with assumptions and constraints. When we released this app, we were free to see behaviours without the assumptions and the constraints.
We learned that contrary to the way agents organised their time, buyers were not particularly interested in follow up calls on Monday following a Saturday open home.
We also learned that buyers were looking at the documents for many properties that matched their needs in the app, then would attend the open homes afterward. This did not map to what was initially expected. We thought buyers would attend an open home, then do the due diligence (accessing the reports) after, but it was the complete opposite in many cases.
These learnings helped us shape the product for the next round of testing, and eventual full scale build. Many of these were totally unexpected, but invaluable to the success of the product evolution.
Post Launch Research
Within four weeks of starting the project, the first property was sold primarily supported by the prototyped document delivery experience. We interviewed the buyer and further confirmed that our hypothesis for the buyers interactions over time was not accurate.
This particular buyer had attended one open home then viewed the reports for that home and others during the week. On the following weekend, the buyer attended one of the other homes (documents already viewed), and made an offer closing the deal within days.
This was a moment of validation of the process we had employed. It showed that the things we had learned in the process were accurate and valid for a live sale.
Further, it validated the prototyping process we use in our Innovation Lab.
Impact to the Business
This project was a key component of a broader strategic play in response to a change in conditions in the real estate market. Whilst there were other factors at play to set the context with clients and with agents (staff), both the prototyping and product build phase were wildly successful.
The new app delivered many positive outcomes for BresicWhitney. The buying cycle was reduced from weeks to one day and there was an increase in meaningful conversations between buyers and agents. Agents were also freed of spending their Monday’s making calls to people who didn’t want to hear from them, allowing them to focus on committed buyers. Buyers demonstrated a preference to BresicWhitney by traversing across properties online before and after their physical inspection. The net result was an increased trust in BresicWhitney, and an increased confidence of qualified buyers to bid at auction. See the full client story here.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.