REA Group (REA) is a global online real estate advertising company that owns realestate.com.au, the most visited property site in Australia. For more than a decade, REA has partnered with Thoughtworks to transform the way they deliver real estate services to customers across Australia and Southeast Asia. What began as an enterprise-wide re-platforming project has led to a long-standing, strategic partnership that has brought technology to the core of REA’s organization, underpinned by a robust engineering culture.
Thoughtworks’ most recent engagement with REA saw the team on a mission to develop Australia’s most accurate property valuation tool. This required building upon an existing valuation model by PropTrack, Australia’s leading Automated Valuation Model (AVM) provider, that was acquired by REA in 2018. PropTrack leverages REA’s unparalleled real estate data and behavioral market intelligence to provide timely and accurate information that consumers and customers can trust.
Thoughtworks brought strong expertise in machine learning engineering to help us accelerate the development of our property valuation models.
Thoughtworks brought strong expertise in machine learning engineering to help us accelerate the development of our property valuation models.
Property prices in Australia have been as volatile as the stock market in recent years, which has been fueled by an unstable economy, the global pandemic and fluctuating economic conditions. Now more than ever, people need access to timely and accurate data on the property market to make better informed decisions on what may be the biggest financial decisions in their lives.
Results
in model accuracy, as defined by the percentage of AVM values within 10% of the following sale price.
end-to-end model training, from 3 to 4 days to 5 hours.
to explain specific model predictions, an improvement in time from 1 to 2 days. Customers can now get answers to their queries in a much shorter turnaround time.
to meet PropTrack's model accuracy target.
When Thoughtworks joined this engagement, PropTrack was on their journey of migrating their AVMs from an on-premise solution to the cloud using Amazon Web Services (AWS). The team experienced many typical challenges that large-scale ML solutions face, such as long feedback cycles and unscalable manual testing. This made experimentation more tedious than it needed to be, and slowed the team down from their goal of improving the AVM.
Thoughtworks brought global experience in data and AI, including time-tested continuous delivery practices, to complement the client’s intimate domain knowledge of applying machine learning in property valuations. Implementing Continuous Delivery for Machine Learning (CD4ML) automates repetitive tasks such as manual testing and software deployments, freeing up the team’s time and effort for higher-value, problem-solving work. In addition, adopting CD4ML has helped to improve delivery flow, reduce friction, and accelerate experimentation.
Thoughtworks also assisted with the implementation of an ML experimentation framework using Metaflow. The experimentation framework has helped the team to improve delivery performance and speed by enabling data scientists to design and run large-scale machine learning training jobs on the cloud without writing any infrastructure code. The framework automated deployments, meaning code and models are deployed to pre-production environments multiple times a day, and can be deployed to the production environment with the click of a button, improving speed and predictability of delivery.
According to a recent report, 64% of companies take more than a month to deploy a new ML model to production, and only the top 1% of companies indicated they can deploy a trained model to production within a day. The joint efforts of this engagement have put PropTrack in this top category.
We were delighted to partner with REA on their ambitious mission to deliver Australia's most accurate valuation model. Thoughtworks Data and AI brought global experience in ML engineering excellence to successfully deliver improved model performance in just 8 months.
We were delighted to partner with REA on their ambitious mission to deliver Australia's most accurate valuation model. Thoughtworks Data and AI brought global experience in ML engineering excellence to successfully deliver improved model performance in just 8 months.
Along the way, Thoughtworks helped embed engineering best practices including pair programming, automated testing, cross-functional teams, and cultivating alignment on engineering practices between data scientists and software engineers.
In just 8 months, PropTrack met the model accuracy target through a 12% improvement in performance. PropTrack has extended this work to improve the model further in addition to applying machine learning and engineering principles across all data science models.
