The move, touted by a company exec at the time as “an exciting advancement,” was intended to streamline the process for homeowners considering selling to Zillow as part of its home-flipping business. Zillow promoted this option as a way to make it convenient to sell a home while minimizing interactions with others during the pandemic. Just eight months later, however, the company is shutting down that business, Zillow Offers, entirely.
Zillow declined a request for an interview with Krishna Rao, the company’s vice president of analytics. In a statement, Zillow spokesperson Viet Shelton told CNN Business the company used the Zestimate for Zillow Offers “the same way we encourage the public to use it: as a starting point.”
“The challenge we faced in Zillow Offers was the ability to accurately forecast the future price of inventory three to six months out, in a market where there were larger and more rapid changes in home values than ever before,” Shelton said.
Indeed, since Zillow entered the home-flipping business in 2018, real estate markets have changed in wildly unpredictable ways. The pandemic led to a temporary housing market freeze, followed by a supply and demand imbalance that caused an unprecedented rise in home prices. This may only have complicated the company’s decision to include the Zestimate — which Zillow points out is not an appraisal, but a “computer-generated estimate of the value of the home today, given the available data” — as part of the Zillow Offers process in more than 20 cities.
Artificial intelligence can look at far more information, far more quickly, than a single human could when considering a fair price for a home, weighing factors like comparable home sales in an area, how many people are looking in a specific neighborhood and so on. Still, “you can have a real estate agent look at a house and in one second pick out one critical factor of the valuation that just doesn’t exist as ones and zeroes in any database,” said Mike DelPrete, a real estate technology strategist and scholar-in-residence at the University of Colorado Boulder.
A key part of Zillow
“Three times a week, we create more than 500,000 unique valuation models, built atop 3.2 terabytes of data, to generate current Zestimates on more than 70 million US homes,” the company wrote in a securities filing in 2011. More than 10 years later, the company publishes Zestimates for more than 100 million US homes.
If you’re looking up homes on Zillow’s website or app, the Zestimate is featured prominently in each listing, whether the home is for sale or not. If the house is currently for sale, a red dot is shown next to the words “House for sale,” and the Zestimate, if it’s available for that home, will appear on the same line.
Zillow has spent years improving the Zestimate, going so far as to run a multi-year data science competition to improve the accuracy of the algorithm behind it. The company awarded a three-person team the $1 million prize in early 2019.
The Zestimate currently has a median error rate of 1.9% for homes that are on the market, Shelton said, meaning Zillow’s estimates for half the homes on the market come within 1.9% of the actual selling price. That percentage of error is much higher — 6.9%, according to Shelton — for off-market homes. Being off by as little as 1.9% on a property with a Zestimate of $500,000 is still nearly $10,000; that figure multiplies over many, many homes in different cities across the United States.
An art, not just a science
It’s one thing to build a model on a website that’s often reasonably accurate. It’s another to then try to use that model in the real world to make very costly bets — and do so at scale, according to Nima Shahbazi, a member of the team that won the Zestimate algorithm competition and CEO of Mindle.AI, which helps companies use AI to make predictions. For instance, if any homes Zillow purchased had hidden problems — such as a missed crack in the foundation — the Zestimate would not be able to predict those issues, he said.
“There are many different parts between a very decent model and deploying the model into production that can go wrong,” he said.
Zillow was using the Zestimate to help it make purchasing decisions for homes it hoped to make a profit off of over time. But Nikhil Malik, an assistant professor of marketing at the University of Southern California, said algorithms tend to be good at making fine-grained, short-term predictions, such as for predicting stock prices a second in advance. But there simply isn’t enough data for an algorithm to learn about longer busts and booms, according to Malik, who researches algorithmic pricing and has studied the Zestimate in particular.
There are also many unquantifiable aspects of putting a price tag on a home, DelPrete noted, such as the value of living in the same neighborhood you grew up in or down the street from your parents. These can vary from person to person, which makes it even harder to outsource a home valuation process to a computer.
“It’s a good tool for what it is,” DelPrete said of the Zestimate, but it’s a mistake to think it can be used to accurately predict house prices now or in the future. He sees it as “almost a toy,” meant more for piquing your curiosity when looking up your home or your neighbor’s home online.
“If you want to do iBuying and you’re going to make thousands of offers every day you have to be really good at valuing homes, not only today but three to six months in the future,” he said. “And that’s an art and a science.”
— CNN’s Anna Bahney contributed to this report.