AI Replaces Appraisers

Ben Taylor Ben Taylor
August 8, 2019 AI & Machine Learning

All the data that matters:

What data actually matters for appraising a property? The number of bedrooms? The number of bathrooms? Absolutely. When it was built? Your neighborhood home prices? That too. There is a long list of things to consider, this is a complicated process for humans and not much has changed with the process for decades.

Something that human appraisers have struggled to consider are all of the unstructured elements on the property. Your landscaping, the view from your porch, the color/style of your neighbor’s fence, the style of your kitchen, etc… Many of these topics have been too “subjective” for influence on your price estimate. Sure, if there are gross quality issues (damaged flooring, etc..) that can go into it, but your choice in tile for the backsplash? The type of rock in the yard? Never. If anything, it may have influenced the property but it was merely the appraiser’s personal bias, not something they could quantify or measure.

Until now…

Like an alien vacuum from space, AI can suck in ALL of the data now. Literally, everything on the property, structured, unstructured, it doesn’t matter anymore. In addition to that, it can also consider a lot more properties than just the 3–5 reference properties typically used. I’m honestly amazed this role wasn’t replaced sooner based on 2014 technology. Sometimes with industries that are slow to innovate, it takes an outsider to come in and churn the industry with new innovation.

Why 2019?

Now in 2019, building hybrid models with structured/unstructured data has become simple enough that an engineer can consider it. A problem that was once a multi-million science project, is now an afternoon curiosity for an engineer. That means, the bar finally got low enough, that this problem became considered low hanging fruit. Uh oh…

The Problem:

Reviewing properties in Utah we have 20,000 properties for sale. Here they are, colored by price percentile (purple=cheap, yellow=$$). You can see the Park City cluster lighting up in all yellow, it is spendy if you want to live 5 min from a ski resort.

No alt text provided for this image

Here is a closer look at the Park City (Utah’s Aspen equivalent) cluster. You can even see my house in this view.

No alt text provided for this image

What Data Is Available?

Feeding in 1,003 categorical variables (e.g. home style, air conditioner type, zip code, etc…), 280 continuous (# of beds, # of bathrooms, etc…), and the unstructured data (images/text) we can build a holistic model. Then for comparable properties, we can run a shotgun experiment to feed the beast with 1–50 similar properties to see what floats to the top. This adds 294 new continuous variables, bringing the grand total of structured variables to 1577. These groups show comparable properties to the property of interest, some are close by, others are far away:

No alt text provided for this image

Building A Model

Considering all of the data that matters we can build a single model using Ziff’s AutoML platform which will automatically build features from structured and unstructured data. This is more of a holistic problem, where we have a lot of different data types being considered.

Validation:

Without having a domain expert in the loop we are seeing an r-value of 0.98. With actual domain expertise and appetite to cycle through this problem, we would expect to go above 0.99+. Here is a plot of our validation between actual and predicted.

No alt text provided for this image

Insight:

Models without insight aren’t very useful. With every model built there is an insight report that can be reviewed to understand what is going on. This report shows what macro features are driving the prediction.

No alt text provided for this image

We can see the image of the realtor on the property offered no lift (0%), but structured offered the most with (52.9%). The text description of the property offered a significant lift at (21%), but not as much as all of the image contributions combined.

Ziff has the only platform right now that can respect multiple image types for the same problem (e.g. images in the home, satellite images, etc…). We had 3 different satellite data resolutions, images in the home, the main image used for the property profile, and the image of all of the realtors involved on the property. It is encouraging to find out that the image of the realtor had no lift, because if it did… what would that mean?

Unstructured Insight:

Looking inside the encoder behavior we can look for quality and possible topics. We can see that the larger resolution satellite images were able to capture more consistent groups of high priced features. The argument here is this is probably capturing foothill and water/mountain proximity. Reviewing the topics by opening up the property IDs in these clusters can answer that question.

No alt text provided for this image

The encoder portion on the far right shows some large clusters of expensive homes, where the cluster on the far left struggles to find features with big trends. Similarly, looking into the description and the main image (below) on the home we also find clusters between high/low priced neural features.

No alt text provided for this image

Similar to the satellite image clusters, we can see that information has been learned from these unstructured elements as well for text and main image. Looking into these clusters gives the data owner strategic insight into what is driving predictions from the unstructured datasets. We haven’t expanded that out in this article, but specific clusters can offer the user very specific topics from the original image/text (or video/audio if that had been included) that a subject matter expert (SME) could react to. Need new risk items for your underwriters to consider? Reviewing embedding clusters is a great approach for finding those.

Feature Importance:

The top feature importance breakdown seems fairly intuitive for anyone that knows anything about selling a house. The exciting thing with this insight is the unstructured elements (e.g. text description, main image, and satellite image) are showing up as three of our top five drivers.

No alt text provided for this image

You can also see the local house comps, as a structured variable, is a really strong feature at #2 and #3. Another one I like is the structured features from local schools. We also have more granular insight with the school data (rating, distance, grade level), but for this report, we have grouped them.

Next Steps:

The real value from the model insight with looking into which properties had the highest prediction errors, topics from embeddings, and feature importance ranks is that this insight can cause discussions from the SMEs. The most successful groups are the ones that can iterate on a problem 5–10 times by including or removing features. Inviting a SME to this problem, we would expect the model performance to improve significantly.

Conclusion:

There is a huge opportunity right now for a tech company to begin the process of automating home/commercial appraisals. The technology support with AutoML and deep-learning is there now, where this is a <24hr curiosity for an engineer instead of a 7 figure high-risk 12–16-month science project. As these types of problems become low hanging fruit, we will see more job disruption. At first, the jobs will be augmented/validated, and then eventually they will be automated (except appraisals that are predicted to have issues e.g. on a ski resort, etc…). In the end, there is no reason why this wouldn’t be completely automated, I’m thinking sooner than later. If you have worked your entire career as a boots-on-the-ground appraiser the winds of change of started.

But how would you know if you were better than a human?

This is pretty straightforward, you could create a human baseline where you have multiple appraisers appraise the same property and you get an understanding of their inter-rater reliability. Use that number as their error, and demonstrate that AI can go below that. Ideally, the bank/creditor has a better understanding of value than just a human appraiser, but like real-estate, they may be slower to innovate. So maybe even reviewing the concept of home value would be worthwhile, but that is another discussion/blog to get to the bottom.

Determining supply/demand pricing in a volatile housing market is something a whole room full of credit SMEs would have to debate with an AI expert in the room.

Before Tesla goes private at $420/share human appraisers might consider a vacation to Denver.

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Ben Taylor

    Tags
    Artificial Intelligence
    Leave a Comment
    Next Post
    Sidechain – The New Dimension in Blockchain World!

    Sidechain - The New Dimension in Blockchain World!

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    More in AI & Machine Learning
    AI & Machine Learning,Future of Work
    AI’s Role in the Future of Work

    Artificial intelligence is shaping the future of work around the world in virtually every field. The role AI will play in employment in the years ahead is dynamic and collaborative. Rather than eliminating jobs altogether, AI will augment the capabilities and resources of employees and businesses, allowing them to do more with less. In more

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    How Can AI Help Improve Legal Services Delivery?

    Everybody is discussing Artificial Intelligence (AI) and machine learning, and some legal professionals are already leveraging these technological capabilities.  AI is not the future expectation; it is the present reality.  Aside from law, AI is widely used in various fields such as transportation and manufacturing, education, employment, defense, health care, business intelligence, robotics, and so

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    5 AI Applications Changing the Energy Industry

    The energy industry faces some significant challenges, but AI applications could help. Increasing demand, population expansion, and climate change necessitate creative solutions that could fundamentally alter how businesses generate and utilize electricity. Industry researchers looking for ways to solve these problems have turned to data and new data-processing technology. Artificial intelligence, in particular — and

    3 MINUTES READ Continue Reading »

    About Us

    Incubated in Harvard Innovation Lab, Experfy specializes in pipelining and deploying the world's best AI and engineering talent at breakneck speed, with exceptional focus on quality and compliance. Enterprises and governments also leverage our award-winning SaaS platform to build their own customized future of work solutions such as talent clouds.

    Join Us At

    Contact Us

    1700 West Park Drive, Suite 190
    Westborough, MA 01581

    Email: support@experfy.com

    Toll Free: (844) EXPERFY or
    (844) 397-3739

    © 2023, Experfy Inc. All rights reserved.